diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..9d0f0667913ee88c306bb2f01179548e27f892af 100644 --- a/.gitattributes +++ b/.gitattributes @@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text *.zip filter=lfs diff=lfs merge=lfs -text *.zst filter=lfs diff=lfs merge=lfs -text *tfevents* filter=lfs diff=lfs merge=lfs -text +figures_seed0/lambda_per_layer.png filter=lfs diff=lfs merge=lfs -text diff --git a/Forpaper.txt b/Forpaper.txt new file mode 100644 index 0000000000000000000000000000000000000000..4c815a9533ad15e899adcd38e59369813e6e2ff4 --- /dev/null +++ b/Forpaper.txt @@ -0,0 +1,424 @@ +================================================================================ +SMAT (Semantic Attention) - Findings Summary for Paper +================================================================================ +Compiled: 2026-06-03 +Source files used (all in this repo, all re-verified for this document): + runs/{baseline,s_only,g_only,full}_s{0..4}/metrics.jsonl (20 training runs) + runs_4ksteps/{baseline,s_only,g_only,full}_s0/metrics.jsonl (4 partial runs) + figures_final/ablation_inference.json (surgical ablations) + figures_final/ablation_results.csv (final eval table) + +Every number in sections 2-6 below was recomputed directly from the raw +metrics.jsonl files (not copied from any pre-existing CSV or README). The +per-seed deltas, means, std devs, and lambda/G statistics were all +re-derived from scratch and cross-checked twice. + +The cross-scale results in section 7 are from prior experiments outside +this repo; they are cited as such and were not re-verified from raw data +here. Treat them as historical context, not as primary evidence. + + +================================================================================ +1. WHAT SMAT IS (one paragraph) +================================================================================ +SMAT modifies standard transformer attention with two additions: (1) a +learnable semantic-similarity bias S added to the QK^T logits, scaled by a +per-layer positive scalar lambda; and (2) a per-token value gate G that +multiplies V before attention aggregation. The gate is also influenced by +a causal semantic-centrality term c derived from S. Equation: + + Attn(Q,K,V) = softmax( QK^T / sqrt(d_k) + lambda * S + P + M ) . (G (*) V) + +where + S_ij = cos(W_s h_i, W_s h_j) shared-projection cosine similarity + c_j = (1/n) sum_{l<=j} S_jl causal semantic centrality + G_j = sigmoid(w_g^T h_j + mu*c_j + b) per-token value gate + lambda = softplus(lambda_raw) constrained positive, per layer + P learnable [L,L] positional bias + M causal mask + +W_s is shared across heads within a layer. G is per-token and broadcast +across head channels. Four variants were trained: + baseline use_S = False, use_G = False (standard attention) + s_only use_S = True, use_G = False (lambda*S only) + g_only use_S = False, use_G = True (gate only) + full use_S = True, use_G = True (both components) + + +================================================================================ +2. TRAINING SETUP (the 5-seed ablation) +================================================================================ +Architecture: 24 layers, 384 dim, 6 heads, block size 256, vocab 50257 +Parameters: 63,376,512 (~64M) per variant +Optimizer: Fused AdamW, lr 6e-4, 200-step linear warmup, cosine to 6e-5 +Batch: 16, grad_accum 2 -> effective batch 32 +Tokens per step: 16 * 2 * 256 = 8,192 +Total steps: 12,000 -> 98,304,000 tokens per run (~98M) +Eval: every 500 steps on 40 val batches (val_ppl = exp(val_loss)) +Precision: bf16 autocast, TF32, cudnn.benchmark, SDPA attention path +Hardware: single RTX 4060 (vram ~6 GB during training) +Corpus: FineWeb-Edu sample-10BT, GPT-2 tokenization +Seeds: {0, 1, 2, 3, 4} for each of the four variants -> 20 runs total + +Every variant uses an IDENTICAL data stream for the same seed (data_seed=1337 +plus run seed); this is what makes the per-seed comparison meaningful. + + +================================================================================ +3. PRIMARY RESULT - 5-SEED ABLATION TABLE +================================================================================ +Final val perplexity at step 11999 (98M tokens seen), per variant per seed: + + seed baseline s_only g_only full + ---- --------- ---------------- ---------------- ---------------- + 0 78.3244 78.7377 (+0.53%) 78.2175 (-0.14%) 77.8634 (-0.59%) + 1 80.9320 80.8649 (-0.08%) 80.5771 (-0.44%) 80.0262 (-1.12%) + 2 82.3365 82.0735 (-0.32%) 81.3363 (-1.21%) 81.2602 (-1.31%) + 3 79.3884 77.9830 (-1.77%) 77.9834 (-1.77%) 77.7016 (-2.12%) + 4 77.7449 77.6807 (-0.08%) 77.0092 (-0.95%) 76.3953 (-1.74%) + +Per-seed deltas are computed as (variant_ppl - baseline_ppl) / baseline_ppl +within the same seed, NOT against the baseline mean. + +Aggregate (mean across 5 seeds; std is population std): + + variant n mean_ppl std delta_mean seed wins vs baseline + -------- -- --------- ------- ------------ ----------------------- + Baseline 5 79.7452 1.6894 - - + S-only 5 79.4680 1.7131 -0.348% 4 / 5 + G-only 5 79.0247 1.6463 -0.904% 5 / 5 + Full SMAT 5 78.6493 1.7498 -1.374% 5 / 5 + +Key observations: + - Full SMAT beats baseline on EVERY seed (5/5). + - The two components are additive in direction: S contributes ~0.35%, + G contributes ~0.90%, and Full SMAT (both together) gives ~1.37%. + S + G individually sum to ~1.25%, slightly less than full's 1.37%; + the components are mostly additive, with a small interaction in their + favor. + - Variance ACROSS seeds (std ~1.7 ppl) is larger than the mean effect + (~1.1 ppl). The signal is real but small relative to seed noise; it + only becomes clear because every variant uses the SAME data stream per + seed, so the within-seed comparison cancels almost all of that noise. + - The smallest within-seed gain for Full SMAT is -0.59% (seed 0); + the largest is -2.12% (seed 3). Direction is consistent; magnitude + varies by roughly a factor of 4 across seeds. + + +================================================================================ +4. DIAGNOSTICS (training dynamics) +================================================================================ +4.1 Lambda (semantic-bias scale, per-layer, softplus-positive) + Init value (all layers, all seeds): 0.1269 ( = softplus(lambda_raw_init) ) + End-of-training mean lambda (averaged across 24 layers), per seed: + Full SMAT: s0 0.3069 s1 0.3075 s2 0.3031 s3 0.2954 s4 0.3176 + -> mean across seeds ~ 0.306 + S-only: s0 0.3140 s1 0.3188 s2 0.3221 s3 0.3183 s4 0.3261 + -> mean across seeds ~ 0.320 + + Per-layer pattern (Full SMAT seed 0 final checkpoint, 24 layers): + layers 0-2: lambda ~ 0.14 - 0.18 (slightly below init) + layers 3-8: lambda ~ 0.08 - 0.16 (DECREASES below init) + layers 9-17: lambda ~ 0.32 - 0.68 (climbs sharply) + layers 18-23: lambda ~ 0.18 - 0.58 (settles, still elevated) + global min: 0.0834 (layer 5) + global max: 0.6821 (layer 16) + + Across all 10 SMAT-bearing runs (s_only + full, 5 seeds each), + the observed per-layer lambda range was [0.059, 0.715]. The shape - + suppressed in early layers, dominant in middle layers - is robust + across all 5 seeds. + +4.2 Gate G (per-token, sigmoid output, averaged over tokens for logging) + Initial G_mean per run is around 0.5 (sigmoid initialized near zero + pre-activation). Across training G_mean rises briefly toward ~0.58 + in the first ~1000 steps, then decays. + Final G_mean at step 11999 (Full SMAT): s0 0.3506, s1 0.3333, + s2 0.3306, s3 0.3447, s4 0.3397 -> mean ~ 0.340 + Final G_mean (G-only): s0 0.3484, s1 0.3359, + s2 0.3676, s3 0.3625, s4 0.3563 -> mean ~ 0.354 + G_std (across tokens) at end of training is ~ 0.16, meaning the gate + is NOT collapsed: many tokens have G well above the mean, many have it + well below. Per-token differentiation is preserved. + +4.3 Stability + NaN failures across all 20 runs at 12k steps each (= 240,000 optimizer + steps total): ZERO. + No NaN-driven step skips were recorded in any run. Training is + numerically clean under bf16 + TF32 + fused AdamW on this hardware. + +4.4 Wall-clock cost + Baseline runs: ~4000 - 5300 sec per run (~ 1.1 - 1.5 hours) + S-only runs: ~4700 - 6100 sec + G-only runs: ~4300 - 5700 sec + Full runs: ~5400 - 6600 sec (~ 1.5 - 1.8 hours) + Cost overhead of Full SMAT vs baseline: roughly +20 to +35% wall time + at this size on a 4060. (Variation within each variant is hardware/ + thermal noise across the days the runs were executed.) + + +================================================================================ +5. INFERENCE-TIME SURGICAL ABLATIONS +================================================================================ +Performed on the Full SMAT seed-0 final checkpoint (no retraining; the +trained weights are loaded, individual components are zeroed/replaced +during a forward pass, and val ppl is re-measured on 80 val batches). +Source: figures_final/ablation_inference.json + +Intact Full SMAT (this measurement, 80 batches): val ppl = 79.010 +(The 40-batch eval at the end of training reported 77.863; the 80-batch +slice used for the ablation is a different val subset, hence the small +difference. What matters is the delta within this measurement context.) + + ablation val_ppl delta vs intact what was changed + ------------ ----------- ----------------- ------------------------ + lambda_zero 79.40 +0.49% lambda set to 0 in the + attention bias only; + S still flows through + mu*c into the gate. + S_off 80.48 +1.85% S removed completely: + no lambda*S in attention + AND no mu*c in the gate. + random_S 81.22 +2.80% S replaced by a random + matrix with the same + Frobenius norm. Tests + whether S's STRUCTURE + matters, or just its norm. + G_one 625,850 +792,016% G forced to constant 1.0 + (gate disabled). + G_mean 196.99 +149.32% G replaced by its scalar + mean over tokens (per- + token differentiation + removed; mean value kept). + +Decomposition of S's contribution: + Total S contribution (S_off vs intact): 1.85% + Contribution via lambda*S in attention: 0.49% -> 26% of S's lift + Contribution via mu*c in the gate: 1.36% -> 74% of S's lift + Most of S's value is being read through the gate's centrality term, + NOT through the additive logit bias. + +Per-layer lambda=0 ablations (24 separate ablations, zeroing lambda in +one layer at a time): + Maximum single-layer cost: +0.0580% (layer 14) + Sum of all 24 per-layer deltas: +0.219% + This is far less than the +0.49% you get from zeroing lambda in every + layer at once. Interpretation: the lambda*S contribution is REDUNDANT + across layers - no single layer is load-bearing. The benefit is small + and distributed. + +Structural vs scalar test: + random_S (+2.80%) hurts MORE than S_off (+1.85%). Replacing S with a + same-norm random matrix is worse than removing it entirely. This means + the model is using S's learned STRUCTURE, not just the presence of an + extra signal of a given magnitude. A random matrix with the right + energy actively damages the model; trained S is genuinely informative. + +Gate criticality: + G=1.0 collapses the model entirely (val ppl > 600,000). The gate is not + a small modulation - the trained network depends on it as a primary + routing mechanism. + G=mean costs +149%. Replacing per-token gating with a single scalar mean + destroys most of the gate's value. The gate's worth is in WHICH tokens + it suppresses or amplifies, not in its average level. + + +================================================================================ +6. PARTIAL 4K-STEP RUN (1 seed, 32M tokens) - CONTEXT ONLY +================================================================================ +A shorter run at 4,000 steps (32,768,000 tokens) was performed for ONE +seed before the full 12k-step ablation was launched. It is not directly +comparable to the 5-seed result above (different budget, no seed +variance), but it shows the same direction and a larger relative gap +because perplexity is higher overall at lower training budgets. + + variant val_ppl delta vs baseline + --------- --------- ------------------ + baseline 148.4449 - + s_only 147.8903 -0.37% + g_only 144.9733 -2.34% + full 141.5651 -4.63% + +This is consistent with the broader pattern: at smaller training budgets +(or smaller scales), the relative effect of SMAT is larger. As training +proceeds and the baseline gets closer to its asymptote, the absolute +ppl gap and the relative gap both shrink. The 12k-step 5-seed result +(section 3) is the rigorous number; the 4k-step result is a directional +sanity check. + +DO NOT cite the 4k-step delta as if it were comparable to the 12k-step +delta. The user has already flagged this as a "partial run, not directly +comparable" in the repo README. + + +================================================================================ +7. CROSS-SCALE CONTEXT (PRIOR EXPERIMENTS, NOT FROM THIS REPO) +================================================================================ +The following are described in this repo's README as prior experiments +done before this codebase. They are NOT re-verified from raw data here +and should be cited as historical context, not as primary evidence of +this paper's findings. If the paper depends on them, the underlying logs +should be located and re-verified before they appear in any final claim. + + scale corpus tokens delta seeds note + ----- ------------------- -------- ------- ----- ------------------- + 7M Shakespeare short mixed 1 directional crossover + around step 150 + 100M FineWeb (smoke) short -7% 1 100-step smoke run + 100M FineWeb 100M -11% 3 of 3 strongest replicated + 100M FineMath 100M -17% 3 of 3 strongest result + 68M FineWeb (SMAT+A90) 20M -22.5% 1 SMAT combined with + BlockAttnRes; out + of scope for V1. + 64M FineWeb (THIS REPO) 98M -1.37% 5 of 5 primary result + +The pattern across these (taking them at face value): direction is +preserved across every scale, every corpus, every seed where data +exists. Magnitude shrinks as the scale and training budget grow, with +the exception of the 100M experiments where the gap is largest. + +If the paper's headline number is from the 100M FineWeb or FineMath +experiments, the V1 evidence is the 5-seed run in section 3 of THIS +document plus those prior experiments. The 5-seed result is the +hardest-to-dispute piece because every variant trained on identical +data and direction is consistent across 5/5 seeds. + + +================================================================================ +8. WHAT'S LOAD-BEARING vs WHAT IS NOT +================================================================================ +Load-bearing (do not remove): + - The gate G. Catastrophic if removed (+792,016%) and badly degraded + if reduced to its mean (+149%). The gate is the single most important + component SMAT adds. + - The structure of S (not just its norm). random_S hurts more than + S_off, so the learned S projection contains real information. + - The mu*c centrality channel from S into the gate. This accounts for + ~74% of S's total contribution. If S is used at all, it should be + used HERE. + +Not load-bearing in any individual layer (and largely redundant overall): + - The per-layer lambda*S term in the attention bias. Removing it in + one layer costs at most 0.06% ppl. Removing it in all layers costs + only 0.49%. The benefit exists but it is small and distributed; no + single layer's lambda is critical, and a uniform lambda across + layers is the wrong abstraction (early layers actively prefer it + near zero, middle layers want it near 0.5-0.7). + +Net implication for V2 (this is what the data argues for): + - Drop the per-layer lambda_raw parameters and the lambda*S term in + attention. Keep S's projection W_s and keep mu*c flowing into the + gate. You retain ~74% of S's lift while removing 24 scalar + parameters and one matrix add per layer. + - If you want lambda to stay, replace per-layer lambda with a + per-layer gate-mix scalar driven by the same per-layer pattern + observed here (suppressed in early layers, elevated in middle + layers). + + +================================================================================ +9. CAVEATS, LIMITATIONS, THINGS NOT TO OVERCLAIM +================================================================================ + - The headline -1.37% at 64M / 98M tokens is real (5/5 seeds, identical + per-seed data streams; one-sided sign test gives p = 1/32 = 0.03125 + under the null hypothesis of no effect), but it is SMALL in absolute + terms (about 1.1 perplexity points). The within-seed effect is what + makes it visible; do not present mean +/- std as if seed noise were + the only consideration. + - Std across seeds (1.65 - 1.75) is larger than the mean effect. + Always quote the WITHIN-SEED comparison or the SIGN TEST when making + the case, not just delta-of-means with overlapping error bars. + - The surgical ablation results in section 5 are from ONE checkpoint + (Full SMAT seed 0). The qualitative conclusions (G is critical, S + routes through mu*c, layer-wise lambda is redundant) are stark + enough that one checkpoint is informative, but it would be more + defensible to repeat on at least one more seed before publication. + - The cross-scale numbers in section 7 are not verified from raw data + in this repo. Do not put them in a results table without separately + locating and confirming the underlying logs. + - All training was on a single RTX 4060 with bf16 + TF32. Numbers may + shift slightly on different hardware; the direction should not. + - Tokenizer is GPT-2 byte-pair. Corpus is FineWeb-Edu sample-10BT. + Results on other corpora (FineMath in particular, see section 7) + may differ in magnitude. + + +================================================================================ +10. SUGGESTED PAPER CLAIMS (ordered by strength of evidence in this repo) +================================================================================ +STRONG (directly supported by 20-run ablation, this repo, this document): + 1. Adding a per-token value gate to attention reduces val ppl by + ~0.9% on FineWeb at 64M params, 5/5 seeds. + 2. Adding both the semantic-similarity bias AND the gate reduces + val ppl by ~1.4% on FineWeb at 64M params, 5/5 seeds. + 3. Training is numerically stable (0 NaN failures over 240,000 + optimizer steps across 20 runs). + 4. lambda exhibits a robust per-layer pattern: suppressed in early + layers, elevated in middle layers, settled in late layers. + This pattern is consistent across all 5 seeds. + +STRONG (supported by surgical-ablation evidence, single checkpoint): + 5. S's contribution flows ~74% through the gate's centrality term + and only ~26% through the additive logit bias. + 6. The trained S has learned STRUCTURE; replacing it with a random + matrix of equal norm hurts more than removing it entirely. + 7. The per-token gate is catastrophic to remove and very expensive + to replace with a mean. Per-token differentiation matters. + +WEAKER (need cross-scale / cross-seed re-verification before paper): + 8. SMAT's relative effect is larger at smaller training budgets and + at certain corpora (FineMath in particular). + 9. SMAT directionality is preserved across scales 7M to 100M. + +DO NOT CLAIM without further work: + - That SMAT is a uniformly large improvement (it is not at this scale). + - That SMAT helps at scales beyond 100M (no data yet). + - That per-layer lambda is essential (the surgical ablation shows it + is largely redundant). + + +================================================================================ +11. RAW NUMBERS (machine-readable, for any tables in the paper) +================================================================================ +12k-step 5-seed ablation, final val ppl: + +baseline = [78.32438343443285, 80.93203870134798, 82.33645607970243, + 79.38843805716219, 77.74486565469935] +s_only = [78.73771250646860, 80.86492907631813, 82.07353893301419, + 77.98296438017636, 77.68072112072338] +g_only = [78.21752726763506, 80.57706364462581, 81.33629330240751, + 77.98339201086276, 77.00922636947199] +full = [77.86344065821494, 80.02616085611275, 81.26017429181104, + 77.70156877103959, 76.39525023929815] + +Means: + baseline 79.74523638546896 + s_only 79.46797320334013 + g_only 79.02470051900063 + full 78.64931896329529 + +Population std: + baseline 1.6894... s_only 1.7131... g_only 1.6463... full 1.7498... + +Sample (n-1) std (if you prefer): + baseline 1.8888 s_only 1.9153 g_only 1.8406 full 1.9563 + +Delta of means vs baseline (%): + s_only -0.3477 g_only -0.9035 full -1.3743 + +Per-layer trained lambda (Full SMAT seed 0, 24 values, from +ablation_inference.json): + [0.1837, 0.1491, 0.1557, 0.1365, 0.1209, 0.0834, 0.1213, 0.0920, + 0.1605, 0.3192, 0.3323, 0.4566, 0.3248, 0.5507, 0.5484, 0.4943, + 0.6821, 0.4693, 0.3914, 0.5843, 0.1961, 0.3536, 0.2781, 0.1833] + (printed to 4 dp; full precision in figures_final/ablation_inference.json) + +Surgical ablation deltas (Full SMAT seed 0, 80-batch eval slice, intact +ppl 79.010): + lambda_zero +0.4903% S_off +1.8547% random_S +2.8032% + G_one +792,015.9609% G_mean +149.3211% + + +================================================================================ +END +================================================================================ diff --git a/HUGGINGFACE_README.md b/HUGGINGFACE_README.md new file mode 100644 index 0000000000000000000000000000000000000000..9e7c30f2108bd16a498f88184841bef1d09dfc3f --- /dev/null +++ b/HUGGINGFACE_README.md @@ -0,0 +1,159 @@ +--- +license: mit +library_name: pytorch +tags: + - language-modeling + - transformer + - attention + - ablation + - research +language: + - en +datasets: + - HuggingFaceFW/fineweb-edu +metrics: + - perplexity +pipeline_tag: text-generation +--- + +# SMAT — Semantic Attention + +Trained checkpoints for **SMAT** (Semantic Attention), a transformer attention +variant with a learnable semantic-similarity bias and per-token value gate. + +- **Code:** [github.com/OutrageouslyBad200/smat](https://github.com/OutrageouslyBad200/smat) +- **Architecture:** 24 layers × 384d × 6 heads, block size 256, ~64 M parameters +- **Tokenizer:** GPT-2 (`tiktoken`, vocab 50 257) +- **Training data:** FineWeb-Edu sample-10BT, 98 M tokens +- **Training compute:** 12 000 optimizer steps, batch 16 × grad_accum 2 (effective 32), + RTX 4060 + +## Equation + +``` +Attn(Q,K,V) = softmax(QK^T/sqrt(d_k) + λ·S + P + M) · (G ⊙ V) +``` + +- `S_ij = cos(W_s h_i, W_s h_j)` — cosine similarity in shared projection +- `c_j = (1/n) Σ_{l≤j} S_jl` — causal semantic centrality +- `G_j = σ(w_g^T h_j + μ·c_j + β)` — per-token value gate +- `λ = softplus(λ_raw)` — constrained positive scalar (per layer) + +## Repository contents + +This HuggingFace repo hosts 20 checkpoints from the 5-seed ablation in +Experiment 6 of the SMAT paper: + +``` +baseline_s0/final.pt s_only_s0/final.pt g_only_s0/final.pt full_s0/final.pt +baseline_s1/final.pt s_only_s1/final.pt g_only_s1/final.pt full_s1/final.pt +baseline_s2/final.pt s_only_s2/final.pt g_only_s2/final.pt full_s2/final.pt +baseline_s3/final.pt s_only_s3/final.pt g_only_s3/final.pt full_s3/final.pt +baseline_s4/final.pt s_only_s4/final.pt g_only_s4/final.pt full_s4/final.pt +``` + +Each variant directory also contains `config.json` and `metrics.jsonl` +(per-step training + eval logs). + +| Variant | `use_S` | `use_G` | Description | +|---------|---------|---------|-------------| +| `baseline` | False | False | Standard attention | +| `s_only` | True | False | Semantic bias only | +| `g_only` | False | True | Value gate only | +| `full` | True | True | Full SMAT | + +## Results + +Validation perplexity on FineWeb-Edu, 5 seeds, 12 000 steps: + +| Variant | Mean ppl | Std | Δ vs baseline | Seed wins | +|---------|----------|-----|---------------|-----------| +| Baseline | 79.75 | 1.69 | — | — | +| S-only | 79.47 | 1.71 | −0.35% | 4/5 | +| G-only | 79.02 | 1.65 | −0.90% | 5/5 | +| **Full SMAT** | **78.65** | 1.75 | **−1.37%** | **5/5** | + +0 NaN failures across 240 000 optimizer steps. + +## Usage + +```bash +pip install torch numpy tiktoken huggingface_hub +git clone https://github.com/OutrageouslyBad200/smat.git +cd smat +``` + +Download a single checkpoint: + +```python +from huggingface_hub import hf_hub_download +ckpt_path = hf_hub_download( + repo_id="OutrageouslyBad200/smat", + filename="full_s0/final.pt", +) +``` + +Load it into the SMAT model: + +```python +import torch +from model import Config, SMATTransformer + +state = torch.load(ckpt_path, map_location="cuda") +cfg = Config(**state["config"]) +model = SMATTransformer(cfg).cuda() +model.load_state_dict(state["state_dict"]) +model.eval() +``` + +Reproduce surgical ablations (Experiment 7): + +```bash +python ablate.py --ckpt full_s0/final.pt --n_batches 80 +``` + +## Surgical-ablation findings (Experiment 7) + +Run on Full SMAT, val ppl 79.010: + +| Ablation | val ppl | Δ | +|----------|---------|---| +| λ=0 (S still drives c) | 79.40 | +0.49% | +| S removed entirely | 80.48 | +1.85% | +| Random S (same norm) | 81.23 | +2.80% | +| G replaced by mean | 196.99 | +149% | +| G forced to 1.0 | 625 850 | catastrophic | + +- The gate **G** is catastrophically essential. +- **S** routes mostly through `μ·c` in the gate (74 % of lift), not through + `λ·S` in attention (26 %). +- Per-token gate differentiation matters: replacing G with its mean costs 149 %. + +## Limitations + +- Small base model (~64 M params); larger-scale runs (100 M on FineWeb / FineMath) + show stronger lifts (−11 % to −17 %) but are not included as released checkpoints. +- Trained only on English FineWeb-Edu sample-10BT — generalization to other + domains untested at this scale. +- Not instruction-tuned, not RLHF'd, no safety filtering. Research artifact only. + +## Citation + +```bibtex +@misc{smat2026, + author = {OutrageouslyBad200}, + title = {SMAT: Semantic Attention}, + year = {2026}, + howpublished = {\url{https://github.com/OutrageouslyBad200/smat}}, +} +``` + +## Contact + +For further information on training runs, intermediate experiments, or the +unpublished paper draft, please contact the creator via +[GitHub](https://github.com/OutrageouslyBad200) or HuggingFace. + +## License + +[MIT License](https://github.com/OutrageouslyBad200/smat/blob/main/LICENSE). diff --git a/all_results.csv b/all_results.csv new file mode 100644 index 0000000000000000000000000000000000000000..4f85e5b849957c66d3f87cf2b8946af81738e52c --- /dev/null +++ b/all_results.csv @@ -0,0 +1,25 @@ +runset,variant,seed,params,final_step,tokens_seen,val_loss,val_ppl,lambda_start,lambda_end,lambda_per_layer_min,lambda_per_layer_max,G_start,G_end,elapsed_s,nan_failures +runs,baseline,0,63376512,11999,98304000,4.360858964920044,78.32438343443285,,,,,,,5309.61736869812,0 +runs,baseline,1,63376512,11999,98304000,4.393609774112702,80.93203870134798,,,,,,,5335.801450252533,0 +runs,baseline,2,63376512,11999,98304000,4.410813975334167,82.33645607970243,,,,,,,4016.196511745453,0 +runs,baseline,3,63376512,11999,98304000,4.374352741241455,79.38843805716219,,,,,,,4001.651510953903,0 +runs,baseline,4,63376512,11999,98304000,4.353432512283325,77.74486565469935,,,,,,,3997.937122106552,0 +runs,full,0,63975624,11999,98304000,4.354956531524659,77.86344065821494,0.12692801654338837,0.3069125845407446,0.08336106687784195,0.6821404099464417,0.5030857535700003,0.3505655957075457,6544.294906139374,0 +runs,full,1,63975624,11999,98304000,4.382353591918945,80.02616085611275,0.12692801654338837,0.3074745563790202,0.09392528235912323,0.6946620941162109,0.4995858383675416,0.3332850156972806,6583.949243545532,0 +runs,full,2,63975624,11999,98304000,4.397656035423279,81.26017429181104,0.12692801654338837,0.30312894998739165,0.11483360826969147,0.6690832376480103,0.48413818205396336,0.33056403052372235,5372.456871032715,0 +runs,full,3,63975624,11999,98304000,4.352875447273254,77.70156877103959,0.12692801654338837,0.29537375799069804,0.09290912002325058,0.6339113712310791,0.4967380625506242,0.3447236562302957,5374.616672039032,0 +runs,full,4,63975624,11999,98304000,4.335920524597168,76.39525023929815,0.12692801654338837,0.3175949805105726,0.09670225530862808,0.6432420015335083,0.49705679963032406,0.3396819991370042,5370.658619880676,0 +runs,g_only,0,63385776,11999,98304000,4.359493756294251,78.21752726763506,,,,,0.4967447916666667,0.3484141031901042,5647.636380910873,0 +runs,g_only,1,63385776,11999,98304000,4.389214038848877,80.57706364462581,,,,,0.5144856770833334,0.3359120686848958,5649.687728404999,0 +runs,g_only,2,63385776,11999,98304000,4.398592329025268,81.33629330240751,,,,,0.5052897135416666,0.3676300048828125,4351.29425239563,0 +runs,g_only,3,63385776,11999,98304000,4.356495881080628,77.98339201086276,,,,,0.4998372395833333,0.362548828125,4340.425580263138,0 +runs,g_only,4,63385776,11999,98304000,4.34392523765564,77.00922636947199,,,,,0.5079752604166666,0.3562774658203125,4339.748939752579,0 +runs,s_only,0,63966360,11999,98304000,4.366122233867645,78.7377125064686,0.12692801654338837,0.31397200624148053,0.06856122612953186,0.7095168232917786,,,6095.362888336182,0 +runs,s_only,1,63966360,11999,98304000,4.392780220508575,80.86492907631813,0.12692801654338837,0.3187686977908015,0.058697257190942764,0.6693422198295593,,,6085.201226711273,0 +runs,s_only,2,63966360,11999,98304000,4.407615661621094,82.07353893301419,0.12692801654338837,0.32208347382644814,0.07567994296550751,0.7152900695800781,,,4696.141266584396,0 +runs,s_only,3,63966360,11999,98304000,4.3564903974533085,77.98296438017636,0.12692801654338837,0.3182733968521158,0.0908360481262207,0.6816989183425903,,,4680.379953622818,0 +runs,s_only,4,63966360,11999,98304000,4.352607107162475,77.68072112072338,0.12692801654338837,0.3260646794612209,0.07901874929666519,0.6664481163024902,,,4688.767492055893,0 +runs_4ksteps,baseline,0,63376512,3999,32768000,5.000214111804962,148.44493951411144,,,,,,,1804.3665130138397,0 +runs_4ksteps,full,0,63975624,3999,32768000,4.9527598261833194,141.5651204177534,0.12692801654338837,0.16720068734139204,,,0.5060585426787535,0.4214151619623105,2212.9264504909515,0 +runs_4ksteps,g_only,0,63385776,3999,32768000,4.976549911499023,144.973346966276,,,,,0.4969075520833333,0.4358113606770833,1914.1351833343506,0 +runs_4ksteps,s_only,0,63966360,3999,32768000,4.996470749378204,147.89029507035235,0.12692801654338837,0.16468850150704384,,,,,2049.8153450489044,0 diff --git a/baseline_s0/config.json b/baseline_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..a5f8c0a1fc2f1fcff84ada40fecf34083846d449 --- /dev/null +++ b/baseline_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s0", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/baseline_s0/final.pt b/baseline_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..dda8dbbb0f19d8b9fada8aadd4dbfaf876ea7f1d --- /dev/null +++ b/baseline_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bce27c7b560e565c25258a6ea84e5173a531c8167830748f786bd4bb96b4541d +size 253572039 diff --git a/baseline_s0/metrics.jsonl b/baseline_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..a33ed018f77d2829462f16285595cadc7ac18668 --- /dev/null +++ b/baseline_s0/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 0, "config": {"variant": "baseline", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s0", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.890632629394531, "grad_norm": 14.004855155944824, "tokens_seen": 8192, "tokens_per_sec": 6666.771125443355, "vram_gb": 5.453331456, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.797861099243164, "grad_norm": 2.4691383838653564, "tokens_seen": 172032, "tokens_per_sec": 17646.147196195, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.06395149230957, "grad_norm": 1.7941702604293823, "tokens_seen": 335872, "tokens_per_sec": 18332.384144842086, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.103279113769531, "grad_norm": 1.5039955377578735, "tokens_seen": 499712, "tokens_per_sec": 18576.41382690032, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.571502685546875, "grad_norm": 0.8192026615142822, "tokens_seen": 663552, "tokens_per_sec": 18702.486145448438, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.3209428787231445, "grad_norm": 0.7388450503349304, "tokens_seen": 827392, "tokens_per_sec": 18756.78800832847, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.097091913223267, "grad_norm": 0.6304831504821777, "tokens_seen": 991232, "tokens_per_sec": 18796.54885566577, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.985159873962402, "grad_norm": 0.5573312640190125, "tokens_seen": 1155072, "tokens_per_sec": 18828.255659326656, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.802034378051758, "grad_norm": 0.776248037815094, "tokens_seen": 1318912, "tokens_per_sec": 18851.40660535627, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.851132869720459, "grad_norm": 0.5454278588294983, "tokens_seen": 1482752, "tokens_per_sec": 18867.68686915481, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.688426971435547, "grad_norm": 0.6350254416465759, "tokens_seen": 1646592, "tokens_per_sec": 18882.344894874557, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.835505723953247, "grad_norm": 0.5815845131874084, "tokens_seen": 1810432, "tokens_per_sec": 18894.891075053823, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.55937933921814, "grad_norm": 0.595177948474884, "tokens_seen": 1974272, "tokens_per_sec": 18900.428607583523, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.656440496444702, "grad_norm": 0.5813316106796265, "tokens_seen": 2138112, "tokens_per_sec": 18905.674913681785, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.423501491546631, "grad_norm": 0.6590389013290405, "tokens_seen": 2301952, "tokens_per_sec": 18906.990910767305, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.564842700958252, "grad_norm": 0.5534185767173767, "tokens_seen": 2465792, "tokens_per_sec": 18911.560056532362, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.360459804534912, "grad_norm": 0.5858584642410278, "tokens_seen": 2629632, "tokens_per_sec": 18915.390758022364, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.402979135513306, "grad_norm": 0.5513539910316467, "tokens_seen": 2793472, "tokens_per_sec": 18917.21588960651, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.5521910190582275, "grad_norm": 0.5159503221511841, "tokens_seen": 2957312, "tokens_per_sec": 18921.131893488946, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.402872085571289, "grad_norm": 0.5543516874313354, "tokens_seen": 3121152, "tokens_per_sec": 18924.239139166682, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.305682897567749, "grad_norm": 0.539954423904419, "tokens_seen": 3284992, "tokens_per_sec": 18926.098135768516, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.42765736579895, "grad_norm": 0.5563998222351074, "tokens_seen": 3448832, "tokens_per_sec": 18928.220290300225, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.364402770996094, "grad_norm": 0.5420711636543274, "tokens_seen": 3612672, "tokens_per_sec": 18929.116998290763, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.415828227996826, "grad_norm": 0.5212138891220093, "tokens_seen": 3776512, "tokens_per_sec": 18930.466654356558, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.201891899108887, "grad_norm": 0.4946546256542206, "tokens_seen": 3940352, "tokens_per_sec": 18931.436594106086, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.082775354385376, "grad_norm": 0.5330259203910828, "tokens_seen": 4104192, "tokens_per_sec": 18932.39806048905, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.303739154338837, "val_ppl": 546.6119602006547, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.269145965576172, "grad_norm": 0.5410400629043579, "tokens_seen": 4268032, "tokens_per_sec": 18674.848458566492, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.19500207901001, "grad_norm": 0.545661211013794, "tokens_seen": 4431872, "tokens_per_sec": 18682.47795593009, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.185184001922607, "grad_norm": 0.5598522424697876, "tokens_seen": 4595712, "tokens_per_sec": 18692.438844385306, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.219585180282593, "grad_norm": 0.6056293845176697, "tokens_seen": 4759552, "tokens_per_sec": 18701.57788587368, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.166798114776611, "grad_norm": 0.5028415322303772, "tokens_seen": 4923392, "tokens_per_sec": 18709.512460277503, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.069556713104248, "grad_norm": 0.48512002825737, "tokens_seen": 5087232, "tokens_per_sec": 18716.49613914132, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.146238327026367, "grad_norm": 0.5354776382446289, "tokens_seen": 5251072, "tokens_per_sec": 18723.89448147724, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.042133808135986, "grad_norm": 0.5581792593002319, "tokens_seen": 5414912, "tokens_per_sec": 18729.471395229502, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.031276702880859, "grad_norm": 0.5070163607597351, "tokens_seen": 5578752, "tokens_per_sec": 18735.174705470057, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.998528242111206, "grad_norm": 0.5411954522132874, "tokens_seen": 5742592, "tokens_per_sec": 18739.920837484424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.0923426151275635, "grad_norm": 0.5311352610588074, "tokens_seen": 5906432, "tokens_per_sec": 18744.134913825266, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.936523675918579, "grad_norm": 0.5572741031646729, "tokens_seen": 6070272, "tokens_per_sec": 18749.20547732706, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.9347124099731445, "grad_norm": 0.6145651340484619, "tokens_seen": 6234112, "tokens_per_sec": 18753.435465854527, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.9417564868927, "grad_norm": 0.5109319090843201, "tokens_seen": 6397952, "tokens_per_sec": 18757.6922719063, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.835493564605713, "grad_norm": 0.5537232756614685, "tokens_seen": 6561792, "tokens_per_sec": 18761.782406029048, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8155059814453125, "grad_norm": 0.5301229357719421, "tokens_seen": 6725632, "tokens_per_sec": 18766.17698145576, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.799767255783081, "grad_norm": 0.5625292658805847, "tokens_seen": 6889472, "tokens_per_sec": 18770.120518783315, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.758581638336182, "grad_norm": 0.5512877702713013, "tokens_seen": 7053312, "tokens_per_sec": 18773.10670861143, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.918421030044556, "grad_norm": 0.5356389284133911, "tokens_seen": 7217152, "tokens_per_sec": 18776.4062838019, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.744070768356323, "grad_norm": 0.5244048237800598, "tokens_seen": 7380992, "tokens_per_sec": 18779.468616867165, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.787522315979004, "grad_norm": 0.5544624328613281, "tokens_seen": 7544832, "tokens_per_sec": 18782.653461142414, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.761897325515747, "grad_norm": 0.5255046486854553, "tokens_seen": 7708672, "tokens_per_sec": 18784.92142230955, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.790459871292114, "grad_norm": 0.5542486906051636, "tokens_seen": 7872512, "tokens_per_sec": 18787.650979253103, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.895646333694458, "grad_norm": 0.5942807793617249, "tokens_seen": 8036352, "tokens_per_sec": 18790.452102009065, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.839163541793823, "grad_norm": 0.5869764685630798, "tokens_seen": 8200192, "tokens_per_sec": 18793.068585064455, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.823627710342407, "val_ppl": 338.1967106415114, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.632595777511597, "grad_norm": 0.5060266852378845, "tokens_seen": 8364032, "tokens_per_sec": 18662.686320779176, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.704880952835083, "grad_norm": 0.5181776881217957, "tokens_seen": 8527872, "tokens_per_sec": 18667.436394486216, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.75211763381958, "grad_norm": 0.5448099374771118, "tokens_seen": 8691712, "tokens_per_sec": 18671.62734124469, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.735692262649536, "grad_norm": 0.5725530982017517, "tokens_seen": 8855552, "tokens_per_sec": 18675.808943812382, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.632856369018555, "grad_norm": 0.5423212647438049, "tokens_seen": 9019392, "tokens_per_sec": 18679.651156271095, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.7276833057403564, "grad_norm": 0.5043532848358154, "tokens_seen": 9183232, "tokens_per_sec": 18683.44379885779, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.553969383239746, "grad_norm": 0.5692328214645386, "tokens_seen": 9347072, "tokens_per_sec": 18687.0839040901, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.582232236862183, "grad_norm": 0.5270517468452454, "tokens_seen": 9510912, "tokens_per_sec": 18690.439626395862, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.648525953292847, "grad_norm": 0.5073367357254028, "tokens_seen": 9674752, "tokens_per_sec": 18693.516537105366, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.527204513549805, "grad_norm": 0.5607810616493225, "tokens_seen": 9838592, "tokens_per_sec": 18695.98579569864, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.711727857589722, "grad_norm": 0.5320408940315247, "tokens_seen": 10002432, "tokens_per_sec": 18698.08402916243, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.676315546035767, "grad_norm": 0.5328497886657715, "tokens_seen": 10166272, "tokens_per_sec": 18699.41547938124, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.644982576370239, "grad_norm": 0.6300049424171448, "tokens_seen": 10330112, "tokens_per_sec": 18700.325848427612, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.4910783767700195, "grad_norm": 0.5817524790763855, "tokens_seen": 10493952, "tokens_per_sec": 18698.72571469087, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.599607229232788, "grad_norm": 0.5805956125259399, "tokens_seen": 10657792, "tokens_per_sec": 18700.54770947154, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.540740966796875, "grad_norm": 0.5638679265975952, "tokens_seen": 10821632, "tokens_per_sec": 18703.130166623054, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.630486488342285, "grad_norm": 0.5575174689292908, "tokens_seen": 10985472, "tokens_per_sec": 18705.963100224482, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.47309136390686, "grad_norm": 0.5427681803703308, "tokens_seen": 11149312, "tokens_per_sec": 18707.80849443616, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.301149129867554, "grad_norm": 0.5845924615859985, "tokens_seen": 11313152, "tokens_per_sec": 18707.71974908225, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.587926626205444, "grad_norm": 0.5609380006790161, "tokens_seen": 11476992, "tokens_per_sec": 18709.957073627476, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.5229833126068115, "grad_norm": 0.5477386713027954, "tokens_seen": 11640832, "tokens_per_sec": 18711.51883523811, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.437740087509155, "grad_norm": 0.576702356338501, "tokens_seen": 11804672, "tokens_per_sec": 18713.88758057427, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.530909776687622, "grad_norm": 0.5459407567977905, "tokens_seen": 11968512, "tokens_per_sec": 18715.971223401903, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.439221143722534, "grad_norm": 0.5624483227729797, "tokens_seen": 12132352, "tokens_per_sec": 18718.42478844885, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.670048952102661, "grad_norm": 0.6188408732414246, "tokens_seen": 12296192, "tokens_per_sec": 18720.399636657334, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.541709470748901, "val_ppl": 255.11373638266824, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.446678400039673, "grad_norm": 0.5876959562301636, "tokens_seen": 12460032, "tokens_per_sec": 18634.120187093216, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.301362991333008, "grad_norm": 0.5615300536155701, "tokens_seen": 12623872, "tokens_per_sec": 18637.458261454678, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.546454429626465, "grad_norm": 0.603606641292572, "tokens_seen": 12787712, "tokens_per_sec": 18639.948946981363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.433995246887207, "grad_norm": 0.5787985324859619, "tokens_seen": 12951552, "tokens_per_sec": 18641.928863290363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.562330722808838, "grad_norm": 0.5898619890213013, "tokens_seen": 13115392, "tokens_per_sec": 18644.734335433775, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.342672109603882, "grad_norm": 0.5484730005264282, "tokens_seen": 13279232, "tokens_per_sec": 18645.477017194593, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.408828973770142, "grad_norm": 0.5654874444007874, "tokens_seen": 13443072, "tokens_per_sec": 18647.404584127184, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.399937868118286, "grad_norm": 0.6125302910804749, "tokens_seen": 13606912, "tokens_per_sec": 18649.061267030498, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.5023722648620605, "grad_norm": 0.5817909836769104, "tokens_seen": 13770752, "tokens_per_sec": 18650.903981849246, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.428601503372192, "grad_norm": 0.6352376341819763, "tokens_seen": 13934592, "tokens_per_sec": 18652.85070010366, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.356788158416748, "grad_norm": 0.636605441570282, "tokens_seen": 14098432, "tokens_per_sec": 18654.178186161735, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.452927350997925, "grad_norm": 0.6962578892707825, "tokens_seen": 14262272, "tokens_per_sec": 18656.215638857233, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.413260459899902, "grad_norm": 0.5968581438064575, "tokens_seen": 14426112, "tokens_per_sec": 18657.556149295124, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.339949131011963, "grad_norm": 0.5607135891914368, "tokens_seen": 14589952, "tokens_per_sec": 18659.527190259414, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.387685060501099, "grad_norm": 0.6948267221450806, "tokens_seen": 14753792, "tokens_per_sec": 18661.33424624319, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.309333086013794, "grad_norm": 0.5636512041091919, "tokens_seen": 14917632, "tokens_per_sec": 18663.09808373012, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.439793586730957, "grad_norm": 0.6187537908554077, "tokens_seen": 15081472, "tokens_per_sec": 18664.681352256954, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.395078897476196, "grad_norm": 0.6145064234733582, "tokens_seen": 15245312, "tokens_per_sec": 18666.7138624681, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.340224742889404, "grad_norm": 0.6207507848739624, "tokens_seen": 15409152, "tokens_per_sec": 18668.55090841829, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.281381130218506, "grad_norm": 0.6182495355606079, "tokens_seen": 15572992, "tokens_per_sec": 18670.17375964289, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.218503475189209, "grad_norm": 0.633520781993866, "tokens_seen": 15736832, "tokens_per_sec": 18672.39896461277, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.206000804901123, "grad_norm": 0.596594512462616, "tokens_seen": 15900672, "tokens_per_sec": 18673.808681454488, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.195952653884888, "grad_norm": 0.6104916930198669, "tokens_seen": 16064512, "tokens_per_sec": 18674.47145538391, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.081684350967407, "grad_norm": 0.6075489521026611, "tokens_seen": 16228352, "tokens_per_sec": 18675.707217400188, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.237996578216553, "grad_norm": 0.594325065612793, "tokens_seen": 16392192, "tokens_per_sec": 18677.08961897874, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.354835951328278, "val_ppl": 211.6292560067132, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.153222322463989, "grad_norm": 0.6648209691047668, "tokens_seen": 16556032, "tokens_per_sec": 18612.303747077956, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.327598571777344, "grad_norm": 0.5867108106613159, "tokens_seen": 16719872, "tokens_per_sec": 18613.928354272026, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.321566820144653, "grad_norm": 0.6753186583518982, "tokens_seen": 16883712, "tokens_per_sec": 18615.993406101057, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.266076326370239, "grad_norm": 0.6467592716217041, "tokens_seen": 17047552, "tokens_per_sec": 18618.05727947894, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.261559247970581, "grad_norm": 0.6250351071357727, "tokens_seen": 17211392, "tokens_per_sec": 18620.07398607387, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.226260185241699, "grad_norm": 0.6696427464485168, "tokens_seen": 17375232, "tokens_per_sec": 18622.284247333326, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.285370826721191, "grad_norm": 0.6200236678123474, "tokens_seen": 17539072, "tokens_per_sec": 18622.907256548828, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.338468313217163, "grad_norm": 0.6192476153373718, "tokens_seen": 17702912, "tokens_per_sec": 18625.138669516593, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.307520151138306, "grad_norm": 0.6227665543556213, "tokens_seen": 17866752, "tokens_per_sec": 18627.075231294726, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.247787952423096, "grad_norm": 0.606657862663269, "tokens_seen": 18030592, "tokens_per_sec": 18628.878196898477, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.223803281784058, "grad_norm": 0.6099840402603149, "tokens_seen": 18194432, "tokens_per_sec": 18630.81892845877, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.134663105010986, "grad_norm": 0.589820921421051, "tokens_seen": 18358272, "tokens_per_sec": 18632.66899926007, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.2502219676971436, "grad_norm": 0.6150110960006714, "tokens_seen": 18522112, "tokens_per_sec": 18634.749876569203, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.178414821624756, "grad_norm": 0.6626871824264526, "tokens_seen": 18685952, "tokens_per_sec": 18636.53964586145, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.186791658401489, "grad_norm": 0.6538658738136292, "tokens_seen": 18849792, "tokens_per_sec": 18638.35429434482, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.162872791290283, "grad_norm": 0.6347496509552002, "tokens_seen": 19013632, "tokens_per_sec": 18639.517382627175, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.108135223388672, "grad_norm": 0.6344709992408752, "tokens_seen": 19177472, "tokens_per_sec": 18641.266707812243, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.20079231262207, "grad_norm": 0.6784117817878723, "tokens_seen": 19341312, "tokens_per_sec": 18643.243862127278, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.285532474517822, "grad_norm": 0.6290812492370605, "tokens_seen": 19505152, "tokens_per_sec": 18644.867213904596, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.18052077293396, "grad_norm": 0.6040476560592651, "tokens_seen": 19668992, "tokens_per_sec": 18646.42640961469, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.26459813117981, "grad_norm": 0.6517846584320068, "tokens_seen": 19832832, "tokens_per_sec": 18648.33661818175, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.085031032562256, "grad_norm": 0.6815958023071289, "tokens_seen": 19996672, "tokens_per_sec": 18649.78319843805, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.113767623901367, "grad_norm": 0.6930291056632996, "tokens_seen": 20160512, "tokens_per_sec": 18651.25075527076, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.072915077209473, "grad_norm": 0.6539711952209473, "tokens_seen": 20324352, "tokens_per_sec": 18652.65214081382, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.234839916229248, "grad_norm": 0.6837641596794128, "tokens_seen": 20488192, "tokens_per_sec": 18654.237945561672, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.2089867234230045, "val_ppl": 182.90862723545237, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.144009351730347, "grad_norm": 0.6335085034370422, "tokens_seen": 20652032, "tokens_per_sec": 18602.87575802873, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.0018630027771, "grad_norm": 0.6551423668861389, "tokens_seen": 20815872, "tokens_per_sec": 18604.586219050965, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.242849826812744, "grad_norm": 0.6995706558227539, "tokens_seen": 20979712, "tokens_per_sec": 18606.453769803273, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.142164945602417, "grad_norm": 0.6745108962059021, "tokens_seen": 21143552, "tokens_per_sec": 18650.768945359247, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 4.9854514598846436, "grad_norm": 0.6425358057022095, "tokens_seen": 21307392, "tokens_per_sec": 18652.222189123866, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.001100778579712, "grad_norm": 0.6719156503677368, "tokens_seen": 21471232, "tokens_per_sec": 18653.723398609924, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.141652345657349, "grad_norm": 0.6743621230125427, "tokens_seen": 21635072, "tokens_per_sec": 18654.90294787402, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.111774921417236, "grad_norm": 0.6835953593254089, "tokens_seen": 21798912, "tokens_per_sec": 18656.255715790103, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.096091270446777, "grad_norm": 0.6759418249130249, "tokens_seen": 21962752, "tokens_per_sec": 18657.506177290947, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.097742557525635, "grad_norm": 0.6613295078277588, "tokens_seen": 22126592, "tokens_per_sec": 18659.01172948027, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.058473348617554, "grad_norm": 0.6728112101554871, "tokens_seen": 22290432, "tokens_per_sec": 18660.477904213625, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.133389949798584, "grad_norm": 0.7223525047302246, "tokens_seen": 22454272, "tokens_per_sec": 18661.712970255645, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.260310649871826, "grad_norm": 0.6331192255020142, "tokens_seen": 22618112, "tokens_per_sec": 18663.314302202267, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.2544474601745605, "grad_norm": 0.7307506203651428, "tokens_seen": 22781952, "tokens_per_sec": 18664.286973633607, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.0196592807769775, "grad_norm": 0.6550894975662231, "tokens_seen": 22945792, "tokens_per_sec": 18665.255910218788, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.294972896575928, "grad_norm": 0.7704381346702576, "tokens_seen": 23109632, "tokens_per_sec": 18666.079954992114, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.015143394470215, "grad_norm": 0.710565984249115, "tokens_seen": 23273472, "tokens_per_sec": 18667.473251100448, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.04838752746582, "grad_norm": 0.6872653365135193, "tokens_seen": 23437312, "tokens_per_sec": 18668.783393478352, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.095804214477539, "grad_norm": 0.6965939998626709, "tokens_seen": 23601152, "tokens_per_sec": 18670.16032540442, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.913230657577515, "grad_norm": 0.6736608743667603, "tokens_seen": 23764992, "tokens_per_sec": 18671.302731386353, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.051895380020142, "grad_norm": 0.6646551489830017, "tokens_seen": 23928832, "tokens_per_sec": 18672.54132106789, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.113583326339722, "grad_norm": 0.7508693337440491, "tokens_seen": 24092672, "tokens_per_sec": 18673.921273545733, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.96597146987915, "grad_norm": 0.7658485174179077, "tokens_seen": 24256512, "tokens_per_sec": 18675.113310183744, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.120446681976318, "grad_norm": 0.7276890277862549, "tokens_seen": 24420352, "tokens_per_sec": 18676.28623505574, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.098109722137451, "grad_norm": 0.6908815503120422, "tokens_seen": 24584192, "tokens_per_sec": 18677.56923636502, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.048342514038086, "val_ppl": 155.76407364052096, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.054649829864502, "grad_norm": 0.8768901228904724, "tokens_seen": 24748032, "tokens_per_sec": 18634.33843395832, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.170622825622559, "grad_norm": 0.6523572206497192, "tokens_seen": 24911872, "tokens_per_sec": 18635.899710063928, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.912797451019287, "grad_norm": 0.6811433434486389, "tokens_seen": 25075712, "tokens_per_sec": 18637.132631377634, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.987507104873657, "grad_norm": 0.8084988594055176, "tokens_seen": 25239552, "tokens_per_sec": 18638.50983827227, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.927613019943237, "grad_norm": 0.7692644000053406, "tokens_seen": 25403392, "tokens_per_sec": 18639.97351564804, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.88690972328186, "grad_norm": 0.6952964067459106, "tokens_seen": 25567232, "tokens_per_sec": 18641.475540411484, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 5.067819595336914, "grad_norm": 0.7125639915466309, "tokens_seen": 25731072, "tokens_per_sec": 18642.80024446895, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.854369640350342, "grad_norm": 0.6983838081359863, "tokens_seen": 25894912, "tokens_per_sec": 18644.14843007259, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.8842692375183105, "grad_norm": 0.7736248970031738, "tokens_seen": 26058752, "tokens_per_sec": 18645.361300768727, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.841411590576172, "grad_norm": 0.665176272392273, "tokens_seen": 26222592, "tokens_per_sec": 18646.586312907435, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.826552391052246, "grad_norm": 0.8079140186309814, "tokens_seen": 26386432, "tokens_per_sec": 18647.854222553553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.878979444503784, "grad_norm": 0.6826104521751404, "tokens_seen": 26550272, "tokens_per_sec": 18649.092483547876, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.003003120422363, "grad_norm": 0.718781054019928, "tokens_seen": 26714112, "tokens_per_sec": 18650.28192018012, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.868488073348999, "grad_norm": 0.7065439820289612, "tokens_seen": 26877952, "tokens_per_sec": 18651.543614763414, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.0461578369140625, "grad_norm": 0.739102840423584, "tokens_seen": 27041792, "tokens_per_sec": 18652.594600071297, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.923418283462524, "grad_norm": 0.7804850339889526, "tokens_seen": 27205632, "tokens_per_sec": 18653.816521544228, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.98117470741272, "grad_norm": 0.7156400084495544, "tokens_seen": 27369472, "tokens_per_sec": 18654.791229230614, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.785350799560547, "grad_norm": 0.7369333505630493, "tokens_seen": 27533312, "tokens_per_sec": 18655.84168524259, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.96619176864624, "grad_norm": 0.9842808246612549, "tokens_seen": 27697152, "tokens_per_sec": 18656.942383353264, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.991742849349976, "grad_norm": 0.7662747502326965, "tokens_seen": 27860992, "tokens_per_sec": 18658.08783941065, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.8157057762146, "grad_norm": 0.7141731977462769, "tokens_seen": 28024832, "tokens_per_sec": 18659.146977241657, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.915035963058472, "grad_norm": 0.7168604731559753, "tokens_seen": 28188672, "tokens_per_sec": 18660.407583707667, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.827199935913086, "grad_norm": 0.701389729976654, "tokens_seen": 28352512, "tokens_per_sec": 18661.35465606922, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.796424627304077, "grad_norm": 0.6912838816642761, "tokens_seen": 28516352, "tokens_per_sec": 18662.45922894929, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.970445871353149, "grad_norm": 0.8318507671356201, "tokens_seen": 28680192, "tokens_per_sec": 18663.325489673283, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.935591053962708, "val_ppl": 139.15536656110362, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.9425647258758545, "grad_norm": 0.667151927947998, "tokens_seen": 28844032, "tokens_per_sec": 18626.57190604436, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.809300899505615, "grad_norm": 0.708935022354126, "tokens_seen": 29007872, "tokens_per_sec": 18627.48333265963, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.857528209686279, "grad_norm": 0.7606891393661499, "tokens_seen": 29171712, "tokens_per_sec": 18628.660436886472, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.9189393520355225, "grad_norm": 0.7539911866188049, "tokens_seen": 29335552, "tokens_per_sec": 18629.843020595694, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.788268327713013, "grad_norm": 0.7098650336265564, "tokens_seen": 29499392, "tokens_per_sec": 18630.98876683901, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.873860836029053, "grad_norm": 0.7236209511756897, "tokens_seen": 29663232, "tokens_per_sec": 18632.03893740553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.928498983383179, "grad_norm": 0.71012282371521, "tokens_seen": 29827072, "tokens_per_sec": 18633.215655962413, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.809976816177368, "grad_norm": 0.6932209134101868, "tokens_seen": 29990912, "tokens_per_sec": 18634.351374859074, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.949077367782593, "grad_norm": 0.7387899160385132, "tokens_seen": 30154752, "tokens_per_sec": 18635.582847631154, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.792922496795654, "grad_norm": 0.7623375654220581, "tokens_seen": 30318592, "tokens_per_sec": 18636.60833845682, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.9240734577178955, "grad_norm": 0.8354504108428955, "tokens_seen": 30482432, "tokens_per_sec": 18637.523555068634, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.868399381637573, "grad_norm": 0.720023512840271, "tokens_seen": 30646272, "tokens_per_sec": 18638.554689125933, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.9505650997161865, "grad_norm": 0.7511132955551147, "tokens_seen": 30810112, "tokens_per_sec": 18639.579404993274, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.926332950592041, "grad_norm": 0.7851009964942932, "tokens_seen": 30973952, "tokens_per_sec": 18640.658042338815, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.901261329650879, "grad_norm": 0.790350079536438, "tokens_seen": 31137792, "tokens_per_sec": 18641.52192601783, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.694547891616821, "grad_norm": 0.8936490416526794, "tokens_seen": 31301632, "tokens_per_sec": 18642.59777198631, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.947315692901611, "grad_norm": 0.7452596426010132, "tokens_seen": 31465472, "tokens_per_sec": 18643.713240761394, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.864606857299805, "grad_norm": 0.8296713829040527, "tokens_seen": 31629312, "tokens_per_sec": 18644.784949088626, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.765476703643799, "grad_norm": 0.7601428627967834, "tokens_seen": 31793152, "tokens_per_sec": 18645.889679739128, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.712690830230713, "grad_norm": 0.7797467112541199, "tokens_seen": 31956992, "tokens_per_sec": 18646.9441934441, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.888458251953125, "grad_norm": 1.365750789642334, "tokens_seen": 32120832, "tokens_per_sec": 18647.764103819285, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.679916858673096, "grad_norm": 0.7348853945732117, "tokens_seen": 32284672, "tokens_per_sec": 18648.704520225765, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.7991766929626465, "grad_norm": 0.7744191884994507, "tokens_seen": 32448512, "tokens_per_sec": 18649.563632907866, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.682670831680298, "grad_norm": 0.6999843120574951, "tokens_seen": 32612352, "tokens_per_sec": 18650.534854387373, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.764566898345947, "grad_norm": 0.7167416214942932, "tokens_seen": 32776192, "tokens_per_sec": 18651.564134484943, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.891586971282959, "val_ppl": 133.1647350641837, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.941775560379028, "grad_norm": 0.7543519735336304, "tokens_seen": 32940032, "tokens_per_sec": 18619.33733784707, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.896595239639282, "grad_norm": 0.732148289680481, "tokens_seen": 33103872, "tokens_per_sec": 18620.50489728729, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.776617527008057, "grad_norm": 0.7762644290924072, "tokens_seen": 33267712, "tokens_per_sec": 18621.56174719167, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.81941032409668, "grad_norm": 0.7725790739059448, "tokens_seen": 33431552, "tokens_per_sec": 18622.46150968061, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.715605020523071, "grad_norm": 0.8160229325294495, "tokens_seen": 33595392, "tokens_per_sec": 18623.541535389963, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.640811204910278, "grad_norm": 0.9230165481567383, "tokens_seen": 33759232, "tokens_per_sec": 18624.661999598513, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.754544734954834, "grad_norm": 0.7705773115158081, "tokens_seen": 33923072, "tokens_per_sec": 18625.658287152233, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.748470306396484, "grad_norm": 0.8215309381484985, "tokens_seen": 34086912, "tokens_per_sec": 18626.7231221248, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.927226305007935, "grad_norm": 0.8233512043952942, "tokens_seen": 34250752, "tokens_per_sec": 18627.73649200837, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.750659704208374, "grad_norm": 0.8366873860359192, "tokens_seen": 34414592, "tokens_per_sec": 18628.77015989832, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.813973665237427, "grad_norm": 0.720691442489624, "tokens_seen": 34578432, "tokens_per_sec": 18629.506156435982, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.82310938835144, "grad_norm": 0.7420302629470825, "tokens_seen": 34742272, "tokens_per_sec": 18630.247092209607, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.8608739376068115, "grad_norm": 0.7592225670814514, "tokens_seen": 34906112, "tokens_per_sec": 18631.092143087386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.880003213882446, "grad_norm": 0.8325053453445435, "tokens_seen": 35069952, "tokens_per_sec": 18632.080568985733, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.854421615600586, "grad_norm": 0.7803013920783997, "tokens_seen": 35233792, "tokens_per_sec": 18632.994584485026, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.691881418228149, "grad_norm": 0.844491183757782, "tokens_seen": 35397632, "tokens_per_sec": 18633.77004711727, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.723642826080322, "grad_norm": 0.8261871933937073, "tokens_seen": 35561472, "tokens_per_sec": 18634.664030951237, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.861575603485107, "grad_norm": 0.7845346927642822, "tokens_seen": 35725312, "tokens_per_sec": 18635.573834024763, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.854814291000366, "grad_norm": 0.7658339142799377, "tokens_seen": 35889152, "tokens_per_sec": 18636.448383301697, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.599551200866699, "grad_norm": 0.78634113073349, "tokens_seen": 36052992, "tokens_per_sec": 18637.181013866462, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.877705097198486, "grad_norm": 0.7602078318595886, "tokens_seen": 36216832, "tokens_per_sec": 18637.8044311695, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.710777759552002, "grad_norm": 0.8385705351829529, "tokens_seen": 36380672, "tokens_per_sec": 18638.75950271859, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.734395265579224, "grad_norm": 0.7861683964729309, "tokens_seen": 36544512, "tokens_per_sec": 18639.394831749258, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.701314449310303, "grad_norm": 0.7201582789421082, "tokens_seen": 36708352, "tokens_per_sec": 18640.237453758662, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.729318380355835, "grad_norm": 0.9218176007270813, "tokens_seen": 36872192, "tokens_per_sec": 18640.904915394487, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.8242013335227965, "val_ppl": 124.48700507167784, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.702590227127075, "grad_norm": 0.770817220211029, "tokens_seen": 37036032, "tokens_per_sec": 18612.109125535815, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.762092590332031, "grad_norm": 0.7649873495101929, "tokens_seen": 37199872, "tokens_per_sec": 18613.012390907617, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.783337354660034, "grad_norm": 0.8070899844169617, "tokens_seen": 37363712, "tokens_per_sec": 18613.869433730615, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.739763021469116, "grad_norm": 0.8113750219345093, "tokens_seen": 37527552, "tokens_per_sec": 18614.63534918292, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.738144159317017, "grad_norm": 0.8122279644012451, "tokens_seen": 37691392, "tokens_per_sec": 18615.556310532214, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.750312328338623, "grad_norm": 0.8077119588851929, "tokens_seen": 37855232, "tokens_per_sec": 18616.463716841055, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.813313961029053, "grad_norm": 0.8755798935890198, "tokens_seen": 38019072, "tokens_per_sec": 18617.42073533932, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.697959899902344, "grad_norm": 0.8010699152946472, "tokens_seen": 38182912, "tokens_per_sec": 18618.440639819815, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.757710218429565, "grad_norm": 0.8981845378875732, "tokens_seen": 38346752, "tokens_per_sec": 18619.09964038067, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.702011585235596, "grad_norm": 0.9134675860404968, "tokens_seen": 38510592, "tokens_per_sec": 18619.570591474283, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.73668909072876, "grad_norm": 0.7923208475112915, "tokens_seen": 38674432, "tokens_per_sec": 18619.76378430014, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.620959997177124, "grad_norm": 0.8486613035202026, "tokens_seen": 38838272, "tokens_per_sec": 18620.0477250271, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.730647563934326, "grad_norm": 0.9372776746749878, "tokens_seen": 39002112, "tokens_per_sec": 18620.332720167768, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.755367755889893, "grad_norm": 0.8557371497154236, "tokens_seen": 39165952, "tokens_per_sec": 18620.571268929005, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.806634426116943, "grad_norm": 0.7818487286567688, "tokens_seen": 39329792, "tokens_per_sec": 18620.561034558672, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.72579288482666, "grad_norm": 0.7692115902900696, "tokens_seen": 39493632, "tokens_per_sec": 18620.8176574095, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.476200103759766, "grad_norm": 0.8416553139686584, "tokens_seen": 39657472, "tokens_per_sec": 18620.738016394957, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.7408952713012695, "grad_norm": 0.8091793656349182, "tokens_seen": 39821312, "tokens_per_sec": 18620.400900244902, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.72725772857666, "grad_norm": 0.7926111221313477, "tokens_seen": 39985152, "tokens_per_sec": 18620.778990562176, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.545568466186523, "grad_norm": 0.7448379993438721, "tokens_seen": 40148992, "tokens_per_sec": 18620.90803477052, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.779897212982178, "grad_norm": 0.8478948473930359, "tokens_seen": 40312832, "tokens_per_sec": 18621.28847161706, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.644714593887329, "grad_norm": 0.7688567042350769, "tokens_seen": 40476672, "tokens_per_sec": 18621.728064301176, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.718647480010986, "grad_norm": 0.8412390947341919, "tokens_seen": 40640512, "tokens_per_sec": 18621.88565326096, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.5802812576293945, "grad_norm": 0.8112427592277527, "tokens_seen": 40804352, "tokens_per_sec": 18622.103655879495, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.785215139389038, "grad_norm": 0.9595121741294861, "tokens_seen": 40968192, "tokens_per_sec": 18622.28885400703, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.779685521125794, "val_ppl": 119.06690013181057, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.6721203327178955, "grad_norm": 0.9006349444389343, "tokens_seen": 41132032, "tokens_per_sec": 18595.30058938178, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.6497766971588135, "grad_norm": 0.820769190788269, "tokens_seen": 41295872, "tokens_per_sec": 18595.46801247012, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.573205232620239, "grad_norm": 1.0130138397216797, "tokens_seen": 41459712, "tokens_per_sec": 18595.6500694205, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.6903722286224365, "grad_norm": 0.8468446731567383, "tokens_seen": 41623552, "tokens_per_sec": 18596.011981651944, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.664802551269531, "grad_norm": 0.7521331906318665, "tokens_seen": 41787392, "tokens_per_sec": 18596.381594407947, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.60447096824646, "grad_norm": 0.805797278881073, "tokens_seen": 41951232, "tokens_per_sec": 18596.699264517792, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.711558103561401, "grad_norm": 0.775863766670227, "tokens_seen": 42115072, "tokens_per_sec": 18597.035620923423, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.556130886077881, "grad_norm": 0.816089391708374, "tokens_seen": 42278912, "tokens_per_sec": 18597.40956817156, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.587226390838623, "grad_norm": 0.8738594055175781, "tokens_seen": 42442752, "tokens_per_sec": 18597.81797100635, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.5324387550354, "grad_norm": 0.8863729238510132, "tokens_seen": 42606592, "tokens_per_sec": 18598.204820250183, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.617767572402954, "grad_norm": 0.8037155270576477, "tokens_seen": 42770432, "tokens_per_sec": 18598.54550067253, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.599399566650391, "grad_norm": 0.8382680416107178, "tokens_seen": 42934272, "tokens_per_sec": 18598.903121430187, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.562868118286133, "grad_norm": 0.8975099921226501, "tokens_seen": 43098112, "tokens_per_sec": 18599.15566411497, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.618177175521851, "grad_norm": 0.871766984462738, "tokens_seen": 43261952, "tokens_per_sec": 18599.67188475775, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.72063946723938, "grad_norm": 0.8351306319236755, "tokens_seen": 43425792, "tokens_per_sec": 18599.987102801493, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.603780031204224, "grad_norm": 0.7723714113235474, "tokens_seen": 43589632, "tokens_per_sec": 18600.009785142534, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.6830220222473145, "grad_norm": 0.8566490411758423, "tokens_seen": 43753472, "tokens_per_sec": 18600.214359720256, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.518950700759888, "grad_norm": 0.8342156410217285, "tokens_seen": 43917312, "tokens_per_sec": 18600.745551280554, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.782988786697388, "grad_norm": 0.7782499194145203, "tokens_seen": 44081152, "tokens_per_sec": 18601.200059726543, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.482271671295166, "grad_norm": 0.8266713619232178, "tokens_seen": 44244992, "tokens_per_sec": 18601.41921646916, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.654188632965088, "grad_norm": 0.8228229880332947, "tokens_seen": 44408832, "tokens_per_sec": 18601.788552392412, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.519973039627075, "grad_norm": 0.8958624601364136, "tokens_seen": 44572672, "tokens_per_sec": 18602.164949603743, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.6652326583862305, "grad_norm": 0.9195429086685181, "tokens_seen": 44736512, "tokens_per_sec": 18602.586756834527, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.5553765296936035, "grad_norm": 0.8280176520347595, "tokens_seen": 44900352, "tokens_per_sec": 18602.98839640205, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.6321892738342285, "grad_norm": 0.8456745743751526, "tokens_seen": 45064192, "tokens_per_sec": 18603.316365904604, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.706741714477539, "val_ppl": 110.69090910955924, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.444919586181641, "grad_norm": 0.8215606808662415, "tokens_seen": 45228032, "tokens_per_sec": 18579.503892993253, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.677463054656982, "grad_norm": 0.832905113697052, "tokens_seen": 45391872, "tokens_per_sec": 18579.85926420057, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.690892219543457, "grad_norm": 0.8105453848838806, "tokens_seen": 45555712, "tokens_per_sec": 18580.0456109993, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.737597465515137, "grad_norm": 0.7879133820533752, "tokens_seen": 45719552, "tokens_per_sec": 18580.416969483922, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.559011220932007, "grad_norm": 0.8267650604248047, "tokens_seen": 45883392, "tokens_per_sec": 18580.86213151301, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.638695478439331, "grad_norm": 0.9265620112419128, "tokens_seen": 46047232, "tokens_per_sec": 18581.095400360242, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.740231990814209, "grad_norm": 0.9221693277359009, "tokens_seen": 46211072, "tokens_per_sec": 18581.602668201067, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.509124755859375, "grad_norm": 0.7815338969230652, "tokens_seen": 46374912, "tokens_per_sec": 18582.000787401244, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.587784767150879, "grad_norm": 0.8695182800292969, "tokens_seen": 46538752, "tokens_per_sec": 18582.526967382626, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.725842237472534, "grad_norm": 0.963461697101593, "tokens_seen": 46702592, "tokens_per_sec": 18583.025874224582, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.5681352615356445, "grad_norm": 0.837469220161438, "tokens_seen": 46866432, "tokens_per_sec": 18583.524481654415, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.721729278564453, "grad_norm": 0.8287349343299866, "tokens_seen": 47030272, "tokens_per_sec": 18583.983593953853, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.602954149246216, "grad_norm": 0.8922120332717896, "tokens_seen": 47194112, "tokens_per_sec": 18584.504628312487, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.634100675582886, "grad_norm": 0.8453561663627625, "tokens_seen": 47357952, "tokens_per_sec": 18585.074109100973, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.499755144119263, "grad_norm": 0.866422176361084, "tokens_seen": 47521792, "tokens_per_sec": 18585.35267841523, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.730268478393555, "grad_norm": 0.8204149007797241, "tokens_seen": 47685632, "tokens_per_sec": 18585.715263324724, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.541012763977051, "grad_norm": 0.8261560201644897, "tokens_seen": 47849472, "tokens_per_sec": 18586.023835461234, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.632730484008789, "grad_norm": 0.8309053778648376, "tokens_seen": 48013312, "tokens_per_sec": 18586.348697493635, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.671115398406982, "grad_norm": 0.8788448572158813, "tokens_seen": 48177152, "tokens_per_sec": 18586.78931276089, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.578191757202148, "grad_norm": 0.8724309802055359, "tokens_seen": 48340992, "tokens_per_sec": 18586.947055230157, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.580864906311035, "grad_norm": 0.8435691595077515, "tokens_seen": 48504832, "tokens_per_sec": 18587.348846912817, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.497784852981567, "grad_norm": 0.8203125596046448, "tokens_seen": 48668672, "tokens_per_sec": 18587.90746052297, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.570899963378906, "grad_norm": 0.9406875371932983, "tokens_seen": 48832512, "tokens_per_sec": 18588.381222468823, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.804361343383789, "grad_norm": 0.8632916808128357, "tokens_seen": 48996352, "tokens_per_sec": 18588.921466904154, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.573354721069336, "grad_norm": 0.8801300525665283, "tokens_seen": 49160192, "tokens_per_sec": 18589.180032400436, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.655693781375885, "val_ppl": 105.18216812026667, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.493156433105469, "grad_norm": 0.809167206287384, "tokens_seen": 49324032, "tokens_per_sec": 18567.33891107843, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.519174575805664, "grad_norm": 0.9052747488021851, "tokens_seen": 49487872, "tokens_per_sec": 18567.622119314954, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.67163872718811, "grad_norm": 0.8174178600311279, "tokens_seen": 49651712, "tokens_per_sec": 18568.014590464183, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.606488227844238, "grad_norm": 0.8753623366355896, "tokens_seen": 49815552, "tokens_per_sec": 18568.3991416733, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.525824546813965, "grad_norm": 0.8372958302497864, "tokens_seen": 49979392, "tokens_per_sec": 18568.77523814249, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.683767795562744, "grad_norm": 0.9547490477561951, "tokens_seen": 50143232, "tokens_per_sec": 18569.315791408037, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.590677738189697, "grad_norm": 0.8641857504844666, "tokens_seen": 50307072, "tokens_per_sec": 18569.684961339088, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.461004734039307, "grad_norm": 0.8277103900909424, "tokens_seen": 50470912, "tokens_per_sec": 18569.97792230701, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.638270378112793, "grad_norm": 0.948759913444519, "tokens_seen": 50634752, "tokens_per_sec": 18570.5016111451, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.569838047027588, "grad_norm": 0.8967357277870178, "tokens_seen": 50798592, "tokens_per_sec": 18570.780480141675, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.5466814041137695, "grad_norm": 0.8806203603744507, "tokens_seen": 50962432, "tokens_per_sec": 18571.23007102386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.484802722930908, "grad_norm": 0.8056567907333374, "tokens_seen": 51126272, "tokens_per_sec": 18571.703038443404, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.617714881896973, "grad_norm": 0.8970366716384888, "tokens_seen": 51290112, "tokens_per_sec": 18572.09463759554, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.563032627105713, "grad_norm": 0.8722556829452515, "tokens_seen": 51453952, "tokens_per_sec": 18572.277742412873, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.507393836975098, "grad_norm": 0.9226962924003601, "tokens_seen": 51617792, "tokens_per_sec": 18572.733100931186, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.4738929271698, "grad_norm": 0.8948053121566772, "tokens_seen": 51781632, "tokens_per_sec": 18573.121476698903, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.559509992599487, "grad_norm": 0.89838045835495, "tokens_seen": 51945472, "tokens_per_sec": 18573.555470619798, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.590914249420166, "grad_norm": 0.9119340181350708, "tokens_seen": 52109312, "tokens_per_sec": 18573.960999758507, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.590933561325073, "grad_norm": 0.8971143364906311, "tokens_seen": 52273152, "tokens_per_sec": 18574.314971657772, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.441056251525879, "grad_norm": 0.8921831846237183, "tokens_seen": 52436992, "tokens_per_sec": 18574.77676280121, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.606031894683838, "grad_norm": 0.8880563378334045, "tokens_seen": 52600832, "tokens_per_sec": 18575.185967229987, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.6566548347473145, "grad_norm": 0.9063526391983032, "tokens_seen": 52764672, "tokens_per_sec": 18575.13167654882, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.497307062149048, "grad_norm": 0.9313164949417114, "tokens_seen": 52928512, "tokens_per_sec": 18575.575904208596, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.501392364501953, "grad_norm": 0.8894153833389282, "tokens_seen": 53092352, "tokens_per_sec": 18576.095067057155, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.529250621795654, "grad_norm": 0.8892995715141296, "tokens_seen": 53256192, "tokens_per_sec": 18576.47773032483, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.6318844079971315, "val_ppl": 102.70742456049241, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.694933891296387, "grad_norm": 0.9101230502128601, "tokens_seen": 53420032, "tokens_per_sec": 18556.38969393369, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.47900390625, "grad_norm": 0.8890600204467773, "tokens_seen": 53583872, "tokens_per_sec": 18556.71720396609, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.687341690063477, "grad_norm": 0.916364312171936, "tokens_seen": 53747712, "tokens_per_sec": 18557.21879214461, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.630258798599243, "grad_norm": 0.9580039381980896, "tokens_seen": 53911552, "tokens_per_sec": 18557.574504299755, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.376768112182617, "grad_norm": 0.9578585028648376, "tokens_seen": 54075392, "tokens_per_sec": 18558.01295143604, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.371206521987915, "grad_norm": 0.8829969763755798, "tokens_seen": 54239232, "tokens_per_sec": 18558.385145469238, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.687062978744507, "grad_norm": 0.8970814347267151, "tokens_seen": 54403072, "tokens_per_sec": 18558.80672022101, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.665821313858032, "grad_norm": 0.8797740936279297, "tokens_seen": 54566912, "tokens_per_sec": 18559.13403477496, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.519096612930298, "grad_norm": 0.8496575951576233, "tokens_seen": 54730752, "tokens_per_sec": 18559.595489483552, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.485321044921875, "grad_norm": 0.9405184984207153, "tokens_seen": 54894592, "tokens_per_sec": 18560.11442878422, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.577546834945679, "grad_norm": 0.9055253267288208, "tokens_seen": 55058432, "tokens_per_sec": 18560.62076702989, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.477223634719849, "grad_norm": 0.9549897909164429, "tokens_seen": 55222272, "tokens_per_sec": 18560.964274711587, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.501967668533325, "grad_norm": 0.9426594376564026, "tokens_seen": 55386112, "tokens_per_sec": 18561.359867553212, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.48242974281311, "grad_norm": 1.0213457345962524, "tokens_seen": 55549952, "tokens_per_sec": 18561.598418943107, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.425533294677734, "grad_norm": 0.8424636125564575, "tokens_seen": 55713792, "tokens_per_sec": 18562.08624764996, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.43634557723999, "grad_norm": 0.8764915466308594, "tokens_seen": 55877632, "tokens_per_sec": 18562.520505807228, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.3635358810424805, "grad_norm": 0.9034444093704224, "tokens_seen": 56041472, "tokens_per_sec": 18562.9471389887, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.567293882369995, "grad_norm": 0.9056126475334167, "tokens_seen": 56205312, "tokens_per_sec": 18563.445028535938, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.516265392303467, "grad_norm": 0.8637042045593262, "tokens_seen": 56369152, "tokens_per_sec": 18563.59284726347, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.651524305343628, "grad_norm": 1.0295426845550537, "tokens_seen": 56532992, "tokens_per_sec": 18563.986915640075, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.579680681228638, "grad_norm": 0.9185755252838135, "tokens_seen": 56696832, "tokens_per_sec": 18564.465894193385, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.420640230178833, "grad_norm": 0.8388102054595947, "tokens_seen": 56860672, "tokens_per_sec": 18564.902611980273, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.528065919876099, "grad_norm": 0.9369996786117554, "tokens_seen": 57024512, "tokens_per_sec": 18565.34198956009, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.3629984855651855, "grad_norm": 0.9470366835594177, "tokens_seen": 57188352, "tokens_per_sec": 18565.65536745868, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.652233600616455, "grad_norm": 0.932612955570221, "tokens_seen": 57352192, "tokens_per_sec": 18566.04725139891, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.576679933071136, "val_ppl": 97.19117674046774, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.459043979644775, "grad_norm": 0.9136372804641724, "tokens_seen": 57516032, "tokens_per_sec": 18546.743552872922, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.4620137214660645, "grad_norm": 0.9500139355659485, "tokens_seen": 57679872, "tokens_per_sec": 18543.93036674128, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.550784587860107, "grad_norm": 0.88311368227005, "tokens_seen": 57843712, "tokens_per_sec": 18544.384155355147, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.558621883392334, "grad_norm": 0.9279803037643433, "tokens_seen": 58007552, "tokens_per_sec": 18544.72256207348, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.540844678878784, "grad_norm": 0.9185247421264648, "tokens_seen": 58171392, "tokens_per_sec": 18545.204619818684, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.747811794281006, "grad_norm": 0.9883248805999756, "tokens_seen": 58335232, "tokens_per_sec": 18545.579909598757, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.515615463256836, "grad_norm": 0.9477130174636841, "tokens_seen": 58499072, "tokens_per_sec": 18546.0573795527, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.457859039306641, "grad_norm": 0.8973203301429749, "tokens_seen": 58662912, "tokens_per_sec": 18546.53504054202, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.578212022781372, "grad_norm": 0.9119975566864014, "tokens_seen": 58826752, "tokens_per_sec": 18546.88795005478, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.636928081512451, "grad_norm": 0.9791064858436584, "tokens_seen": 58990592, "tokens_per_sec": 18547.366971964224, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.587172269821167, "grad_norm": 0.9395557641983032, "tokens_seen": 59154432, "tokens_per_sec": 18547.727267329297, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.399141550064087, "grad_norm": 0.9620897173881531, "tokens_seen": 59318272, "tokens_per_sec": 18547.958729099228, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.447195291519165, "grad_norm": 0.9341991543769836, "tokens_seen": 59482112, "tokens_per_sec": 18548.399972057963, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.504668235778809, "grad_norm": 1.050785779953003, "tokens_seen": 59645952, "tokens_per_sec": 18548.803252469283, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.361506462097168, "grad_norm": 0.9154180884361267, "tokens_seen": 59809792, "tokens_per_sec": 18549.26375061954, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.60476541519165, "grad_norm": 0.9173752069473267, "tokens_seen": 59973632, "tokens_per_sec": 18549.66164047521, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.422069072723389, "grad_norm": 0.9473037123680115, "tokens_seen": 60137472, "tokens_per_sec": 18550.17655989191, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.388154983520508, "grad_norm": 0.8888207674026489, "tokens_seen": 60301312, "tokens_per_sec": 18550.541675138582, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.350804805755615, "grad_norm": 0.9484627842903137, "tokens_seen": 60465152, "tokens_per_sec": 18550.9837752449, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.47506856918335, "grad_norm": 0.9107881784439087, "tokens_seen": 60628992, "tokens_per_sec": 18551.420418472855, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.588514089584351, "grad_norm": 0.9617636203765869, "tokens_seen": 60792832, "tokens_per_sec": 18551.923957564988, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.624690294265747, "grad_norm": 0.9840243458747864, "tokens_seen": 60956672, "tokens_per_sec": 18552.074185082827, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.420677423477173, "grad_norm": 0.9298626184463501, "tokens_seen": 61120512, "tokens_per_sec": 18552.519969387064, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.480180978775024, "grad_norm": 0.977173924446106, "tokens_seen": 61284352, "tokens_per_sec": 18552.950507796333, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.534392356872559, "grad_norm": 0.935020923614502, "tokens_seen": 61448192, "tokens_per_sec": 18553.393572617308, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.60050185918808, "val_ppl": 99.53425529009606, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.51125431060791, "grad_norm": 0.8843566179275513, "tokens_seen": 61612032, "tokens_per_sec": 18536.28400013138, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.584105491638184, "grad_norm": 0.9548847675323486, "tokens_seen": 61775872, "tokens_per_sec": 18536.703303800434, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.276291847229004, "grad_norm": 0.8766446113586426, "tokens_seen": 61939712, "tokens_per_sec": 18537.10124372479, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.516156196594238, "grad_norm": 0.9589840173721313, "tokens_seen": 62103552, "tokens_per_sec": 18537.545461983238, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.362742185592651, "grad_norm": 0.984714150428772, "tokens_seen": 62267392, "tokens_per_sec": 18538.06599493716, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.543301582336426, "grad_norm": 1.0087417364120483, "tokens_seen": 62431232, "tokens_per_sec": 18538.584020356182, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.362884521484375, "grad_norm": 0.9882856607437134, "tokens_seen": 62595072, "tokens_per_sec": 18539.039816546596, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.512427806854248, "grad_norm": 0.9114844799041748, "tokens_seen": 62758912, "tokens_per_sec": 18539.310944052748, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.4767186641693115, "grad_norm": 1.0302023887634277, "tokens_seen": 62922752, "tokens_per_sec": 18539.740763201196, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.532094955444336, "grad_norm": 0.977031946182251, "tokens_seen": 63086592, "tokens_per_sec": 18540.06105457487, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.497029781341553, "grad_norm": 0.9943826794624329, "tokens_seen": 63250432, "tokens_per_sec": 18540.500639053676, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.351253032684326, "grad_norm": 0.9518991708755493, "tokens_seen": 63414272, "tokens_per_sec": 18540.89894189459, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.481576919555664, "grad_norm": 1.043532371520996, "tokens_seen": 63578112, "tokens_per_sec": 18541.315849912902, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.4231486320495605, "grad_norm": 0.979995608329773, "tokens_seen": 63741952, "tokens_per_sec": 18541.57521915072, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.490717887878418, "grad_norm": 0.9609515070915222, "tokens_seen": 63905792, "tokens_per_sec": 18541.94611569416, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.553723335266113, "grad_norm": 0.9810828566551208, "tokens_seen": 64069632, "tokens_per_sec": 18542.469856368898, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.312367916107178, "grad_norm": 0.9482528567314148, "tokens_seen": 64233472, "tokens_per_sec": 18542.89195329045, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.469425916671753, "grad_norm": 1.009134292602539, "tokens_seen": 64397312, "tokens_per_sec": 18543.32754314411, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.524807453155518, "grad_norm": 1.0139073133468628, "tokens_seen": 64561152, "tokens_per_sec": 18543.808255975848, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.301063299179077, "grad_norm": 0.9870384931564331, "tokens_seen": 64724992, "tokens_per_sec": 18544.280917792603, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.3623292446136475, "grad_norm": 0.9942115545272827, "tokens_seen": 64888832, "tokens_per_sec": 18544.584961517234, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.4721457958221436, "grad_norm": 1.0096616744995117, "tokens_seen": 65052672, "tokens_per_sec": 18545.073035358047, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.515677213668823, "grad_norm": 0.9724459648132324, "tokens_seen": 65216512, "tokens_per_sec": 18545.57487728584, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.417470932006836, "grad_norm": 0.9116675853729248, "tokens_seen": 65380352, "tokens_per_sec": 18546.08482971661, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.392369270324707, "grad_norm": 1.0186703205108643, "tokens_seen": 65544192, "tokens_per_sec": 18546.52954203236, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.52443813085556, "val_ppl": 92.24408216181949, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.424145936965942, "grad_norm": 0.9239085912704468, "tokens_seen": 65708032, "tokens_per_sec": 18530.508007846027, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.596339225769043, "grad_norm": 1.0620381832122803, "tokens_seen": 65871872, "tokens_per_sec": 18530.971157920598, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.5171098709106445, "grad_norm": 1.0318098068237305, "tokens_seen": 66035712, "tokens_per_sec": 18531.267249378172, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.4052042961120605, "grad_norm": 1.0288389921188354, "tokens_seen": 66199552, "tokens_per_sec": 18531.55006054758, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.52506685256958, "grad_norm": 0.9720541834831238, "tokens_seen": 66363392, "tokens_per_sec": 18532.14390209769, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.34982967376709, "grad_norm": 0.8863577842712402, "tokens_seen": 66527232, "tokens_per_sec": 18532.656424404628, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.510200262069702, "grad_norm": 0.964465856552124, "tokens_seen": 66691072, "tokens_per_sec": 18533.200871727156, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.49341893196106, "grad_norm": 1.035891056060791, "tokens_seen": 66854912, "tokens_per_sec": 18533.73827107976, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.565616846084595, "grad_norm": 0.9936989545822144, "tokens_seen": 67018752, "tokens_per_sec": 18534.29849039802, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.222253203392029, "grad_norm": 0.9376404881477356, "tokens_seen": 67182592, "tokens_per_sec": 18534.81235642685, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.396277666091919, "grad_norm": 0.9701206684112549, "tokens_seen": 67346432, "tokens_per_sec": 18535.355887734055, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.26693868637085, "grad_norm": 0.9407998323440552, "tokens_seen": 67510272, "tokens_per_sec": 18535.877199425944, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.4105353355407715, "grad_norm": 1.0055954456329346, "tokens_seen": 67674112, "tokens_per_sec": 18536.37572912078, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.328751564025879, "grad_norm": 0.9907688498497009, "tokens_seen": 67837952, "tokens_per_sec": 18536.848793216162, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.36266016960144, "grad_norm": 0.9425933957099915, "tokens_seen": 68001792, "tokens_per_sec": 18537.33990739011, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.425410270690918, "grad_norm": 1.0560107231140137, "tokens_seen": 68165632, "tokens_per_sec": 18537.796300829996, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.447124004364014, "grad_norm": 1.050581932067871, "tokens_seen": 68329472, "tokens_per_sec": 18538.355547895706, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.494417905807495, "grad_norm": 1.0534712076187134, "tokens_seen": 68493312, "tokens_per_sec": 18538.750948983838, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.431558132171631, "grad_norm": 0.9303742051124573, "tokens_seen": 68657152, "tokens_per_sec": 18539.30862029284, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.386359930038452, "grad_norm": 0.8694852590560913, "tokens_seen": 68820992, "tokens_per_sec": 18539.92750292959, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.524695158004761, "grad_norm": 1.016943097114563, "tokens_seen": 68984832, "tokens_per_sec": 18540.482243574166, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.194749593734741, "grad_norm": 0.9289159774780273, "tokens_seen": 69148672, "tokens_per_sec": 18540.94001262277, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.362478256225586, "grad_norm": 1.0748718976974487, "tokens_seen": 69312512, "tokens_per_sec": 18541.43948722769, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.307236194610596, "grad_norm": 0.9245467782020569, "tokens_seen": 69476352, "tokens_per_sec": 18541.87406045446, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.616661787033081, "grad_norm": 0.9835972785949707, "tokens_seen": 69640192, "tokens_per_sec": 18542.31699334477, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.512775695323944, "val_ppl": 91.17454035417592, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.426785707473755, "grad_norm": 0.9536045789718628, "tokens_seen": 69804032, "tokens_per_sec": 18527.343013810725, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.460023880004883, "grad_norm": 1.0676088333129883, "tokens_seen": 69967872, "tokens_per_sec": 18527.78637919585, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.524279356002808, "grad_norm": 1.0527456998825073, "tokens_seen": 70131712, "tokens_per_sec": 18528.239473244164, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.368939161300659, "grad_norm": 0.9429740309715271, "tokens_seen": 70295552, "tokens_per_sec": 18528.835703324035, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.49690318107605, "grad_norm": 1.039046287536621, "tokens_seen": 70459392, "tokens_per_sec": 18529.274730356115, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.452924489974976, "grad_norm": 0.9971879124641418, "tokens_seen": 70623232, "tokens_per_sec": 18529.85624433559, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.483038425445557, "grad_norm": 1.1435314416885376, "tokens_seen": 70787072, "tokens_per_sec": 18530.302865838315, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.3364996910095215, "grad_norm": 1.052681803703308, "tokens_seen": 70950912, "tokens_per_sec": 18530.797960439984, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.395498275756836, "grad_norm": 0.9428715705871582, "tokens_seen": 71114752, "tokens_per_sec": 18531.328360403597, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.375422477722168, "grad_norm": 1.0539147853851318, "tokens_seen": 71278592, "tokens_per_sec": 18531.781297763202, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.342244863510132, "grad_norm": 1.005510926246643, "tokens_seen": 71442432, "tokens_per_sec": 18532.255425785843, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.431304931640625, "grad_norm": 0.9613327383995056, "tokens_seen": 71606272, "tokens_per_sec": 18532.860855039355, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.326010465621948, "grad_norm": 1.1476541757583618, "tokens_seen": 71770112, "tokens_per_sec": 18533.353037811095, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.435685396194458, "grad_norm": 0.9437397718429565, "tokens_seen": 71933952, "tokens_per_sec": 18533.87243392311, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.4379308223724365, "grad_norm": 1.090957522392273, "tokens_seen": 72097792, "tokens_per_sec": 18534.40792865497, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.421525239944458, "grad_norm": 1.0010873079299927, "tokens_seen": 72261632, "tokens_per_sec": 18534.82721018884, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.426444053649902, "grad_norm": 1.0174909830093384, "tokens_seen": 72425472, "tokens_per_sec": 18535.272923691482, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.459243297576904, "grad_norm": 0.9702380895614624, "tokens_seen": 72589312, "tokens_per_sec": 18535.760710674454, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.2800209522247314, "grad_norm": 1.0212429761886597, "tokens_seen": 72753152, "tokens_per_sec": 18536.35698056368, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.3819193840026855, "grad_norm": 1.0633260011672974, "tokens_seen": 72916992, "tokens_per_sec": 18536.97345617705, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.469841003417969, "grad_norm": 1.1403956413269043, "tokens_seen": 73080832, "tokens_per_sec": 18537.549991398344, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.267824172973633, "grad_norm": 1.0881813764572144, "tokens_seen": 73244672, "tokens_per_sec": 18538.014439449966, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.504760265350342, "grad_norm": 1.0590039491653442, "tokens_seen": 73408512, "tokens_per_sec": 18538.56484904383, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.3727192878723145, "grad_norm": 1.0644599199295044, "tokens_seen": 73572352, "tokens_per_sec": 18539.16153580938, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.327488660812378, "grad_norm": 1.008020281791687, "tokens_seen": 73736192, "tokens_per_sec": 18539.58811821438, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.481220602989197, "val_ppl": 88.34243793679042, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.360975980758667, "grad_norm": 1.0369420051574707, "tokens_seen": 73900032, "tokens_per_sec": 18525.529583730884, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.320831775665283, "grad_norm": 1.0029659271240234, "tokens_seen": 74063872, "tokens_per_sec": 18526.119690391042, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.340886116027832, "grad_norm": 1.0799869298934937, "tokens_seen": 74227712, "tokens_per_sec": 18526.706422343173, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.3098273277282715, "grad_norm": 0.9786790609359741, "tokens_seen": 74391552, "tokens_per_sec": 18527.25559990589, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.266291618347168, "grad_norm": 0.9594520330429077, "tokens_seen": 74555392, "tokens_per_sec": 18527.821902294505, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.354236602783203, "grad_norm": 1.0846583843231201, "tokens_seen": 74719232, "tokens_per_sec": 18528.387916840617, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.499434471130371, "grad_norm": 1.04557204246521, "tokens_seen": 74883072, "tokens_per_sec": 18528.956240614847, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.321939468383789, "grad_norm": 1.0534536838531494, "tokens_seen": 75046912, "tokens_per_sec": 18529.551365657135, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.465654373168945, "grad_norm": 1.0724987983703613, "tokens_seen": 75210752, "tokens_per_sec": 18530.076679309008, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.422461986541748, "grad_norm": 1.279974102973938, "tokens_seen": 75374592, "tokens_per_sec": 18530.54715468176, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.467588186264038, "grad_norm": 1.12867271900177, "tokens_seen": 75538432, "tokens_per_sec": 18529.65041144722, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.46806788444519, "grad_norm": 1.0767149925231934, "tokens_seen": 75702272, "tokens_per_sec": 18529.933050037573, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.393189430236816, "grad_norm": 1.0030152797698975, "tokens_seen": 75866112, "tokens_per_sec": 18530.488296716685, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.338684558868408, "grad_norm": 0.9527992606163025, "tokens_seen": 76029952, "tokens_per_sec": 18530.93832223827, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.458083629608154, "grad_norm": 1.0315014123916626, "tokens_seen": 76193792, "tokens_per_sec": 18531.435843137104, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.3252387046813965, "grad_norm": 0.9897311329841614, "tokens_seen": 76357632, "tokens_per_sec": 18531.880161222674, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.4799439907073975, "grad_norm": 1.0726253986358643, "tokens_seen": 76521472, "tokens_per_sec": 18532.286936384156, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.541705131530762, "grad_norm": 1.1226394176483154, "tokens_seen": 76685312, "tokens_per_sec": 18532.567094554393, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.449329376220703, "grad_norm": 1.0562435388565063, "tokens_seen": 76849152, "tokens_per_sec": 18533.042435493113, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.305680274963379, "grad_norm": 1.0865671634674072, "tokens_seen": 77012992, "tokens_per_sec": 18533.54247347765, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.539138555526733, "grad_norm": 1.185103178024292, "tokens_seen": 77176832, "tokens_per_sec": 18534.040475683938, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.416462659835815, "grad_norm": 1.0566926002502441, "tokens_seen": 77340672, "tokens_per_sec": 18534.59669519516, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.352176666259766, "grad_norm": 1.0409653186798096, "tokens_seen": 77504512, "tokens_per_sec": 18535.053127397303, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.41366720199585, "grad_norm": 1.0432549715042114, "tokens_seen": 77668352, "tokens_per_sec": 18535.642203169726, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.350035190582275, "grad_norm": 1.0149537324905396, "tokens_seen": 77832192, "tokens_per_sec": 18536.134073361398, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.447317492961884, "val_ppl": 85.39755692603819, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.206342697143555, "grad_norm": 0.9861244559288025, "tokens_seen": 77996032, "tokens_per_sec": 18522.808750228134, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.527679681777954, "grad_norm": 1.0829424858093262, "tokens_seen": 78159872, "tokens_per_sec": 18523.427799976518, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.4984636306762695, "grad_norm": 1.0017842054367065, "tokens_seen": 78323712, "tokens_per_sec": 18523.90609682233, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.267892837524414, "grad_norm": 1.1290279626846313, "tokens_seen": 78487552, "tokens_per_sec": 18524.363361428866, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.351733684539795, "grad_norm": 1.1342087984085083, "tokens_seen": 78651392, "tokens_per_sec": 18524.86516820898, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.45286750793457, "grad_norm": 1.0646607875823975, "tokens_seen": 78815232, "tokens_per_sec": 18525.321649535334, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.3181703090667725, "grad_norm": 1.1909843683242798, "tokens_seen": 78979072, "tokens_per_sec": 18525.734364625532, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.4327075481414795, "grad_norm": 1.0331611633300781, "tokens_seen": 79142912, "tokens_per_sec": 18526.14497976016, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.6218483448028564, "grad_norm": 1.1388273239135742, "tokens_seen": 79306752, "tokens_per_sec": 18526.59339427765, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.266204833984375, "grad_norm": 0.9451889395713806, "tokens_seen": 79470592, "tokens_per_sec": 18527.114499915468, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.2762980461120605, "grad_norm": 0.971755862236023, "tokens_seen": 79634432, "tokens_per_sec": 18527.40290971181, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.446271181106567, "grad_norm": 1.1211217641830444, "tokens_seen": 79798272, "tokens_per_sec": 18527.808675488417, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.288689851760864, "grad_norm": 1.0141730308532715, "tokens_seen": 79962112, "tokens_per_sec": 18528.287261035974, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.368178367614746, "grad_norm": 1.0780675411224365, "tokens_seen": 80125952, "tokens_per_sec": 18528.905950383734, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.3298304080963135, "grad_norm": 1.1184618473052979, "tokens_seen": 80289792, "tokens_per_sec": 18529.349463095165, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.488492012023926, "grad_norm": 1.1071417331695557, "tokens_seen": 80453632, "tokens_per_sec": 18529.861496145026, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.121268272399902, "grad_norm": 1.0189471244812012, "tokens_seen": 80617472, "tokens_per_sec": 18530.24806849769, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.460531711578369, "grad_norm": 1.106054425239563, "tokens_seen": 80781312, "tokens_per_sec": 18530.77900388852, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.510333061218262, "grad_norm": 1.1284055709838867, "tokens_seen": 80945152, "tokens_per_sec": 18531.284207009332, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.302734851837158, "grad_norm": 1.026017665863037, "tokens_seen": 81108992, "tokens_per_sec": 18531.77610626706, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.436864376068115, "grad_norm": 1.019885540008545, "tokens_seen": 81272832, "tokens_per_sec": 18532.264607470028, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.578466415405273, "grad_norm": 1.1509627103805542, "tokens_seen": 81436672, "tokens_per_sec": 18532.742056460716, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.483678817749023, "grad_norm": 1.0257943868637085, "tokens_seen": 81600512, "tokens_per_sec": 18533.259489810356, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.350959300994873, "grad_norm": 1.0166957378387451, "tokens_seen": 81764352, "tokens_per_sec": 18533.67838144287, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.463154554367065, "grad_norm": 1.050276756286621, "tokens_seen": 81928192, "tokens_per_sec": 18534.21902855311, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.43799855709076, "val_ppl": 84.60543914932893, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.262304782867432, "grad_norm": 1.0727800130844116, "tokens_seen": 82092032, "tokens_per_sec": 18521.59275831651, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.24626088142395, "grad_norm": 1.0234944820404053, "tokens_seen": 82255872, "tokens_per_sec": 18522.064196441537, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.270499229431152, "grad_norm": 0.9818622469902039, "tokens_seen": 82419712, "tokens_per_sec": 18522.593914028184, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.604130029678345, "grad_norm": 1.10483980178833, "tokens_seen": 82583552, "tokens_per_sec": 18523.071846472325, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.574373483657837, "grad_norm": 1.0765979290008545, "tokens_seen": 82747392, "tokens_per_sec": 18523.621054957195, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.320065975189209, "grad_norm": 1.3085788488388062, "tokens_seen": 82911232, "tokens_per_sec": 18524.115038614615, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.334651708602905, "grad_norm": 1.1784138679504395, "tokens_seen": 83075072, "tokens_per_sec": 18524.499642343086, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.310887098312378, "grad_norm": 1.1834520101547241, "tokens_seen": 83238912, "tokens_per_sec": 18524.93078601516, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.300715923309326, "grad_norm": 1.1112650632858276, "tokens_seen": 83402752, "tokens_per_sec": 18525.397328858387, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.322278261184692, "grad_norm": 1.0417014360427856, "tokens_seen": 83566592, "tokens_per_sec": 18525.86158488385, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.363404273986816, "grad_norm": 1.2809770107269287, "tokens_seen": 83730432, "tokens_per_sec": 18526.32432961572, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.342404842376709, "grad_norm": 1.1129424571990967, "tokens_seen": 83894272, "tokens_per_sec": 18526.79059444411, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.413541793823242, "grad_norm": 1.1588995456695557, "tokens_seen": 84058112, "tokens_per_sec": 18527.24534160199, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.388892412185669, "grad_norm": 1.1605333089828491, "tokens_seen": 84221952, "tokens_per_sec": 18527.698967493707, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.288039684295654, "grad_norm": 1.0632611513137817, "tokens_seen": 84385792, "tokens_per_sec": 18528.130613765083, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.248725652694702, "grad_norm": 1.0548995733261108, "tokens_seen": 84549632, "tokens_per_sec": 18528.526427170993, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.315858364105225, "grad_norm": 1.1679577827453613, "tokens_seen": 84713472, "tokens_per_sec": 18529.03783614178, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.412982940673828, "grad_norm": 1.1988269090652466, "tokens_seen": 84877312, "tokens_per_sec": 18529.52886923167, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.2614665031433105, "grad_norm": 1.118200421333313, "tokens_seen": 85041152, "tokens_per_sec": 18529.988188670824, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.36414361000061, "grad_norm": 1.1298068761825562, "tokens_seen": 85204992, "tokens_per_sec": 18530.401874634365, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.417123556137085, "grad_norm": 1.2401491403579712, "tokens_seen": 85368832, "tokens_per_sec": 18530.887013408974, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.433330059051514, "grad_norm": 1.214778184890747, "tokens_seen": 85532672, "tokens_per_sec": 18531.29505959296, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.450512170791626, "grad_norm": 1.2383365631103516, "tokens_seen": 85696512, "tokens_per_sec": 18531.73932663394, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.340073585510254, "grad_norm": 1.1039284467697144, "tokens_seen": 85860352, "tokens_per_sec": 18532.13422997635, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.437248229980469, "grad_norm": 1.0979968309402466, "tokens_seen": 86024192, "tokens_per_sec": 18532.573135047965, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.4456297874450685, "val_ppl": 85.25355255076212, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.242414236068726, "grad_norm": 1.2190343141555786, "tokens_seen": 86188032, "tokens_per_sec": 18520.358781824598, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.421136140823364, "grad_norm": 1.3679721355438232, "tokens_seen": 86351872, "tokens_per_sec": 18520.66081418803, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.357821941375732, "grad_norm": 1.1724077463150024, "tokens_seen": 86515712, "tokens_per_sec": 18521.119444573553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.362497091293335, "grad_norm": 1.1136484146118164, "tokens_seen": 86679552, "tokens_per_sec": 18521.582174332383, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.385737895965576, "grad_norm": 1.0467414855957031, "tokens_seen": 86843392, "tokens_per_sec": 18522.002637428915, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.2076802253723145, "grad_norm": 1.084492564201355, "tokens_seen": 87007232, "tokens_per_sec": 18522.42824934485, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.216795206069946, "grad_norm": 1.0988364219665527, "tokens_seen": 87171072, "tokens_per_sec": 18522.92272790908, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.3593645095825195, "grad_norm": 1.1682112216949463, "tokens_seen": 87334912, "tokens_per_sec": 18523.368701530948, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.44258189201355, "grad_norm": 1.0425161123275757, "tokens_seen": 87498752, "tokens_per_sec": 18523.72074405276, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.2647411823272705, "grad_norm": 1.2019821405410767, "tokens_seen": 87662592, "tokens_per_sec": 18524.147387477267, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.469586133956909, "grad_norm": 1.0688011646270752, "tokens_seen": 87826432, "tokens_per_sec": 18524.536913140626, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.405364274978638, "grad_norm": 1.0509027242660522, "tokens_seen": 87990272, "tokens_per_sec": 18524.962244587943, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.2984631061553955, "grad_norm": 1.1221035718917847, "tokens_seen": 88154112, "tokens_per_sec": 18525.40829078658, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.39806342124939, "grad_norm": 1.1553776264190674, "tokens_seen": 88317952, "tokens_per_sec": 18525.725484103623, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.4759581089019775, "grad_norm": 1.211578369140625, "tokens_seen": 88481792, "tokens_per_sec": 18526.183163725225, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.262625694274902, "grad_norm": 1.2793546915054321, "tokens_seen": 88645632, "tokens_per_sec": 18526.63559216374, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.3660664558410645, "grad_norm": 1.0596868991851807, "tokens_seen": 88809472, "tokens_per_sec": 18527.080051770765, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.459820508956909, "grad_norm": 1.1749087572097778, "tokens_seen": 88973312, "tokens_per_sec": 18527.45591827094, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.294307470321655, "grad_norm": 1.1006624698638916, "tokens_seen": 89137152, "tokens_per_sec": 18527.787035871545, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.392316579818726, "grad_norm": 1.1760411262512207, "tokens_seen": 89300992, "tokens_per_sec": 18528.172370189804, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.315049648284912, "grad_norm": 1.1152663230895996, "tokens_seen": 89464832, "tokens_per_sec": 18528.620211937287, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.397035360336304, "grad_norm": 1.1730670928955078, "tokens_seen": 89628672, "tokens_per_sec": 18528.95166954605, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.446075677871704, "grad_norm": 1.178944706916809, "tokens_seen": 89792512, "tokens_per_sec": 18529.35687542593, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.353684186935425, "grad_norm": 1.2782338857650757, "tokens_seen": 89956352, "tokens_per_sec": 18529.792541688428, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.373977184295654, "grad_norm": 1.1483206748962402, "tokens_seen": 90120192, "tokens_per_sec": 18530.221146565225, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.40200662612915, "val_ppl": 81.61447419894579, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.224282264709473, "grad_norm": 1.1518833637237549, "tokens_seen": 90284032, "tokens_per_sec": 18518.438795130085, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.2250401973724365, "grad_norm": 1.0411157608032227, "tokens_seen": 90447872, "tokens_per_sec": 18518.890157564463, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.445651054382324, "grad_norm": 1.230961799621582, "tokens_seen": 90611712, "tokens_per_sec": 18519.23724232254, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.370028734207153, "grad_norm": 1.1917309761047363, "tokens_seen": 90775552, "tokens_per_sec": 18519.670691389754, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.441600322723389, "grad_norm": 1.1586114168167114, "tokens_seen": 90939392, "tokens_per_sec": 18520.110001333192, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.319808483123779, "grad_norm": 1.037383794784546, "tokens_seen": 91103232, "tokens_per_sec": 18520.580985285487, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.196626901626587, "grad_norm": 1.0948857069015503, "tokens_seen": 91267072, "tokens_per_sec": 18521.080699675887, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.271643161773682, "grad_norm": 1.4524503946304321, "tokens_seen": 91430912, "tokens_per_sec": 18521.480640355974, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.357658863067627, "grad_norm": 1.0600119829177856, "tokens_seen": 91594752, "tokens_per_sec": 18521.931493269243, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.361290454864502, "grad_norm": 1.0935081243515015, "tokens_seen": 91758592, "tokens_per_sec": 18522.359911024578, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.354551315307617, "grad_norm": 1.0479321479797363, "tokens_seen": 91922432, "tokens_per_sec": 18522.797741012364, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.194082260131836, "grad_norm": 1.0422662496566772, "tokens_seen": 92086272, "tokens_per_sec": 18523.250099242385, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.384793996810913, "grad_norm": 1.2585968971252441, "tokens_seen": 92250112, "tokens_per_sec": 18523.680982481867, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.383350133895874, "grad_norm": 1.1784422397613525, "tokens_seen": 92413952, "tokens_per_sec": 18524.12454878101, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.390413045883179, "grad_norm": 1.1186062097549438, "tokens_seen": 92577792, "tokens_per_sec": 18524.4361044246, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.238890886306763, "grad_norm": 1.1566340923309326, "tokens_seen": 92741632, "tokens_per_sec": 18524.84258858232, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.3053202629089355, "grad_norm": 1.203363299369812, "tokens_seen": 92905472, "tokens_per_sec": 18525.252409050943, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.208630561828613, "grad_norm": 1.1534889936447144, "tokens_seen": 93069312, "tokens_per_sec": 18525.628522740775, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.39574670791626, "grad_norm": 1.1130598783493042, "tokens_seen": 93233152, "tokens_per_sec": 18526.00466729764, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.2900354862213135, "grad_norm": 1.0797978639602661, "tokens_seen": 93396992, "tokens_per_sec": 18526.41283078822, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.32354736328125, "grad_norm": 1.116310715675354, "tokens_seen": 93560832, "tokens_per_sec": 18526.671628699307, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.16841983795166, "grad_norm": 1.227197527885437, "tokens_seen": 93724672, "tokens_per_sec": 18527.041313160866, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.401111602783203, "grad_norm": 1.1154595613479614, "tokens_seen": 93888512, "tokens_per_sec": 18527.45606591601, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.280885219573975, "grad_norm": 1.440141201019287, "tokens_seen": 94052352, "tokens_per_sec": 18527.849297394736, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.293467283248901, "grad_norm": 1.15973699092865, "tokens_seen": 94216192, "tokens_per_sec": 18528.137544587986, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.408300161361694, "val_ppl": 82.12973748087546, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.31578803062439, "grad_norm": 1.1233372688293457, "tokens_seen": 94380032, "tokens_per_sec": 18517.0452981984, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.441195964813232, "grad_norm": 1.241367220878601, "tokens_seen": 94543872, "tokens_per_sec": 18517.396050892712, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.38175630569458, "grad_norm": 1.1109975576400757, "tokens_seen": 94707712, "tokens_per_sec": 18517.811639515287, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.185446739196777, "grad_norm": 1.1267613172531128, "tokens_seen": 94871552, "tokens_per_sec": 18518.011896105636, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.309538841247559, "grad_norm": 1.1861224174499512, "tokens_seen": 95035392, "tokens_per_sec": 18518.43361818912, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.376643896102905, "grad_norm": 1.153005599975586, "tokens_seen": 95199232, "tokens_per_sec": 18518.806948394642, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.1437835693359375, "grad_norm": 1.0291732549667358, "tokens_seen": 95363072, "tokens_per_sec": 18519.217737085455, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.290805339813232, "grad_norm": 1.1794538497924805, "tokens_seen": 95526912, "tokens_per_sec": 18519.570126426763, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.26739764213562, "grad_norm": 1.114917516708374, "tokens_seen": 95690752, "tokens_per_sec": 18519.992606756412, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.279391765594482, "grad_norm": 1.1266099214553833, "tokens_seen": 95854592, "tokens_per_sec": 18520.394235664106, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.248436212539673, "grad_norm": 1.0944488048553467, "tokens_seen": 96018432, "tokens_per_sec": 18520.834503744223, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.4237024784088135, "grad_norm": 1.1189268827438354, "tokens_seen": 96182272, "tokens_per_sec": 18521.227157407237, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.358163595199585, "grad_norm": 1.1602025032043457, "tokens_seen": 96346112, "tokens_per_sec": 18521.659071225655, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.356736183166504, "grad_norm": 1.2474735975265503, "tokens_seen": 96509952, "tokens_per_sec": 18522.05251409645, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.400278806686401, "grad_norm": 1.1194031238555908, "tokens_seen": 96673792, "tokens_per_sec": 18522.135821820724, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.309163570404053, "grad_norm": 1.2480125427246094, "tokens_seen": 96837632, "tokens_per_sec": 18522.388285371817, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.348254919052124, "grad_norm": 1.1754730939865112, "tokens_seen": 97001472, "tokens_per_sec": 18522.71184946154, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.1405839920043945, "grad_norm": 1.1904990673065186, "tokens_seen": 97165312, "tokens_per_sec": 18523.028425260716, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.487298011779785, "grad_norm": 1.4051294326782227, "tokens_seen": 97329152, "tokens_per_sec": 18523.40187993249, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.26797080039978, "grad_norm": 1.3429830074310303, "tokens_seen": 97492992, "tokens_per_sec": 18523.64878563017, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.441081523895264, "grad_norm": 1.260815978050232, "tokens_seen": 97656832, "tokens_per_sec": 18523.98237031494, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.149876356124878, "grad_norm": 1.2542825937271118, "tokens_seen": 97820672, "tokens_per_sec": 18524.356726200138, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.342633485794067, "grad_norm": 1.1650941371917725, "tokens_seen": 97984512, "tokens_per_sec": 18524.69859373019, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.226228713989258, "grad_norm": 1.1364530324935913, "tokens_seen": 98148352, "tokens_per_sec": 18525.021726952742, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.360858964920044, "val_ppl": 78.32438343443285, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5309.61736869812, "nan_failures": 0} diff --git a/baseline_s1/config.json b/baseline_s1/config.json new file mode 100644 index 0000000000000000000000000000000000000000..6a2b0999ec516fea3fdab1a627fd19bbc2ad9252 --- /dev/null +++ b/baseline_s1/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 1, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s1", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1001340, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/baseline_s1/final.pt b/baseline_s1/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..6d395156aab2d77453fc317c64fa352cbaca8976 --- /dev/null +++ b/baseline_s1/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a5d08149cc3d598b4b31279b1e19ed2a79854bed1e1212d1d3094bfd50d0c0d +size 253572039 diff --git a/baseline_s1/metrics.jsonl b/baseline_s1/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..1a005bca3667b7567649c7d2b495ced02ab8c146 --- /dev/null +++ b/baseline_s1/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 1, "config": {"variant": "baseline", "seed": 1, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s1", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1001340, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.894187927246094, "grad_norm": 12.840143203735352, "tokens_seen": 8192, "tokens_per_sec": 5038.473402494093, "vram_gb": 5.453331456, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.821617126464844, "grad_norm": 2.1904776096343994, "tokens_seen": 172032, "tokens_per_sec": 16730.05962263407, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.177618026733398, "grad_norm": 1.6886783838272095, "tokens_seen": 335872, "tokens_per_sec": 17681.4287867992, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.279356002807617, "grad_norm": 1.4504493474960327, "tokens_seen": 499712, "tokens_per_sec": 18040.06378613553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.422396659851074, "grad_norm": 0.9481536149978638, "tokens_seen": 663552, "tokens_per_sec": 18201.812056704075, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.305180311203003, "grad_norm": 0.6560132503509521, "tokens_seen": 827392, "tokens_per_sec": 18299.626403782593, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.089128494262695, "grad_norm": 0.6177870035171509, "tokens_seen": 991232, "tokens_per_sec": 18352.979985504357, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.8202126026153564, "grad_norm": 0.6152971386909485, "tokens_seen": 1155072, "tokens_per_sec": 18403.89487225117, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.791837930679321, "grad_norm": 0.6325405240058899, "tokens_seen": 1318912, "tokens_per_sec": 18442.040979530146, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.77067232131958, "grad_norm": 0.6821605563163757, "tokens_seen": 1482752, "tokens_per_sec": 18471.96765609028, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.616251468658447, "grad_norm": 0.6107667684555054, "tokens_seen": 1646592, "tokens_per_sec": 18495.34686507943, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.611367702484131, "grad_norm": 0.6281615495681763, "tokens_seen": 1810432, "tokens_per_sec": 18516.243762652008, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.625713586807251, "grad_norm": 0.5048328638076782, "tokens_seen": 1974272, "tokens_per_sec": 18529.491242727207, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.628416061401367, "grad_norm": 0.5055111646652222, "tokens_seen": 2138112, "tokens_per_sec": 18543.510759209054, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.710697889328003, "grad_norm": 0.5401038527488708, "tokens_seen": 2301952, "tokens_per_sec": 18553.499150349588, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.545255661010742, "grad_norm": 0.5461310744285583, "tokens_seen": 2465792, "tokens_per_sec": 18563.34779140678, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.475483417510986, "grad_norm": 0.5658100843429565, "tokens_seen": 2629632, "tokens_per_sec": 18569.820353131723, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.502255201339722, "grad_norm": 0.574661135673523, "tokens_seen": 2793472, "tokens_per_sec": 18578.238042829664, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.2481138706207275, "grad_norm": 0.636875331401825, "tokens_seen": 2957312, "tokens_per_sec": 18583.50268529955, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.308951377868652, "grad_norm": 0.4929552376270294, "tokens_seen": 3121152, "tokens_per_sec": 18591.305851038305, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.273540019989014, "grad_norm": 0.48576274514198303, "tokens_seen": 3284992, "tokens_per_sec": 18596.00480178879, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.371373891830444, "grad_norm": 0.5051372647285461, "tokens_seen": 3448832, "tokens_per_sec": 18603.86133869488, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.273222208023071, "grad_norm": 0.622732937335968, "tokens_seen": 3612672, "tokens_per_sec": 18608.12483492838, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.3453521728515625, "grad_norm": 0.5336393713951111, "tokens_seen": 3776512, "tokens_per_sec": 18611.32757068699, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.203033447265625, "grad_norm": 0.5022684335708618, "tokens_seen": 3940352, "tokens_per_sec": 18614.278583880197, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.339316129684448, "grad_norm": 0.579375684261322, "tokens_seen": 4104192, "tokens_per_sec": 18617.39114156921, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.261846256256104, "val_ppl": 524.185828636835, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.181748867034912, "grad_norm": 0.5547939538955688, "tokens_seen": 4268032, "tokens_per_sec": 18364.496422303557, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.273858308792114, "grad_norm": 0.5561439990997314, "tokens_seen": 4431872, "tokens_per_sec": 18375.457890180944, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.163022756576538, "grad_norm": 0.5117971897125244, "tokens_seen": 4595712, "tokens_per_sec": 18385.64855290047, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.104215383529663, "grad_norm": 0.5477694869041443, "tokens_seen": 4759552, "tokens_per_sec": 18395.243915144765, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.155248641967773, "grad_norm": 0.6027102470397949, "tokens_seen": 4923392, "tokens_per_sec": 18404.60649459174, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.266623020172119, "grad_norm": 0.4858063757419586, "tokens_seen": 5087232, "tokens_per_sec": 18413.77848154606, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.263182640075684, "grad_norm": 0.5760160684585571, "tokens_seen": 5251072, "tokens_per_sec": 18420.81343208522, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.986729383468628, "grad_norm": 0.506699800491333, "tokens_seen": 5414912, "tokens_per_sec": 18428.14470178424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.013251304626465, "grad_norm": 0.5355015993118286, "tokens_seen": 5578752, "tokens_per_sec": 18433.872691300116, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.990871906280518, "grad_norm": 0.5134819149971008, "tokens_seen": 5742592, "tokens_per_sec": 18439.96911644617, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.074056148529053, "grad_norm": 0.5060974359512329, "tokens_seen": 5906432, "tokens_per_sec": 18446.14759607787, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.9969751834869385, "grad_norm": 0.5432641506195068, "tokens_seen": 6070272, "tokens_per_sec": 18451.430585827773, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.054317474365234, "grad_norm": 0.5135830044746399, "tokens_seen": 6234112, "tokens_per_sec": 18456.853499477893, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.031311988830566, "grad_norm": 0.5114948153495789, "tokens_seen": 6397952, "tokens_per_sec": 18460.541868993296, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.951602458953857, "grad_norm": 0.5624085068702698, "tokens_seen": 6561792, "tokens_per_sec": 18466.250932117535, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8575873374938965, "grad_norm": 0.5391199588775635, "tokens_seen": 6725632, "tokens_per_sec": 18469.840951845214, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.790021657943726, "grad_norm": 0.5543391108512878, "tokens_seen": 6889472, "tokens_per_sec": 18474.763206268846, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.696231365203857, "grad_norm": 0.5137075781822205, "tokens_seen": 7053312, "tokens_per_sec": 18478.68577561882, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.773521184921265, "grad_norm": 0.553370475769043, "tokens_seen": 7217152, "tokens_per_sec": 18482.406288203183, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.697640419006348, "grad_norm": 0.5467625856399536, "tokens_seen": 7380992, "tokens_per_sec": 18486.688461986374, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.890371322631836, "grad_norm": 0.5559662580490112, "tokens_seen": 7544832, "tokens_per_sec": 18490.951250129554, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.796257019042969, "grad_norm": 0.5262491703033447, "tokens_seen": 7708672, "tokens_per_sec": 18494.652073810386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.936986446380615, "grad_norm": 0.5311614274978638, "tokens_seen": 7872512, "tokens_per_sec": 18497.718742902918, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.638291597366333, "grad_norm": 0.5078580379486084, "tokens_seen": 8036352, "tokens_per_sec": 18501.33628448945, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.7745277881622314, "grad_norm": 0.5372287034988403, "tokens_seen": 8200192, "tokens_per_sec": 18502.678676789244, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.7938099026679994, "val_ppl": 328.2612885425845, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.684814453125, "grad_norm": 0.5746755003929138, "tokens_seen": 8364032, "tokens_per_sec": 18375.21401076702, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.737382411956787, "grad_norm": 0.5710068941116333, "tokens_seen": 8527872, "tokens_per_sec": 18379.83444157579, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.79574728012085, "grad_norm": 0.5461016297340393, "tokens_seen": 8691712, "tokens_per_sec": 18383.82758603925, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.581888914108276, "grad_norm": 0.5116097927093506, "tokens_seen": 8855552, "tokens_per_sec": 18389.403937935618, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.803101301193237, "grad_norm": 0.5633887648582458, "tokens_seen": 9019392, "tokens_per_sec": 18393.83957437279, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.661193609237671, "grad_norm": 0.5810731053352356, "tokens_seen": 9183232, "tokens_per_sec": 18397.900750736673, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.69517707824707, "grad_norm": 0.49330267310142517, "tokens_seen": 9347072, "tokens_per_sec": 18402.644122354242, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.6915435791015625, "grad_norm": 0.579338550567627, "tokens_seen": 9510912, "tokens_per_sec": 18406.374789384205, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.724461078643799, "grad_norm": 0.5912520885467529, "tokens_seen": 9674752, "tokens_per_sec": 18410.58741132684, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.662317752838135, "grad_norm": 0.5563398003578186, "tokens_seen": 9838592, "tokens_per_sec": 18413.72980572908, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.844480991363525, "grad_norm": 0.6378459930419922, "tokens_seen": 10002432, "tokens_per_sec": 18417.77238287711, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.605411052703857, "grad_norm": 0.5775927305221558, "tokens_seen": 10166272, "tokens_per_sec": 18421.35824617458, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.609335660934448, "grad_norm": 0.632107138633728, "tokens_seen": 10330112, "tokens_per_sec": 18424.831607452044, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.4945244789123535, "grad_norm": 0.5340181589126587, "tokens_seen": 10493952, "tokens_per_sec": 18428.475284957625, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.6335532665252686, "grad_norm": 0.5771287083625793, "tokens_seen": 10657792, "tokens_per_sec": 18431.885532629683, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.4951910972595215, "grad_norm": 0.5894888639450073, "tokens_seen": 10821632, "tokens_per_sec": 18434.97993978432, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.744264602661133, "grad_norm": 0.621690571308136, "tokens_seen": 10985472, "tokens_per_sec": 18438.52790245704, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.565986394882202, "grad_norm": 0.5504023432731628, "tokens_seen": 11149312, "tokens_per_sec": 18441.55521659536, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.489603281021118, "grad_norm": 0.5714759826660156, "tokens_seen": 11313152, "tokens_per_sec": 18444.359298019735, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.702593803405762, "grad_norm": 0.554024338722229, "tokens_seen": 11476992, "tokens_per_sec": 18446.52241051214, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.458469390869141, "grad_norm": 0.5259249210357666, "tokens_seen": 11640832, "tokens_per_sec": 18449.052357065346, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.596973180770874, "grad_norm": 0.5479153394699097, "tokens_seen": 11804672, "tokens_per_sec": 18452.00408156276, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.617449760437012, "grad_norm": 0.6469538807868958, "tokens_seen": 11968512, "tokens_per_sec": 18453.99889226624, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.504939079284668, "grad_norm": 0.537784218788147, "tokens_seen": 12132352, "tokens_per_sec": 18456.585389359814, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.320697069168091, "grad_norm": 0.648520290851593, "tokens_seen": 12296192, "tokens_per_sec": 18459.705501139437, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.517850840091706, "val_ppl": 249.09910768144974, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.503247261047363, "grad_norm": 0.5666484832763672, "tokens_seen": 12460032, "tokens_per_sec": 18375.03453148202, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.509313583374023, "grad_norm": 0.6438779830932617, "tokens_seen": 12623872, "tokens_per_sec": 18378.140217234097, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.479535102844238, "grad_norm": 0.5880498290061951, "tokens_seen": 12787712, "tokens_per_sec": 18381.21735402415, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.162854909896851, "grad_norm": 0.5506569743156433, "tokens_seen": 12951552, "tokens_per_sec": 18383.922807070932, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.255427837371826, "grad_norm": 0.6520641446113586, "tokens_seen": 13115392, "tokens_per_sec": 18386.840143441772, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.267862796783447, "grad_norm": 0.5807978510856628, "tokens_seen": 13279232, "tokens_per_sec": 18389.706031593225, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.269646644592285, "grad_norm": 0.5493749976158142, "tokens_seen": 13443072, "tokens_per_sec": 18391.98836933801, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.4070799350738525, "grad_norm": 0.6387667059898376, "tokens_seen": 13606912, "tokens_per_sec": 18395.031005845496, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.206947088241577, "grad_norm": 0.5771621465682983, "tokens_seen": 13770752, "tokens_per_sec": 18397.451549632937, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.2906622886657715, "grad_norm": 0.5524643063545227, "tokens_seen": 13934592, "tokens_per_sec": 18400.171841965257, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.363339185714722, "grad_norm": 0.5894514322280884, "tokens_seen": 14098432, "tokens_per_sec": 18402.828597027612, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.303325653076172, "grad_norm": 0.5640892386436462, "tokens_seen": 14262272, "tokens_per_sec": 18405.321121792913, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.356955051422119, "grad_norm": 0.598112940788269, "tokens_seen": 14426112, "tokens_per_sec": 18407.581422722338, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.347827911376953, "grad_norm": 0.5825461149215698, "tokens_seen": 14589952, "tokens_per_sec": 18408.679635644457, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.455654144287109, "grad_norm": 0.6322556138038635, "tokens_seen": 14753792, "tokens_per_sec": 18410.809968527377, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.441610336303711, "grad_norm": 0.5888528823852539, "tokens_seen": 14917632, "tokens_per_sec": 18412.84167834649, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.22222900390625, "grad_norm": 0.6147303581237793, "tokens_seen": 15081472, "tokens_per_sec": 18414.886449836486, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.302528142929077, "grad_norm": 0.648514986038208, "tokens_seen": 15245312, "tokens_per_sec": 18417.336368108947, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.126223802566528, "grad_norm": 0.5771087408065796, "tokens_seen": 15409152, "tokens_per_sec": 18419.72828895489, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.364521503448486, "grad_norm": 0.6324231028556824, "tokens_seen": 15572992, "tokens_per_sec": 18421.905852159816, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.388929843902588, "grad_norm": 0.6150810122489929, "tokens_seen": 15736832, "tokens_per_sec": 18424.575312255343, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.504941463470459, "grad_norm": 0.5936684608459473, "tokens_seen": 15900672, "tokens_per_sec": 18425.90153286068, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.152280330657959, "grad_norm": 0.5890950560569763, "tokens_seen": 16064512, "tokens_per_sec": 18427.876175132547, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.267764091491699, "grad_norm": 0.6288110613822937, "tokens_seen": 16228352, "tokens_per_sec": 18429.775602471276, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.321403980255127, "grad_norm": 0.6355517506599426, "tokens_seen": 16392192, "tokens_per_sec": 18431.720669840346, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.367719411849976, "val_ppl": 214.3734123295739, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.237840175628662, "grad_norm": 0.6292460560798645, "tokens_seen": 16556032, "tokens_per_sec": 18368.386230926222, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.309171915054321, "grad_norm": 0.6496156454086304, "tokens_seen": 16719872, "tokens_per_sec": 18370.709842243967, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.256886959075928, "grad_norm": 0.594859778881073, "tokens_seen": 16883712, "tokens_per_sec": 18372.81183130382, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.295757055282593, "grad_norm": 0.629502534866333, "tokens_seen": 17047552, "tokens_per_sec": 18375.482580776745, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.309129238128662, "grad_norm": 0.573697030544281, "tokens_seen": 17211392, "tokens_per_sec": 18377.783243264486, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.1898980140686035, "grad_norm": 0.6288768649101257, "tokens_seen": 17375232, "tokens_per_sec": 18380.207138720227, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.161802291870117, "grad_norm": 0.639483630657196, "tokens_seen": 17539072, "tokens_per_sec": 18382.70736496262, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.205959320068359, "grad_norm": 0.6188419461250305, "tokens_seen": 17702912, "tokens_per_sec": 18384.761125602337, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.217644691467285, "grad_norm": 0.6265055537223816, "tokens_seen": 17866752, "tokens_per_sec": 18386.723476164236, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.1751768589019775, "grad_norm": 0.5923656821250916, "tokens_seen": 18030592, "tokens_per_sec": 18389.030175755415, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.309700012207031, "grad_norm": 0.668271541595459, "tokens_seen": 18194432, "tokens_per_sec": 18391.21291461139, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.112881660461426, "grad_norm": 0.5922240614891052, "tokens_seen": 18358272, "tokens_per_sec": 18393.48465552014, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.087188005447388, "grad_norm": 0.6091412305831909, "tokens_seen": 18522112, "tokens_per_sec": 18396.129057283433, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.204813480377197, "grad_norm": 0.6420775651931763, "tokens_seen": 18685952, "tokens_per_sec": 18398.324796608125, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.133959770202637, "grad_norm": 0.6308857798576355, "tokens_seen": 18849792, "tokens_per_sec": 18400.576169966313, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.024588108062744, "grad_norm": 0.6202092170715332, "tokens_seen": 19013632, "tokens_per_sec": 18402.406290677733, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.1899425983428955, "grad_norm": 0.5998408794403076, "tokens_seen": 19177472, "tokens_per_sec": 18404.58545133386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.26208233833313, "grad_norm": 0.6411885023117065, "tokens_seen": 19341312, "tokens_per_sec": 18406.824133574377, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.220269441604614, "grad_norm": 0.6687451004981995, "tokens_seen": 19505152, "tokens_per_sec": 18408.29360951948, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.140031576156616, "grad_norm": 0.72568678855896, "tokens_seen": 19668992, "tokens_per_sec": 18410.297986579506, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.159706115722656, "grad_norm": 0.6051239967346191, "tokens_seen": 19832832, "tokens_per_sec": 18412.692391028228, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.149464130401611, "grad_norm": 0.665452778339386, "tokens_seen": 19996672, "tokens_per_sec": 18415.045995487737, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.185205459594727, "grad_norm": 0.6657267808914185, "tokens_seen": 20160512, "tokens_per_sec": 18416.750964217656, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.159993886947632, "grad_norm": 0.6403266191482544, "tokens_seen": 20324352, "tokens_per_sec": 18418.94452628407, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.177808046340942, "grad_norm": 0.6784225106239319, "tokens_seen": 20488192, "tokens_per_sec": 18421.05572697749, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.204453074932099, "val_ppl": 182.08126072521628, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.272786617279053, "grad_norm": 0.809325098991394, "tokens_seen": 20652032, "tokens_per_sec": 18371.28642745425, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.214205741882324, "grad_norm": 0.6230910420417786, "tokens_seen": 20815872, "tokens_per_sec": 18373.80628206537, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.183841228485107, "grad_norm": 0.6753566265106201, "tokens_seen": 20979712, "tokens_per_sec": 18376.189167287364, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.028142929077148, "grad_norm": 0.6503043174743652, "tokens_seen": 21143552, "tokens_per_sec": 18378.340266883893, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.0963454246521, "grad_norm": 0.6317532062530518, "tokens_seen": 21307392, "tokens_per_sec": 18380.170041423873, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.023433208465576, "grad_norm": 0.7034384608268738, "tokens_seen": 21471232, "tokens_per_sec": 18382.19867805796, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.216521263122559, "grad_norm": 0.7439193725585938, "tokens_seen": 21635072, "tokens_per_sec": 18384.50050285584, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.049310207366943, "grad_norm": 0.6533716917037964, "tokens_seen": 21798912, "tokens_per_sec": 18386.907504823386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.062217712402344, "grad_norm": 0.659913957118988, "tokens_seen": 21962752, "tokens_per_sec": 18388.44512011733, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.199546575546265, "grad_norm": 0.7093247771263123, "tokens_seen": 22126592, "tokens_per_sec": 18390.594831325594, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.052448034286499, "grad_norm": 0.7064862251281738, "tokens_seen": 22290432, "tokens_per_sec": 18392.70064870651, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.212583780288696, "grad_norm": 0.7314108610153198, "tokens_seen": 22454272, "tokens_per_sec": 18394.82665714338, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.034427642822266, "grad_norm": 0.6999198198318481, "tokens_seen": 22618112, "tokens_per_sec": 18396.53161113348, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.1576197147369385, "grad_norm": 0.6835367679595947, "tokens_seen": 22781952, "tokens_per_sec": 18397.566969496365, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.899679183959961, "grad_norm": 0.6852870583534241, "tokens_seen": 22945792, "tokens_per_sec": 18397.360952207553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.220712423324585, "grad_norm": 0.6683334708213806, "tokens_seen": 23109632, "tokens_per_sec": 18397.78983757767, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.0773561000823975, "grad_norm": 0.7100439667701721, "tokens_seen": 23273472, "tokens_per_sec": 18398.208744002517, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.01195764541626, "grad_norm": 0.660786509513855, "tokens_seen": 23437312, "tokens_per_sec": 18398.323973140814, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.083988666534424, "grad_norm": 0.6965491771697998, "tokens_seen": 23601152, "tokens_per_sec": 18398.86593279591, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.062253475189209, "grad_norm": 0.6950525045394897, "tokens_seen": 23764992, "tokens_per_sec": 18400.670429493242, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.010495662689209, "grad_norm": 0.7276315689086914, "tokens_seen": 23928832, "tokens_per_sec": 18399.797132043772, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.0966269969940186, "grad_norm": 0.7003159523010254, "tokens_seen": 24092672, "tokens_per_sec": 18399.291271929756, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.880216836929321, "grad_norm": 0.6962426900863647, "tokens_seen": 24256512, "tokens_per_sec": 18401.58171142417, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.0023276805877686, "grad_norm": 0.6698529124259949, "tokens_seen": 24420352, "tokens_per_sec": 18403.813352537327, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.9825897216796875, "grad_norm": 0.7161815762519836, "tokens_seen": 24584192, "tokens_per_sec": 18405.382146045056, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.063517677783966, "val_ppl": 158.14584514288788, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.025374412536621, "grad_norm": 0.7259272336959839, "tokens_seen": 24748032, "tokens_per_sec": 18362.95492697424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.863132476806641, "grad_norm": 0.6766065359115601, "tokens_seen": 24911872, "tokens_per_sec": 18364.88911496963, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.007089614868164, "grad_norm": 0.6909694671630859, "tokens_seen": 25075712, "tokens_per_sec": 18366.552993539466, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.868147134780884, "grad_norm": 0.676908016204834, "tokens_seen": 25239552, "tokens_per_sec": 18368.589206108823, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.022034168243408, "grad_norm": 0.7120152115821838, "tokens_seen": 25403392, "tokens_per_sec": 18370.967623530567, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 5.023714065551758, "grad_norm": 0.712931752204895, "tokens_seen": 25567232, "tokens_per_sec": 18373.04975301971, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.976207256317139, "grad_norm": 0.7531589865684509, "tokens_seen": 25731072, "tokens_per_sec": 18375.15440078855, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.9825780391693115, "grad_norm": 0.7254669666290283, "tokens_seen": 25894912, "tokens_per_sec": 18377.000720493583, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 5.0306408405303955, "grad_norm": 0.7669152021408081, "tokens_seen": 26058752, "tokens_per_sec": 18379.127934405533, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.80053973197937, "grad_norm": 0.6783765554428101, "tokens_seen": 26222592, "tokens_per_sec": 18380.938715538436, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.014438152313232, "grad_norm": 0.7352532744407654, "tokens_seen": 26386432, "tokens_per_sec": 18382.965727114475, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.03941011428833, "grad_norm": 0.7127110362052917, "tokens_seen": 26550272, "tokens_per_sec": 18384.69832918564, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.929201602935791, "grad_norm": 0.7434890866279602, "tokens_seen": 26714112, "tokens_per_sec": 18386.626691933387, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.85127592086792, "grad_norm": 0.7333239316940308, "tokens_seen": 26877952, "tokens_per_sec": 18388.55585688186, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.037408828735352, "grad_norm": 0.7122877240180969, "tokens_seen": 27041792, "tokens_per_sec": 18390.39697231659, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 5.059391260147095, "grad_norm": 0.7254337668418884, "tokens_seen": 27205632, "tokens_per_sec": 18392.448423260204, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.9044811725616455, "grad_norm": 0.6949459314346313, "tokens_seen": 27369472, "tokens_per_sec": 18394.403411711115, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.817488193511963, "grad_norm": 0.7590793371200562, "tokens_seen": 27533312, "tokens_per_sec": 18396.439956593513, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.0969812870025635, "grad_norm": 0.7607041001319885, "tokens_seen": 27697152, "tokens_per_sec": 18397.536853686066, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 5.173400163650513, "grad_norm": 0.7021132111549377, "tokens_seen": 27860992, "tokens_per_sec": 18399.21374914504, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.069242000579834, "grad_norm": 0.7173861265182495, "tokens_seen": 28024832, "tokens_per_sec": 18400.973102168893, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.875608205795288, "grad_norm": 0.7268418073654175, "tokens_seen": 28188672, "tokens_per_sec": 18402.9458749931, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 5.004350900650024, "grad_norm": 0.7606379389762878, "tokens_seen": 28352512, "tokens_per_sec": 18404.9161517329, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.8811140060424805, "grad_norm": 0.7646462917327881, "tokens_seen": 28516352, "tokens_per_sec": 18406.553573307647, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.909530401229858, "grad_norm": 0.8235291242599487, "tokens_seen": 28680192, "tokens_per_sec": 18408.337211453156, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 5.008232998847961, "val_ppl": 149.64008819305587, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.8629937171936035, "grad_norm": 0.7788772583007812, "tokens_seen": 28844032, "tokens_per_sec": 18372.887939019143, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.972301959991455, "grad_norm": 0.8082265853881836, "tokens_seen": 29007872, "tokens_per_sec": 18374.733686225278, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.949596405029297, "grad_norm": 0.7786532044410706, "tokens_seen": 29171712, "tokens_per_sec": 18376.61021610887, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.999843120574951, "grad_norm": 0.7442296743392944, "tokens_seen": 29335552, "tokens_per_sec": 18378.702658733808, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 5.071067810058594, "grad_norm": 0.8040263056755066, "tokens_seen": 29499392, "tokens_per_sec": 18380.575326225706, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.935228109359741, "grad_norm": 0.7418686747550964, "tokens_seen": 29663232, "tokens_per_sec": 18382.51136221324, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.7935919761657715, "grad_norm": 0.7183381915092468, "tokens_seen": 29827072, "tokens_per_sec": 18384.38694017108, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.749658823013306, "grad_norm": 0.8348175287246704, "tokens_seen": 29990912, "tokens_per_sec": 18386.468635053756, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.749053478240967, "grad_norm": 0.7269952893257141, "tokens_seen": 30154752, "tokens_per_sec": 18388.278352076497, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.964040756225586, "grad_norm": 0.7151952981948853, "tokens_seen": 30318592, "tokens_per_sec": 18390.020529685386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.914679527282715, "grad_norm": 0.7746522426605225, "tokens_seen": 30482432, "tokens_per_sec": 18391.796746782762, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.933616399765015, "grad_norm": 0.6956624388694763, "tokens_seen": 30646272, "tokens_per_sec": 18393.39267438475, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.890732049942017, "grad_norm": 0.758348822593689, "tokens_seen": 30810112, "tokens_per_sec": 18395.113468984353, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.795104026794434, "grad_norm": 0.7714309692382812, "tokens_seen": 30973952, "tokens_per_sec": 18396.88845737766, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.797221660614014, "grad_norm": 0.7681964635848999, "tokens_seen": 31137792, "tokens_per_sec": 18398.67180852437, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.867315053939819, "grad_norm": 0.6974779963493347, "tokens_seen": 31301632, "tokens_per_sec": 18400.42165703826, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.943953514099121, "grad_norm": 0.7060205340385437, "tokens_seen": 31465472, "tokens_per_sec": 18402.067724324832, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.835214138031006, "grad_norm": 0.8060503602027893, "tokens_seen": 31629312, "tokens_per_sec": 18403.978987131995, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.693636894226074, "grad_norm": 0.6705571413040161, "tokens_seen": 31793152, "tokens_per_sec": 18405.676423239045, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.808753728866577, "grad_norm": 0.7899183630943298, "tokens_seen": 31956992, "tokens_per_sec": 18407.16671317489, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.917898178100586, "grad_norm": 0.864263117313385, "tokens_seen": 32120832, "tokens_per_sec": 18408.93286339114, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.751324653625488, "grad_norm": 0.7781925797462463, "tokens_seen": 32284672, "tokens_per_sec": 18410.54490430156, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.985816240310669, "grad_norm": 0.7019535303115845, "tokens_seen": 32448512, "tokens_per_sec": 18412.21974618481, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 5.040502309799194, "grad_norm": 0.7753507494926453, "tokens_seen": 32612352, "tokens_per_sec": 18413.777362523626, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.949225187301636, "grad_norm": 0.7835791110992432, "tokens_seen": 32776192, "tokens_per_sec": 18415.36926700145, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.880603563785553, "val_ppl": 131.71013536826186, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.585585355758667, "grad_norm": 0.7253314256668091, "tokens_seen": 32940032, "tokens_per_sec": 18384.4849541762, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.874008417129517, "grad_norm": 0.8101338148117065, "tokens_seen": 33103872, "tokens_per_sec": 18386.215553637245, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.924072742462158, "grad_norm": 0.7533227801322937, "tokens_seen": 33267712, "tokens_per_sec": 18387.705520777967, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.7094011306762695, "grad_norm": 0.7575594186782837, "tokens_seen": 33431552, "tokens_per_sec": 18389.364843173502, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.852265119552612, "grad_norm": 0.840298593044281, "tokens_seen": 33595392, "tokens_per_sec": 18391.11253758893, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.848249435424805, "grad_norm": 0.7941919565200806, "tokens_seen": 33759232, "tokens_per_sec": 18392.633915744802, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.898233413696289, "grad_norm": 0.7355437874794006, "tokens_seen": 33923072, "tokens_per_sec": 18394.204763556525, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.735962867736816, "grad_norm": 0.7535868883132935, "tokens_seen": 34086912, "tokens_per_sec": 18395.864793478268, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 5.009842395782471, "grad_norm": 0.837282121181488, "tokens_seen": 34250752, "tokens_per_sec": 18397.608190195748, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.687723159790039, "grad_norm": 0.7706795930862427, "tokens_seen": 34414592, "tokens_per_sec": 18398.988329604806, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.758594751358032, "grad_norm": 0.8233644366264343, "tokens_seen": 34578432, "tokens_per_sec": 18400.40149939274, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.716007709503174, "grad_norm": 0.7540460824966431, "tokens_seen": 34742272, "tokens_per_sec": 18401.856004168545, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.769834995269775, "grad_norm": 0.7533568739891052, "tokens_seen": 34906112, "tokens_per_sec": 18403.387099022297, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.889520645141602, "grad_norm": 0.8348531126976013, "tokens_seen": 35069952, "tokens_per_sec": 18405.0566143722, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.92801308631897, "grad_norm": 0.7431093454360962, "tokens_seen": 35233792, "tokens_per_sec": 18406.534529410095, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.724565267562866, "grad_norm": 0.7806497812271118, "tokens_seen": 35397632, "tokens_per_sec": 18408.229819399545, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.77405047416687, "grad_norm": 0.7937305569648743, "tokens_seen": 35561472, "tokens_per_sec": 18410.153088923787, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.867125749588013, "grad_norm": 0.8035933375358582, "tokens_seen": 35725312, "tokens_per_sec": 18411.690914244566, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.739198207855225, "grad_norm": 0.7739276885986328, "tokens_seen": 35889152, "tokens_per_sec": 18413.059421824222, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.835389137268066, "grad_norm": 0.7729043960571289, "tokens_seen": 36052992, "tokens_per_sec": 18414.67507718989, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.816931486129761, "grad_norm": 0.8517364263534546, "tokens_seen": 36216832, "tokens_per_sec": 18416.12059673384, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.730810165405273, "grad_norm": 0.8050373196601868, "tokens_seen": 36380672, "tokens_per_sec": 18417.708927279375, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.545767784118652, "grad_norm": 0.8044047355651855, "tokens_seen": 36544512, "tokens_per_sec": 18419.0331180866, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.696388006210327, "grad_norm": 0.7882777452468872, "tokens_seen": 36708352, "tokens_per_sec": 18420.38959577544, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.973212242126465, "grad_norm": 0.7570145130157471, "tokens_seen": 36872192, "tokens_per_sec": 18421.84707744352, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.7787797212600704, "val_ppl": 118.9590981805263, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.8236329555511475, "grad_norm": 0.7827323079109192, "tokens_seen": 37036032, "tokens_per_sec": 18394.4761160146, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.8043084144592285, "grad_norm": 0.7521243095397949, "tokens_seen": 37199872, "tokens_per_sec": 18395.823519685375, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.676626443862915, "grad_norm": 0.8124701976776123, "tokens_seen": 37363712, "tokens_per_sec": 18397.38956625444, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.849327325820923, "grad_norm": 0.8126922249794006, "tokens_seen": 37527552, "tokens_per_sec": 18398.9192626111, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.679932594299316, "grad_norm": 0.7786033749580383, "tokens_seen": 37691392, "tokens_per_sec": 18400.512859909828, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.681302547454834, "grad_norm": 0.8242138028144836, "tokens_seen": 37855232, "tokens_per_sec": 18402.00851551015, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.843323469161987, "grad_norm": 0.8279966711997986, "tokens_seen": 38019072, "tokens_per_sec": 18403.581974467455, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.725597381591797, "grad_norm": 0.8106165528297424, "tokens_seen": 38182912, "tokens_per_sec": 18405.221585654115, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.823702335357666, "grad_norm": 0.7830932140350342, "tokens_seen": 38346752, "tokens_per_sec": 18406.76331849758, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.789394378662109, "grad_norm": 0.8176937699317932, "tokens_seen": 38510592, "tokens_per_sec": 18408.19375879984, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.799789905548096, "grad_norm": 0.8631911873817444, "tokens_seen": 38674432, "tokens_per_sec": 18409.463297918614, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.75557279586792, "grad_norm": 0.847510814666748, "tokens_seen": 38838272, "tokens_per_sec": 18411.018489250902, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.826492786407471, "grad_norm": 0.7846702337265015, "tokens_seen": 39002112, "tokens_per_sec": 18412.58560433979, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.633442163467407, "grad_norm": 0.8847705721855164, "tokens_seen": 39165952, "tokens_per_sec": 18414.257417715573, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.604918718338013, "grad_norm": 0.8196356892585754, "tokens_seen": 39329792, "tokens_per_sec": 18415.693537165374, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.768158912658691, "grad_norm": 0.757093071937561, "tokens_seen": 39493632, "tokens_per_sec": 18417.03337621623, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.766262531280518, "grad_norm": 0.8420349955558777, "tokens_seen": 39657472, "tokens_per_sec": 18418.41391130412, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.8484625816345215, "grad_norm": 0.7339353561401367, "tokens_seen": 39821312, "tokens_per_sec": 18419.82357940678, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.618065595626831, "grad_norm": 0.8402555584907532, "tokens_seen": 39985152, "tokens_per_sec": 18421.086593567627, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.742518663406372, "grad_norm": 0.790288507938385, "tokens_seen": 40148992, "tokens_per_sec": 18422.4351105433, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.786834001541138, "grad_norm": 0.843224823474884, "tokens_seen": 40312832, "tokens_per_sec": 18423.468759257164, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.537976264953613, "grad_norm": 0.8019604682922363, "tokens_seen": 40476672, "tokens_per_sec": 18424.577945033707, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.7404375076293945, "grad_norm": 0.8436595797538757, "tokens_seen": 40640512, "tokens_per_sec": 18425.87484046934, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.839502573013306, "grad_norm": 0.7803347706794739, "tokens_seen": 40804352, "tokens_per_sec": 18427.169417884434, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.7403693199157715, "grad_norm": 0.891021192073822, "tokens_seen": 40968192, "tokens_per_sec": 18428.406168051897, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.753770291805267, "val_ppl": 116.02089356248884, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.689184904098511, "grad_norm": 0.8232014775276184, "tokens_seen": 41132032, "tokens_per_sec": 18403.668112263524, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.62475061416626, "grad_norm": 0.8230831027030945, "tokens_seen": 41295872, "tokens_per_sec": 18404.957303675834, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.763581991195679, "grad_norm": 0.8229972720146179, "tokens_seen": 41459712, "tokens_per_sec": 18406.15078531061, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.7832348346710205, "grad_norm": 0.8392459750175476, "tokens_seen": 41623552, "tokens_per_sec": 18407.14870461612, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.644877910614014, "grad_norm": 0.8448570966720581, "tokens_seen": 41787392, "tokens_per_sec": 18408.39677992496, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.772445917129517, "grad_norm": 0.783693790435791, "tokens_seen": 41951232, "tokens_per_sec": 18409.77366158147, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.7211384773254395, "grad_norm": 0.7809682488441467, "tokens_seen": 42115072, "tokens_per_sec": 18411.0563756664, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.748126745223999, "grad_norm": 0.8985596895217896, "tokens_seen": 42278912, "tokens_per_sec": 18412.181860374654, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.799334287643433, "grad_norm": 0.9654952883720398, "tokens_seen": 42442752, "tokens_per_sec": 18413.25604162471, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.7622621059417725, "grad_norm": 0.8738468885421753, "tokens_seen": 42606592, "tokens_per_sec": 18414.564592798735, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.570423126220703, "grad_norm": 0.8961275219917297, "tokens_seen": 42770432, "tokens_per_sec": 18415.740478358366, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.88964581489563, "grad_norm": 0.9090605974197388, "tokens_seen": 42934272, "tokens_per_sec": 18417.0112015509, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.652756214141846, "grad_norm": 0.8114741444587708, "tokens_seen": 43098112, "tokens_per_sec": 18418.284090573834, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.556955814361572, "grad_norm": 0.7975722551345825, "tokens_seen": 43261952, "tokens_per_sec": 18419.48571233427, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.671111345291138, "grad_norm": 0.8579062819480896, "tokens_seen": 43425792, "tokens_per_sec": 18420.734599728952, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.67230224609375, "grad_norm": 0.8669515252113342, "tokens_seen": 43589632, "tokens_per_sec": 18421.88253418481, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.74619722366333, "grad_norm": 0.8562427759170532, "tokens_seen": 43753472, "tokens_per_sec": 18423.140208357636, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.503636121749878, "grad_norm": 0.8045363426208496, "tokens_seen": 43917312, "tokens_per_sec": 18424.217474616886, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.544143199920654, "grad_norm": 0.9054698348045349, "tokens_seen": 44081152, "tokens_per_sec": 18425.618854369382, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.563178777694702, "grad_norm": 0.8149958848953247, "tokens_seen": 44244992, "tokens_per_sec": 18426.629595728635, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.642456531524658, "grad_norm": 0.8221347332000732, "tokens_seen": 44408832, "tokens_per_sec": 18427.790582314898, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.796669960021973, "grad_norm": 0.9078685641288757, "tokens_seen": 44572672, "tokens_per_sec": 18428.967905016456, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.66872501373291, "grad_norm": 0.8423323035240173, "tokens_seen": 44736512, "tokens_per_sec": 18429.384323124745, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.686805963516235, "grad_norm": 0.8692070245742798, "tokens_seen": 44900352, "tokens_per_sec": 18430.423980933203, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.662116527557373, "grad_norm": 0.8604173064231873, "tokens_seen": 45064192, "tokens_per_sec": 18431.342561313013, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.694090926647187, "val_ppl": 109.29940229072533, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.493906736373901, "grad_norm": 0.7823935747146606, "tokens_seen": 45228032, "tokens_per_sec": 18408.819246816416, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.633123397827148, "grad_norm": 0.895406186580658, "tokens_seen": 45391872, "tokens_per_sec": 18409.650502966688, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.65361475944519, "grad_norm": 0.8193708062171936, "tokens_seen": 45555712, "tokens_per_sec": 18410.697694539555, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.501162528991699, "grad_norm": 0.8036696910858154, "tokens_seen": 45719552, "tokens_per_sec": 18411.795681625976, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.721282482147217, "grad_norm": 0.8874931931495667, "tokens_seen": 45883392, "tokens_per_sec": 18412.765363311675, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.69982385635376, "grad_norm": 0.8523518443107605, "tokens_seen": 46047232, "tokens_per_sec": 18413.940460540663, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.648630619049072, "grad_norm": 0.820733368396759, "tokens_seen": 46211072, "tokens_per_sec": 18415.16062682498, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.755548477172852, "grad_norm": 0.8695284128189087, "tokens_seen": 46374912, "tokens_per_sec": 18416.10719063355, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.776244401931763, "grad_norm": 0.9893088340759277, "tokens_seen": 46538752, "tokens_per_sec": 18417.30087250464, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.622916221618652, "grad_norm": 0.8059424161911011, "tokens_seen": 46702592, "tokens_per_sec": 18418.355350902475, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.5098936557769775, "grad_norm": 0.8019989728927612, "tokens_seen": 46866432, "tokens_per_sec": 18419.28822371563, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.6483869552612305, "grad_norm": 0.8717887997627258, "tokens_seen": 47030272, "tokens_per_sec": 18420.287144373957, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.684269905090332, "grad_norm": 0.8067510724067688, "tokens_seen": 47194112, "tokens_per_sec": 18421.371961112472, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.624741554260254, "grad_norm": 0.798240065574646, "tokens_seen": 47357952, "tokens_per_sec": 18422.524274771826, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.59620475769043, "grad_norm": 0.8836538195610046, "tokens_seen": 47521792, "tokens_per_sec": 18423.58416335333, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.5805418491363525, "grad_norm": 0.7827699184417725, "tokens_seen": 47685632, "tokens_per_sec": 18424.70589721835, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.56845235824585, "grad_norm": 0.814288854598999, "tokens_seen": 47849472, "tokens_per_sec": 18425.766042443967, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.571362495422363, "grad_norm": 0.8334901332855225, "tokens_seen": 48013312, "tokens_per_sec": 18426.754862270718, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.549460411071777, "grad_norm": 0.8802002668380737, "tokens_seen": 48177152, "tokens_per_sec": 18427.78138538144, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.532215595245361, "grad_norm": 0.847817063331604, "tokens_seen": 48340992, "tokens_per_sec": 18428.825860365956, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.64236044883728, "grad_norm": 0.8718186020851135, "tokens_seen": 48504832, "tokens_per_sec": 18429.80639323351, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.604209899902344, "grad_norm": 0.8700405359268188, "tokens_seen": 48668672, "tokens_per_sec": 18430.89764883665, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.756610631942749, "grad_norm": 0.923457682132721, "tokens_seen": 48832512, "tokens_per_sec": 18431.970475188937, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.536556720733643, "grad_norm": 0.7889070510864258, "tokens_seen": 48996352, "tokens_per_sec": 18432.96624190814, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.581989288330078, "grad_norm": 0.8931621313095093, "tokens_seen": 49160192, "tokens_per_sec": 18434.048847097394, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.705582880973816, "val_ppl": 110.56271106996358, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.640796661376953, "grad_norm": 0.8820377588272095, "tokens_seen": 49324032, "tokens_per_sec": 18413.198865037823, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.528033494949341, "grad_norm": 0.8387547731399536, "tokens_seen": 49487872, "tokens_per_sec": 18414.2293448365, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.4497599601745605, "grad_norm": 0.8063119053840637, "tokens_seen": 49651712, "tokens_per_sec": 18415.2370365065, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.474146842956543, "grad_norm": 0.8878888487815857, "tokens_seen": 49815552, "tokens_per_sec": 18416.248160883548, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.4737279415130615, "grad_norm": 0.8675033450126648, "tokens_seen": 49979392, "tokens_per_sec": 18417.3179989266, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.578215599060059, "grad_norm": 0.7915452718734741, "tokens_seen": 50143232, "tokens_per_sec": 18418.294101829164, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.505557298660278, "grad_norm": 0.8760988712310791, "tokens_seen": 50307072, "tokens_per_sec": 18419.334686916955, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.503898859024048, "grad_norm": 0.8606467843055725, "tokens_seen": 50470912, "tokens_per_sec": 18420.404451680966, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.628217935562134, "grad_norm": 0.9184629321098328, "tokens_seen": 50634752, "tokens_per_sec": 18421.309953212953, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.555878162384033, "grad_norm": 0.900288462638855, "tokens_seen": 50798592, "tokens_per_sec": 18422.175680526834, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.591746807098389, "grad_norm": 0.8824641108512878, "tokens_seen": 50962432, "tokens_per_sec": 18423.143255593877, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.538883686065674, "grad_norm": 0.9373509883880615, "tokens_seen": 51126272, "tokens_per_sec": 18423.945473290412, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.5627758502960205, "grad_norm": 1.0183614492416382, "tokens_seen": 51290112, "tokens_per_sec": 18424.619874667602, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.616122722625732, "grad_norm": 0.8518578410148621, "tokens_seen": 51453952, "tokens_per_sec": 18425.42064203097, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.4911394119262695, "grad_norm": 0.8738527297973633, "tokens_seen": 51617792, "tokens_per_sec": 18426.383745074494, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.633519172668457, "grad_norm": 0.8803289532661438, "tokens_seen": 51781632, "tokens_per_sec": 18427.385224628913, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.483866453170776, "grad_norm": 0.8810064792633057, "tokens_seen": 51945472, "tokens_per_sec": 18428.172336722004, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.550987243652344, "grad_norm": 0.9633132219314575, "tokens_seen": 52109312, "tokens_per_sec": 18429.16283420441, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.599100589752197, "grad_norm": 0.9412763118743896, "tokens_seen": 52273152, "tokens_per_sec": 18430.159970644712, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.530834197998047, "grad_norm": 0.9893215298652649, "tokens_seen": 52436992, "tokens_per_sec": 18430.993981870488, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.599327564239502, "grad_norm": 0.9306963086128235, "tokens_seen": 52600832, "tokens_per_sec": 18431.761394790763, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.549819707870483, "grad_norm": 0.8385119438171387, "tokens_seen": 52764672, "tokens_per_sec": 18432.71264111658, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.63485050201416, "grad_norm": 0.8813527226448059, "tokens_seen": 52928512, "tokens_per_sec": 18433.61098407244, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.4166412353515625, "grad_norm": 0.8698263764381409, "tokens_seen": 53092352, "tokens_per_sec": 18434.47727779397, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.53356409072876, "grad_norm": 0.8961759805679321, "tokens_seen": 53256192, "tokens_per_sec": 18435.378045405356, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.643070948123932, "val_ppl": 103.86281565241346, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.4874818325042725, "grad_norm": 0.8104704022407532, "tokens_seen": 53420032, "tokens_per_sec": 18416.151252213524, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.641347408294678, "grad_norm": 0.976381242275238, "tokens_seen": 53583872, "tokens_per_sec": 18417.085595473643, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.624234914779663, "grad_norm": 0.9155158400535583, "tokens_seen": 53747712, "tokens_per_sec": 18417.921968931103, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.535748243331909, "grad_norm": 0.8936362862586975, "tokens_seen": 53911552, "tokens_per_sec": 18418.783026627774, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.3416736125946045, "grad_norm": 0.8972247838973999, "tokens_seen": 54075392, "tokens_per_sec": 18419.741214158494, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.760455369949341, "grad_norm": 0.9396589398384094, "tokens_seen": 54239232, "tokens_per_sec": 18420.8132559378, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.5656418800354, "grad_norm": 0.9407448768615723, "tokens_seen": 54403072, "tokens_per_sec": 18421.725793480236, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.515501976013184, "grad_norm": 0.903817355632782, "tokens_seen": 54566912, "tokens_per_sec": 18422.677127048784, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.504255294799805, "grad_norm": 0.8363887071609497, "tokens_seen": 54730752, "tokens_per_sec": 18423.580337017673, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.5149266719818115, "grad_norm": 0.9185793399810791, "tokens_seen": 54894592, "tokens_per_sec": 18424.351342317666, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.52471399307251, "grad_norm": 0.9138901829719543, "tokens_seen": 55058432, "tokens_per_sec": 18425.282587931713, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.508490800857544, "grad_norm": 0.8962960839271545, "tokens_seen": 55222272, "tokens_per_sec": 18426.18925957908, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.57140588760376, "grad_norm": 0.8465431928634644, "tokens_seen": 55386112, "tokens_per_sec": 18427.07921204638, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.565364360809326, "grad_norm": 0.8760858774185181, "tokens_seen": 55549952, "tokens_per_sec": 18427.976792603764, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.52027153968811, "grad_norm": 1.034516453742981, "tokens_seen": 55713792, "tokens_per_sec": 18428.887490240926, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.679469347000122, "grad_norm": 1.0469037294387817, "tokens_seen": 55877632, "tokens_per_sec": 18429.833692423195, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.574259996414185, "grad_norm": 0.9010772705078125, "tokens_seen": 56041472, "tokens_per_sec": 18430.633055759186, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.48986554145813, "grad_norm": 0.8773576617240906, "tokens_seen": 56205312, "tokens_per_sec": 18431.429288734013, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.521826982498169, "grad_norm": 0.9444363117218018, "tokens_seen": 56369152, "tokens_per_sec": 18432.225451918155, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.505570411682129, "grad_norm": 0.9812315106391907, "tokens_seen": 56532992, "tokens_per_sec": 18433.132840127477, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.442934274673462, "grad_norm": 0.9642930030822754, "tokens_seen": 56696832, "tokens_per_sec": 18433.992520972424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.433999061584473, "grad_norm": 0.8804433941841125, "tokens_seen": 56860672, "tokens_per_sec": 18434.866059835873, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.507890224456787, "grad_norm": 0.8741114735603333, "tokens_seen": 57024512, "tokens_per_sec": 18435.707107675862, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.3641884326934814, "grad_norm": 0.8976360559463501, "tokens_seen": 57188352, "tokens_per_sec": 18436.464663418847, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.3672194480896, "grad_norm": 0.8986959457397461, "tokens_seen": 57352192, "tokens_per_sec": 18437.228341926173, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.581812167167664, "val_ppl": 97.6912668035634, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.537680625915527, "grad_norm": 0.9389032125473022, "tokens_seen": 57516032, "tokens_per_sec": 18419.387336524946, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.62532377243042, "grad_norm": 0.9173671007156372, "tokens_seen": 57679872, "tokens_per_sec": 18420.205043308062, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.532129764556885, "grad_norm": 0.9636743664741516, "tokens_seen": 57843712, "tokens_per_sec": 18420.975206412702, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.432359218597412, "grad_norm": 0.8740593194961548, "tokens_seen": 58007552, "tokens_per_sec": 18421.65746953056, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.565988063812256, "grad_norm": 0.936962902545929, "tokens_seen": 58171392, "tokens_per_sec": 18422.55843741578, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.496807098388672, "grad_norm": 1.1419473886489868, "tokens_seen": 58335232, "tokens_per_sec": 18423.33200875705, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.44028639793396, "grad_norm": 0.922793984413147, "tokens_seen": 58499072, "tokens_per_sec": 18424.278097362636, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.384308815002441, "grad_norm": 0.9677154421806335, "tokens_seen": 58662912, "tokens_per_sec": 18425.094431509195, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.595977306365967, "grad_norm": 0.9719270467758179, "tokens_seen": 58826752, "tokens_per_sec": 18425.870703793265, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.477461814880371, "grad_norm": 0.9124245047569275, "tokens_seen": 58990592, "tokens_per_sec": 18426.724149345664, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.567506313323975, "grad_norm": 0.9285746812820435, "tokens_seen": 59154432, "tokens_per_sec": 18427.567439694725, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.479340553283691, "grad_norm": 1.0544360876083374, "tokens_seen": 59318272, "tokens_per_sec": 18428.323699492576, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.523635387420654, "grad_norm": 0.9846721887588501, "tokens_seen": 59482112, "tokens_per_sec": 18429.11154182997, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.608293294906616, "grad_norm": 0.8793529868125916, "tokens_seen": 59645952, "tokens_per_sec": 18429.932166418945, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.431826829910278, "grad_norm": 0.943513810634613, "tokens_seen": 59809792, "tokens_per_sec": 18430.65935964995, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.495611667633057, "grad_norm": 0.8635718822479248, "tokens_seen": 59973632, "tokens_per_sec": 18431.582727965306, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.524722337722778, "grad_norm": 0.9736462831497192, "tokens_seen": 60137472, "tokens_per_sec": 18432.353967253242, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.534096717834473, "grad_norm": 0.8870933651924133, "tokens_seen": 60301312, "tokens_per_sec": 18433.13568929247, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.37765645980835, "grad_norm": 0.9300444722175598, "tokens_seen": 60465152, "tokens_per_sec": 18433.958886938854, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.379376411437988, "grad_norm": 0.8665372133255005, "tokens_seen": 60628992, "tokens_per_sec": 18434.806642784617, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.561888933181763, "grad_norm": 0.9357874989509583, "tokens_seen": 60792832, "tokens_per_sec": 18435.610513400134, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.597761869430542, "grad_norm": 0.9785711765289307, "tokens_seen": 60956672, "tokens_per_sec": 18436.346495758455, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.445774078369141, "grad_norm": 1.0312180519104004, "tokens_seen": 61120512, "tokens_per_sec": 18437.081233334295, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.688880205154419, "grad_norm": 1.0038031339645386, "tokens_seen": 61284352, "tokens_per_sec": 18437.84866218208, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.32108998298645, "grad_norm": 0.9057358503341675, "tokens_seen": 61448192, "tokens_per_sec": 18438.62947716582, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.560253763198853, "val_ppl": 95.60773847751126, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.550484657287598, "grad_norm": 0.9005586504936218, "tokens_seen": 61612032, "tokens_per_sec": 18421.864606484196, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.422646522521973, "grad_norm": 0.9395201206207275, "tokens_seen": 61775872, "tokens_per_sec": 18422.652179064553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.432652473449707, "grad_norm": 0.9596764445304871, "tokens_seen": 61939712, "tokens_per_sec": 18423.420567023506, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.3822808265686035, "grad_norm": 0.8612409234046936, "tokens_seen": 62103552, "tokens_per_sec": 18424.18519755999, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.405117511749268, "grad_norm": 0.9674244523048401, "tokens_seen": 62267392, "tokens_per_sec": 18424.86324029415, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.38833475112915, "grad_norm": 0.9208038449287415, "tokens_seen": 62431232, "tokens_per_sec": 18425.539126000163, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.434474229812622, "grad_norm": 0.9791646003723145, "tokens_seen": 62595072, "tokens_per_sec": 18426.310206180704, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.50715970993042, "grad_norm": 1.0352091789245605, "tokens_seen": 62758912, "tokens_per_sec": 18427.02986286867, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.399357318878174, "grad_norm": 1.051440954208374, "tokens_seen": 62922752, "tokens_per_sec": 18427.8091912985, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.4943623542785645, "grad_norm": 0.9584020972251892, "tokens_seen": 63086592, "tokens_per_sec": 18428.541333012796, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.545749664306641, "grad_norm": 1.2217762470245361, "tokens_seen": 63250432, "tokens_per_sec": 18429.273262698363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.369208574295044, "grad_norm": 0.9213349223136902, "tokens_seen": 63414272, "tokens_per_sec": 18429.94663085697, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.595259428024292, "grad_norm": 1.020620346069336, "tokens_seen": 63578112, "tokens_per_sec": 18430.58637099028, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.455068826675415, "grad_norm": 0.934378445148468, "tokens_seen": 63741952, "tokens_per_sec": 18431.29868595743, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.501249313354492, "grad_norm": 1.0529571771621704, "tokens_seen": 63905792, "tokens_per_sec": 18432.02937766568, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.359876871109009, "grad_norm": 1.0149513483047485, "tokens_seen": 64069632, "tokens_per_sec": 18432.792541515744, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.512950420379639, "grad_norm": 0.985651433467865, "tokens_seen": 64233472, "tokens_per_sec": 18433.45051485939, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.457021951675415, "grad_norm": 1.06819748878479, "tokens_seen": 64397312, "tokens_per_sec": 18433.973010084854, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.409718036651611, "grad_norm": 1.078031063079834, "tokens_seen": 64561152, "tokens_per_sec": 18434.642376543277, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.581737041473389, "grad_norm": 1.013733983039856, "tokens_seen": 64724992, "tokens_per_sec": 18435.264136727204, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.4541916847229, "grad_norm": 0.9898834228515625, "tokens_seen": 64888832, "tokens_per_sec": 18435.98482870768, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.4669036865234375, "grad_norm": 0.9708373546600342, "tokens_seen": 65052672, "tokens_per_sec": 18436.634635876177, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.537695407867432, "grad_norm": 1.0239157676696777, "tokens_seen": 65216512, "tokens_per_sec": 18437.252387327233, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.2623450756073, "grad_norm": 0.964472234249115, "tokens_seen": 65380352, "tokens_per_sec": 18437.89020034322, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.362938165664673, "grad_norm": 1.0608606338500977, "tokens_seen": 65544192, "tokens_per_sec": 18438.82285320203, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.550552105903625, "val_ppl": 94.68466985185302, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.63040018081665, "grad_norm": 0.9648984670639038, "tokens_seen": 65708032, "tokens_per_sec": 18423.39722418724, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.421513557434082, "grad_norm": 0.9887478351593018, "tokens_seen": 65871872, "tokens_per_sec": 18424.101262763026, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.314100742340088, "grad_norm": 0.9030269384384155, "tokens_seen": 66035712, "tokens_per_sec": 18424.619427276775, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.404408931732178, "grad_norm": 0.9098779559135437, "tokens_seen": 66199552, "tokens_per_sec": 18425.169592465332, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.399646997451782, "grad_norm": 0.951650857925415, "tokens_seen": 66363392, "tokens_per_sec": 18425.78050186115, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.441912651062012, "grad_norm": 0.8991469740867615, "tokens_seen": 66527232, "tokens_per_sec": 18426.545892299688, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.416296482086182, "grad_norm": 0.9451498985290527, "tokens_seen": 66691072, "tokens_per_sec": 18427.22934363726, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.519892454147339, "grad_norm": 1.1868444681167603, "tokens_seen": 66854912, "tokens_per_sec": 18427.792952861662, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.443144798278809, "grad_norm": 0.9498934149742126, "tokens_seen": 67018752, "tokens_per_sec": 18428.42299232298, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.501646041870117, "grad_norm": 1.111618995666504, "tokens_seen": 67182592, "tokens_per_sec": 18428.783476241788, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.393707275390625, "grad_norm": 1.0536807775497437, "tokens_seen": 67346432, "tokens_per_sec": 18429.474923760114, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.444730758666992, "grad_norm": 1.0304899215698242, "tokens_seen": 67510272, "tokens_per_sec": 18430.106835395512, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.34383487701416, "grad_norm": 0.9836363792419434, "tokens_seen": 67674112, "tokens_per_sec": 18430.74073272778, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.421511173248291, "grad_norm": 0.9392022490501404, "tokens_seen": 67837952, "tokens_per_sec": 18431.378327332975, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.592230796813965, "grad_norm": 1.108741044998169, "tokens_seen": 68001792, "tokens_per_sec": 18431.99679145276, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.2331132888793945, "grad_norm": 0.916892945766449, "tokens_seen": 68165632, "tokens_per_sec": 18432.5583723187, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.4176764488220215, "grad_norm": 1.06046462059021, "tokens_seen": 68329472, "tokens_per_sec": 18433.148143988845, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.414526462554932, "grad_norm": 0.9836248755455017, "tokens_seen": 68493312, "tokens_per_sec": 18433.7309597894, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.371909141540527, "grad_norm": 0.9757658243179321, "tokens_seen": 68657152, "tokens_per_sec": 18434.313596052016, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.5593907833099365, "grad_norm": 0.9495131373405457, "tokens_seen": 68820992, "tokens_per_sec": 18434.9471013756, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.376095294952393, "grad_norm": 0.9887415766716003, "tokens_seen": 68984832, "tokens_per_sec": 18435.503955530705, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.394641399383545, "grad_norm": 0.915250301361084, "tokens_seen": 69148672, "tokens_per_sec": 18435.96264764746, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.4169392585754395, "grad_norm": 0.9401070475578308, "tokens_seen": 69312512, "tokens_per_sec": 18436.62929807819, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.54074501991272, "grad_norm": 0.9756420254707336, "tokens_seen": 69476352, "tokens_per_sec": 18437.061336582366, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.497002601623535, "grad_norm": 1.1135746240615845, "tokens_seen": 69640192, "tokens_per_sec": 18437.584256130165, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.498786544799804, "val_ppl": 89.90796579156104, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.434922456741333, "grad_norm": 1.0586432218551636, "tokens_seen": 69804032, "tokens_per_sec": 18422.761935772745, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.376718997955322, "grad_norm": 1.0487321615219116, "tokens_seen": 69967872, "tokens_per_sec": 18423.39948455047, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.367098331451416, "grad_norm": 1.1831133365631104, "tokens_seen": 70131712, "tokens_per_sec": 18423.994651015848, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.494734287261963, "grad_norm": 1.043217658996582, "tokens_seen": 70295552, "tokens_per_sec": 18424.646591398177, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.546333312988281, "grad_norm": 1.0791162252426147, "tokens_seen": 70459392, "tokens_per_sec": 18425.226806295846, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.507094383239746, "grad_norm": 0.9592376351356506, "tokens_seen": 70623232, "tokens_per_sec": 18425.79972916602, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.4156882762908936, "grad_norm": 1.012393832206726, "tokens_seen": 70787072, "tokens_per_sec": 18426.390913629923, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.446887493133545, "grad_norm": 1.02488112449646, "tokens_seen": 70950912, "tokens_per_sec": 18427.007383109314, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.3778557777404785, "grad_norm": 1.0657484531402588, "tokens_seen": 71114752, "tokens_per_sec": 18427.689653632042, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.363209247589111, "grad_norm": 1.0398809909820557, "tokens_seen": 71278592, "tokens_per_sec": 18428.356556058603, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.4514501094818115, "grad_norm": 0.9363350868225098, "tokens_seen": 71442432, "tokens_per_sec": 18429.00859776116, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.581944704055786, "grad_norm": 1.057296633720398, "tokens_seen": 71606272, "tokens_per_sec": 18429.503309117965, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.40882420539856, "grad_norm": 1.0694557428359985, "tokens_seen": 71770112, "tokens_per_sec": 18430.066840009844, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.1667633056640625, "grad_norm": 0.9832227826118469, "tokens_seen": 71933952, "tokens_per_sec": 18430.634434531155, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.418431997299194, "grad_norm": 1.0515304803848267, "tokens_seen": 72097792, "tokens_per_sec": 18431.215377638502, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.396117925643921, "grad_norm": 1.0576263666152954, "tokens_seen": 72261632, "tokens_per_sec": 18431.7504403605, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.247247695922852, "grad_norm": 1.0547566413879395, "tokens_seen": 72425472, "tokens_per_sec": 18432.348402547526, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.392363548278809, "grad_norm": 1.0736896991729736, "tokens_seen": 72589312, "tokens_per_sec": 18432.820471476414, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.506488084793091, "grad_norm": 0.9953778386116028, "tokens_seen": 72753152, "tokens_per_sec": 18433.254623368688, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.5683910846710205, "grad_norm": 1.0209622383117676, "tokens_seen": 72916992, "tokens_per_sec": 18433.780773662147, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.3841552734375, "grad_norm": 1.3720812797546387, "tokens_seen": 73080832, "tokens_per_sec": 18434.345994634346, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.402979850769043, "grad_norm": 1.0042974948883057, "tokens_seen": 73244672, "tokens_per_sec": 18434.87466609185, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.412542104721069, "grad_norm": 1.112115740776062, "tokens_seen": 73408512, "tokens_per_sec": 18435.433676552802, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.446013927459717, "grad_norm": 0.9967105984687805, "tokens_seen": 73572352, "tokens_per_sec": 18436.023109902457, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.431023597717285, "grad_norm": 1.0289982557296753, "tokens_seen": 73736192, "tokens_per_sec": 18436.691543790443, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.468240427970886, "val_ppl": 87.2031477200904, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.457679748535156, "grad_norm": 1.0063871145248413, "tokens_seen": 73900032, "tokens_per_sec": 18422.898143501126, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.4054296016693115, "grad_norm": 1.0911014080047607, "tokens_seen": 74063872, "tokens_per_sec": 18423.352902124032, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.44287633895874, "grad_norm": 0.9490364193916321, "tokens_seen": 74227712, "tokens_per_sec": 18423.95021546553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.454412221908569, "grad_norm": 1.1100270748138428, "tokens_seen": 74391552, "tokens_per_sec": 18424.502993580623, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.375012397766113, "grad_norm": 1.105296015739441, "tokens_seen": 74555392, "tokens_per_sec": 18425.169295287506, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.242608547210693, "grad_norm": 0.995853066444397, "tokens_seen": 74719232, "tokens_per_sec": 18425.75103120882, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.481356143951416, "grad_norm": 1.0394648313522339, "tokens_seen": 74883072, "tokens_per_sec": 18426.349635335373, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.537768840789795, "grad_norm": 1.0209605693817139, "tokens_seen": 75046912, "tokens_per_sec": 18426.930250393016, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.344168424606323, "grad_norm": 1.0286493301391602, "tokens_seen": 75210752, "tokens_per_sec": 18427.49706838343, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.456279039382935, "grad_norm": 1.0570851564407349, "tokens_seen": 75374592, "tokens_per_sec": 18427.890321131505, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.3606953620910645, "grad_norm": 0.9351013898849487, "tokens_seen": 75538432, "tokens_per_sec": 18428.44686074773, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.3787219524383545, "grad_norm": 1.1839936971664429, "tokens_seen": 75702272, "tokens_per_sec": 18429.020527399432, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.4730963706970215, "grad_norm": 0.9849770069122314, "tokens_seen": 75866112, "tokens_per_sec": 18429.638946594267, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.465344429016113, "grad_norm": 1.3061041831970215, "tokens_seen": 76029952, "tokens_per_sec": 18430.260193244874, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.5456764698028564, "grad_norm": 1.1325103044509888, "tokens_seen": 76193792, "tokens_per_sec": 18430.822884981593, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.30602240562439, "grad_norm": 1.1020435094833374, "tokens_seen": 76357632, "tokens_per_sec": 18431.46620579368, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.29430365562439, "grad_norm": 1.089595079421997, "tokens_seen": 76521472, "tokens_per_sec": 18432.072311281572, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.484872579574585, "grad_norm": 1.17606520652771, "tokens_seen": 76685312, "tokens_per_sec": 18432.576640535335, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.397442817687988, "grad_norm": 1.0211820602416992, "tokens_seen": 76849152, "tokens_per_sec": 18433.086939391538, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.3438286781311035, "grad_norm": 1.0792158842086792, "tokens_seen": 77012992, "tokens_per_sec": 18433.5562991382, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.245734930038452, "grad_norm": 1.052577018737793, "tokens_seen": 77176832, "tokens_per_sec": 18434.158010737432, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.340087652206421, "grad_norm": 1.0470874309539795, "tokens_seen": 77340672, "tokens_per_sec": 18434.759332414476, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.325073957443237, "grad_norm": 1.086035966873169, "tokens_seen": 77504512, "tokens_per_sec": 18435.31590086358, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.400963068008423, "grad_norm": 1.003617525100708, "tokens_seen": 77668352, "tokens_per_sec": 18435.88692408571, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.369438409805298, "grad_norm": 1.085357904434204, "tokens_seen": 77832192, "tokens_per_sec": 18436.493262766802, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.46136965751648, "val_ppl": 86.60604852392251, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.4205405712127686, "grad_norm": 1.0910136699676514, "tokens_seen": 77996032, "tokens_per_sec": 18423.301306769186, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.288652658462524, "grad_norm": 1.0002684593200684, "tokens_seen": 78159872, "tokens_per_sec": 18423.810671660693, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.413511276245117, "grad_norm": 1.1686103343963623, "tokens_seen": 78323712, "tokens_per_sec": 18424.331796397953, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.4614293575286865, "grad_norm": 1.2622629404067993, "tokens_seen": 78487552, "tokens_per_sec": 18425.01493358254, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.3535895347595215, "grad_norm": 1.0650379657745361, "tokens_seen": 78651392, "tokens_per_sec": 18425.59811181447, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.379322290420532, "grad_norm": 1.0923397541046143, "tokens_seen": 78815232, "tokens_per_sec": 18426.206252045635, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.297040700912476, "grad_norm": 1.0662877559661865, "tokens_seen": 78979072, "tokens_per_sec": 18426.783871950578, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.535320997238159, "grad_norm": 1.0928244590759277, "tokens_seen": 79142912, "tokens_per_sec": 18427.373102635454, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.336510896682739, "grad_norm": 0.9579651355743408, "tokens_seen": 79306752, "tokens_per_sec": 18427.869446857057, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.418427467346191, "grad_norm": 1.0762789249420166, "tokens_seen": 79470592, "tokens_per_sec": 18428.437639523236, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.507986545562744, "grad_norm": 1.087748646736145, "tokens_seen": 79634432, "tokens_per_sec": 18429.03127698051, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.241403341293335, "grad_norm": 1.0200115442276, "tokens_seen": 79798272, "tokens_per_sec": 18429.542124867578, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.458376407623291, "grad_norm": 1.0419362783432007, "tokens_seen": 79962112, "tokens_per_sec": 18430.1127349954, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.358199834823608, "grad_norm": 1.031014084815979, "tokens_seen": 80125952, "tokens_per_sec": 18430.550407418363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.458502769470215, "grad_norm": 1.0055639743804932, "tokens_seen": 80289792, "tokens_per_sec": 18431.121331354894, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.297142744064331, "grad_norm": 1.0346343517303467, "tokens_seen": 80453632, "tokens_per_sec": 18431.693960949106, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.498861312866211, "grad_norm": 1.0881925821304321, "tokens_seen": 80617472, "tokens_per_sec": 18432.254675669785, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.541049957275391, "grad_norm": 1.124534249305725, "tokens_seen": 80781312, "tokens_per_sec": 18432.800741240557, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.359213829040527, "grad_norm": 1.048356533050537, "tokens_seen": 80945152, "tokens_per_sec": 18433.298830828848, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.333211421966553, "grad_norm": 1.0020229816436768, "tokens_seen": 81108992, "tokens_per_sec": 18433.8604397579, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.315706014633179, "grad_norm": 1.1078985929489136, "tokens_seen": 81272832, "tokens_per_sec": 18434.34789796552, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.274598598480225, "grad_norm": 1.0382325649261475, "tokens_seen": 81436672, "tokens_per_sec": 18434.93430425869, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.3425610065460205, "grad_norm": 1.1569410562515259, "tokens_seen": 81600512, "tokens_per_sec": 18435.46112157988, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.41151237487793, "grad_norm": 0.9842793941497803, "tokens_seen": 81764352, "tokens_per_sec": 18436.065154223445, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.3414599895477295, "grad_norm": 1.1094610691070557, "tokens_seen": 81928192, "tokens_per_sec": 18436.451856988584, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.426782476902008, "val_ppl": 83.66179962024201, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.653463840484619, "grad_norm": 1.1557427644729614, "tokens_seen": 82092032, "tokens_per_sec": 18423.862997707947, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.251923084259033, "grad_norm": 1.1315499544143677, "tokens_seen": 82255872, "tokens_per_sec": 18424.32927721654, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.489863634109497, "grad_norm": 1.132185935974121, "tokens_seen": 82419712, "tokens_per_sec": 18424.86311913115, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.240051746368408, "grad_norm": 1.0840575695037842, "tokens_seen": 82583552, "tokens_per_sec": 18425.38923200798, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.311962842941284, "grad_norm": 1.0763393640518188, "tokens_seen": 82747392, "tokens_per_sec": 18425.97249440238, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.3431172370910645, "grad_norm": 1.0813225507736206, "tokens_seen": 82911232, "tokens_per_sec": 18426.485267594464, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.273225784301758, "grad_norm": 1.1790179014205933, "tokens_seen": 83075072, "tokens_per_sec": 18426.988349071766, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.382808685302734, "grad_norm": 1.0337700843811035, "tokens_seen": 83238912, "tokens_per_sec": 18427.560665189674, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.278599500656128, "grad_norm": 1.0264933109283447, "tokens_seen": 83402752, "tokens_per_sec": 18428.11509577281, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.475861549377441, "grad_norm": 1.126380443572998, "tokens_seen": 83566592, "tokens_per_sec": 18428.6096467602, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.401804685592651, "grad_norm": 1.076601505279541, "tokens_seen": 83730432, "tokens_per_sec": 18429.101119499075, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.337006092071533, "grad_norm": 1.1204980611801147, "tokens_seen": 83894272, "tokens_per_sec": 18429.590359869275, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.451336145401001, "grad_norm": 0.9906603097915649, "tokens_seen": 84058112, "tokens_per_sec": 18430.01605840198, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.354597091674805, "grad_norm": 1.1166490316390991, "tokens_seen": 84221952, "tokens_per_sec": 18430.51691247051, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.27429461479187, "grad_norm": 1.1255241632461548, "tokens_seen": 84385792, "tokens_per_sec": 18431.039600304088, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.315690040588379, "grad_norm": 1.063291311264038, "tokens_seen": 84549632, "tokens_per_sec": 18431.494612639537, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.473184823989868, "grad_norm": 1.1258984804153442, "tokens_seen": 84713472, "tokens_per_sec": 18431.973569743554, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.35502552986145, "grad_norm": 1.0038120746612549, "tokens_seen": 84877312, "tokens_per_sec": 18432.424905107357, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.368607044219971, "grad_norm": 1.0954557657241821, "tokens_seen": 85041152, "tokens_per_sec": 18432.906791744663, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.466428279876709, "grad_norm": 1.211106538772583, "tokens_seen": 85204992, "tokens_per_sec": 18433.289189210598, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.414173126220703, "grad_norm": 1.2740131616592407, "tokens_seen": 85368832, "tokens_per_sec": 18433.82423391363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.436994314193726, "grad_norm": 1.0677627325057983, "tokens_seen": 85532672, "tokens_per_sec": 18434.33100959546, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.253016710281372, "grad_norm": 1.1313732862472534, "tokens_seen": 85696512, "tokens_per_sec": 18434.79449418034, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.3505027294158936, "grad_norm": 1.1639642715454102, "tokens_seen": 85860352, "tokens_per_sec": 18435.2737712978, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.524708986282349, "grad_norm": 1.553203821182251, "tokens_seen": 86024192, "tokens_per_sec": 18435.772893309386, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.419735825061798, "val_ppl": 83.0743363016174, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.366194486618042, "grad_norm": 1.2611932754516602, "tokens_seen": 86188032, "tokens_per_sec": 18423.789010834196, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.458855390548706, "grad_norm": 1.0727870464324951, "tokens_seen": 86351872, "tokens_per_sec": 18424.233412300327, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.427987813949585, "grad_norm": 1.1260946989059448, "tokens_seen": 86515712, "tokens_per_sec": 18424.703854974417, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.3069634437561035, "grad_norm": 1.064809799194336, "tokens_seen": 86679552, "tokens_per_sec": 18425.02295580026, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.254749059677124, "grad_norm": 1.209596037864685, "tokens_seen": 86843392, "tokens_per_sec": 18425.479359182325, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.357758283615112, "grad_norm": 1.1082417964935303, "tokens_seen": 87007232, "tokens_per_sec": 18425.968373622363, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.4616217613220215, "grad_norm": 1.1264456510543823, "tokens_seen": 87171072, "tokens_per_sec": 18426.493550015395, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.272802352905273, "grad_norm": 1.0808922052383423, "tokens_seen": 87334912, "tokens_per_sec": 18426.92703947969, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.290154695510864, "grad_norm": 1.1549686193466187, "tokens_seen": 87498752, "tokens_per_sec": 18427.42446231731, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.340330123901367, "grad_norm": 1.0331494808197021, "tokens_seen": 87662592, "tokens_per_sec": 18427.888802028603, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.356918573379517, "grad_norm": 1.232961893081665, "tokens_seen": 87826432, "tokens_per_sec": 18428.37963563008, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.428253173828125, "grad_norm": 1.2047741413116455, "tokens_seen": 87990272, "tokens_per_sec": 18428.830703696043, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.316914081573486, "grad_norm": 1.1244949102401733, "tokens_seen": 88154112, "tokens_per_sec": 18429.244299148086, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.354756832122803, "grad_norm": 1.051016926765442, "tokens_seen": 88317952, "tokens_per_sec": 18429.786904918834, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.419533014297485, "grad_norm": 1.1279449462890625, "tokens_seen": 88481792, "tokens_per_sec": 18430.211549896838, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.503332138061523, "grad_norm": 1.1705259084701538, "tokens_seen": 88645632, "tokens_per_sec": 18430.684316195806, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.209577798843384, "grad_norm": 1.0451644659042358, "tokens_seen": 88809472, "tokens_per_sec": 18431.233239398785, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.396439075469971, "grad_norm": 1.0630947351455688, "tokens_seen": 88973312, "tokens_per_sec": 18431.715825570478, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.228466987609863, "grad_norm": 1.2111681699752808, "tokens_seen": 89137152, "tokens_per_sec": 18432.1693509424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.3472325801849365, "grad_norm": 1.0631523132324219, "tokens_seen": 89300992, "tokens_per_sec": 18432.47054079391, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.353399753570557, "grad_norm": 1.1462600231170654, "tokens_seen": 89464832, "tokens_per_sec": 18432.806599477466, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.300130844116211, "grad_norm": 1.1310851573944092, "tokens_seen": 89628672, "tokens_per_sec": 18433.240615265997, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.319273233413696, "grad_norm": 1.2214276790618896, "tokens_seen": 89792512, "tokens_per_sec": 18433.80133405445, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.511113882064819, "grad_norm": 1.1918303966522217, "tokens_seen": 89956352, "tokens_per_sec": 18434.378845714164, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.3060972690582275, "grad_norm": 1.0618619918823242, "tokens_seen": 90120192, "tokens_per_sec": 18434.884081313827, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.405412697792054, "val_ppl": 81.89293290274028, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.124361038208008, "grad_norm": 1.1023285388946533, "tokens_seen": 90284032, "tokens_per_sec": 18423.486378253543, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.327439785003662, "grad_norm": 1.2988940477371216, "tokens_seen": 90447872, "tokens_per_sec": 18423.927652926308, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.395828723907471, "grad_norm": 1.1864160299301147, "tokens_seen": 90611712, "tokens_per_sec": 18424.401903045557, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.395392417907715, "grad_norm": 1.1385681629180908, "tokens_seen": 90775552, "tokens_per_sec": 18424.850357001185, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.415147542953491, "grad_norm": 1.1372493505477905, "tokens_seen": 90939392, "tokens_per_sec": 18425.3618732891, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.397192716598511, "grad_norm": 1.0900081396102905, "tokens_seen": 91103232, "tokens_per_sec": 18425.642541595633, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.254412889480591, "grad_norm": 1.1076457500457764, "tokens_seen": 91267072, "tokens_per_sec": 18426.06671151078, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.288542032241821, "grad_norm": 1.1882699728012085, "tokens_seen": 91430912, "tokens_per_sec": 18426.402122827396, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.316462993621826, "grad_norm": 1.130786418914795, "tokens_seen": 91594752, "tokens_per_sec": 18426.90414208644, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.527584791183472, "grad_norm": 1.6722304821014404, "tokens_seen": 91758592, "tokens_per_sec": 18427.314215223832, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.370784521102905, "grad_norm": 1.2523579597473145, "tokens_seen": 91922432, "tokens_per_sec": 18427.831179911722, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.286251783370972, "grad_norm": 1.102992057800293, "tokens_seen": 92086272, "tokens_per_sec": 18428.312400616534, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.300320625305176, "grad_norm": 1.1658213138580322, "tokens_seen": 92250112, "tokens_per_sec": 18428.794194538594, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.190799713134766, "grad_norm": 1.2625094652175903, "tokens_seen": 92413952, "tokens_per_sec": 18429.25538209911, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.398316860198975, "grad_norm": 1.0783652067184448, "tokens_seen": 92577792, "tokens_per_sec": 18429.75131292484, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.214247941970825, "grad_norm": 1.1072310209274292, "tokens_seen": 92741632, "tokens_per_sec": 18430.25771617861, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.397917032241821, "grad_norm": 1.1087604761123657, "tokens_seen": 92905472, "tokens_per_sec": 18430.73736574155, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.289757966995239, "grad_norm": 1.12772536277771, "tokens_seen": 93069312, "tokens_per_sec": 18431.22275629877, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.340849161148071, "grad_norm": 1.2233415842056274, "tokens_seen": 93233152, "tokens_per_sec": 18431.660215284806, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.388938665390015, "grad_norm": 1.1994907855987549, "tokens_seen": 93396992, "tokens_per_sec": 18432.122810640667, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.09438681602478, "grad_norm": 1.1303315162658691, "tokens_seen": 93560832, "tokens_per_sec": 18432.601357024865, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.200803756713867, "grad_norm": 1.0537351369857788, "tokens_seen": 93724672, "tokens_per_sec": 18433.069816540465, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.328047752380371, "grad_norm": 1.2378721237182617, "tokens_seen": 93888512, "tokens_per_sec": 18433.525756388488, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.379939317703247, "grad_norm": 1.028870940208435, "tokens_seen": 94052352, "tokens_per_sec": 18433.796411866322, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.5152387619018555, "grad_norm": 1.3192704916000366, "tokens_seen": 94216192, "tokens_per_sec": 18434.061942684944, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.397926378250122, "val_ppl": 81.28214536676627, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.29772686958313, "grad_norm": 1.0613763332366943, "tokens_seen": 94380032, "tokens_per_sec": 18423.01714385783, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.200323581695557, "grad_norm": 1.0542608499526978, "tokens_seen": 94543872, "tokens_per_sec": 18423.45564982117, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.340201139450073, "grad_norm": 1.1385197639465332, "tokens_seen": 94707712, "tokens_per_sec": 18423.953073470493, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.3809239864349365, "grad_norm": 1.1533259153366089, "tokens_seen": 94871552, "tokens_per_sec": 18424.382054551843, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.187847137451172, "grad_norm": 1.205197811126709, "tokens_seen": 95035392, "tokens_per_sec": 18424.68204706331, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.437176465988159, "grad_norm": 1.231892466545105, "tokens_seen": 95199232, "tokens_per_sec": 18425.19448212108, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.380670547485352, "grad_norm": 1.1916946172714233, "tokens_seen": 95363072, "tokens_per_sec": 18425.70400745014, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.371730327606201, "grad_norm": 1.1669714450836182, "tokens_seen": 95526912, "tokens_per_sec": 18426.18979091855, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.289565086364746, "grad_norm": 1.1673372983932495, "tokens_seen": 95690752, "tokens_per_sec": 18426.63682042946, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.314181327819824, "grad_norm": 1.071200966835022, "tokens_seen": 95854592, "tokens_per_sec": 18427.06576596127, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.165143013000488, "grad_norm": 1.112831950187683, "tokens_seen": 96018432, "tokens_per_sec": 18427.581768483553, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.399565935134888, "grad_norm": 1.1564584970474243, "tokens_seen": 96182272, "tokens_per_sec": 18428.05294540504, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.278162240982056, "grad_norm": 1.164716124534607, "tokens_seen": 96346112, "tokens_per_sec": 18428.596907040424, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.266835451126099, "grad_norm": 1.1416853666305542, "tokens_seen": 96509952, "tokens_per_sec": 18429.151400067036, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.246927738189697, "grad_norm": 1.106881022453308, "tokens_seen": 96673792, "tokens_per_sec": 18429.62900781301, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.331687688827515, "grad_norm": 1.10366690158844, "tokens_seen": 96837632, "tokens_per_sec": 18430.123442298736, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.299016952514648, "grad_norm": 1.1289337873458862, "tokens_seen": 97001472, "tokens_per_sec": 18430.60308222165, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.310561418533325, "grad_norm": 1.1369224786758423, "tokens_seen": 97165312, "tokens_per_sec": 18431.065005314933, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.271977663040161, "grad_norm": 1.1172997951507568, "tokens_seen": 97329152, "tokens_per_sec": 18431.568663327682, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.410719394683838, "grad_norm": 1.2612080574035645, "tokens_seen": 97492992, "tokens_per_sec": 18431.986382169714, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.243111610412598, "grad_norm": 1.1136391162872314, "tokens_seen": 97656832, "tokens_per_sec": 18432.397373945158, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.3208394050598145, "grad_norm": 1.1956816911697388, "tokens_seen": 97820672, "tokens_per_sec": 18432.925753526513, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.227377891540527, "grad_norm": 1.1604074239730835, "tokens_seen": 97984512, "tokens_per_sec": 18433.39208778366, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.3223230838775635, "grad_norm": 1.1837916374206543, "tokens_seen": 98148352, "tokens_per_sec": 18433.872093363538, "vram_gb": 5.96580096, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.393609774112702, "val_ppl": 80.93203870134798, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5335.801450252533, "nan_failures": 0} diff --git a/baseline_s2/config.json b/baseline_s2/config.json new file mode 100644 index 0000000000000000000000000000000000000000..cf388673ced8f92ced25b4391b09a4454add3b02 --- /dev/null +++ b/baseline_s2/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 2, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s2", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 2001343, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/baseline_s2/final.pt b/baseline_s2/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..00cdbaa0545b7d484d22cd160017b48aff109da1 --- /dev/null +++ b/baseline_s2/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b59023116afb5ef8093dfc4e71d3ee798e4ee2a88a5db75fa8c695bae1a795de +size 253572039 diff --git a/baseline_s2/metrics.jsonl b/baseline_s2/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..3845a258a71479318d764c20f64d284a068ac4ad --- /dev/null +++ b/baseline_s2/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 2, "config": {"variant": "baseline", "seed": 2, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s2", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 2001343, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.874977111816406, "grad_norm": 14.162490844726562, "tokens_seen": 8192, "tokens_per_sec": 10576.685669959337, "vram_gb": 4.843584512, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.890010833740234, "grad_norm": 2.4616966247558594, "tokens_seen": 172032, "tokens_per_sec": 23283.6211220412, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.093172073364258, "grad_norm": 1.8079652786254883, "tokens_seen": 335872, "tokens_per_sec": 24010.349519933374, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.195852279663086, "grad_norm": 1.5270484685897827, "tokens_seen": 499712, "tokens_per_sec": 24275.87667265951, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.5602827072143555, "grad_norm": 0.811616063117981, "tokens_seen": 663552, "tokens_per_sec": 24383.292547963083, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.250953674316406, "grad_norm": 0.8797574043273926, "tokens_seen": 827392, "tokens_per_sec": 24477.48467848282, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.156132221221924, "grad_norm": 0.6613376140594482, "tokens_seen": 991232, "tokens_per_sec": 24534.315850368876, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 7.027589559555054, "grad_norm": 0.6040841937065125, "tokens_seen": 1155072, "tokens_per_sec": 24570.366193719, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.737114429473877, "grad_norm": 0.6133050322532654, "tokens_seen": 1318912, "tokens_per_sec": 24599.03993416822, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.889482498168945, "grad_norm": 0.9533765316009521, "tokens_seen": 1482752, "tokens_per_sec": 24619.85126192801, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.63266658782959, "grad_norm": 0.6073065400123596, "tokens_seen": 1646592, "tokens_per_sec": 24636.134187884476, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.637905120849609, "grad_norm": 0.6424582004547119, "tokens_seen": 1810432, "tokens_per_sec": 24648.209130413994, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.591235160827637, "grad_norm": 0.5358360409736633, "tokens_seen": 1974272, "tokens_per_sec": 24657.67243393739, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.416498899459839, "grad_norm": 0.5511308312416077, "tokens_seen": 2138112, "tokens_per_sec": 24665.367912781436, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.522326469421387, "grad_norm": 0.514664888381958, "tokens_seen": 2301952, "tokens_per_sec": 24668.302655057698, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.655339241027832, "grad_norm": 0.5024561882019043, "tokens_seen": 2465792, "tokens_per_sec": 24673.5550998131, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.608323335647583, "grad_norm": 0.5455635786056519, "tokens_seen": 2629632, "tokens_per_sec": 24678.09584416002, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.474260091781616, "grad_norm": 0.6570676565170288, "tokens_seen": 2793472, "tokens_per_sec": 24681.276828991562, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.404645204544067, "grad_norm": 0.519804835319519, "tokens_seen": 2957312, "tokens_per_sec": 24680.150862555252, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.280712604522705, "grad_norm": 0.5167493224143982, "tokens_seen": 3121152, "tokens_per_sec": 24684.558484101075, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.477793455123901, "grad_norm": 0.546838104724884, "tokens_seen": 3284992, "tokens_per_sec": 24686.31338609714, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.312530040740967, "grad_norm": 0.5195452570915222, "tokens_seen": 3448832, "tokens_per_sec": 24687.78223205816, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.315897226333618, "grad_norm": 0.5197346210479736, "tokens_seen": 3612672, "tokens_per_sec": 24690.706847018566, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.258857250213623, "grad_norm": 0.5051612854003906, "tokens_seen": 3776512, "tokens_per_sec": 24693.636997342397, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.502858638763428, "grad_norm": 0.4978979229927063, "tokens_seen": 3940352, "tokens_per_sec": 24695.379831980197, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.3362298011779785, "grad_norm": 0.5116076469421387, "tokens_seen": 4104192, "tokens_per_sec": 24697.69233244728, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.360678195953369, "grad_norm": 0.608462393283844, "tokens_seen": 4268032, "tokens_per_sec": 24699.011995059427, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.116555690765381, "grad_norm": 0.4948306977748871, "tokens_seen": 4431872, "tokens_per_sec": 24700.277699575752, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.078150272369385, "grad_norm": 0.5737776160240173, "tokens_seen": 4595712, "tokens_per_sec": 24702.18374949712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 5.928973436355591, "grad_norm": 0.5688516497612, "tokens_seen": 4759552, "tokens_per_sec": 24701.66006535864, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.0702972412109375, "grad_norm": 0.536319375038147, "tokens_seen": 4923392, "tokens_per_sec": 24702.517097859363, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.23584246635437, "grad_norm": 0.5653117299079895, "tokens_seen": 5087232, "tokens_per_sec": 24701.584700286716, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.143592357635498, "grad_norm": 0.5144650936126709, "tokens_seen": 5251072, "tokens_per_sec": 24701.499583951754, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.143932580947876, "grad_norm": 0.5244831442832947, "tokens_seen": 5414912, "tokens_per_sec": 24702.333585104374, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.095804452896118, "grad_norm": 0.4797462224960327, "tokens_seen": 5578752, "tokens_per_sec": 24702.550851429674, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.985368728637695, "grad_norm": 0.4971744418144226, "tokens_seen": 5742592, "tokens_per_sec": 24702.20654659178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.072377681732178, "grad_norm": 0.67547607421875, "tokens_seen": 5906432, "tokens_per_sec": 24702.591127779935, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.024243116378784, "grad_norm": 0.551299512386322, "tokens_seen": 6070272, "tokens_per_sec": 24702.091666736942, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.048907041549683, "grad_norm": 0.5160964131355286, "tokens_seen": 6234112, "tokens_per_sec": 24701.203113052663, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.815036058425903, "grad_norm": 0.5538373589515686, "tokens_seen": 6397952, "tokens_per_sec": 24700.912931793635, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.920495510101318, "grad_norm": 0.5552422404289246, "tokens_seen": 6561792, "tokens_per_sec": 24699.488447122876, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.874448537826538, "grad_norm": 0.5230735540390015, "tokens_seen": 6725632, "tokens_per_sec": 24699.24532013714, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.929415702819824, "grad_norm": 0.5186138153076172, "tokens_seen": 6889472, "tokens_per_sec": 24699.55238367275, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.906992435455322, "grad_norm": 0.5314434766769409, "tokens_seen": 7053312, "tokens_per_sec": 24698.434565091866, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.999906063079834, "grad_norm": 0.582237184047699, "tokens_seen": 7217152, "tokens_per_sec": 24699.015834092814, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.797523498535156, "grad_norm": 0.515197217464447, "tokens_seen": 7380992, "tokens_per_sec": 24697.815955219845, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.952146768569946, "grad_norm": 0.5145859718322754, "tokens_seen": 7544832, "tokens_per_sec": 24696.744872469546, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.859717845916748, "grad_norm": 0.525251030921936, "tokens_seen": 7708672, "tokens_per_sec": 24697.018945080683, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.756378412246704, "grad_norm": 0.5307116508483887, "tokens_seen": 7872512, "tokens_per_sec": 24696.30226231165, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.844521760940552, "grad_norm": 0.5299025774002075, "tokens_seen": 8036352, "tokens_per_sec": 24695.376806335684, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.749675750732422, "grad_norm": 0.515627384185791, "tokens_seen": 8200192, "tokens_per_sec": 24695.23800361781, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.753756403923035, "val_ppl": 315.3731067975142, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.703342437744141, "grad_norm": 0.6231306791305542, "tokens_seen": 8364032, "tokens_per_sec": 24610.74737105661, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.840808868408203, "grad_norm": 0.5276055335998535, "tokens_seen": 8527872, "tokens_per_sec": 24610.91931081718, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.742414712905884, "grad_norm": 1.167891502380371, "tokens_seen": 8691712, "tokens_per_sec": 24611.78706695492, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.793543577194214, "grad_norm": 0.5542780160903931, "tokens_seen": 8855552, "tokens_per_sec": 24611.974160420254, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.627833127975464, "grad_norm": 0.5340262651443481, "tokens_seen": 9019392, "tokens_per_sec": 24612.081153852247, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.711627721786499, "grad_norm": 0.5691092610359192, "tokens_seen": 9183232, "tokens_per_sec": 24613.029290459293, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.768693208694458, "grad_norm": 0.6275193691253662, "tokens_seen": 9347072, "tokens_per_sec": 24613.188697160618, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.692526817321777, "grad_norm": 0.556128978729248, "tokens_seen": 9510912, "tokens_per_sec": 24613.631951325984, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.5273706912994385, "grad_norm": 0.5076292753219604, "tokens_seen": 9674752, "tokens_per_sec": 24614.65357969051, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.572821378707886, "grad_norm": 0.5165883898735046, "tokens_seen": 9838592, "tokens_per_sec": 24615.083522816654, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.6382505893707275, "grad_norm": 0.5509685277938843, "tokens_seen": 10002432, "tokens_per_sec": 24614.982064590626, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.562063455581665, "grad_norm": 0.5729836821556091, "tokens_seen": 10166272, "tokens_per_sec": 24615.303443748493, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.508618116378784, "grad_norm": 0.6202326416969299, "tokens_seen": 10330112, "tokens_per_sec": 24615.959898204594, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.47882604598999, "grad_norm": 0.5907869338989258, "tokens_seen": 10493952, "tokens_per_sec": 24616.909048794492, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.616560935974121, "grad_norm": 0.5832404494285583, "tokens_seen": 10657792, "tokens_per_sec": 24616.763721261992, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.492021083831787, "grad_norm": 0.8590800762176514, "tokens_seen": 10821632, "tokens_per_sec": 24617.473572426006, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.678251504898071, "grad_norm": 0.5995529890060425, "tokens_seen": 10985472, "tokens_per_sec": 24618.64506164333, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.502358436584473, "grad_norm": 0.6038060784339905, "tokens_seen": 11149312, "tokens_per_sec": 24619.047116000187, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.506100177764893, "grad_norm": 0.57606440782547, "tokens_seen": 11313152, "tokens_per_sec": 24618.108479757062, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.423551559448242, "grad_norm": 0.5866327881813049, "tokens_seen": 11476992, "tokens_per_sec": 24618.160306369053, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.475375652313232, "grad_norm": 0.6042300462722778, "tokens_seen": 11640832, "tokens_per_sec": 24618.972023277463, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.515129089355469, "grad_norm": 0.548153281211853, "tokens_seen": 11804672, "tokens_per_sec": 24620.240348835516, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.44817590713501, "grad_norm": 0.5912075042724609, "tokens_seen": 11968512, "tokens_per_sec": 24620.898394754833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.3770012855529785, "grad_norm": 0.5926820039749146, "tokens_seen": 12132352, "tokens_per_sec": 24620.877040758296, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.639180898666382, "grad_norm": 0.6016820669174194, "tokens_seen": 12296192, "tokens_per_sec": 24620.783700287273, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.4307544231414795, "grad_norm": 0.5770927667617798, "tokens_seen": 12460032, "tokens_per_sec": 24621.40691828361, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.436166524887085, "grad_norm": 0.5773724317550659, "tokens_seen": 12623872, "tokens_per_sec": 24620.818457044315, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.625874996185303, "grad_norm": 0.5645002722740173, "tokens_seen": 12787712, "tokens_per_sec": 24620.48502156347, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.5142199993133545, "grad_norm": 0.5800223350524902, "tokens_seen": 12951552, "tokens_per_sec": 24620.994435983706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.42619514465332, "grad_norm": 0.585349440574646, "tokens_seen": 13115392, "tokens_per_sec": 24622.05806594702, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.392378807067871, "grad_norm": 0.5649492144584656, "tokens_seen": 13279232, "tokens_per_sec": 24622.64194350726, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.465084075927734, "grad_norm": 0.6216118931770325, "tokens_seen": 13443072, "tokens_per_sec": 24622.6297396579, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.183169364929199, "grad_norm": 0.5595034956932068, "tokens_seen": 13606912, "tokens_per_sec": 24621.14589316646, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.464940547943115, "grad_norm": 0.598300039768219, "tokens_seen": 13770752, "tokens_per_sec": 24621.461849233256, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.536661148071289, "grad_norm": 0.6346196532249451, "tokens_seen": 13934592, "tokens_per_sec": 24622.461637923985, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.341026544570923, "grad_norm": 0.6089397668838501, "tokens_seen": 14098432, "tokens_per_sec": 24622.26131891014, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.391558647155762, "grad_norm": 0.6031805276870728, "tokens_seen": 14262272, "tokens_per_sec": 24622.26493148574, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.256462812423706, "grad_norm": 0.5822959542274475, "tokens_seen": 14426112, "tokens_per_sec": 24621.937822282227, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.289092063903809, "grad_norm": 0.603330671787262, "tokens_seen": 14589952, "tokens_per_sec": 24622.440442485193, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.467414855957031, "grad_norm": 0.6448376178741455, "tokens_seen": 14753792, "tokens_per_sec": 24623.587547410876, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.502924919128418, "grad_norm": 0.619753897190094, "tokens_seen": 14917632, "tokens_per_sec": 24623.28617160315, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.446031093597412, "grad_norm": 0.6018189191818237, "tokens_seen": 15081472, "tokens_per_sec": 24624.30650757488, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.460203409194946, "grad_norm": 0.6511768698692322, "tokens_seen": 15245312, "tokens_per_sec": 24624.6298282717, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.1998982429504395, "grad_norm": 0.5997490286827087, "tokens_seen": 15409152, "tokens_per_sec": 24625.167350756692, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.262648105621338, "grad_norm": 0.6571085453033447, "tokens_seen": 15572992, "tokens_per_sec": 24624.725299153888, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.451582431793213, "grad_norm": 0.5957452654838562, "tokens_seen": 15736832, "tokens_per_sec": 24625.73021193307, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.573643684387207, "grad_norm": 0.6633651852607727, "tokens_seen": 15900672, "tokens_per_sec": 24626.389510964837, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.204231262207031, "grad_norm": 0.6156212687492371, "tokens_seen": 16064512, "tokens_per_sec": 24626.69487912781, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.185438632965088, "grad_norm": 0.6358019709587097, "tokens_seen": 16228352, "tokens_per_sec": 24627.03922796013, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.139295816421509, "grad_norm": 0.644256055355072, "tokens_seen": 16392192, "tokens_per_sec": 24626.726253430923, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.310031747817993, "val_ppl": 202.35665266834712, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.241345643997192, "grad_norm": 0.6134127974510193, "tokens_seen": 16556032, "tokens_per_sec": 24585.06655584008, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.321048021316528, "grad_norm": 0.6390129327774048, "tokens_seen": 16719872, "tokens_per_sec": 24585.285853922913, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.383711576461792, "grad_norm": 0.6168883442878723, "tokens_seen": 16883712, "tokens_per_sec": 24586.193329549264, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.396591424942017, "grad_norm": 0.6189345717430115, "tokens_seen": 17047552, "tokens_per_sec": 24586.65540455008, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.2224273681640625, "grad_norm": 0.6223505735397339, "tokens_seen": 17211392, "tokens_per_sec": 24587.31893319113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.3651206493377686, "grad_norm": 0.7262228727340698, "tokens_seen": 17375232, "tokens_per_sec": 24587.886818679704, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.148095607757568, "grad_norm": 0.5782901644706726, "tokens_seen": 17539072, "tokens_per_sec": 24589.096986590586, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.191933870315552, "grad_norm": 0.584703266620636, "tokens_seen": 17702912, "tokens_per_sec": 24589.12217385741, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.327138185501099, "grad_norm": 0.6357983946800232, "tokens_seen": 17866752, "tokens_per_sec": 24589.59524605466, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.095118045806885, "grad_norm": 0.6567632555961609, "tokens_seen": 18030592, "tokens_per_sec": 24590.53496427676, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.190495014190674, "grad_norm": 0.6443833708763123, "tokens_seen": 18194432, "tokens_per_sec": 24590.549305564167, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.2992682456970215, "grad_norm": 0.6294499635696411, "tokens_seen": 18358272, "tokens_per_sec": 24591.174257394985, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.110235691070557, "grad_norm": 0.666668713092804, "tokens_seen": 18522112, "tokens_per_sec": 24591.3964827604, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.144646883010864, "grad_norm": 0.6419041156768799, "tokens_seen": 18685952, "tokens_per_sec": 24591.455910311415, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.353468418121338, "grad_norm": 0.6295685768127441, "tokens_seen": 18849792, "tokens_per_sec": 24591.4139048393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.307669401168823, "grad_norm": 0.7147087454795837, "tokens_seen": 19013632, "tokens_per_sec": 24591.301555839033, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.138740062713623, "grad_norm": 0.6659388542175293, "tokens_seen": 19177472, "tokens_per_sec": 24592.2246159448, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.146076679229736, "grad_norm": 0.6242967247962952, "tokens_seen": 19341312, "tokens_per_sec": 24592.884104988538, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.140064001083374, "grad_norm": 0.7296732068061829, "tokens_seen": 19505152, "tokens_per_sec": 24593.605239502464, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.116482496261597, "grad_norm": 0.6414076089859009, "tokens_seen": 19668992, "tokens_per_sec": 24594.53908779057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.299939155578613, "grad_norm": 0.7074671983718872, "tokens_seen": 19832832, "tokens_per_sec": 24594.928346657878, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.130852222442627, "grad_norm": 0.6261174082756042, "tokens_seen": 19996672, "tokens_per_sec": 24594.272946714274, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.074695587158203, "grad_norm": 0.6291102766990662, "tokens_seen": 20160512, "tokens_per_sec": 24594.79757072381, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.063986301422119, "grad_norm": 0.6134341955184937, "tokens_seen": 20324352, "tokens_per_sec": 24595.459906182976, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.182329177856445, "grad_norm": 0.6169524788856506, "tokens_seen": 20488192, "tokens_per_sec": 24595.88839937161, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.0018627643585205, "grad_norm": 0.6388098001480103, "tokens_seen": 20652032, "tokens_per_sec": 24596.332206412964, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.0955729484558105, "grad_norm": 0.6674990653991699, "tokens_seen": 20815872, "tokens_per_sec": 24596.685473326677, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.1890928745269775, "grad_norm": 0.6890289783477783, "tokens_seen": 20979712, "tokens_per_sec": 24597.206180748493, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.172515392303467, "grad_norm": 0.6899364590644836, "tokens_seen": 21143552, "tokens_per_sec": 24597.29576220353, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.054567813873291, "grad_norm": 0.6108677983283997, "tokens_seen": 21307392, "tokens_per_sec": 24597.91303196681, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.0008344650268555, "grad_norm": 0.6745784282684326, "tokens_seen": 21471232, "tokens_per_sec": 24597.824747799372, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.111222505569458, "grad_norm": 0.6862103343009949, "tokens_seen": 21635072, "tokens_per_sec": 24597.667838045403, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.162845611572266, "grad_norm": 0.6333869099617004, "tokens_seen": 21798912, "tokens_per_sec": 24597.659130750162, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.14229154586792, "grad_norm": 0.6544219851493835, "tokens_seen": 21962752, "tokens_per_sec": 24597.949248489927, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.157154083251953, "grad_norm": 0.6658969521522522, "tokens_seen": 22126592, "tokens_per_sec": 24598.07368670381, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.015000104904175, "grad_norm": 0.6737167239189148, "tokens_seen": 22290432, "tokens_per_sec": 24598.775922509987, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.100646495819092, "grad_norm": 0.6789868474006653, "tokens_seen": 22454272, "tokens_per_sec": 24598.896195919508, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.055103302001953, "grad_norm": 0.6890611052513123, "tokens_seen": 22618112, "tokens_per_sec": 24599.08467494635, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 4.970836400985718, "grad_norm": 0.7093930244445801, "tokens_seen": 22781952, "tokens_per_sec": 24599.82323930419, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.007469654083252, "grad_norm": 0.6643649339675903, "tokens_seen": 22945792, "tokens_per_sec": 24600.337465058943, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.996825456619263, "grad_norm": 0.6867532730102539, "tokens_seen": 23109632, "tokens_per_sec": 24599.619852789077, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.139064788818359, "grad_norm": 0.6453619599342346, "tokens_seen": 23273472, "tokens_per_sec": 24600.174144054596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.888462543487549, "grad_norm": 0.6971294283866882, "tokens_seen": 23437312, "tokens_per_sec": 24600.03124168168, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.052058696746826, "grad_norm": 0.6853320002555847, "tokens_seen": 23601152, "tokens_per_sec": 24600.721329208354, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.018959045410156, "grad_norm": 0.7244731187820435, "tokens_seen": 23764992, "tokens_per_sec": 24601.176396043902, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.075683832168579, "grad_norm": 0.6981412768363953, "tokens_seen": 23928832, "tokens_per_sec": 24601.34530159307, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.193588733673096, "grad_norm": 0.7387233972549438, "tokens_seen": 24092672, "tokens_per_sec": 24601.037595791837, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.903010845184326, "grad_norm": 0.6888434886932373, "tokens_seen": 24256512, "tokens_per_sec": 24601.22946178796, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.0096375942230225, "grad_norm": 0.6840689182281494, "tokens_seen": 24420352, "tokens_per_sec": 24600.73492349107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.916057109832764, "grad_norm": 0.7491065263748169, "tokens_seen": 24584192, "tokens_per_sec": 24600.63136811106, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.067261648178101, "val_ppl": 158.73904827962306, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.10599946975708, "grad_norm": 0.7082916498184204, "tokens_seen": 24748032, "tokens_per_sec": 24572.85433143827, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.039037466049194, "grad_norm": 0.7172942757606506, "tokens_seen": 24911872, "tokens_per_sec": 24573.434004277835, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.980351209640503, "grad_norm": 0.708868682384491, "tokens_seen": 25075712, "tokens_per_sec": 24572.779607262666, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.876773834228516, "grad_norm": 0.7128223776817322, "tokens_seen": 25239552, "tokens_per_sec": 24573.31144579352, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.0038228034973145, "grad_norm": 0.6784937381744385, "tokens_seen": 25403392, "tokens_per_sec": 24573.14585411981, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.759237051010132, "grad_norm": 0.6783036589622498, "tokens_seen": 25567232, "tokens_per_sec": 24573.320905597262, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.896729230880737, "grad_norm": 0.6497519016265869, "tokens_seen": 25731072, "tokens_per_sec": 24573.4363459843, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.854261159896851, "grad_norm": 0.7514152526855469, "tokens_seen": 25894912, "tokens_per_sec": 24573.027786048315, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.978490591049194, "grad_norm": 0.6664456129074097, "tokens_seen": 26058752, "tokens_per_sec": 24573.559310783367, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.903182744979858, "grad_norm": 0.6782830953598022, "tokens_seen": 26222592, "tokens_per_sec": 24573.60036172883, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.966947793960571, "grad_norm": 0.7131545543670654, "tokens_seen": 26386432, "tokens_per_sec": 24574.074659443373, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.028950929641724, "grad_norm": 0.7789819836616516, "tokens_seen": 26550272, "tokens_per_sec": 24573.879897288418, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.100800037384033, "grad_norm": 0.7389815449714661, "tokens_seen": 26714112, "tokens_per_sec": 24573.785905561333, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.879584789276123, "grad_norm": 0.7394940853118896, "tokens_seen": 26877952, "tokens_per_sec": 24571.671205947434, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.771418809890747, "grad_norm": 0.703143835067749, "tokens_seen": 27041792, "tokens_per_sec": 24570.423139030532, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.907673120498657, "grad_norm": 0.9366303086280823, "tokens_seen": 27205632, "tokens_per_sec": 24569.69121594934, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.920375347137451, "grad_norm": 0.6926969289779663, "tokens_seen": 27369472, "tokens_per_sec": 24568.86054002755, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.851568937301636, "grad_norm": 0.7284768223762512, "tokens_seen": 27533312, "tokens_per_sec": 24568.709622835668, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.977431297302246, "grad_norm": 0.7240822315216064, "tokens_seen": 27697152, "tokens_per_sec": 24568.156172068895, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.88345742225647, "grad_norm": 0.712985098361969, "tokens_seen": 27860992, "tokens_per_sec": 24566.949770957217, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.0099241733551025, "grad_norm": 0.6871935129165649, "tokens_seen": 28024832, "tokens_per_sec": 24566.85398492059, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.935109615325928, "grad_norm": 0.7320567965507507, "tokens_seen": 28188672, "tokens_per_sec": 24566.930670402664, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.934609413146973, "grad_norm": 0.7230526208877563, "tokens_seen": 28352512, "tokens_per_sec": 24566.00937614484, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.954260587692261, "grad_norm": 0.740764856338501, "tokens_seen": 28516352, "tokens_per_sec": 24565.988590183744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.858606576919556, "grad_norm": 0.7185389995574951, "tokens_seen": 28680192, "tokens_per_sec": 24565.161980624376, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.902589797973633, "grad_norm": 0.7085665464401245, "tokens_seen": 28844032, "tokens_per_sec": 24564.684324258444, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.874493360519409, "grad_norm": 0.6937246322631836, "tokens_seen": 29007872, "tokens_per_sec": 24563.63739903914, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.815959692001343, "grad_norm": 0.733386218547821, "tokens_seen": 29171712, "tokens_per_sec": 24563.50192467204, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.890920639038086, "grad_norm": 0.6826701164245605, "tokens_seen": 29335552, "tokens_per_sec": 24562.730430303152, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.762225866317749, "grad_norm": 0.7098303437232971, "tokens_seen": 29499392, "tokens_per_sec": 24562.086203696672, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.879634380340576, "grad_norm": 0.834397554397583, "tokens_seen": 29663232, "tokens_per_sec": 24559.75837823144, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.811228275299072, "grad_norm": 0.7428311705589294, "tokens_seen": 29827072, "tokens_per_sec": 24558.70694335513, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.881338119506836, "grad_norm": 0.7625472545623779, "tokens_seen": 29990912, "tokens_per_sec": 24557.621548866937, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.90925669670105, "grad_norm": 0.7301552891731262, "tokens_seen": 30154752, "tokens_per_sec": 24557.218096287073, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.834270238876343, "grad_norm": 0.8343570232391357, "tokens_seen": 30318592, "tokens_per_sec": 24554.957750624024, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.992587327957153, "grad_norm": 0.7334856390953064, "tokens_seen": 30482432, "tokens_per_sec": 24553.36186294478, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.878865718841553, "grad_norm": 0.7541519999504089, "tokens_seen": 30646272, "tokens_per_sec": 24552.067778455104, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.8163721561431885, "grad_norm": 0.7325838804244995, "tokens_seen": 30810112, "tokens_per_sec": 24549.596765555158, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.822922706604004, "grad_norm": 0.784505307674408, "tokens_seen": 30973952, "tokens_per_sec": 24547.49673922654, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.9284749031066895, "grad_norm": 0.7721770405769348, "tokens_seen": 31137792, "tokens_per_sec": 24546.26890160444, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.850111484527588, "grad_norm": 0.6757882833480835, "tokens_seen": 31301632, "tokens_per_sec": 24544.36973586665, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 5.016287565231323, "grad_norm": 0.7522569894790649, "tokens_seen": 31465472, "tokens_per_sec": 24543.42860158258, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.661356210708618, "grad_norm": 0.7341118454933167, "tokens_seen": 31629312, "tokens_per_sec": 24542.69604326713, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.773056745529175, "grad_norm": 0.7550076246261597, "tokens_seen": 31793152, "tokens_per_sec": 24540.654563390664, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.775173187255859, "grad_norm": 0.7016341090202332, "tokens_seen": 31956992, "tokens_per_sec": 24539.77394136705, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.832955837249756, "grad_norm": 0.7667298316955566, "tokens_seen": 32120832, "tokens_per_sec": 24539.035813236384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.946750640869141, "grad_norm": 0.7320140600204468, "tokens_seen": 32284672, "tokens_per_sec": 24537.916794195564, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.728057384490967, "grad_norm": 0.736677885055542, "tokens_seen": 32448512, "tokens_per_sec": 24537.709418912615, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.655433893203735, "grad_norm": 0.7233144640922546, "tokens_seen": 32612352, "tokens_per_sec": 24537.844147835767, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.880748987197876, "grad_norm": 0.7636198401451111, "tokens_seen": 32776192, "tokens_per_sec": 24538.142118330208, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.879926943778992, "val_ppl": 131.62104779821973, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.915621280670166, "grad_norm": 0.7567387223243713, "tokens_seen": 32940032, "tokens_per_sec": 24516.64221542132, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.677643299102783, "grad_norm": 0.7310529351234436, "tokens_seen": 33103872, "tokens_per_sec": 24516.737518421345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.728933095932007, "grad_norm": 0.7621753811836243, "tokens_seen": 33267712, "tokens_per_sec": 24516.96125739576, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.853359699249268, "grad_norm": 0.7752558588981628, "tokens_seen": 33431552, "tokens_per_sec": 24516.815908614313, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.733830213546753, "grad_norm": 0.7836071252822876, "tokens_seen": 33595392, "tokens_per_sec": 24517.065272562897, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.927579164505005, "grad_norm": 0.7827466130256653, "tokens_seen": 33759232, "tokens_per_sec": 24517.041514975183, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.840081214904785, "grad_norm": 0.7961176633834839, "tokens_seen": 33923072, "tokens_per_sec": 24517.141196470602, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.758086442947388, "grad_norm": 0.6739274263381958, "tokens_seen": 34086912, "tokens_per_sec": 24517.544384215566, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.8409411907196045, "grad_norm": 0.7149121165275574, "tokens_seen": 34250752, "tokens_per_sec": 24517.687449736368, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.6837077140808105, "grad_norm": 0.7462505102157593, "tokens_seen": 34414592, "tokens_per_sec": 24517.71560632497, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.724596977233887, "grad_norm": 0.7249518632888794, "tokens_seen": 34578432, "tokens_per_sec": 24517.79130565763, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.763152599334717, "grad_norm": 0.7029643058776855, "tokens_seen": 34742272, "tokens_per_sec": 24517.94309095551, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.648346424102783, "grad_norm": 0.7510204315185547, "tokens_seen": 34906112, "tokens_per_sec": 24517.860382074425, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.723224878311157, "grad_norm": 0.7540858387947083, "tokens_seen": 35069952, "tokens_per_sec": 24518.25859204959, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.7886059284210205, "grad_norm": 0.8295822143554688, "tokens_seen": 35233792, "tokens_per_sec": 24518.477483626164, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.851625442504883, "grad_norm": 0.7535659074783325, "tokens_seen": 35397632, "tokens_per_sec": 24518.81816734528, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.811367511749268, "grad_norm": 0.7646371126174927, "tokens_seen": 35561472, "tokens_per_sec": 24518.919525348963, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.696643352508545, "grad_norm": 0.7548817992210388, "tokens_seen": 35725312, "tokens_per_sec": 24519.115149646557, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.675223350524902, "grad_norm": 0.7847955226898193, "tokens_seen": 35889152, "tokens_per_sec": 24519.087573046338, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.733495712280273, "grad_norm": 0.8676465749740601, "tokens_seen": 36052992, "tokens_per_sec": 24519.358482978427, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.785974740982056, "grad_norm": 0.7757711410522461, "tokens_seen": 36216832, "tokens_per_sec": 24519.600307558947, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.769365310668945, "grad_norm": 0.7305030226707458, "tokens_seen": 36380672, "tokens_per_sec": 24519.924268780334, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.9003424644470215, "grad_norm": 0.7937370538711548, "tokens_seen": 36544512, "tokens_per_sec": 24520.20483344267, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.852762460708618, "grad_norm": 0.7752267718315125, "tokens_seen": 36708352, "tokens_per_sec": 24520.450561786078, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.715139389038086, "grad_norm": 0.7664779424667358, "tokens_seen": 36872192, "tokens_per_sec": 24520.4193073116, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.836533308029175, "grad_norm": 0.8346341848373413, "tokens_seen": 37036032, "tokens_per_sec": 24520.52705773024, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.783336400985718, "grad_norm": 0.7829397916793823, "tokens_seen": 37199872, "tokens_per_sec": 24520.745197487424, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.783078193664551, "grad_norm": 0.7921072840690613, "tokens_seen": 37363712, "tokens_per_sec": 24520.93425221973, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.616779565811157, "grad_norm": 0.8043190836906433, "tokens_seen": 37527552, "tokens_per_sec": 24521.105389419132, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.634896516799927, "grad_norm": 0.7654120326042175, "tokens_seen": 37691392, "tokens_per_sec": 24521.120825197744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.678593158721924, "grad_norm": 0.7461925745010376, "tokens_seen": 37855232, "tokens_per_sec": 24521.31525759948, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.719116687774658, "grad_norm": 0.8256919384002686, "tokens_seen": 38019072, "tokens_per_sec": 24521.329007616117, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.872925758361816, "grad_norm": 0.815895676612854, "tokens_seen": 38182912, "tokens_per_sec": 24521.589223134048, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.891138076782227, "grad_norm": 0.8470778465270996, "tokens_seen": 38346752, "tokens_per_sec": 24521.958151283972, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.719234943389893, "grad_norm": 0.7950282692909241, "tokens_seen": 38510592, "tokens_per_sec": 24522.211162325697, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.5935680866241455, "grad_norm": 0.9736528992652893, "tokens_seen": 38674432, "tokens_per_sec": 24522.651844171043, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.906435251235962, "grad_norm": 0.7936067581176758, "tokens_seen": 38838272, "tokens_per_sec": 24522.92311011735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.538063287734985, "grad_norm": 0.9360958933830261, "tokens_seen": 39002112, "tokens_per_sec": 24523.131136013304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.510082721710205, "grad_norm": 0.7415186762809753, "tokens_seen": 39165952, "tokens_per_sec": 24523.39601816345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.790353536605835, "grad_norm": 0.7896305918693542, "tokens_seen": 39329792, "tokens_per_sec": 24523.28640988211, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.693849086761475, "grad_norm": 0.8350623846054077, "tokens_seen": 39493632, "tokens_per_sec": 24523.47112892938, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.765537977218628, "grad_norm": 0.7810299396514893, "tokens_seen": 39657472, "tokens_per_sec": 24523.766583408087, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.790981292724609, "grad_norm": 0.8484138250350952, "tokens_seen": 39821312, "tokens_per_sec": 24523.859064042088, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.630990266799927, "grad_norm": 0.8047155141830444, "tokens_seen": 39985152, "tokens_per_sec": 24524.20392920337, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.714361190795898, "grad_norm": 0.7784938216209412, "tokens_seen": 40148992, "tokens_per_sec": 24524.451084103854, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.679460763931274, "grad_norm": 0.7992149591445923, "tokens_seen": 40312832, "tokens_per_sec": 24524.599277670113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.665088653564453, "grad_norm": 0.778285562992096, "tokens_seen": 40476672, "tokens_per_sec": 24524.421284107368, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.663004398345947, "grad_norm": 0.7617591023445129, "tokens_seen": 40640512, "tokens_per_sec": 24524.570180916788, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.636382341384888, "grad_norm": 0.8003793358802795, "tokens_seen": 40804352, "tokens_per_sec": 24524.759648077623, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.557705879211426, "grad_norm": 0.7794813513755798, "tokens_seen": 40968192, "tokens_per_sec": 24525.033014641605, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.756368184089661, "val_ppl": 116.32269520103802, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.820950984954834, "grad_norm": 0.820298433303833, "tokens_seen": 41132032, "tokens_per_sec": 24508.29697365601, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.632599353790283, "grad_norm": 0.8340275287628174, "tokens_seen": 41295872, "tokens_per_sec": 24508.01560097739, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.6473002433776855, "grad_norm": 0.763243556022644, "tokens_seen": 41459712, "tokens_per_sec": 24508.20426458192, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.569807052612305, "grad_norm": 0.8097470998764038, "tokens_seen": 41623552, "tokens_per_sec": 24508.518231520615, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.671289920806885, "grad_norm": 0.7865845561027527, "tokens_seen": 41787392, "tokens_per_sec": 24508.712099333297, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.506438493728638, "grad_norm": 0.8245251178741455, "tokens_seen": 41951232, "tokens_per_sec": 24508.851374196194, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.827539920806885, "grad_norm": 0.9569913148880005, "tokens_seen": 42115072, "tokens_per_sec": 24509.109234078536, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.8611369132995605, "grad_norm": 0.8340169787406921, "tokens_seen": 42278912, "tokens_per_sec": 24509.398508453472, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.640013217926025, "grad_norm": 0.7655485272407532, "tokens_seen": 42442752, "tokens_per_sec": 24509.58449651451, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.653957843780518, "grad_norm": 0.8055598139762878, "tokens_seen": 42606592, "tokens_per_sec": 24509.76660302003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.7389726638793945, "grad_norm": 0.8344480991363525, "tokens_seen": 42770432, "tokens_per_sec": 24510.0221749412, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.609429836273193, "grad_norm": 0.7932127714157104, "tokens_seen": 42934272, "tokens_per_sec": 24510.1516044434, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.463584661483765, "grad_norm": 0.8778133392333984, "tokens_seen": 43098112, "tokens_per_sec": 24510.244155587174, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.803248405456543, "grad_norm": 0.8196386694908142, "tokens_seen": 43261952, "tokens_per_sec": 24510.29452881881, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.714314937591553, "grad_norm": 0.8656349778175354, "tokens_seen": 43425792, "tokens_per_sec": 24510.72969799035, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.7415595054626465, "grad_norm": 0.8194333910942078, "tokens_seen": 43589632, "tokens_per_sec": 24510.931794506196, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.487332344055176, "grad_norm": 0.784777045249939, "tokens_seen": 43753472, "tokens_per_sec": 24511.268660683767, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.7312726974487305, "grad_norm": 0.8610876798629761, "tokens_seen": 43917312, "tokens_per_sec": 24511.232688269316, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.50031590461731, "grad_norm": 0.8338241577148438, "tokens_seen": 44081152, "tokens_per_sec": 24511.50821118165, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.553075790405273, "grad_norm": 0.8263097405433655, "tokens_seen": 44244992, "tokens_per_sec": 24511.908951723217, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.7137205600738525, "grad_norm": 0.8390830755233765, "tokens_seen": 44408832, "tokens_per_sec": 24512.121142884833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.56077766418457, "grad_norm": 0.8299065828323364, "tokens_seen": 44572672, "tokens_per_sec": 24512.33209591962, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.769161701202393, "grad_norm": 0.825066864490509, "tokens_seen": 44736512, "tokens_per_sec": 24512.76106492775, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.558732032775879, "grad_norm": 0.8091856241226196, "tokens_seen": 44900352, "tokens_per_sec": 24513.173957462976, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.88421630859375, "grad_norm": 0.8990447521209717, "tokens_seen": 45064192, "tokens_per_sec": 24513.41450814556, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.624439477920532, "grad_norm": 0.8343185186386108, "tokens_seen": 45228032, "tokens_per_sec": 24513.289337807975, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.584733724594116, "grad_norm": 0.811570405960083, "tokens_seen": 45391872, "tokens_per_sec": 24513.50371541788, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.6671576499938965, "grad_norm": 0.7736892104148865, "tokens_seen": 45555712, "tokens_per_sec": 24513.716976154537, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.595664024353027, "grad_norm": 0.7819673418998718, "tokens_seen": 45719552, "tokens_per_sec": 24513.893260053715, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.766069173812866, "grad_norm": 0.7857593297958374, "tokens_seen": 45883392, "tokens_per_sec": 24514.15943321717, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.689734220504761, "grad_norm": 0.8252121210098267, "tokens_seen": 46047232, "tokens_per_sec": 24514.506498523373, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.540770053863525, "grad_norm": 0.8108976483345032, "tokens_seen": 46211072, "tokens_per_sec": 24514.592273107766, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.573038578033447, "grad_norm": 0.827280580997467, "tokens_seen": 46374912, "tokens_per_sec": 24514.592050747582, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.673779487609863, "grad_norm": 0.8981935977935791, "tokens_seen": 46538752, "tokens_per_sec": 24514.66045860422, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.575953006744385, "grad_norm": 0.8380823731422424, "tokens_seen": 46702592, "tokens_per_sec": 24514.57831114678, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.535695552825928, "grad_norm": 0.8268945813179016, "tokens_seen": 46866432, "tokens_per_sec": 24514.85292616989, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.570342540740967, "grad_norm": 0.7735924124717712, "tokens_seen": 47030272, "tokens_per_sec": 24514.99436111102, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.354124069213867, "grad_norm": 0.8242924809455872, "tokens_seen": 47194112, "tokens_per_sec": 24515.109466999096, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.554733514785767, "grad_norm": 0.8506794571876526, "tokens_seen": 47357952, "tokens_per_sec": 24515.078450282013, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.6448962688446045, "grad_norm": 0.8927487730979919, "tokens_seen": 47521792, "tokens_per_sec": 24514.787524949243, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.598513126373291, "grad_norm": 0.8390204310417175, "tokens_seen": 47685632, "tokens_per_sec": 24514.953778368214, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.614759683609009, "grad_norm": 0.8065266609191895, "tokens_seen": 47849472, "tokens_per_sec": 24515.017096331758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.6591432094573975, "grad_norm": 0.8398613333702087, "tokens_seen": 48013312, "tokens_per_sec": 24515.032740829593, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.654690265655518, "grad_norm": 0.861840546131134, "tokens_seen": 48177152, "tokens_per_sec": 24514.905178017743, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.803696155548096, "grad_norm": 0.8361665606498718, "tokens_seen": 48340992, "tokens_per_sec": 24514.94188847472, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.652481317520142, "grad_norm": 0.8887232542037964, "tokens_seen": 48504832, "tokens_per_sec": 24515.01940652063, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.616170644760132, "grad_norm": 0.8319665789604187, "tokens_seen": 48668672, "tokens_per_sec": 24515.245244812104, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.654339551925659, "grad_norm": 0.7925450801849365, "tokens_seen": 48832512, "tokens_per_sec": 24515.405368440715, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.5205066204071045, "grad_norm": 0.8933059573173523, "tokens_seen": 48996352, "tokens_per_sec": 24515.699170054137, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.546364784240723, "grad_norm": 0.8034866452217102, "tokens_seen": 49160192, "tokens_per_sec": 24515.852758052337, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.6399352073669435, "val_ppl": 103.53763888970232, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.4759697914123535, "grad_norm": 0.8018413782119751, "tokens_seen": 49324032, "tokens_per_sec": 24501.73407218896, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.623022079467773, "grad_norm": 0.8084707856178284, "tokens_seen": 49487872, "tokens_per_sec": 24501.865872196995, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.524055480957031, "grad_norm": 0.8346961736679077, "tokens_seen": 49651712, "tokens_per_sec": 24502.002745159905, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.592864274978638, "grad_norm": 0.8174338340759277, "tokens_seen": 49815552, "tokens_per_sec": 24502.259229573545, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.558851003646851, "grad_norm": 0.9129741191864014, "tokens_seen": 49979392, "tokens_per_sec": 24502.18970167305, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.462734699249268, "grad_norm": 0.8007805943489075, "tokens_seen": 50143232, "tokens_per_sec": 24502.426906648234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.553126096725464, "grad_norm": 0.8951355814933777, "tokens_seen": 50307072, "tokens_per_sec": 24502.63523574614, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.6351704597473145, "grad_norm": 0.7736890912055969, "tokens_seen": 50470912, "tokens_per_sec": 24502.852230296856, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.538120746612549, "grad_norm": 0.860214352607727, "tokens_seen": 50634752, "tokens_per_sec": 24503.09941945288, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.595583438873291, "grad_norm": 0.8037154078483582, "tokens_seen": 50798592, "tokens_per_sec": 24503.138938474538, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.5126166343688965, "grad_norm": 0.818429708480835, "tokens_seen": 50962432, "tokens_per_sec": 24503.428702924743, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.459501504898071, "grad_norm": 0.8560409545898438, "tokens_seen": 51126272, "tokens_per_sec": 24503.68384569921, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.450350522994995, "grad_norm": 0.8430068492889404, "tokens_seen": 51290112, "tokens_per_sec": 24503.94117356806, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.745669841766357, "grad_norm": 0.892916738986969, "tokens_seen": 51453952, "tokens_per_sec": 24504.10784342895, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.730035781860352, "grad_norm": 0.8590502142906189, "tokens_seen": 51617792, "tokens_per_sec": 24504.355133834277, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.494525671005249, "grad_norm": 0.8846338391304016, "tokens_seen": 51781632, "tokens_per_sec": 24504.084003480533, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.629751920700073, "grad_norm": 0.876926600933075, "tokens_seen": 51945472, "tokens_per_sec": 24498.145899600244, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.5112199783325195, "grad_norm": 0.8236797451972961, "tokens_seen": 52109312, "tokens_per_sec": 24490.710270439762, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.55948543548584, "grad_norm": 0.816844642162323, "tokens_seen": 52273152, "tokens_per_sec": 24483.90016232857, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.384910583496094, "grad_norm": 0.8963565230369568, "tokens_seen": 52436992, "tokens_per_sec": 24478.32062741113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.574521541595459, "grad_norm": 0.8985388875007629, "tokens_seen": 52600832, "tokens_per_sec": 24478.36091098127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.753279209136963, "grad_norm": 0.9969784021377563, "tokens_seen": 52764672, "tokens_per_sec": 24478.620903699037, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.541424989700317, "grad_norm": 0.8806348443031311, "tokens_seen": 52928512, "tokens_per_sec": 24478.90738547855, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.586740016937256, "grad_norm": 1.020916223526001, "tokens_seen": 53092352, "tokens_per_sec": 24479.17094162775, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.4079673290252686, "grad_norm": 0.8775977492332458, "tokens_seen": 53256192, "tokens_per_sec": 24479.055710183184, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.672013759613037, "grad_norm": 0.8982191681861877, "tokens_seen": 53420032, "tokens_per_sec": 24479.18325680896, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.591898202896118, "grad_norm": 1.0285700559616089, "tokens_seen": 53583872, "tokens_per_sec": 24479.32508387921, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.585550308227539, "grad_norm": 0.8909537196159363, "tokens_seen": 53747712, "tokens_per_sec": 24479.326752221943, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.4449427127838135, "grad_norm": 0.8810632824897766, "tokens_seen": 53911552, "tokens_per_sec": 24479.445642109018, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.633819818496704, "grad_norm": 0.8738259673118591, "tokens_seen": 54075392, "tokens_per_sec": 24479.662051326082, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.475740909576416, "grad_norm": 0.9018210172653198, "tokens_seen": 54239232, "tokens_per_sec": 24480.025701727314, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.583945989608765, "grad_norm": 0.9333981275558472, "tokens_seen": 54403072, "tokens_per_sec": 24480.323189435407, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.680750370025635, "grad_norm": 0.9101545214653015, "tokens_seen": 54566912, "tokens_per_sec": 24480.285072839226, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.516995191574097, "grad_norm": 0.903434693813324, "tokens_seen": 54730752, "tokens_per_sec": 24480.372029303315, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.604161739349365, "grad_norm": 0.847373366355896, "tokens_seen": 54894592, "tokens_per_sec": 24480.52645402109, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.61495041847229, "grad_norm": 0.8580666780471802, "tokens_seen": 55058432, "tokens_per_sec": 24480.383331267523, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.581175327301025, "grad_norm": 0.8824209570884705, "tokens_seen": 55222272, "tokens_per_sec": 24480.47161109433, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.604783535003662, "grad_norm": 0.8723942637443542, "tokens_seen": 55386112, "tokens_per_sec": 24480.626283659723, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.556421756744385, "grad_norm": 0.8314032554626465, "tokens_seen": 55549952, "tokens_per_sec": 24480.6378366662, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.576638221740723, "grad_norm": 0.91450035572052, "tokens_seen": 55713792, "tokens_per_sec": 24480.807380364957, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.54016375541687, "grad_norm": 0.8807561993598938, "tokens_seen": 55877632, "tokens_per_sec": 24480.788976337208, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.623892784118652, "grad_norm": 0.8227481842041016, "tokens_seen": 56041472, "tokens_per_sec": 24480.806502595617, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.591876029968262, "grad_norm": 0.901369571685791, "tokens_seen": 56205312, "tokens_per_sec": 24480.983724677026, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.478687286376953, "grad_norm": 0.8759716153144836, "tokens_seen": 56369152, "tokens_per_sec": 24481.178634346998, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.581440210342407, "grad_norm": 0.9645659923553467, "tokens_seen": 56532992, "tokens_per_sec": 24480.981112036694, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.409404993057251, "grad_norm": 0.8722048401832581, "tokens_seen": 56696832, "tokens_per_sec": 24481.158697445113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.76798415184021, "grad_norm": 0.9814342856407166, "tokens_seen": 56860672, "tokens_per_sec": 24481.22748349393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.5283331871032715, "grad_norm": 0.856285035610199, "tokens_seen": 57024512, "tokens_per_sec": 24481.5823717449, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.481014251708984, "grad_norm": 0.9061179161071777, "tokens_seen": 57188352, "tokens_per_sec": 24481.881705993044, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.580734491348267, "grad_norm": 0.9050378799438477, "tokens_seen": 57352192, "tokens_per_sec": 24482.17525092614, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.584457731246948, "val_ppl": 97.95005748260107, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.468124866485596, "grad_norm": 0.9060422778129578, "tokens_seen": 57516032, "tokens_per_sec": 24470.20985237128, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.586272716522217, "grad_norm": 0.9031558632850647, "tokens_seen": 57679872, "tokens_per_sec": 24470.56070333582, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.61792778968811, "grad_norm": 0.8974457383155823, "tokens_seen": 57843712, "tokens_per_sec": 24470.81525894147, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.47887110710144, "grad_norm": 0.9323126673698425, "tokens_seen": 58007552, "tokens_per_sec": 24470.92834552109, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.49921441078186, "grad_norm": 0.9553396701812744, "tokens_seen": 58171392, "tokens_per_sec": 24471.029577315785, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.441729307174683, "grad_norm": 0.9082965850830078, "tokens_seen": 58335232, "tokens_per_sec": 24471.18119037388, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.586694240570068, "grad_norm": 0.8571968674659729, "tokens_seen": 58499072, "tokens_per_sec": 24471.44107575724, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.441089630126953, "grad_norm": 0.873968243598938, "tokens_seen": 58662912, "tokens_per_sec": 24471.669560834314, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.493950128555298, "grad_norm": 0.9225819706916809, "tokens_seen": 58826752, "tokens_per_sec": 24471.926757883222, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.5565454959869385, "grad_norm": 0.9411314129829407, "tokens_seen": 58990592, "tokens_per_sec": 24472.013105966627, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.5815911293029785, "grad_norm": 0.8495245575904846, "tokens_seen": 59154432, "tokens_per_sec": 24472.348456127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.39885687828064, "grad_norm": 0.9505347609519958, "tokens_seen": 59318272, "tokens_per_sec": 24472.69338994607, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.591885805130005, "grad_norm": 0.937684953212738, "tokens_seen": 59482112, "tokens_per_sec": 24472.95708467519, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.465270519256592, "grad_norm": 0.9518610239028931, "tokens_seen": 59645952, "tokens_per_sec": 24473.285856469254, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.463434457778931, "grad_norm": 0.8860647082328796, "tokens_seen": 59809792, "tokens_per_sec": 24473.592963693278, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.622486352920532, "grad_norm": 0.9273859858512878, "tokens_seen": 59973632, "tokens_per_sec": 24473.839891226522, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.395779848098755, "grad_norm": 0.8786794543266296, "tokens_seen": 60137472, "tokens_per_sec": 24474.150808644903, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.383533716201782, "grad_norm": 0.9407999515533447, "tokens_seen": 60301312, "tokens_per_sec": 24474.487666055727, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.597370147705078, "grad_norm": 0.9236119985580444, "tokens_seen": 60465152, "tokens_per_sec": 24474.7510334096, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.453288316726685, "grad_norm": 0.8926072120666504, "tokens_seen": 60628992, "tokens_per_sec": 24475.06565950429, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.673529624938965, "grad_norm": 0.8642966151237488, "tokens_seen": 60792832, "tokens_per_sec": 24475.349021901824, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.428603172302246, "grad_norm": 0.8777105212211609, "tokens_seen": 60956672, "tokens_per_sec": 24475.59879087124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.543842554092407, "grad_norm": 0.8346220850944519, "tokens_seen": 61120512, "tokens_per_sec": 24475.90355310251, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.5133960247039795, "grad_norm": 0.8482956886291504, "tokens_seen": 61284352, "tokens_per_sec": 24476.215207235997, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.479380130767822, "grad_norm": 0.9565541744232178, "tokens_seen": 61448192, "tokens_per_sec": 24476.42706941948, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.36482310295105, "grad_norm": 0.872641384601593, "tokens_seen": 61612032, "tokens_per_sec": 24476.767854406415, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.480591058731079, "grad_norm": 0.9106304049491882, "tokens_seen": 61775872, "tokens_per_sec": 24477.079241765765, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.591200351715088, "grad_norm": 0.8532452583312988, "tokens_seen": 61939712, "tokens_per_sec": 24477.34716177575, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.503696918487549, "grad_norm": 0.9858893752098083, "tokens_seen": 62103552, "tokens_per_sec": 24477.613549755504, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.475251197814941, "grad_norm": 0.9522062540054321, "tokens_seen": 62267392, "tokens_per_sec": 24477.935171232744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.711445569992065, "grad_norm": 0.9662061929702759, "tokens_seen": 62431232, "tokens_per_sec": 24478.18959209912, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.446849822998047, "grad_norm": 0.9629977941513062, "tokens_seen": 62595072, "tokens_per_sec": 24478.472771295576, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.372482776641846, "grad_norm": 0.8803629279136658, "tokens_seen": 62758912, "tokens_per_sec": 24478.701593872695, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.330411195755005, "grad_norm": 0.8660396337509155, "tokens_seen": 62922752, "tokens_per_sec": 24479.020162030785, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.473123550415039, "grad_norm": 0.8515709638595581, "tokens_seen": 63086592, "tokens_per_sec": 24479.260606521322, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.530218839645386, "grad_norm": 0.9134182929992676, "tokens_seen": 63250432, "tokens_per_sec": 24479.507512613345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.4269208908081055, "grad_norm": 0.9428609609603882, "tokens_seen": 63414272, "tokens_per_sec": 24479.777471558013, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.567101240158081, "grad_norm": 0.9645809531211853, "tokens_seen": 63578112, "tokens_per_sec": 24480.006749197644, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.6094536781311035, "grad_norm": 0.9968925714492798, "tokens_seen": 63741952, "tokens_per_sec": 24480.325305518756, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.502329349517822, "grad_norm": 0.9109309911727905, "tokens_seen": 63905792, "tokens_per_sec": 24480.529368496846, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.4581618309021, "grad_norm": 0.8758007884025574, "tokens_seen": 64069632, "tokens_per_sec": 24480.819856273825, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.360724687576294, "grad_norm": 0.8524029850959778, "tokens_seen": 64233472, "tokens_per_sec": 24481.0689941265, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.419090747833252, "grad_norm": 0.960829496383667, "tokens_seen": 64397312, "tokens_per_sec": 24481.335330735063, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.481651306152344, "grad_norm": 1.0039254426956177, "tokens_seen": 64561152, "tokens_per_sec": 24481.623063409075, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.4671430587768555, "grad_norm": 0.9481309652328491, "tokens_seen": 64724992, "tokens_per_sec": 24481.874385728417, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.561169147491455, "grad_norm": 0.9447068572044373, "tokens_seen": 64888832, "tokens_per_sec": 24482.083032821258, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.442570686340332, "grad_norm": 0.9372158050537109, "tokens_seen": 65052672, "tokens_per_sec": 24482.35376698519, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.515752792358398, "grad_norm": 0.8922737240791321, "tokens_seen": 65216512, "tokens_per_sec": 24482.591791764706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.477459907531738, "grad_norm": 0.9565081596374512, "tokens_seen": 65380352, "tokens_per_sec": 24482.850281014034, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.444262742996216, "grad_norm": 0.9264848828315735, "tokens_seen": 65544192, "tokens_per_sec": 24483.11327457707, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.4794742345809935, "val_ppl": 88.18829412918758, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.493948459625244, "grad_norm": 0.8920753002166748, "tokens_seen": 65708032, "tokens_per_sec": 24472.676101286826, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.52025294303894, "grad_norm": 0.8749822378158569, "tokens_seen": 65871872, "tokens_per_sec": 24472.9678096153, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.395020484924316, "grad_norm": 0.928347110748291, "tokens_seen": 66035712, "tokens_per_sec": 24473.22235166162, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.455122470855713, "grad_norm": 0.8929117918014526, "tokens_seen": 66199552, "tokens_per_sec": 24473.257255261207, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.5019025802612305, "grad_norm": 0.9514235854148865, "tokens_seen": 66363392, "tokens_per_sec": 24473.461388212414, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.536475658416748, "grad_norm": 0.9454032778739929, "tokens_seen": 66527232, "tokens_per_sec": 24473.76622249501, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.445774793624878, "grad_norm": 0.9316017627716064, "tokens_seen": 66691072, "tokens_per_sec": 24474.058630745483, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.565812587738037, "grad_norm": 0.9278233647346497, "tokens_seen": 66854912, "tokens_per_sec": 24474.322537227967, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.551246166229248, "grad_norm": 1.0394235849380493, "tokens_seen": 67018752, "tokens_per_sec": 24474.59842209776, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.499769449234009, "grad_norm": 0.9201073050498962, "tokens_seen": 67182592, "tokens_per_sec": 24474.27491980291, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.521439075469971, "grad_norm": 0.976344883441925, "tokens_seen": 67346432, "tokens_per_sec": 24474.521248280675, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.419908046722412, "grad_norm": 0.9932031631469727, "tokens_seen": 67510272, "tokens_per_sec": 24474.75296550767, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.474217176437378, "grad_norm": 0.9347466230392456, "tokens_seen": 67674112, "tokens_per_sec": 24475.05474026804, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.453847885131836, "grad_norm": 0.9453269243240356, "tokens_seen": 67837952, "tokens_per_sec": 24475.330503694786, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.5001726150512695, "grad_norm": 0.97548907995224, "tokens_seen": 68001792, "tokens_per_sec": 24475.586694755857, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.440595865249634, "grad_norm": 0.9283581376075745, "tokens_seen": 68165632, "tokens_per_sec": 24475.798305458327, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.475255727767944, "grad_norm": 0.9446901082992554, "tokens_seen": 68329472, "tokens_per_sec": 24476.090471730768, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.4817259311676025, "grad_norm": 0.9370797276496887, "tokens_seen": 68493312, "tokens_per_sec": 24476.374657354543, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.445523738861084, "grad_norm": 0.9654633402824402, "tokens_seen": 68657152, "tokens_per_sec": 24476.62455754325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.3891379833221436, "grad_norm": 0.9761630892753601, "tokens_seen": 68820992, "tokens_per_sec": 24476.863202424764, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.591430187225342, "grad_norm": 0.9842066168785095, "tokens_seen": 68984832, "tokens_per_sec": 24477.086047818888, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.474409341812134, "grad_norm": 1.0112273693084717, "tokens_seen": 69148672, "tokens_per_sec": 24477.37413427434, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.416067361831665, "grad_norm": 0.9839879274368286, "tokens_seen": 69312512, "tokens_per_sec": 24477.61208077007, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.36656641960144, "grad_norm": 0.9239302277565002, "tokens_seen": 69476352, "tokens_per_sec": 24477.87901748315, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.405592441558838, "grad_norm": 1.074846625328064, "tokens_seen": 69640192, "tokens_per_sec": 24478.090479015493, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.515607833862305, "grad_norm": 0.9531385898590088, "tokens_seen": 69804032, "tokens_per_sec": 24478.316935072486, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.593299150466919, "grad_norm": 1.0367300510406494, "tokens_seen": 69967872, "tokens_per_sec": 24478.612895350107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.316596984863281, "grad_norm": 1.0037420988082886, "tokens_seen": 70131712, "tokens_per_sec": 24478.827161875626, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.490124940872192, "grad_norm": 0.9468274116516113, "tokens_seen": 70295552, "tokens_per_sec": 24479.033858642157, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.377636194229126, "grad_norm": 0.9965645670890808, "tokens_seen": 70459392, "tokens_per_sec": 24479.333568256974, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.404886722564697, "grad_norm": 0.9534224271774292, "tokens_seen": 70623232, "tokens_per_sec": 24479.569251600977, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.40445876121521, "grad_norm": 1.0175572633743286, "tokens_seen": 70787072, "tokens_per_sec": 24479.777906295814, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.459975719451904, "grad_norm": 0.9356381297111511, "tokens_seen": 70950912, "tokens_per_sec": 24480.059437060387, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.322762489318848, "grad_norm": 1.0148468017578125, "tokens_seen": 71114752, "tokens_per_sec": 24480.234909647425, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.202543020248413, "grad_norm": 0.9490100145339966, "tokens_seen": 71278592, "tokens_per_sec": 24480.50183211048, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.386466026306152, "grad_norm": 0.9872127771377563, "tokens_seen": 71442432, "tokens_per_sec": 24480.725009768965, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.43009614944458, "grad_norm": 0.9575414657592773, "tokens_seen": 71606272, "tokens_per_sec": 24480.93834820261, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.534462213516235, "grad_norm": 0.9915619492530823, "tokens_seen": 71770112, "tokens_per_sec": 24481.186336411924, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.531801700592041, "grad_norm": 0.9803833961486816, "tokens_seen": 71933952, "tokens_per_sec": 24481.446835022914, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.348299264907837, "grad_norm": 0.9328469038009644, "tokens_seen": 72097792, "tokens_per_sec": 24481.64158806703, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.368246555328369, "grad_norm": 0.9753528237342834, "tokens_seen": 72261632, "tokens_per_sec": 24481.880332981087, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.512716293334961, "grad_norm": 0.976791262626648, "tokens_seen": 72425472, "tokens_per_sec": 24482.08606399422, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.401990652084351, "grad_norm": 0.9171484112739563, "tokens_seen": 72589312, "tokens_per_sec": 24482.35626104471, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.403055191040039, "grad_norm": 0.9831825494766235, "tokens_seen": 72753152, "tokens_per_sec": 24482.548514507514, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.480915546417236, "grad_norm": 0.942884624004364, "tokens_seen": 72916992, "tokens_per_sec": 24482.81561185735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.357619285583496, "grad_norm": 0.954590916633606, "tokens_seen": 73080832, "tokens_per_sec": 24482.721455208906, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.438993215560913, "grad_norm": 0.9485318064689636, "tokens_seen": 73244672, "tokens_per_sec": 24482.739335801194, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.419190406799316, "grad_norm": 0.9547813534736633, "tokens_seen": 73408512, "tokens_per_sec": 24482.927993992347, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.338192939758301, "grad_norm": 0.9265035390853882, "tokens_seen": 73572352, "tokens_per_sec": 24482.856209586156, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.371409177780151, "grad_norm": 0.9478694200515747, "tokens_seen": 73736192, "tokens_per_sec": 24483.052278079093, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.508366084098816, "val_ppl": 90.77338120556041, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.3755998611450195, "grad_norm": 0.9999054074287415, "tokens_seen": 73900032, "tokens_per_sec": 24473.85635352142, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.543805360794067, "grad_norm": 0.9502936005592346, "tokens_seen": 74063872, "tokens_per_sec": 24474.099593611332, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.409498691558838, "grad_norm": 0.9935785531997681, "tokens_seen": 74227712, "tokens_per_sec": 24474.347015203428, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.3127875328063965, "grad_norm": 0.9283362627029419, "tokens_seen": 74391552, "tokens_per_sec": 24474.54105387105, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.454383373260498, "grad_norm": 1.003671407699585, "tokens_seen": 74555392, "tokens_per_sec": 24474.81438263242, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.3764684200286865, "grad_norm": 0.9993801116943359, "tokens_seen": 74719232, "tokens_per_sec": 24475.081579671085, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.310529947280884, "grad_norm": 0.9417620897293091, "tokens_seen": 74883072, "tokens_per_sec": 24475.321159352854, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.449219226837158, "grad_norm": 0.9549633860588074, "tokens_seen": 75046912, "tokens_per_sec": 24475.513337014865, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.370728969573975, "grad_norm": 1.1801429986953735, "tokens_seen": 75210752, "tokens_per_sec": 24475.78822446194, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.485527992248535, "grad_norm": 1.0845731496810913, "tokens_seen": 75374592, "tokens_per_sec": 24475.741409346, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.337197780609131, "grad_norm": 1.1048955917358398, "tokens_seen": 75538432, "tokens_per_sec": 24475.87868718193, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.362997055053711, "grad_norm": 0.9680981040000916, "tokens_seen": 75702272, "tokens_per_sec": 24475.97506592361, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.370051145553589, "grad_norm": 0.9722657203674316, "tokens_seen": 75866112, "tokens_per_sec": 24476.184532462292, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.256371259689331, "grad_norm": 0.9374227523803711, "tokens_seen": 76029952, "tokens_per_sec": 24476.427007927156, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.495760440826416, "grad_norm": 1.1272203922271729, "tokens_seen": 76193792, "tokens_per_sec": 24476.65816467945, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.346290111541748, "grad_norm": 1.0628111362457275, "tokens_seen": 76357632, "tokens_per_sec": 24476.839394963168, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.364584445953369, "grad_norm": 1.0336202383041382, "tokens_seen": 76521472, "tokens_per_sec": 24477.09219404409, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.125462293624878, "grad_norm": 0.9373244643211365, "tokens_seen": 76685312, "tokens_per_sec": 24477.29197518848, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.5110924243927, "grad_norm": 1.0035814046859741, "tokens_seen": 76849152, "tokens_per_sec": 24477.533216148087, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.422543525695801, "grad_norm": 0.9512774348258972, "tokens_seen": 77012992, "tokens_per_sec": 24477.73607038596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.3750457763671875, "grad_norm": 1.0572847127914429, "tokens_seen": 77176832, "tokens_per_sec": 24477.959963847054, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.283337831497192, "grad_norm": 0.9444621205329895, "tokens_seen": 77340672, "tokens_per_sec": 24478.18281672818, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.451351642608643, "grad_norm": 0.9980995059013367, "tokens_seen": 77504512, "tokens_per_sec": 24478.42149931269, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.262932777404785, "grad_norm": 1.071226716041565, "tokens_seen": 77668352, "tokens_per_sec": 24478.634489567994, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.425966024398804, "grad_norm": 1.039326548576355, "tokens_seen": 77832192, "tokens_per_sec": 24478.825632347038, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.2903454303741455, "grad_norm": 1.0273237228393555, "tokens_seen": 77996032, "tokens_per_sec": 24479.105012721328, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.266018867492676, "grad_norm": 1.021854281425476, "tokens_seen": 78159872, "tokens_per_sec": 24479.304560206703, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.3123109340667725, "grad_norm": 0.9333472847938538, "tokens_seen": 78323712, "tokens_per_sec": 24479.502285591654, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.230960369110107, "grad_norm": 0.9719789624214172, "tokens_seen": 78487552, "tokens_per_sec": 24479.705998538673, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.383977651596069, "grad_norm": 1.049391508102417, "tokens_seen": 78651392, "tokens_per_sec": 24479.92927719054, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.47630763053894, "grad_norm": 0.9659077525138855, "tokens_seen": 78815232, "tokens_per_sec": 24480.168525408884, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.40911865234375, "grad_norm": 0.9808695912361145, "tokens_seen": 78979072, "tokens_per_sec": 24480.349203572645, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.2528908252716064, "grad_norm": 0.9353129863739014, "tokens_seen": 79142912, "tokens_per_sec": 24480.611315880236, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.227078437805176, "grad_norm": 1.0632247924804688, "tokens_seen": 79306752, "tokens_per_sec": 24480.787343013875, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.246172666549683, "grad_norm": 1.0141479969024658, "tokens_seen": 79470592, "tokens_per_sec": 24481.033670011217, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.257999897003174, "grad_norm": 1.0266785621643066, "tokens_seen": 79634432, "tokens_per_sec": 24481.25369161303, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.359278202056885, "grad_norm": 1.055638313293457, "tokens_seen": 79798272, "tokens_per_sec": 24481.44538740035, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.2054643630981445, "grad_norm": 0.9822818636894226, "tokens_seen": 79962112, "tokens_per_sec": 24481.67670230816, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.358777761459351, "grad_norm": 1.00542414188385, "tokens_seen": 80125952, "tokens_per_sec": 24481.885920491462, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.414834499359131, "grad_norm": 1.052749752998352, "tokens_seen": 80289792, "tokens_per_sec": 24482.091323178905, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.430934906005859, "grad_norm": 1.0657284259796143, "tokens_seen": 80453632, "tokens_per_sec": 24482.27584443671, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.173696756362915, "grad_norm": 1.002269983291626, "tokens_seen": 80617472, "tokens_per_sec": 24482.42387140426, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.431265115737915, "grad_norm": 1.1223138570785522, "tokens_seen": 80781312, "tokens_per_sec": 24482.66003797726, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.524593114852905, "grad_norm": 1.0754725933074951, "tokens_seen": 80945152, "tokens_per_sec": 24482.88700095703, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.334794282913208, "grad_norm": 1.0279356241226196, "tokens_seen": 81108992, "tokens_per_sec": 24483.072176526803, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.39650559425354, "grad_norm": 0.9922977089881897, "tokens_seen": 81272832, "tokens_per_sec": 24483.271495431592, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.3446338176727295, "grad_norm": 1.1475461721420288, "tokens_seen": 81436672, "tokens_per_sec": 24483.485339832543, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.2145161628723145, "grad_norm": 0.9779347777366638, "tokens_seen": 81600512, "tokens_per_sec": 24483.66358022065, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.327863454818726, "grad_norm": 1.0001087188720703, "tokens_seen": 81764352, "tokens_per_sec": 24483.769393164024, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.2259840965271, "grad_norm": 0.9996048808097839, "tokens_seen": 81928192, "tokens_per_sec": 24484.062270186107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.4160161972045895, "val_ppl": 82.76590466715272, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.50252890586853, "grad_norm": 1.0552048683166504, "tokens_seen": 82092032, "tokens_per_sec": 24475.7191134514, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.527415752410889, "grad_norm": 1.1073106527328491, "tokens_seen": 82255872, "tokens_per_sec": 24475.91503135149, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.51752233505249, "grad_norm": 1.0558364391326904, "tokens_seen": 82419712, "tokens_per_sec": 24476.119663256864, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.351218938827515, "grad_norm": 1.0858515501022339, "tokens_seen": 82583552, "tokens_per_sec": 24476.34458396284, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.495257139205933, "grad_norm": 1.0089246034622192, "tokens_seen": 82747392, "tokens_per_sec": 24476.54670076463, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.297830820083618, "grad_norm": 1.1015830039978027, "tokens_seen": 82911232, "tokens_per_sec": 24476.73650691183, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.288839101791382, "grad_norm": 1.062648057937622, "tokens_seen": 83075072, "tokens_per_sec": 24476.943528438245, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.391778230667114, "grad_norm": 1.0411062240600586, "tokens_seen": 83238912, "tokens_per_sec": 24476.976514296926, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.392980575561523, "grad_norm": 1.043286919593811, "tokens_seen": 83402752, "tokens_per_sec": 24476.908228701086, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.298938274383545, "grad_norm": 0.9764354228973389, "tokens_seen": 83566592, "tokens_per_sec": 24477.104285448433, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.396192789077759, "grad_norm": 1.1140309572219849, "tokens_seen": 83730432, "tokens_per_sec": 24477.303031030053, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.319678783416748, "grad_norm": 1.0527541637420654, "tokens_seen": 83894272, "tokens_per_sec": 24477.47324421644, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.340498447418213, "grad_norm": 1.0257288217544556, "tokens_seen": 84058112, "tokens_per_sec": 24477.54922334717, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.468936920166016, "grad_norm": 1.047622561454773, "tokens_seen": 84221952, "tokens_per_sec": 24477.73903045399, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.314990043640137, "grad_norm": 1.0911738872528076, "tokens_seen": 84385792, "tokens_per_sec": 24477.929180100553, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.501439571380615, "grad_norm": 1.1381049156188965, "tokens_seen": 84549632, "tokens_per_sec": 24478.12608068116, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.30764627456665, "grad_norm": 1.0434004068374634, "tokens_seen": 84713472, "tokens_per_sec": 24478.350471069483, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.335675239562988, "grad_norm": 0.9891902208328247, "tokens_seen": 84877312, "tokens_per_sec": 24478.545357268908, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.447521686553955, "grad_norm": 1.1385565996170044, "tokens_seen": 85041152, "tokens_per_sec": 24478.730030943763, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.448023557662964, "grad_norm": 1.0239466428756714, "tokens_seen": 85204992, "tokens_per_sec": 24478.660745349436, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.266997814178467, "grad_norm": 1.0584872961044312, "tokens_seen": 85368832, "tokens_per_sec": 24478.846856756103, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.208497524261475, "grad_norm": 1.014259934425354, "tokens_seen": 85532672, "tokens_per_sec": 24479.0113440665, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.359752655029297, "grad_norm": 1.0987173318862915, "tokens_seen": 85696512, "tokens_per_sec": 24479.075783820404, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.382753849029541, "grad_norm": 1.141056776046753, "tokens_seen": 85860352, "tokens_per_sec": 24479.27907151091, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.267534494400024, "grad_norm": 0.9480759501457214, "tokens_seen": 86024192, "tokens_per_sec": 24479.46869522381, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.237181663513184, "grad_norm": 0.9935454726219177, "tokens_seen": 86188032, "tokens_per_sec": 24479.67749821535, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.106071472167969, "grad_norm": 1.060187816619873, "tokens_seen": 86351872, "tokens_per_sec": 24479.864131971444, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.3605687618255615, "grad_norm": 1.0137348175048828, "tokens_seen": 86515712, "tokens_per_sec": 24480.05761879017, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.416379928588867, "grad_norm": 1.1179536581039429, "tokens_seen": 86679552, "tokens_per_sec": 24480.2546530771, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.461820125579834, "grad_norm": 1.117631435394287, "tokens_seen": 86843392, "tokens_per_sec": 24480.45184703543, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.394059181213379, "grad_norm": 1.0567073822021484, "tokens_seen": 87007232, "tokens_per_sec": 24480.650687637593, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.380119323730469, "grad_norm": 1.014029860496521, "tokens_seen": 87171072, "tokens_per_sec": 24480.84643181031, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.342506408691406, "grad_norm": 1.0401487350463867, "tokens_seen": 87334912, "tokens_per_sec": 24481.023079365634, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.447023391723633, "grad_norm": 1.0486857891082764, "tokens_seen": 87498752, "tokens_per_sec": 24481.21581338179, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.458795070648193, "grad_norm": 1.1159414052963257, "tokens_seen": 87662592, "tokens_per_sec": 24481.38504528819, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.341056823730469, "grad_norm": 0.9819040894508362, "tokens_seen": 87826432, "tokens_per_sec": 24481.559427293072, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.211644411087036, "grad_norm": 0.9566699266433716, "tokens_seen": 87990272, "tokens_per_sec": 24481.792968445214, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.322206497192383, "grad_norm": 1.090599775314331, "tokens_seen": 88154112, "tokens_per_sec": 24481.765941147143, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.447514772415161, "grad_norm": 1.0537899732589722, "tokens_seen": 88317952, "tokens_per_sec": 24481.652393819604, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.398430347442627, "grad_norm": 1.1234420537948608, "tokens_seen": 88481792, "tokens_per_sec": 24481.7748633003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.327817440032959, "grad_norm": 1.0108712911605835, "tokens_seen": 88645632, "tokens_per_sec": 24481.838959607758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.32096529006958, "grad_norm": 1.1202921867370605, "tokens_seen": 88809472, "tokens_per_sec": 24481.96635510135, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.313750743865967, "grad_norm": 1.0223416090011597, "tokens_seen": 88973312, "tokens_per_sec": 24482.162137198357, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.44697380065918, "grad_norm": 1.1094775199890137, "tokens_seen": 89137152, "tokens_per_sec": 24482.329771961085, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.493303298950195, "grad_norm": 1.0239607095718384, "tokens_seen": 89300992, "tokens_per_sec": 24482.558183878744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.533982276916504, "grad_norm": 1.3979029655456543, "tokens_seen": 89464832, "tokens_per_sec": 24482.572672630846, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.5428147315979, "grad_norm": 1.058861494064331, "tokens_seen": 89628672, "tokens_per_sec": 24482.750957206903, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.297233581542969, "grad_norm": 1.0395632982254028, "tokens_seen": 89792512, "tokens_per_sec": 24482.909856170336, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.174307107925415, "grad_norm": 1.009901523590088, "tokens_seen": 89956352, "tokens_per_sec": 24483.133318128945, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.401012182235718, "grad_norm": 1.132346749305725, "tokens_seen": 90120192, "tokens_per_sec": 24483.2650033201, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.413574838638306, "val_ppl": 82.56408986818008, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.344550848007202, "grad_norm": 1.181947112083435, "tokens_seen": 90284032, "tokens_per_sec": 24475.696953604234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.281188488006592, "grad_norm": 1.086022973060608, "tokens_seen": 90447872, "tokens_per_sec": 24475.738997314038, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.402621507644653, "grad_norm": 1.1770095825195312, "tokens_seen": 90611712, "tokens_per_sec": 24475.900977363213, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.30818510055542, "grad_norm": 1.103590488433838, "tokens_seen": 90775552, "tokens_per_sec": 24476.045239920146, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.443477153778076, "grad_norm": 1.0651010274887085, "tokens_seen": 90939392, "tokens_per_sec": 24476.256897283885, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.180776596069336, "grad_norm": 1.0254735946655273, "tokens_seen": 91103232, "tokens_per_sec": 24476.352055913296, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.3441386222839355, "grad_norm": 1.0619837045669556, "tokens_seen": 91267072, "tokens_per_sec": 24476.692576445603, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.361436128616333, "grad_norm": 1.0346746444702148, "tokens_seen": 91430912, "tokens_per_sec": 24476.844778527553, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.347273826599121, "grad_norm": 1.0439114570617676, "tokens_seen": 91594752, "tokens_per_sec": 24477.021586534178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.193650960922241, "grad_norm": 1.0521879196166992, "tokens_seen": 91758592, "tokens_per_sec": 24477.24870551081, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.145205974578857, "grad_norm": 1.027746319770813, "tokens_seen": 91922432, "tokens_per_sec": 24477.405488504937, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.339918613433838, "grad_norm": 1.0884170532226562, "tokens_seen": 92086272, "tokens_per_sec": 24477.609916103484, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.362668514251709, "grad_norm": 1.149849772453308, "tokens_seen": 92250112, "tokens_per_sec": 24477.805296083065, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.326626300811768, "grad_norm": 1.1545889377593994, "tokens_seen": 92413952, "tokens_per_sec": 24477.981707271683, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.3646111488342285, "grad_norm": 1.1172500848770142, "tokens_seen": 92577792, "tokens_per_sec": 24477.935702819897, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.23856782913208, "grad_norm": 1.0810848474502563, "tokens_seen": 92741632, "tokens_per_sec": 24478.080034584207, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.421144723892212, "grad_norm": 1.167443037033081, "tokens_seen": 92905472, "tokens_per_sec": 24478.232957281496, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.322118759155273, "grad_norm": 1.1111334562301636, "tokens_seen": 93069312, "tokens_per_sec": 24478.44684515574, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.3463194370269775, "grad_norm": 1.1282020807266235, "tokens_seen": 93233152, "tokens_per_sec": 24478.62054356349, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.382818937301636, "grad_norm": 1.2457711696624756, "tokens_seen": 93396992, "tokens_per_sec": 24478.831756580355, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.342031002044678, "grad_norm": 1.074489712715149, "tokens_seen": 93560832, "tokens_per_sec": 24479.00579967174, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.211063623428345, "grad_norm": 1.1002117395401, "tokens_seen": 93724672, "tokens_per_sec": 24479.195803198945, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.280836820602417, "grad_norm": 1.1272344589233398, "tokens_seen": 93888512, "tokens_per_sec": 24479.384238075563, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.253172159194946, "grad_norm": 1.0849241018295288, "tokens_seen": 94052352, "tokens_per_sec": 24479.553384897867, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.365793943405151, "grad_norm": 1.0643718242645264, "tokens_seen": 94216192, "tokens_per_sec": 24479.734972054775, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.46038031578064, "grad_norm": 1.0802422761917114, "tokens_seen": 94380032, "tokens_per_sec": 24479.90963083727, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.340405464172363, "grad_norm": 1.1611359119415283, "tokens_seen": 94543872, "tokens_per_sec": 24480.071471427982, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.298735857009888, "grad_norm": 1.1346782445907593, "tokens_seen": 94707712, "tokens_per_sec": 24480.221947804952, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.503776550292969, "grad_norm": 1.2460527420043945, "tokens_seen": 94871552, "tokens_per_sec": 24480.43801765229, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.213443756103516, "grad_norm": 1.0966144800186157, "tokens_seen": 95035392, "tokens_per_sec": 24480.624698812506, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.327162265777588, "grad_norm": 1.0618914365768433, "tokens_seen": 95199232, "tokens_per_sec": 24480.802378604425, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.257949590682983, "grad_norm": 0.9889277815818787, "tokens_seen": 95363072, "tokens_per_sec": 24480.981647033634, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.295352935791016, "grad_norm": 1.0708129405975342, "tokens_seen": 95526912, "tokens_per_sec": 24481.122240651264, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.319768667221069, "grad_norm": 1.0388715267181396, "tokens_seen": 95690752, "tokens_per_sec": 24481.330553662232, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.352288722991943, "grad_norm": 1.4199049472808838, "tokens_seen": 95854592, "tokens_per_sec": 24481.332499034586, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.30495548248291, "grad_norm": 1.0445582866668701, "tokens_seen": 96018432, "tokens_per_sec": 24481.511014654865, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.341633081436157, "grad_norm": 1.1087520122528076, "tokens_seen": 96182272, "tokens_per_sec": 24481.65633912028, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.232495307922363, "grad_norm": 1.1965806484222412, "tokens_seen": 96346112, "tokens_per_sec": 24481.849927578984, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.425271034240723, "grad_norm": 1.1268514394760132, "tokens_seen": 96509952, "tokens_per_sec": 24482.051355017684, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.3334925174713135, "grad_norm": 1.1196104288101196, "tokens_seen": 96673792, "tokens_per_sec": 24482.153124414068, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.491137742996216, "grad_norm": 1.0808634757995605, "tokens_seen": 96837632, "tokens_per_sec": 24482.35469923003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.2704017162323, "grad_norm": 1.028992772102356, "tokens_seen": 97001472, "tokens_per_sec": 24482.5229021786, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.226804256439209, "grad_norm": 1.1155728101730347, "tokens_seen": 97165312, "tokens_per_sec": 24482.70227258155, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.349846601486206, "grad_norm": 1.1481457948684692, "tokens_seen": 97329152, "tokens_per_sec": 24482.874739660263, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.3060081005096436, "grad_norm": 1.0700905323028564, "tokens_seen": 97492992, "tokens_per_sec": 24482.921407557213, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.216139316558838, "grad_norm": 1.0883899927139282, "tokens_seen": 97656832, "tokens_per_sec": 24483.17881705499, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.208914518356323, "grad_norm": 1.2391968965530396, "tokens_seen": 97820672, "tokens_per_sec": 24483.354445870016, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.308138608932495, "grad_norm": 1.1028587818145752, "tokens_seen": 97984512, "tokens_per_sec": 24483.503990958096, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.296247243881226, "grad_norm": 1.2196780443191528, "tokens_seen": 98148352, "tokens_per_sec": 24483.712653982886, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.410813975334167, "val_ppl": 82.33645607970243, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4016.196511745453, "nan_failures": 0} diff --git a/baseline_s3/config.json b/baseline_s3/config.json new file mode 100644 index 0000000000000000000000000000000000000000..76a542c18a5e35f21fc88e5cde6ca47b40117eee --- /dev/null +++ b/baseline_s3/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 3, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s3", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 3001346, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/baseline_s3/final.pt b/baseline_s3/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..ec7b7f900cd5322c965f58505aca4472ad11b878 --- /dev/null +++ b/baseline_s3/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22e76aba2e7217efe0857e02a66e135844cc511ee89d9cec34be1a6354b0f5ea +size 253572039 diff --git a/baseline_s3/metrics.jsonl b/baseline_s3/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..450c788a616664ef13626c504a2c4efe7e85f858 --- /dev/null +++ b/baseline_s3/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 3, "config": {"variant": "baseline", "seed": 3, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s3", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 3001346, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.911354064941406, "grad_norm": 13.482648849487305, "tokens_seen": 8192, "tokens_per_sec": 10437.12582208939, "vram_gb": 4.843584512, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.912246704101562, "grad_norm": 2.201179027557373, "tokens_seen": 172032, "tokens_per_sec": 23331.36755875326, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.160577774047852, "grad_norm": 1.8154380321502686, "tokens_seen": 335872, "tokens_per_sec": 24061.62390600899, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.219590187072754, "grad_norm": 1.622032642364502, "tokens_seen": 499712, "tokens_per_sec": 24277.181093923078, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.554261207580566, "grad_norm": 0.9493601322174072, "tokens_seen": 663552, "tokens_per_sec": 24356.08631835554, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.117005348205566, "grad_norm": 0.6969177722930908, "tokens_seen": 827392, "tokens_per_sec": 24290.56817882781, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.203577280044556, "grad_norm": 0.6757397055625916, "tokens_seen": 991232, "tokens_per_sec": 24370.40044816143, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.914400577545166, "grad_norm": 0.6661721467971802, "tokens_seen": 1155072, "tokens_per_sec": 24410.831738265133, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.861088752746582, "grad_norm": 0.6689569354057312, "tokens_seen": 1318912, "tokens_per_sec": 24452.435027918866, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.762676954269409, "grad_norm": 0.6935739517211914, "tokens_seen": 1482752, "tokens_per_sec": 24470.194067386616, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.692569017410278, "grad_norm": 0.6281939744949341, "tokens_seen": 1646592, "tokens_per_sec": 24497.388659378077, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.601470708847046, "grad_norm": 0.5605858564376831, "tokens_seen": 1810432, "tokens_per_sec": 24513.305873579124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.626667499542236, "grad_norm": 0.5443022847175598, "tokens_seen": 1974272, "tokens_per_sec": 24529.809488763116, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.643258333206177, "grad_norm": 0.5726404786109924, "tokens_seen": 2138112, "tokens_per_sec": 24543.2510660645, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.461520433425903, "grad_norm": 0.6336855292320251, "tokens_seen": 2301952, "tokens_per_sec": 24552.5668695706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.3438239097595215, "grad_norm": 0.5151188373565674, "tokens_seen": 2465792, "tokens_per_sec": 24557.760348065418, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.437052965164185, "grad_norm": 0.46169421076774597, "tokens_seen": 2629632, "tokens_per_sec": 24565.14436353469, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.500261068344116, "grad_norm": 0.5670393109321594, "tokens_seen": 2793472, "tokens_per_sec": 24570.05975498225, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.277272462844849, "grad_norm": 0.5822749733924866, "tokens_seen": 2957312, "tokens_per_sec": 24578.64724560488, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.312659740447998, "grad_norm": 0.5546301007270813, "tokens_seen": 3121152, "tokens_per_sec": 24583.570290726, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.385031223297119, "grad_norm": 0.533118486404419, "tokens_seen": 3284992, "tokens_per_sec": 24591.082105068883, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.295218467712402, "grad_norm": 0.49712073802948, "tokens_seen": 3448832, "tokens_per_sec": 24596.002232700044, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.243422269821167, "grad_norm": 0.6104999780654907, "tokens_seen": 3612672, "tokens_per_sec": 24602.222025325504, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.358970642089844, "grad_norm": 0.5457327365875244, "tokens_seen": 3776512, "tokens_per_sec": 24605.580640252152, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.2149834632873535, "grad_norm": 0.5126753449440002, "tokens_seen": 3940352, "tokens_per_sec": 24610.40717423977, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.2556116580963135, "grad_norm": 0.5277386903762817, "tokens_seen": 4104192, "tokens_per_sec": 24613.35934663441, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.2657647132873535, "grad_norm": 0.5442155003547668, "tokens_seen": 4268032, "tokens_per_sec": 24616.09869482885, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.172645807266235, "grad_norm": 0.5554518699645996, "tokens_seen": 4431872, "tokens_per_sec": 24617.39344518576, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.2168450355529785, "grad_norm": 0.5370852947235107, "tokens_seen": 4595712, "tokens_per_sec": 24615.493951192, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.194319725036621, "grad_norm": 0.5299283862113953, "tokens_seen": 4759552, "tokens_per_sec": 24614.83181677584, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.099829435348511, "grad_norm": 0.4799792170524597, "tokens_seen": 4923392, "tokens_per_sec": 24613.93644343279, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.01069188117981, "grad_norm": 0.5377359390258789, "tokens_seen": 5087232, "tokens_per_sec": 24615.401071863824, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.096529722213745, "grad_norm": 0.5443730354309082, "tokens_seen": 5251072, "tokens_per_sec": 24618.379791152758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.899020671844482, "grad_norm": 0.5155149102210999, "tokens_seen": 5414912, "tokens_per_sec": 24618.221733804014, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.025396108627319, "grad_norm": 0.5108160972595215, "tokens_seen": 5578752, "tokens_per_sec": 24618.097904527007, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 6.072484254837036, "grad_norm": 0.5106810331344604, "tokens_seen": 5742592, "tokens_per_sec": 24619.319531502875, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.8644118309021, "grad_norm": 0.5152288675308228, "tokens_seen": 5906432, "tokens_per_sec": 24619.548057769833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.048597097396851, "grad_norm": 0.5501119494438171, "tokens_seen": 6070272, "tokens_per_sec": 24621.560267241617, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.93516731262207, "grad_norm": 0.5626537203788757, "tokens_seen": 6234112, "tokens_per_sec": 24619.765113888865, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.013225078582764, "grad_norm": 0.517265260219574, "tokens_seen": 6397952, "tokens_per_sec": 24620.03013987326, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.936314582824707, "grad_norm": 0.5139294862747192, "tokens_seen": 6561792, "tokens_per_sec": 24620.73384089829, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.858428239822388, "grad_norm": 0.5512032508850098, "tokens_seen": 6725632, "tokens_per_sec": 24622.51165960205, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.885348320007324, "grad_norm": 0.5867959260940552, "tokens_seen": 6889472, "tokens_per_sec": 24619.38952063999, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.860715389251709, "grad_norm": 0.5553728342056274, "tokens_seen": 7053312, "tokens_per_sec": 24621.1150062594, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.96933650970459, "grad_norm": 0.5181498527526855, "tokens_seen": 7217152, "tokens_per_sec": 24621.407704140107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.7708306312561035, "grad_norm": 0.49878236651420593, "tokens_seen": 7380992, "tokens_per_sec": 24619.775167218006, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.798146963119507, "grad_norm": 0.5636610388755798, "tokens_seen": 7544832, "tokens_per_sec": 24620.077685063712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.6470787525177, "grad_norm": 0.511813223361969, "tokens_seen": 7708672, "tokens_per_sec": 24621.28926900483, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.817524194717407, "grad_norm": 0.5937578082084656, "tokens_seen": 7872512, "tokens_per_sec": 24622.758814790748, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.733757972717285, "grad_norm": 0.48853084444999695, "tokens_seen": 8036352, "tokens_per_sec": 24623.816189863905, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.906726360321045, "grad_norm": 0.6287133097648621, "tokens_seen": 8200192, "tokens_per_sec": 24625.899900208184, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.858997368812561, "val_ppl": 350.37267328213653, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.740155220031738, "grad_norm": 0.5097010731697083, "tokens_seen": 8364032, "tokens_per_sec": 24543.375588715848, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.648838043212891, "grad_norm": 0.508368730545044, "tokens_seen": 8527872, "tokens_per_sec": 24545.87454875705, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.625211000442505, "grad_norm": 0.5395720601081848, "tokens_seen": 8691712, "tokens_per_sec": 24547.58050609822, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.69926905632019, "grad_norm": 0.5290780067443848, "tokens_seen": 8855552, "tokens_per_sec": 24549.607887312577, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.664479970932007, "grad_norm": 0.5746238231658936, "tokens_seen": 9019392, "tokens_per_sec": 24550.826858573753, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.753530263900757, "grad_norm": 0.5309387445449829, "tokens_seen": 9183232, "tokens_per_sec": 24552.849401122017, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.64748477935791, "grad_norm": 0.5282455086708069, "tokens_seen": 9347072, "tokens_per_sec": 24555.000211730207, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.70552659034729, "grad_norm": 0.5719210505485535, "tokens_seen": 9510912, "tokens_per_sec": 24556.710222750095, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.632343053817749, "grad_norm": 0.539936900138855, "tokens_seen": 9674752, "tokens_per_sec": 24556.715949886, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.700811862945557, "grad_norm": 0.5910608768463135, "tokens_seen": 9838592, "tokens_per_sec": 24558.103630297614, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.508405447006226, "grad_norm": 0.5339952707290649, "tokens_seen": 10002432, "tokens_per_sec": 24558.79374165583, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.547796010971069, "grad_norm": 0.5311015844345093, "tokens_seen": 10166272, "tokens_per_sec": 24561.252375638545, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.7204062938690186, "grad_norm": 0.5439292788505554, "tokens_seen": 10330112, "tokens_per_sec": 24563.30574936643, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.568471670150757, "grad_norm": 0.5612390637397766, "tokens_seen": 10493952, "tokens_per_sec": 24563.902451480357, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.6690146923065186, "grad_norm": 0.5791947841644287, "tokens_seen": 10657792, "tokens_per_sec": 24563.23402140692, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.55771803855896, "grad_norm": 0.5815996527671814, "tokens_seen": 10821632, "tokens_per_sec": 24561.522951586885, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.62592887878418, "grad_norm": 0.5826106071472168, "tokens_seen": 10985472, "tokens_per_sec": 24563.86116596326, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.4661407470703125, "grad_norm": 0.5935617089271545, "tokens_seen": 11149312, "tokens_per_sec": 24565.355415828457, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.431966304779053, "grad_norm": 0.5928869247436523, "tokens_seen": 11313152, "tokens_per_sec": 24567.272962299747, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.6185479164123535, "grad_norm": 0.5954329967498779, "tokens_seen": 11476992, "tokens_per_sec": 24568.20375878196, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.488710641860962, "grad_norm": 0.5799027681350708, "tokens_seen": 11640832, "tokens_per_sec": 24568.909336862744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.37755823135376, "grad_norm": 0.5535475611686707, "tokens_seen": 11804672, "tokens_per_sec": 24570.43302528717, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.548089504241943, "grad_norm": 0.6309473514556885, "tokens_seen": 11968512, "tokens_per_sec": 24572.24933324366, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.3899009227752686, "grad_norm": 0.5760068297386169, "tokens_seen": 12132352, "tokens_per_sec": 24573.68955034672, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.50634503364563, "grad_norm": 0.5876681804656982, "tokens_seen": 12296192, "tokens_per_sec": 24574.159591425363, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.497336387634277, "grad_norm": 0.5913726091384888, "tokens_seen": 12460032, "tokens_per_sec": 24575.931698599587, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.363976955413818, "grad_norm": 0.6115069389343262, "tokens_seen": 12623872, "tokens_per_sec": 24576.457446612778, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.422857046127319, "grad_norm": 0.6043065786361694, "tokens_seen": 12787712, "tokens_per_sec": 24577.191759322428, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.499262094497681, "grad_norm": 0.5634514689445496, "tokens_seen": 12951552, "tokens_per_sec": 24578.808907344384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.276113271713257, "grad_norm": 0.5883635878562927, "tokens_seen": 13115392, "tokens_per_sec": 24580.153900913137, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.4775145053863525, "grad_norm": 0.5829293131828308, "tokens_seen": 13279232, "tokens_per_sec": 24581.917068806135, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.355524301528931, "grad_norm": 0.5868350863456726, "tokens_seen": 13443072, "tokens_per_sec": 24582.58500205892, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.377467155456543, "grad_norm": 0.6087629199028015, "tokens_seen": 13606912, "tokens_per_sec": 24583.253796011246, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.386178731918335, "grad_norm": 0.5726994872093201, "tokens_seen": 13770752, "tokens_per_sec": 24584.51986612256, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.543962001800537, "grad_norm": 0.6099837422370911, "tokens_seen": 13934592, "tokens_per_sec": 24585.604487821027, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.391173362731934, "grad_norm": 0.6272041201591492, "tokens_seen": 14098432, "tokens_per_sec": 24584.80716729353, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.340357780456543, "grad_norm": 0.5890907049179077, "tokens_seen": 14262272, "tokens_per_sec": 24585.48140928577, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.274436950683594, "grad_norm": 0.571284294128418, "tokens_seen": 14426112, "tokens_per_sec": 24586.251712628764, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.4244372844696045, "grad_norm": 0.6184350252151489, "tokens_seen": 14589952, "tokens_per_sec": 24587.88534668833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.331043243408203, "grad_norm": 0.5960714221000671, "tokens_seen": 14753792, "tokens_per_sec": 24589.01331688114, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.283771276473999, "grad_norm": 0.5860661864280701, "tokens_seen": 14917632, "tokens_per_sec": 24589.04644480325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.446109771728516, "grad_norm": 0.6219528913497925, "tokens_seen": 15081472, "tokens_per_sec": 24590.50626800951, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.40294075012207, "grad_norm": 0.6225642561912537, "tokens_seen": 15245312, "tokens_per_sec": 24591.10998307314, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.290568590164185, "grad_norm": 0.5779489874839783, "tokens_seen": 15409152, "tokens_per_sec": 24591.41629758746, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.218706369400024, "grad_norm": 0.6492091417312622, "tokens_seen": 15572992, "tokens_per_sec": 24592.397987718192, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.31547212600708, "grad_norm": 0.5943756699562073, "tokens_seen": 15736832, "tokens_per_sec": 24592.511879433365, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.43107533454895, "grad_norm": 0.6080855131149292, "tokens_seen": 15900672, "tokens_per_sec": 24592.72452089061, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.372580528259277, "grad_norm": 0.6229556798934937, "tokens_seen": 16064512, "tokens_per_sec": 24593.421602055474, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.106626749038696, "grad_norm": 0.6007239818572998, "tokens_seen": 16228352, "tokens_per_sec": 24593.97927698454, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.356024265289307, "grad_norm": 0.5595954656600952, "tokens_seen": 16392192, "tokens_per_sec": 24593.26181725513, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.364930629730225, "val_ppl": 213.77640443970952, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.384335279464722, "grad_norm": 0.6126500368118286, "tokens_seen": 16556032, "tokens_per_sec": 24551.291328975076, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.413780689239502, "grad_norm": 0.6348196864128113, "tokens_seen": 16719872, "tokens_per_sec": 24552.193375908057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.299625635147095, "grad_norm": 0.6069801449775696, "tokens_seen": 16883712, "tokens_per_sec": 24552.424597208046, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.210429906845093, "grad_norm": 0.610422670841217, "tokens_seen": 17047552, "tokens_per_sec": 24553.8836525472, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.301542282104492, "grad_norm": 0.5997354984283447, "tokens_seen": 17211392, "tokens_per_sec": 24554.356296588907, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.328405857086182, "grad_norm": 0.6045523285865784, "tokens_seen": 17375232, "tokens_per_sec": 24554.969934456127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.146477937698364, "grad_norm": 0.6171650290489197, "tokens_seen": 17539072, "tokens_per_sec": 24555.846521792406, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.124380111694336, "grad_norm": 0.632176399230957, "tokens_seen": 17702912, "tokens_per_sec": 24557.070816849442, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.173550844192505, "grad_norm": 0.6427507400512695, "tokens_seen": 17866752, "tokens_per_sec": 24557.897727795156, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.356361389160156, "grad_norm": 0.6485978960990906, "tokens_seen": 18030592, "tokens_per_sec": 24558.711004841836, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.275235176086426, "grad_norm": 0.6113370656967163, "tokens_seen": 18194432, "tokens_per_sec": 24559.308305663384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.266589879989624, "grad_norm": 0.6860294938087463, "tokens_seen": 18358272, "tokens_per_sec": 24560.111879863172, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.137717008590698, "grad_norm": 0.6383017897605896, "tokens_seen": 18522112, "tokens_per_sec": 24561.185124188134, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.231019735336304, "grad_norm": 0.8840989470481873, "tokens_seen": 18685952, "tokens_per_sec": 24562.04555107894, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.288254499435425, "grad_norm": 0.6287373900413513, "tokens_seen": 18849792, "tokens_per_sec": 24563.395610747306, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.191359996795654, "grad_norm": 0.616662859916687, "tokens_seen": 19013632, "tokens_per_sec": 24563.565046969175, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.006460189819336, "grad_norm": 0.6814315319061279, "tokens_seen": 19177472, "tokens_per_sec": 24563.50082956372, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.210414886474609, "grad_norm": 0.6572176814079285, "tokens_seen": 19341312, "tokens_per_sec": 24563.935382255782, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.150375604629517, "grad_norm": 0.6839007139205933, "tokens_seen": 19505152, "tokens_per_sec": 24564.628685692656, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.09917950630188, "grad_norm": 0.6085502505302429, "tokens_seen": 19668992, "tokens_per_sec": 24565.41436960541, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.126974105834961, "grad_norm": 0.6733583211898804, "tokens_seen": 19832832, "tokens_per_sec": 24566.50813273648, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.150729179382324, "grad_norm": 0.6412805318832397, "tokens_seen": 19996672, "tokens_per_sec": 24567.373988862873, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.0988500118255615, "grad_norm": 0.6312854290008545, "tokens_seen": 20160512, "tokens_per_sec": 24568.559923036377, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.113121747970581, "grad_norm": 0.6100379824638367, "tokens_seen": 20324352, "tokens_per_sec": 24569.403300128142, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.122089385986328, "grad_norm": 0.6065712571144104, "tokens_seen": 20488192, "tokens_per_sec": 24569.362306074967, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.235216856002808, "grad_norm": 0.6321358680725098, "tokens_seen": 20652032, "tokens_per_sec": 24570.553232143495, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.089504718780518, "grad_norm": 0.6114442348480225, "tokens_seen": 20815872, "tokens_per_sec": 24571.45393841359, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.199800729751587, "grad_norm": 0.7981336116790771, "tokens_seen": 20979712, "tokens_per_sec": 24572.067910986054, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.049329519271851, "grad_norm": 0.6358224749565125, "tokens_seen": 21143552, "tokens_per_sec": 24572.097563177107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.118076324462891, "grad_norm": 0.6359771490097046, "tokens_seen": 21307392, "tokens_per_sec": 24573.138329309742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.023731708526611, "grad_norm": 0.6685887575149536, "tokens_seen": 21471232, "tokens_per_sec": 24573.87703742057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 4.973085641860962, "grad_norm": 0.7845972776412964, "tokens_seen": 21635072, "tokens_per_sec": 24574.60927917194, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.09281587600708, "grad_norm": 0.6482133865356445, "tokens_seen": 21798912, "tokens_per_sec": 24575.265122912264, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.075612306594849, "grad_norm": 0.6603434085845947, "tokens_seen": 21962752, "tokens_per_sec": 24576.12960848495, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.061408519744873, "grad_norm": 0.6678443551063538, "tokens_seen": 22126592, "tokens_per_sec": 24577.172117462662, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.1273863315582275, "grad_norm": 0.7287176847457886, "tokens_seen": 22290432, "tokens_per_sec": 24577.30172231803, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.098883867263794, "grad_norm": 0.6660458445549011, "tokens_seen": 22454272, "tokens_per_sec": 24577.29721780396, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.118534564971924, "grad_norm": 0.7150636911392212, "tokens_seen": 22618112, "tokens_per_sec": 24577.504669688413, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.046951532363892, "grad_norm": 0.692216694355011, "tokens_seen": 22781952, "tokens_per_sec": 24577.50696942246, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.009326219558716, "grad_norm": 0.6900954246520996, "tokens_seen": 22945792, "tokens_per_sec": 24578.26630394778, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.181638479232788, "grad_norm": 0.7218508124351501, "tokens_seen": 23109632, "tokens_per_sec": 24579.214970019617, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.1084513664245605, "grad_norm": 0.6804895997047424, "tokens_seen": 23273472, "tokens_per_sec": 24579.76506738497, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.966926336288452, "grad_norm": 0.7022415995597839, "tokens_seen": 23437312, "tokens_per_sec": 24580.052967800584, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.01946234703064, "grad_norm": 0.6733329892158508, "tokens_seen": 23601152, "tokens_per_sec": 24580.904136371828, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.872828722000122, "grad_norm": 0.6934012770652771, "tokens_seen": 23764992, "tokens_per_sec": 24581.305570070428, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.996318817138672, "grad_norm": 0.6941967010498047, "tokens_seen": 23928832, "tokens_per_sec": 24582.45259881363, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.040229558944702, "grad_norm": 0.7026066184043884, "tokens_seen": 24092672, "tokens_per_sec": 24583.23309899854, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.870196580886841, "grad_norm": 0.708983302116394, "tokens_seen": 24256512, "tokens_per_sec": 24584.172626864936, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.803300142288208, "grad_norm": 1.0227357149124146, "tokens_seen": 24420352, "tokens_per_sec": 24584.520705582152, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.082240581512451, "grad_norm": 0.8084367513656616, "tokens_seen": 24584192, "tokens_per_sec": 24585.11537556129, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.039610362052917, "val_ppl": 154.40983937047102, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.1266937255859375, "grad_norm": 0.7193552255630493, "tokens_seen": 24748032, "tokens_per_sec": 24557.4324302804, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.858883619308472, "grad_norm": 0.6626811623573303, "tokens_seen": 24911872, "tokens_per_sec": 24557.97255509783, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.933661699295044, "grad_norm": 0.7541456818580627, "tokens_seen": 25075712, "tokens_per_sec": 24558.490225174693, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.8879780769348145, "grad_norm": 0.6917605400085449, "tokens_seen": 25239552, "tokens_per_sec": 24558.828997591063, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.040010690689087, "grad_norm": 0.7027620077133179, "tokens_seen": 25403392, "tokens_per_sec": 24559.32838225994, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.793644428253174, "grad_norm": 0.7073972821235657, "tokens_seen": 25567232, "tokens_per_sec": 24559.89394661877, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.9199583530426025, "grad_norm": 0.6837719082832336, "tokens_seen": 25731072, "tokens_per_sec": 24560.03926627573, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.870213985443115, "grad_norm": 0.7309454083442688, "tokens_seen": 25894912, "tokens_per_sec": 24560.942590715767, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.896177053451538, "grad_norm": 0.7543188333511353, "tokens_seen": 26058752, "tokens_per_sec": 24561.410689808083, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.867273569107056, "grad_norm": 0.710351288318634, "tokens_seen": 26222592, "tokens_per_sec": 24562.03071053837, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.968404531478882, "grad_norm": 0.7660538554191589, "tokens_seen": 26386432, "tokens_per_sec": 24562.90386095108, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.977293252944946, "grad_norm": 0.7286449074745178, "tokens_seen": 26550272, "tokens_per_sec": 24563.6868209927, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.007090330123901, "grad_norm": 0.6868886351585388, "tokens_seen": 26714112, "tokens_per_sec": 24564.386888654946, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.817754745483398, "grad_norm": 0.661026120185852, "tokens_seen": 26877952, "tokens_per_sec": 24564.825474363, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.07469367980957, "grad_norm": 0.6682150363922119, "tokens_seen": 27041792, "tokens_per_sec": 24565.440545589234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.928258180618286, "grad_norm": 0.6921682953834534, "tokens_seen": 27205632, "tokens_per_sec": 24565.66180752518, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.903677463531494, "grad_norm": 0.7522450685501099, "tokens_seen": 27369472, "tokens_per_sec": 24566.415677893758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.932016372680664, "grad_norm": 0.712076723575592, "tokens_seen": 27533312, "tokens_per_sec": 24566.846239682167, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.071798086166382, "grad_norm": 0.796807587146759, "tokens_seen": 27697152, "tokens_per_sec": 24567.13965608547, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.983360290527344, "grad_norm": 0.7490798234939575, "tokens_seen": 27860992, "tokens_per_sec": 24568.069877330017, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.9429261684417725, "grad_norm": 0.7616471648216248, "tokens_seen": 28024832, "tokens_per_sec": 24568.635145839566, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.728876829147339, "grad_norm": 0.6820592284202576, "tokens_seen": 28188672, "tokens_per_sec": 24569.601532801822, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.946894407272339, "grad_norm": 0.698785126209259, "tokens_seen": 28352512, "tokens_per_sec": 24569.669009805293, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.970077037811279, "grad_norm": 0.7499350905418396, "tokens_seen": 28516352, "tokens_per_sec": 24570.385672012027, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 5.0810956954956055, "grad_norm": 0.7151134610176086, "tokens_seen": 28680192, "tokens_per_sec": 24570.108387752807, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.8384552001953125, "grad_norm": 0.7141432166099548, "tokens_seen": 28844032, "tokens_per_sec": 24570.874400663928, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.85482931137085, "grad_norm": 0.6847703456878662, "tokens_seen": 29007872, "tokens_per_sec": 24571.149153566515, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.921826362609863, "grad_norm": 0.7778295278549194, "tokens_seen": 29171712, "tokens_per_sec": 24571.72504872002, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.930681228637695, "grad_norm": 0.7069521546363831, "tokens_seen": 29335552, "tokens_per_sec": 24572.47938492163, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.909407615661621, "grad_norm": 0.7395805716514587, "tokens_seen": 29499392, "tokens_per_sec": 24571.58951864127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.8991289138793945, "grad_norm": 0.6911380290985107, "tokens_seen": 29663232, "tokens_per_sec": 24571.93516426082, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.805415153503418, "grad_norm": 0.718784511089325, "tokens_seen": 29827072, "tokens_per_sec": 24571.46319829821, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 5.021611213684082, "grad_norm": 0.7363243103027344, "tokens_seen": 29990912, "tokens_per_sec": 24571.86665415011, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.846915245056152, "grad_norm": 0.7205684781074524, "tokens_seen": 30154752, "tokens_per_sec": 24571.375211470364, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 5.013228416442871, "grad_norm": 0.747108519077301, "tokens_seen": 30318592, "tokens_per_sec": 24571.70166150392, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.973392486572266, "grad_norm": 0.775900661945343, "tokens_seen": 30482432, "tokens_per_sec": 24571.37550083772, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.885074853897095, "grad_norm": 0.753386378288269, "tokens_seen": 30646272, "tokens_per_sec": 24571.662380154405, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.818946123123169, "grad_norm": 0.8002954721450806, "tokens_seen": 30810112, "tokens_per_sec": 24571.716557412576, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.775094509124756, "grad_norm": 0.7605034708976746, "tokens_seen": 30973952, "tokens_per_sec": 24572.413257959466, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.918137550354004, "grad_norm": 0.7599372267723083, "tokens_seen": 31137792, "tokens_per_sec": 24572.82187352826, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 5.0340800285339355, "grad_norm": 0.7758492231369019, "tokens_seen": 31301632, "tokens_per_sec": 24573.46940010411, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.922229290008545, "grad_norm": 0.7765550017356873, "tokens_seen": 31465472, "tokens_per_sec": 24573.987134718893, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.886836051940918, "grad_norm": 0.7090150713920593, "tokens_seen": 31629312, "tokens_per_sec": 24574.49653629256, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.919006109237671, "grad_norm": 0.7274752259254456, "tokens_seen": 31793152, "tokens_per_sec": 24575.093687302866, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.901299238204956, "grad_norm": 0.7411814332008362, "tokens_seen": 31956992, "tokens_per_sec": 24575.413975839052, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.8880226612091064, "grad_norm": 0.7140030860900879, "tokens_seen": 32120832, "tokens_per_sec": 24576.254712854123, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.780137300491333, "grad_norm": 0.801620364189148, "tokens_seen": 32284672, "tokens_per_sec": 24576.939685803085, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.7614452838897705, "grad_norm": 0.8051900863647461, "tokens_seen": 32448512, "tokens_per_sec": 24577.241077112267, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.848931550979614, "grad_norm": 0.7349522709846497, "tokens_seen": 32612352, "tokens_per_sec": 24577.878958641075, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.781958103179932, "grad_norm": 0.7552201747894287, "tokens_seen": 32776192, "tokens_per_sec": 24577.927184002965, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.901021122932434, "val_ppl": 134.42697609384186, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.729276180267334, "grad_norm": 0.7616763114929199, "tokens_seen": 32940032, "tokens_per_sec": 24557.26158796053, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 5.1088645458221436, "grad_norm": 0.8247085213661194, "tokens_seen": 33103872, "tokens_per_sec": 24557.063938749558, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.7706708908081055, "grad_norm": 0.7887023091316223, "tokens_seen": 33267712, "tokens_per_sec": 24557.26453623688, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.927911996841431, "grad_norm": 0.842589795589447, "tokens_seen": 33431552, "tokens_per_sec": 24557.632305730207, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.843879222869873, "grad_norm": 0.7709522843360901, "tokens_seen": 33595392, "tokens_per_sec": 24557.66483578389, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.810997247695923, "grad_norm": 0.7564613819122314, "tokens_seen": 33759232, "tokens_per_sec": 24557.71931702345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.831172466278076, "grad_norm": 0.7140018343925476, "tokens_seen": 33923072, "tokens_per_sec": 24558.25446500231, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.772277355194092, "grad_norm": 0.8433713912963867, "tokens_seen": 34086912, "tokens_per_sec": 24558.57388502772, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.757306337356567, "grad_norm": 0.7313010096549988, "tokens_seen": 34250752, "tokens_per_sec": 24558.87033996291, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.855717182159424, "grad_norm": 0.830576479434967, "tokens_seen": 34414592, "tokens_per_sec": 24559.47059184794, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.685349464416504, "grad_norm": 0.7814226150512695, "tokens_seen": 34578432, "tokens_per_sec": 24559.50420417344, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.8711326122283936, "grad_norm": 0.8229392170906067, "tokens_seen": 34742272, "tokens_per_sec": 24559.869873873766, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.891014575958252, "grad_norm": 0.7617362141609192, "tokens_seen": 34906112, "tokens_per_sec": 24560.22397624177, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.795403957366943, "grad_norm": 0.7520343065261841, "tokens_seen": 35069952, "tokens_per_sec": 24560.422308585745, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.720175743103027, "grad_norm": 0.7287963032722473, "tokens_seen": 35233792, "tokens_per_sec": 24561.054975680017, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.772054195404053, "grad_norm": 0.7718043327331543, "tokens_seen": 35397632, "tokens_per_sec": 24561.52312592025, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.87743353843689, "grad_norm": 0.7975549101829529, "tokens_seen": 35561472, "tokens_per_sec": 24562.418683508113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.919607162475586, "grad_norm": 0.7883322834968567, "tokens_seen": 35725312, "tokens_per_sec": 24562.60309648427, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.793355226516724, "grad_norm": 0.7699375748634338, "tokens_seen": 35889152, "tokens_per_sec": 24562.721222998523, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.700901985168457, "grad_norm": 0.7571103572845459, "tokens_seen": 36052992, "tokens_per_sec": 24562.805472549193, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 5.015223979949951, "grad_norm": 0.7784684896469116, "tokens_seen": 36216832, "tokens_per_sec": 24563.18333467103, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.913760423660278, "grad_norm": 0.79611736536026, "tokens_seen": 36380672, "tokens_per_sec": 24563.774129800808, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.786235094070435, "grad_norm": 0.7560778260231018, "tokens_seen": 36544512, "tokens_per_sec": 24564.096413358227, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.983619689941406, "grad_norm": 0.7874147295951843, "tokens_seen": 36708352, "tokens_per_sec": 24564.285048174304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.921640157699585, "grad_norm": 0.835500180721283, "tokens_seen": 36872192, "tokens_per_sec": 24564.37305836662, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.777589559555054, "grad_norm": 0.8815386295318604, "tokens_seen": 37036032, "tokens_per_sec": 24564.503812013518, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.81678318977356, "grad_norm": 0.8002142906188965, "tokens_seen": 37199872, "tokens_per_sec": 24565.048761845, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.957895994186401, "grad_norm": 0.8981053829193115, "tokens_seen": 37363712, "tokens_per_sec": 24565.60141348231, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.821664810180664, "grad_norm": 0.7593609690666199, "tokens_seen": 37527552, "tokens_per_sec": 24565.846185615585, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.817573070526123, "grad_norm": 0.7521754503250122, "tokens_seen": 37691392, "tokens_per_sec": 24566.346627656316, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.76095175743103, "grad_norm": 0.8371963500976562, "tokens_seen": 37855232, "tokens_per_sec": 24566.754769656047, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.747479438781738, "grad_norm": 0.7579405903816223, "tokens_seen": 38019072, "tokens_per_sec": 24567.275436191216, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.947268962860107, "grad_norm": 0.7802871465682983, "tokens_seen": 38182912, "tokens_per_sec": 24568.004252407794, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.632295370101929, "grad_norm": 0.778892993927002, "tokens_seen": 38346752, "tokens_per_sec": 24568.324791474384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.649071216583252, "grad_norm": 0.7802520990371704, "tokens_seen": 38510592, "tokens_per_sec": 24567.994944131715, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.905259132385254, "grad_norm": 0.8112468719482422, "tokens_seen": 38674432, "tokens_per_sec": 24568.23196298904, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.663633823394775, "grad_norm": 0.8157180547714233, "tokens_seen": 38838272, "tokens_per_sec": 24568.6137254574, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.808084011077881, "grad_norm": 0.7628568410873413, "tokens_seen": 39002112, "tokens_per_sec": 24569.05402981369, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.762412071228027, "grad_norm": 0.7826682925224304, "tokens_seen": 39165952, "tokens_per_sec": 24569.464263167774, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.775581121444702, "grad_norm": 0.789571225643158, "tokens_seen": 39329792, "tokens_per_sec": 24569.843905602793, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.629353046417236, "grad_norm": 0.7909398674964905, "tokens_seen": 39493632, "tokens_per_sec": 24570.51498261937, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.742955684661865, "grad_norm": 0.7532638907432556, "tokens_seen": 39657472, "tokens_per_sec": 24571.198742815137, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.775170803070068, "grad_norm": 0.801179051399231, "tokens_seen": 39821312, "tokens_per_sec": 24571.35138162864, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.783709764480591, "grad_norm": 0.8067519068717957, "tokens_seen": 39985152, "tokens_per_sec": 24571.708334232735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.585752487182617, "grad_norm": 0.8029886484146118, "tokens_seen": 40148992, "tokens_per_sec": 24572.29452860168, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.802632808685303, "grad_norm": 0.8031827211380005, "tokens_seen": 40312832, "tokens_per_sec": 24572.844484638317, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.673274993896484, "grad_norm": 0.7778631448745728, "tokens_seen": 40476672, "tokens_per_sec": 24572.976553839526, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.741719007492065, "grad_norm": 0.8235094547271729, "tokens_seen": 40640512, "tokens_per_sec": 24573.43507917196, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.702023506164551, "grad_norm": 0.7626457810401917, "tokens_seen": 40804352, "tokens_per_sec": 24573.968891605175, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.590241193771362, "grad_norm": 0.8674400448799133, "tokens_seen": 40968192, "tokens_per_sec": 24574.212025991736, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.787466144561767, "val_ppl": 119.99692825288234, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.763078689575195, "grad_norm": 0.810836136341095, "tokens_seen": 41132032, "tokens_per_sec": 24557.199203088596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.625031232833862, "grad_norm": 0.8005406260490417, "tokens_seen": 41295872, "tokens_per_sec": 24557.55686982568, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.6735875606536865, "grad_norm": 0.9289167523384094, "tokens_seen": 41459712, "tokens_per_sec": 24557.65643631219, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.7325358390808105, "grad_norm": 0.8205482959747314, "tokens_seen": 41623552, "tokens_per_sec": 24557.996722391446, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.632418155670166, "grad_norm": 0.8715790510177612, "tokens_seen": 41787392, "tokens_per_sec": 24558.274199766744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.748739242553711, "grad_norm": 0.7504488229751587, "tokens_seen": 41951232, "tokens_per_sec": 24558.93237026931, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.67062520980835, "grad_norm": 0.8609955906867981, "tokens_seen": 42115072, "tokens_per_sec": 24559.304115678606, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.784565687179565, "grad_norm": 0.7770110368728638, "tokens_seen": 42278912, "tokens_per_sec": 24559.24061233966, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.518381834030151, "grad_norm": 0.8082246780395508, "tokens_seen": 42442752, "tokens_per_sec": 24559.6982864762, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.5518639087677, "grad_norm": 0.8400576710700989, "tokens_seen": 42606592, "tokens_per_sec": 24560.23058486865, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.7531445026397705, "grad_norm": 0.7900371551513672, "tokens_seen": 42770432, "tokens_per_sec": 24560.451903694953, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.570917129516602, "grad_norm": 0.8358391523361206, "tokens_seen": 42934272, "tokens_per_sec": 24560.88966698255, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.715034008026123, "grad_norm": 0.8336214423179626, "tokens_seen": 43098112, "tokens_per_sec": 24561.356478439648, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.552292585372925, "grad_norm": 0.8090770244598389, "tokens_seen": 43261952, "tokens_per_sec": 24561.49826880304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.752288818359375, "grad_norm": 0.8824611902236938, "tokens_seen": 43425792, "tokens_per_sec": 24561.563573961346, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.605779409408569, "grad_norm": 0.8006929755210876, "tokens_seen": 43589632, "tokens_per_sec": 24561.68946238101, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.583779335021973, "grad_norm": 0.7925569415092468, "tokens_seen": 43753472, "tokens_per_sec": 24561.970698363657, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.626313209533691, "grad_norm": 0.8268255591392517, "tokens_seen": 43917312, "tokens_per_sec": 24561.946190503102, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.745260000228882, "grad_norm": 0.819965660572052, "tokens_seen": 44081152, "tokens_per_sec": 24561.748834544585, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.676152229309082, "grad_norm": 0.8126147389411926, "tokens_seen": 44244992, "tokens_per_sec": 24562.104811461573, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.8282530307769775, "grad_norm": 0.8495223522186279, "tokens_seen": 44408832, "tokens_per_sec": 24562.502349083614, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.546512126922607, "grad_norm": 0.8177617788314819, "tokens_seen": 44572672, "tokens_per_sec": 24562.94545006536, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.599451541900635, "grad_norm": 0.7927690148353577, "tokens_seen": 44736512, "tokens_per_sec": 24563.28277160061, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.588232517242432, "grad_norm": 0.7914522290229797, "tokens_seen": 44900352, "tokens_per_sec": 24563.76080635308, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.790547847747803, "grad_norm": 0.9544269442558289, "tokens_seen": 45064192, "tokens_per_sec": 24561.137965360784, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.802496671676636, "grad_norm": 0.8767352104187012, "tokens_seen": 45228032, "tokens_per_sec": 24561.540598911724, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.688390016555786, "grad_norm": 0.9283465147018433, "tokens_seen": 45391872, "tokens_per_sec": 24561.2855383336, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.7110984325408936, "grad_norm": 0.9917014241218567, "tokens_seen": 45555712, "tokens_per_sec": 24561.75821680117, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.546030521392822, "grad_norm": 0.8396859765052795, "tokens_seen": 45719552, "tokens_per_sec": 24562.08751819736, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.733628749847412, "grad_norm": 0.9262025356292725, "tokens_seen": 45883392, "tokens_per_sec": 24562.72831447707, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.621158123016357, "grad_norm": 0.878244161605835, "tokens_seen": 46047232, "tokens_per_sec": 24562.952121591596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.60165548324585, "grad_norm": 0.8861499428749084, "tokens_seen": 46211072, "tokens_per_sec": 24563.509141049977, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.656599998474121, "grad_norm": 0.8419275283813477, "tokens_seen": 46374912, "tokens_per_sec": 24563.94195968645, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.761638879776001, "grad_norm": 0.8412174582481384, "tokens_seen": 46538752, "tokens_per_sec": 24564.324622731423, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.643034219741821, "grad_norm": 0.8530125021934509, "tokens_seen": 46702592, "tokens_per_sec": 24564.692031905925, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.631712198257446, "grad_norm": 0.8150501847267151, "tokens_seen": 46866432, "tokens_per_sec": 24564.830654370886, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.651925563812256, "grad_norm": 0.8890335559844971, "tokens_seen": 47030272, "tokens_per_sec": 24564.878167034472, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.574819803237915, "grad_norm": 0.7858060598373413, "tokens_seen": 47194112, "tokens_per_sec": 24565.206078460178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.666507005691528, "grad_norm": 0.8522087335586548, "tokens_seen": 47357952, "tokens_per_sec": 24565.85718548863, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.657425165176392, "grad_norm": 0.8416123986244202, "tokens_seen": 47521792, "tokens_per_sec": 24566.277601405385, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.483811378479004, "grad_norm": 0.8692891001701355, "tokens_seen": 47685632, "tokens_per_sec": 24566.70343769053, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.5635082721710205, "grad_norm": 0.837816596031189, "tokens_seen": 47849472, "tokens_per_sec": 24567.199370961833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.503920078277588, "grad_norm": 0.8261966705322266, "tokens_seen": 48013312, "tokens_per_sec": 24567.51589762182, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.494473695755005, "grad_norm": 0.8461592793464661, "tokens_seen": 48177152, "tokens_per_sec": 24567.809125691758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.551688194274902, "grad_norm": 0.8957417011260986, "tokens_seen": 48340992, "tokens_per_sec": 24568.102474748706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.549750804901123, "grad_norm": 0.8344432711601257, "tokens_seen": 48504832, "tokens_per_sec": 24568.09125756895, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.699535608291626, "grad_norm": 0.8270236849784851, "tokens_seen": 48668672, "tokens_per_sec": 24568.50865745045, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.629924297332764, "grad_norm": 0.8236134648323059, "tokens_seen": 48832512, "tokens_per_sec": 24569.044962165128, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.473614454269409, "grad_norm": 0.8784738183021545, "tokens_seen": 48996352, "tokens_per_sec": 24569.32049335496, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.697652101516724, "grad_norm": 0.8447684645652771, "tokens_seen": 49160192, "tokens_per_sec": 24569.81084962473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.733228063583374, "val_ppl": 113.66187855554257, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.790809869766235, "grad_norm": 0.8601894974708557, "tokens_seen": 49324032, "tokens_per_sec": 24555.8539196157, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.577046632766724, "grad_norm": 0.8182151913642883, "tokens_seen": 49487872, "tokens_per_sec": 24555.82552651628, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.504237413406372, "grad_norm": 0.8855736255645752, "tokens_seen": 49651712, "tokens_per_sec": 24556.256351577154, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.537166595458984, "grad_norm": 0.7914032936096191, "tokens_seen": 49815552, "tokens_per_sec": 24556.693904991072, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.674764156341553, "grad_norm": 0.8376719355583191, "tokens_seen": 49979392, "tokens_per_sec": 24557.024250927592, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.699067115783691, "grad_norm": 0.8981291055679321, "tokens_seen": 50143232, "tokens_per_sec": 24557.500567081148, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.652570486068726, "grad_norm": 0.8484107851982117, "tokens_seen": 50307072, "tokens_per_sec": 24558.056061794203, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.534645080566406, "grad_norm": 0.8474447131156921, "tokens_seen": 50470912, "tokens_per_sec": 24558.309265479395, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.607831001281738, "grad_norm": 0.8846114277839661, "tokens_seen": 50634752, "tokens_per_sec": 24558.699336591744, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.577167510986328, "grad_norm": 1.0577880144119263, "tokens_seen": 50798592, "tokens_per_sec": 24559.305782217143, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.596758842468262, "grad_norm": 0.8844712376594543, "tokens_seen": 50962432, "tokens_per_sec": 24559.45249805995, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.688304424285889, "grad_norm": 0.814367413520813, "tokens_seen": 51126272, "tokens_per_sec": 24559.85603434454, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.576444864273071, "grad_norm": 0.8269041776657104, "tokens_seen": 51290112, "tokens_per_sec": 24560.353425919333, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.560689449310303, "grad_norm": 0.8673679828643799, "tokens_seen": 51453952, "tokens_per_sec": 24560.806675975018, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.681925058364868, "grad_norm": 0.8144948482513428, "tokens_seen": 51617792, "tokens_per_sec": 24561.235743844336, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.527637004852295, "grad_norm": 0.8780878186225891, "tokens_seen": 51781632, "tokens_per_sec": 24561.56352132761, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.649170398712158, "grad_norm": 0.9051763415336609, "tokens_seen": 51945472, "tokens_per_sec": 24562.074116032578, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.775933742523193, "grad_norm": 0.9472296237945557, "tokens_seen": 52109312, "tokens_per_sec": 24562.56591642596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.317969799041748, "grad_norm": 0.8681222796440125, "tokens_seen": 52273152, "tokens_per_sec": 24563.05016514351, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.544894218444824, "grad_norm": 0.8114728927612305, "tokens_seen": 52436992, "tokens_per_sec": 24563.45667485813, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.488154411315918, "grad_norm": 0.8798807263374329, "tokens_seen": 52600832, "tokens_per_sec": 24563.900953337612, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.655445098876953, "grad_norm": 0.8372685313224792, "tokens_seen": 52764672, "tokens_per_sec": 24564.078003665003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.527412414550781, "grad_norm": 0.8967610597610474, "tokens_seen": 52928512, "tokens_per_sec": 24564.651644853548, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.564213275909424, "grad_norm": 0.9795519113540649, "tokens_seen": 53092352, "tokens_per_sec": 24565.16826574268, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.508893728256226, "grad_norm": 0.8228693604469299, "tokens_seen": 53256192, "tokens_per_sec": 24565.681191819323, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.620414733886719, "grad_norm": 0.8856927156448364, "tokens_seen": 53420032, "tokens_per_sec": 24566.080779349933, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.533585786819458, "grad_norm": 0.8526291251182556, "tokens_seen": 53583872, "tokens_per_sec": 24566.416708944915, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.678381443023682, "grad_norm": 0.8541836738586426, "tokens_seen": 53747712, "tokens_per_sec": 24566.56599690519, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.614883661270142, "grad_norm": 0.8962562680244446, "tokens_seen": 53911552, "tokens_per_sec": 24566.807627307207, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.515305995941162, "grad_norm": 0.8826895952224731, "tokens_seen": 54075392, "tokens_per_sec": 24567.197903994325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.547547101974487, "grad_norm": 0.9766806364059448, "tokens_seen": 54239232, "tokens_per_sec": 24566.92184904675, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.415295600891113, "grad_norm": 0.9247039556503296, "tokens_seen": 54403072, "tokens_per_sec": 24567.348813975794, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.4638991355896, "grad_norm": 0.8287384510040283, "tokens_seen": 54566912, "tokens_per_sec": 24567.770457951705, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.487519025802612, "grad_norm": 0.908710241317749, "tokens_seen": 54730752, "tokens_per_sec": 24568.023336636918, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.796737909317017, "grad_norm": 0.959037721157074, "tokens_seen": 54894592, "tokens_per_sec": 24568.50941103346, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.442226886749268, "grad_norm": 0.8769466876983643, "tokens_seen": 55058432, "tokens_per_sec": 24569.029232584464, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.488398551940918, "grad_norm": 0.8771674036979675, "tokens_seen": 55222272, "tokens_per_sec": 24569.3352889783, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.590463399887085, "grad_norm": 0.9472689032554626, "tokens_seen": 55386112, "tokens_per_sec": 24569.84778931509, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.548926115036011, "grad_norm": 0.822056770324707, "tokens_seen": 55549952, "tokens_per_sec": 24570.114470440538, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.706373691558838, "grad_norm": 0.9413663744926453, "tokens_seen": 55713792, "tokens_per_sec": 24570.637593555388, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.596577882766724, "grad_norm": 0.9236465692520142, "tokens_seen": 55877632, "tokens_per_sec": 24571.033058091285, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.58313512802124, "grad_norm": 0.9463653564453125, "tokens_seen": 56041472, "tokens_per_sec": 24571.498697891824, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.580459356307983, "grad_norm": 0.915077805519104, "tokens_seen": 56205312, "tokens_per_sec": 24571.947244021776, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.578424453735352, "grad_norm": 0.9038037061691284, "tokens_seen": 56369152, "tokens_per_sec": 24572.34522005968, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.566767692565918, "grad_norm": 0.886100709438324, "tokens_seen": 56532992, "tokens_per_sec": 24572.793737200234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.495244026184082, "grad_norm": 0.9324176907539368, "tokens_seen": 56696832, "tokens_per_sec": 24572.999025466343, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.388635158538818, "grad_norm": 0.8160023093223572, "tokens_seen": 56860672, "tokens_per_sec": 24573.358106886288, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.452684164047241, "grad_norm": 0.8766239881515503, "tokens_seen": 57024512, "tokens_per_sec": 24573.646719555818, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.628742933273315, "grad_norm": 0.984987735748291, "tokens_seen": 57188352, "tokens_per_sec": 24573.94543223884, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.41638970375061, "grad_norm": 0.8323292136192322, "tokens_seen": 57352192, "tokens_per_sec": 24574.450904342884, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.625931262969971, "val_ppl": 102.09780873282851, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.506452560424805, "grad_norm": 0.8402889966964722, "tokens_seen": 57516032, "tokens_per_sec": 24562.34716831953, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.487674951553345, "grad_norm": 0.9215455055236816, "tokens_seen": 57679872, "tokens_per_sec": 24562.800366587362, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.555610656738281, "grad_norm": 0.9379121661186218, "tokens_seen": 57843712, "tokens_per_sec": 24563.045591694954, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.49553656578064, "grad_norm": 0.9733635783195496, "tokens_seen": 58007552, "tokens_per_sec": 24563.403054570106, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.491103410720825, "grad_norm": 0.868748128414154, "tokens_seen": 58171392, "tokens_per_sec": 24563.44465521381, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.622875213623047, "grad_norm": 0.9154259562492371, "tokens_seen": 58335232, "tokens_per_sec": 24563.74073334446, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.5255091190338135, "grad_norm": 0.9620804786682129, "tokens_seen": 58499072, "tokens_per_sec": 24563.713033697088, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.690763473510742, "grad_norm": 0.9206262826919556, "tokens_seen": 58662912, "tokens_per_sec": 24563.908483141775, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.508254051208496, "grad_norm": 0.8641611933708191, "tokens_seen": 58826752, "tokens_per_sec": 24564.274629622556, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.424942255020142, "grad_norm": 0.8692377209663391, "tokens_seen": 58990592, "tokens_per_sec": 24564.28577929742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.456017732620239, "grad_norm": 0.9022359848022461, "tokens_seen": 59154432, "tokens_per_sec": 24564.776330911733, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.5890796184539795, "grad_norm": 0.8638601899147034, "tokens_seen": 59318272, "tokens_per_sec": 24565.294561349296, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.421114444732666, "grad_norm": 1.0057135820388794, "tokens_seen": 59482112, "tokens_per_sec": 24565.429475716926, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.391459941864014, "grad_norm": 0.8580179810523987, "tokens_seen": 59645952, "tokens_per_sec": 24565.931281772308, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.6424455642700195, "grad_norm": 0.9342143535614014, "tokens_seen": 59809792, "tokens_per_sec": 24566.323423443748, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.648534774780273, "grad_norm": 0.9438406229019165, "tokens_seen": 59973632, "tokens_per_sec": 24566.557475705966, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.446889638900757, "grad_norm": 0.8946565985679626, "tokens_seen": 60137472, "tokens_per_sec": 24567.04182298766, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.426235198974609, "grad_norm": 0.9043591618537903, "tokens_seen": 60301312, "tokens_per_sec": 24567.459999080616, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.474580764770508, "grad_norm": 0.8958013653755188, "tokens_seen": 60465152, "tokens_per_sec": 24567.530692893793, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.604887962341309, "grad_norm": 0.9010693430900574, "tokens_seen": 60628992, "tokens_per_sec": 24567.901170621983, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.4649951457977295, "grad_norm": 0.9003762006759644, "tokens_seen": 60792832, "tokens_per_sec": 24568.361512992604, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.524059772491455, "grad_norm": 0.9438246488571167, "tokens_seen": 60956672, "tokens_per_sec": 24568.534143683853, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.599240779876709, "grad_norm": 0.9208192825317383, "tokens_seen": 61120512, "tokens_per_sec": 24568.96919288214, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.526740550994873, "grad_norm": 0.925715982913971, "tokens_seen": 61284352, "tokens_per_sec": 24569.387516486528, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.59163236618042, "grad_norm": 0.9019239544868469, "tokens_seen": 61448192, "tokens_per_sec": 24569.805754913927, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.405539512634277, "grad_norm": 0.8919208645820618, "tokens_seen": 61612032, "tokens_per_sec": 24570.10135269682, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.5050153732299805, "grad_norm": 0.9169324040412903, "tokens_seen": 61775872, "tokens_per_sec": 24570.50361829599, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.538411855697632, "grad_norm": 0.8688304424285889, "tokens_seen": 61939712, "tokens_per_sec": 24570.698088853198, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.467194557189941, "grad_norm": 0.8813076615333557, "tokens_seen": 62103552, "tokens_per_sec": 24571.067922597937, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.4122068881988525, "grad_norm": 0.9776194095611572, "tokens_seen": 62267392, "tokens_per_sec": 24571.38430857861, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.324151992797852, "grad_norm": 0.9568731188774109, "tokens_seen": 62431232, "tokens_per_sec": 24571.65019293072, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.526385307312012, "grad_norm": 0.9245277047157288, "tokens_seen": 62595072, "tokens_per_sec": 24572.014486251006, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.427597522735596, "grad_norm": 0.9759580492973328, "tokens_seen": 62758912, "tokens_per_sec": 24572.234814521857, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.52930760383606, "grad_norm": 0.928102970123291, "tokens_seen": 62922752, "tokens_per_sec": 24572.638804889873, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.4679694175720215, "grad_norm": 0.9256324768066406, "tokens_seen": 63086592, "tokens_per_sec": 24573.155400721505, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.363451242446899, "grad_norm": 0.9401569962501526, "tokens_seen": 63250432, "tokens_per_sec": 24573.42835031423, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.353309392929077, "grad_norm": 0.9154642820358276, "tokens_seen": 63414272, "tokens_per_sec": 24573.559388452817, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.438617467880249, "grad_norm": 0.9463784098625183, "tokens_seen": 63578112, "tokens_per_sec": 24573.723826659058, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.4695725440979, "grad_norm": 0.8829452991485596, "tokens_seen": 63741952, "tokens_per_sec": 24574.09821860222, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.44088077545166, "grad_norm": 0.9814178347587585, "tokens_seen": 63905792, "tokens_per_sec": 24574.357347906178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.55776572227478, "grad_norm": 0.9231176376342773, "tokens_seen": 64069632, "tokens_per_sec": 24574.85595578725, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.491929769515991, "grad_norm": 0.9494239091873169, "tokens_seen": 64233472, "tokens_per_sec": 24575.278667171933, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.25283670425415, "grad_norm": 0.8984648585319519, "tokens_seen": 64397312, "tokens_per_sec": 24575.370592084386, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.551907062530518, "grad_norm": 1.0185041427612305, "tokens_seen": 64561152, "tokens_per_sec": 24575.713072410428, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.553531646728516, "grad_norm": 1.0278171300888062, "tokens_seen": 64724992, "tokens_per_sec": 24576.084459535923, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.340596437454224, "grad_norm": 0.9629780650138855, "tokens_seen": 64888832, "tokens_per_sec": 24576.499989952714, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.4963765144348145, "grad_norm": 0.9379900693893433, "tokens_seen": 65052672, "tokens_per_sec": 24576.992293233587, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.548729419708252, "grad_norm": 0.9439013004302979, "tokens_seen": 65216512, "tokens_per_sec": 24577.17211048557, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.428126573562622, "grad_norm": 0.9188264012336731, "tokens_seen": 65380352, "tokens_per_sec": 24577.55032132127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.489837884902954, "grad_norm": 0.8990498185157776, "tokens_seen": 65544192, "tokens_per_sec": 24577.872110752178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.512760353088379, "val_ppl": 91.17314154363078, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.5364344120025635, "grad_norm": 0.9689050912857056, "tokens_seen": 65708032, "tokens_per_sec": 24567.475732044986, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.553670406341553, "grad_norm": 0.9564383625984192, "tokens_seen": 65871872, "tokens_per_sec": 24567.90020702093, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.508331298828125, "grad_norm": 0.9477137923240662, "tokens_seen": 66035712, "tokens_per_sec": 24568.25205163791, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.416638135910034, "grad_norm": 0.9998306632041931, "tokens_seen": 66199552, "tokens_per_sec": 24568.533920787093, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.315126419067383, "grad_norm": 0.9497788548469543, "tokens_seen": 66363392, "tokens_per_sec": 24568.633071730885, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.735356092453003, "grad_norm": 1.0171719789505005, "tokens_seen": 66527232, "tokens_per_sec": 24568.99542043894, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.4247705936431885, "grad_norm": 0.9617504477500916, "tokens_seen": 66691072, "tokens_per_sec": 24569.388469338657, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.408440113067627, "grad_norm": 0.9161290526390076, "tokens_seen": 66854912, "tokens_per_sec": 24569.792697753597, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.505300283432007, "grad_norm": 1.0500593185424805, "tokens_seen": 67018752, "tokens_per_sec": 24570.178254339233, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.462691783905029, "grad_norm": 0.9232648015022278, "tokens_seen": 67182592, "tokens_per_sec": 24570.154626055933, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.4796953201293945, "grad_norm": 1.018170952796936, "tokens_seen": 67346432, "tokens_per_sec": 24570.44065895952, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.605698108673096, "grad_norm": 0.986452043056488, "tokens_seen": 67510272, "tokens_per_sec": 24570.785439574123, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.438554763793945, "grad_norm": 0.9131793975830078, "tokens_seen": 67674112, "tokens_per_sec": 24571.159865600952, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.392441749572754, "grad_norm": 0.9334810376167297, "tokens_seen": 67837952, "tokens_per_sec": 24571.19023118076, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.420666694641113, "grad_norm": 0.9711953401565552, "tokens_seen": 68001792, "tokens_per_sec": 24571.41567731276, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.39618706703186, "grad_norm": 0.9970296025276184, "tokens_seen": 68165632, "tokens_per_sec": 24571.71059342083, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.482784271240234, "grad_norm": 1.0020406246185303, "tokens_seen": 68329472, "tokens_per_sec": 24571.861530827962, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.3427770137786865, "grad_norm": 0.9957478642463684, "tokens_seen": 68493312, "tokens_per_sec": 24572.141739594656, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.424437999725342, "grad_norm": 0.9583022594451904, "tokens_seen": 68657152, "tokens_per_sec": 24572.50376283785, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.498407363891602, "grad_norm": 1.0104141235351562, "tokens_seen": 68820992, "tokens_per_sec": 24572.875724572532, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.478360176086426, "grad_norm": 1.0074046850204468, "tokens_seen": 68984832, "tokens_per_sec": 24573.256665921992, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.433752536773682, "grad_norm": 0.9514246582984924, "tokens_seen": 69148672, "tokens_per_sec": 24573.633173740534, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.509211778640747, "grad_norm": 1.0067046880722046, "tokens_seen": 69312512, "tokens_per_sec": 24573.969679975486, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.4628541469573975, "grad_norm": 1.0969009399414062, "tokens_seen": 69476352, "tokens_per_sec": 24574.360346622583, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.429898977279663, "grad_norm": 1.0400490760803223, "tokens_seen": 69640192, "tokens_per_sec": 24574.795798571384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.437654256820679, "grad_norm": 1.0150710344314575, "tokens_seen": 69804032, "tokens_per_sec": 24575.036639203227, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.423808813095093, "grad_norm": 0.9071522951126099, "tokens_seen": 69967872, "tokens_per_sec": 24575.405914486546, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.397938966751099, "grad_norm": 1.0629574060440063, "tokens_seen": 70131712, "tokens_per_sec": 24575.8680172774, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.511873245239258, "grad_norm": 1.0573651790618896, "tokens_seen": 70295552, "tokens_per_sec": 24575.628585697497, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.344486236572266, "grad_norm": 0.9742872714996338, "tokens_seen": 70459392, "tokens_per_sec": 24576.04741054882, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.449488401412964, "grad_norm": 0.9860809445381165, "tokens_seen": 70623232, "tokens_per_sec": 24576.339130705528, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.462501764297485, "grad_norm": 0.9509527683258057, "tokens_seen": 70787072, "tokens_per_sec": 24576.496592234907, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.479590177536011, "grad_norm": 0.9628742337226868, "tokens_seen": 70950912, "tokens_per_sec": 24576.75084085644, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.49006986618042, "grad_norm": 0.9376156330108643, "tokens_seen": 71114752, "tokens_per_sec": 24576.968274049013, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.443716764450073, "grad_norm": 0.9596404433250427, "tokens_seen": 71278592, "tokens_per_sec": 24576.997826551094, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.462041854858398, "grad_norm": 1.0019396543502808, "tokens_seen": 71442432, "tokens_per_sec": 24577.007013252252, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.372870445251465, "grad_norm": 0.9841032028198242, "tokens_seen": 71606272, "tokens_per_sec": 24577.095880693596, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.454787015914917, "grad_norm": 0.9796621799468994, "tokens_seen": 71770112, "tokens_per_sec": 24577.35488913034, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.294104337692261, "grad_norm": 0.9585615992546082, "tokens_seen": 71933952, "tokens_per_sec": 24577.45778366099, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.478723764419556, "grad_norm": 1.0348397493362427, "tokens_seen": 72097792, "tokens_per_sec": 24577.64222870128, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.514003276824951, "grad_norm": 0.9657214283943176, "tokens_seen": 72261632, "tokens_per_sec": 24577.85797815232, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.452703475952148, "grad_norm": 0.981033205986023, "tokens_seen": 72425472, "tokens_per_sec": 24578.002753223536, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.36250638961792, "grad_norm": 0.9247066378593445, "tokens_seen": 72589312, "tokens_per_sec": 24578.21302287852, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.565526962280273, "grad_norm": 1.001090407371521, "tokens_seen": 72753152, "tokens_per_sec": 24578.307822778468, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.433549165725708, "grad_norm": 0.9579930901527405, "tokens_seen": 72916992, "tokens_per_sec": 24578.562488703927, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.436217546463013, "grad_norm": 1.0393757820129395, "tokens_seen": 73080832, "tokens_per_sec": 24578.66609661397, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.287868976593018, "grad_norm": 0.9980840682983398, "tokens_seen": 73244672, "tokens_per_sec": 24578.809711612746, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.454008102416992, "grad_norm": 0.9532633423805237, "tokens_seen": 73408512, "tokens_per_sec": 24578.909209186484, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.462407112121582, "grad_norm": 1.1248537302017212, "tokens_seen": 73572352, "tokens_per_sec": 24579.203256483735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.5518128871917725, "grad_norm": 0.9771884679794312, "tokens_seen": 73736192, "tokens_per_sec": 24579.34393369325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.48013961315155, "val_ppl": 88.24699225634747, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.355995178222656, "grad_norm": 1.03940749168396, "tokens_seen": 73900032, "tokens_per_sec": 24569.920702494106, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.464417934417725, "grad_norm": 0.9754484295845032, "tokens_seen": 74063872, "tokens_per_sec": 24570.11119900552, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.23694634437561, "grad_norm": 0.9975242614746094, "tokens_seen": 74227712, "tokens_per_sec": 24570.277014636995, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.41987419128418, "grad_norm": 1.014875054359436, "tokens_seen": 74391552, "tokens_per_sec": 24570.47746888323, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.51497483253479, "grad_norm": 1.019769310951233, "tokens_seen": 74555392, "tokens_per_sec": 24570.631303423168, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.486588954925537, "grad_norm": 0.9327249526977539, "tokens_seen": 74719232, "tokens_per_sec": 24570.841001260247, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.266944885253906, "grad_norm": 0.9822763204574585, "tokens_seen": 74883072, "tokens_per_sec": 24570.94389981745, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.562884569168091, "grad_norm": 1.201082468032837, "tokens_seen": 75046912, "tokens_per_sec": 24571.230351423656, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.34697151184082, "grad_norm": 0.9719794392585754, "tokens_seen": 75210752, "tokens_per_sec": 24571.396876137915, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.4363086223602295, "grad_norm": 0.9846988320350647, "tokens_seen": 75374592, "tokens_per_sec": 24571.279673851157, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.40809965133667, "grad_norm": 1.0666612386703491, "tokens_seen": 75538432, "tokens_per_sec": 24571.47057156435, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.423817873001099, "grad_norm": 1.0891042947769165, "tokens_seen": 75702272, "tokens_per_sec": 24571.536112400612, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.434835433959961, "grad_norm": 1.0474587678909302, "tokens_seen": 75866112, "tokens_per_sec": 24571.276258887858, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.43110990524292, "grad_norm": 1.0011317729949951, "tokens_seen": 76029952, "tokens_per_sec": 24571.385794578575, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.318530797958374, "grad_norm": 1.2036466598510742, "tokens_seen": 76193792, "tokens_per_sec": 24571.469701386563, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.543054580688477, "grad_norm": 1.0885818004608154, "tokens_seen": 76357632, "tokens_per_sec": 24571.574104453524, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.5564117431640625, "grad_norm": 1.075007677078247, "tokens_seen": 76521472, "tokens_per_sec": 24571.627646264355, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.377203464508057, "grad_norm": 1.09181547164917, "tokens_seen": 76685312, "tokens_per_sec": 24571.66897396027, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.444385766983032, "grad_norm": 1.0126854181289673, "tokens_seen": 76849152, "tokens_per_sec": 24571.869726200297, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.418699741363525, "grad_norm": 1.092335820198059, "tokens_seen": 77012992, "tokens_per_sec": 24572.04513520822, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.40367317199707, "grad_norm": 1.1012828350067139, "tokens_seen": 77176832, "tokens_per_sec": 24572.23415521559, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.283282041549683, "grad_norm": 1.0052032470703125, "tokens_seen": 77340672, "tokens_per_sec": 24572.23797468154, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.506150484085083, "grad_norm": 1.0618816614151, "tokens_seen": 77504512, "tokens_per_sec": 24572.377520390626, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.561147451400757, "grad_norm": 1.155683159828186, "tokens_seen": 77668352, "tokens_per_sec": 24572.626372268624, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.3066487312316895, "grad_norm": 1.0219534635543823, "tokens_seen": 77832192, "tokens_per_sec": 24572.65380712584, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.424508094787598, "grad_norm": 1.0704668760299683, "tokens_seen": 77996032, "tokens_per_sec": 24572.706227151826, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.379008769989014, "grad_norm": 0.9958764314651489, "tokens_seen": 78159872, "tokens_per_sec": 24572.86936710925, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.318497180938721, "grad_norm": 0.9532411098480225, "tokens_seen": 78323712, "tokens_per_sec": 24572.96315071982, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.402504920959473, "grad_norm": 1.0696626901626587, "tokens_seen": 78487552, "tokens_per_sec": 24573.09294967797, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.6060967445373535, "grad_norm": 1.0690884590148926, "tokens_seen": 78651392, "tokens_per_sec": 24573.371146532987, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.432697296142578, "grad_norm": 1.0961546897888184, "tokens_seen": 78815232, "tokens_per_sec": 24573.473259546223, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.268503189086914, "grad_norm": 1.0847116708755493, "tokens_seen": 78979072, "tokens_per_sec": 24573.603275941525, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.508608818054199, "grad_norm": 1.027641773223877, "tokens_seen": 79142912, "tokens_per_sec": 24573.899600564197, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.422302961349487, "grad_norm": 1.0198968648910522, "tokens_seen": 79306752, "tokens_per_sec": 24574.121072785416, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.279837131500244, "grad_norm": 1.0011730194091797, "tokens_seen": 79470592, "tokens_per_sec": 24574.357104482016, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.347539901733398, "grad_norm": 0.9777579307556152, "tokens_seen": 79634432, "tokens_per_sec": 24574.495053739585, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.336123943328857, "grad_norm": 1.077436089515686, "tokens_seen": 79798272, "tokens_per_sec": 24574.685868582197, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.409180164337158, "grad_norm": 1.0340721607208252, "tokens_seen": 79962112, "tokens_per_sec": 24574.951789256203, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.331556797027588, "grad_norm": 0.9924361109733582, "tokens_seen": 80125952, "tokens_per_sec": 24575.15785018518, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.347098112106323, "grad_norm": 1.1186200380325317, "tokens_seen": 80289792, "tokens_per_sec": 24575.456101698244, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.321405410766602, "grad_norm": 1.0777034759521484, "tokens_seen": 80453632, "tokens_per_sec": 24575.671492783433, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.341282606124878, "grad_norm": 0.9728928804397583, "tokens_seen": 80617472, "tokens_per_sec": 24575.763570747647, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.334009170532227, "grad_norm": 1.039110541343689, "tokens_seen": 80781312, "tokens_per_sec": 24575.956216089944, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.266481399536133, "grad_norm": 1.0231553316116333, "tokens_seen": 80945152, "tokens_per_sec": 24576.15685862248, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.3550872802734375, "grad_norm": 1.0504813194274902, "tokens_seen": 81108992, "tokens_per_sec": 24576.359601988046, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.356899261474609, "grad_norm": 1.015514612197876, "tokens_seen": 81272832, "tokens_per_sec": 24576.55903465354, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.323024034500122, "grad_norm": 1.1579875946044922, "tokens_seen": 81436672, "tokens_per_sec": 24576.767712326888, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.506969451904297, "grad_norm": 1.2170027494430542, "tokens_seen": 81600512, "tokens_per_sec": 24577.05879602932, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.269843578338623, "grad_norm": 1.1013047695159912, "tokens_seen": 81764352, "tokens_per_sec": 24577.23702792686, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.412874937057495, "grad_norm": 1.048855185508728, "tokens_seen": 81928192, "tokens_per_sec": 24577.37277104742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.477726316452026, "val_ppl": 88.03428284988154, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.2730231285095215, "grad_norm": 1.1502470970153809, "tokens_seen": 82092032, "tokens_per_sec": 24569.028604425992, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.373356342315674, "grad_norm": 1.0664398670196533, "tokens_seen": 82255872, "tokens_per_sec": 24569.27101937382, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.265395879745483, "grad_norm": 1.0210407972335815, "tokens_seen": 82419712, "tokens_per_sec": 24569.430044664885, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.210334062576294, "grad_norm": 0.9724225997924805, "tokens_seen": 82583552, "tokens_per_sec": 24569.54647148169, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.519334316253662, "grad_norm": 1.099232792854309, "tokens_seen": 82747392, "tokens_per_sec": 24569.780256495247, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.416517972946167, "grad_norm": 1.032089114189148, "tokens_seen": 82911232, "tokens_per_sec": 24569.940946541818, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.291411638259888, "grad_norm": 1.030493140220642, "tokens_seen": 83075072, "tokens_per_sec": 24570.109184178313, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.421454429626465, "grad_norm": 1.0746334791183472, "tokens_seen": 83238912, "tokens_per_sec": 24570.192690613723, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.517526388168335, "grad_norm": 1.0745725631713867, "tokens_seen": 83402752, "tokens_per_sec": 24570.450358658607, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.50605034828186, "grad_norm": 1.052222728729248, "tokens_seen": 83566592, "tokens_per_sec": 24570.636936053124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.414307117462158, "grad_norm": 1.1146736145019531, "tokens_seen": 83730432, "tokens_per_sec": 24570.886648371805, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.2718589305877686, "grad_norm": 1.068208932876587, "tokens_seen": 83894272, "tokens_per_sec": 24570.74267139766, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.4186155796051025, "grad_norm": 1.1450669765472412, "tokens_seen": 84058112, "tokens_per_sec": 24570.86120483375, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.434090614318848, "grad_norm": 1.155576229095459, "tokens_seen": 84221952, "tokens_per_sec": 24571.128911596235, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.314829349517822, "grad_norm": 1.1069746017456055, "tokens_seen": 84385792, "tokens_per_sec": 24571.281319828187, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.454233169555664, "grad_norm": 1.0230902433395386, "tokens_seen": 84549632, "tokens_per_sec": 24571.39689972003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.267475843429565, "grad_norm": 1.035740613937378, "tokens_seen": 84713472, "tokens_per_sec": 24571.643950409678, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.307044982910156, "grad_norm": 1.0165340900421143, "tokens_seen": 84877312, "tokens_per_sec": 24571.793585915646, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.447336912155151, "grad_norm": 1.0408896207809448, "tokens_seen": 85041152, "tokens_per_sec": 24572.046885956006, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.4064483642578125, "grad_norm": 1.0747730731964111, "tokens_seen": 85204992, "tokens_per_sec": 24572.258117543533, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.250371932983398, "grad_norm": 0.981550931930542, "tokens_seen": 85368832, "tokens_per_sec": 24572.481890815237, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.258545875549316, "grad_norm": 1.0724118947982788, "tokens_seen": 85532672, "tokens_per_sec": 24572.635517172654, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.464334726333618, "grad_norm": 1.2869890928268433, "tokens_seen": 85696512, "tokens_per_sec": 24572.82809642403, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.582787990570068, "grad_norm": 1.1194325685501099, "tokens_seen": 85860352, "tokens_per_sec": 24573.12871903952, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.414665222167969, "grad_norm": 1.19082772731781, "tokens_seen": 86024192, "tokens_per_sec": 24573.359012506633, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.316178798675537, "grad_norm": 1.0919349193572998, "tokens_seen": 86188032, "tokens_per_sec": 24573.232715131016, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.2827277183532715, "grad_norm": 1.0508360862731934, "tokens_seen": 86351872, "tokens_per_sec": 24573.447746013004, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.246007204055786, "grad_norm": 1.1070396900177002, "tokens_seen": 86515712, "tokens_per_sec": 24573.726549240004, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.444232940673828, "grad_norm": 1.0348299741744995, "tokens_seen": 86679552, "tokens_per_sec": 24573.84249742064, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.25981593132019, "grad_norm": 1.031789779663086, "tokens_seen": 86843392, "tokens_per_sec": 24574.090255810344, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.414460182189941, "grad_norm": 1.0743526220321655, "tokens_seen": 87007232, "tokens_per_sec": 24574.222575355183, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.27017068862915, "grad_norm": 1.08877432346344, "tokens_seen": 87171072, "tokens_per_sec": 24574.281506183695, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.342546224594116, "grad_norm": 1.1709848642349243, "tokens_seen": 87334912, "tokens_per_sec": 24574.42995818739, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.414527416229248, "grad_norm": 1.1752973794937134, "tokens_seen": 87498752, "tokens_per_sec": 24574.503334352255, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.423760175704956, "grad_norm": 1.078330397605896, "tokens_seen": 87662592, "tokens_per_sec": 24574.68929634471, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.322376012802124, "grad_norm": 1.0072120428085327, "tokens_seen": 87826432, "tokens_per_sec": 24574.783212727238, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.266982793807983, "grad_norm": 1.0353261232376099, "tokens_seen": 87990272, "tokens_per_sec": 24574.930733771133, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.3646159172058105, "grad_norm": 1.088854432106018, "tokens_seen": 88154112, "tokens_per_sec": 24574.973489198863, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.3552470207214355, "grad_norm": 1.1298619508743286, "tokens_seen": 88317952, "tokens_per_sec": 24575.162604224322, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.391925811767578, "grad_norm": 1.104914903640747, "tokens_seen": 88481792, "tokens_per_sec": 24575.03444754704, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.3324408531188965, "grad_norm": 1.1653573513031006, "tokens_seen": 88645632, "tokens_per_sec": 24574.982130481254, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.489473342895508, "grad_norm": 1.2048283815383911, "tokens_seen": 88809472, "tokens_per_sec": 24575.249210591308, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.433444261550903, "grad_norm": 1.2023210525512695, "tokens_seen": 88973312, "tokens_per_sec": 24575.447706030434, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.437352418899536, "grad_norm": 1.0574326515197754, "tokens_seen": 89137152, "tokens_per_sec": 24575.590577540468, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.294679641723633, "grad_norm": 1.102941870689392, "tokens_seen": 89300992, "tokens_per_sec": 24574.861194203117, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.17605447769165, "grad_norm": 1.0927637815475464, "tokens_seen": 89464832, "tokens_per_sec": 24575.011387067076, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.442110538482666, "grad_norm": 1.045587182044983, "tokens_seen": 89628672, "tokens_per_sec": 24575.19509411979, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.365128755569458, "grad_norm": 1.1398160457611084, "tokens_seen": 89792512, "tokens_per_sec": 24575.38006906625, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.3493828773498535, "grad_norm": 1.2844008207321167, "tokens_seen": 89956352, "tokens_per_sec": 24575.384187701653, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.340261697769165, "grad_norm": 1.1772379875183105, "tokens_seen": 90120192, "tokens_per_sec": 24575.57390603182, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.389680480957031, "val_ppl": 80.61465694692906, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.296415328979492, "grad_norm": 1.0684317350387573, "tokens_seen": 90284032, "tokens_per_sec": 24568.062720429545, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.305863857269287, "grad_norm": 1.0368210077285767, "tokens_seen": 90447872, "tokens_per_sec": 24568.274962548367, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.413646697998047, "grad_norm": 1.0707345008850098, "tokens_seen": 90611712, "tokens_per_sec": 24568.409750305393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.159250259399414, "grad_norm": 1.1308432817459106, "tokens_seen": 90775552, "tokens_per_sec": 24568.540942487634, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.392862796783447, "grad_norm": 1.309278130531311, "tokens_seen": 90939392, "tokens_per_sec": 24568.759922827358, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.323166847229004, "grad_norm": 1.00667405128479, "tokens_seen": 91103232, "tokens_per_sec": 24568.87307471712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.348700761795044, "grad_norm": 1.096488118171692, "tokens_seen": 91267072, "tokens_per_sec": 24569.07065659964, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.364522218704224, "grad_norm": 1.0241166353225708, "tokens_seen": 91430912, "tokens_per_sec": 24569.143249761903, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.317832946777344, "grad_norm": 1.1306363344192505, "tokens_seen": 91594752, "tokens_per_sec": 24569.31691870758, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.241659641265869, "grad_norm": 1.1130969524383545, "tokens_seen": 91758592, "tokens_per_sec": 24569.447948185058, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.289601564407349, "grad_norm": 1.1215360164642334, "tokens_seen": 91922432, "tokens_per_sec": 24569.630501796997, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.34674596786499, "grad_norm": 1.096208095550537, "tokens_seen": 92086272, "tokens_per_sec": 24569.85053862913, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.363373279571533, "grad_norm": 1.2648913860321045, "tokens_seen": 92250112, "tokens_per_sec": 24570.05379306127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.299260854721069, "grad_norm": 1.0890264511108398, "tokens_seen": 92413952, "tokens_per_sec": 24570.210428235383, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.297800064086914, "grad_norm": 1.133524775505066, "tokens_seen": 92577792, "tokens_per_sec": 24570.19537465379, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.319580078125, "grad_norm": 1.1310354471206665, "tokens_seen": 92741632, "tokens_per_sec": 24570.41178847567, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.394543647766113, "grad_norm": 1.2037456035614014, "tokens_seen": 92905472, "tokens_per_sec": 24570.60466212822, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.4945995807647705, "grad_norm": 1.1252998113632202, "tokens_seen": 93069312, "tokens_per_sec": 24570.60186241632, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.318876266479492, "grad_norm": 1.1509530544281006, "tokens_seen": 93233152, "tokens_per_sec": 24570.650940919215, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.403559923171997, "grad_norm": 1.1283400058746338, "tokens_seen": 93396992, "tokens_per_sec": 24570.691767229506, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.312460422515869, "grad_norm": 1.0611052513122559, "tokens_seen": 93560832, "tokens_per_sec": 24570.893613182754, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.345593452453613, "grad_norm": 1.062735915184021, "tokens_seen": 93724672, "tokens_per_sec": 24570.80173385501, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.380504846572876, "grad_norm": 1.1572434902191162, "tokens_seen": 93888512, "tokens_per_sec": 24571.018838100525, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.372962951660156, "grad_norm": 1.09675931930542, "tokens_seen": 94052352, "tokens_per_sec": 24571.18576185455, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.276216745376587, "grad_norm": 1.1638970375061035, "tokens_seen": 94216192, "tokens_per_sec": 24571.271232338197, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.186398029327393, "grad_norm": 1.205876350402832, "tokens_seen": 94380032, "tokens_per_sec": 24571.442903934083, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.445651054382324, "grad_norm": 1.0969171524047852, "tokens_seen": 94543872, "tokens_per_sec": 24571.506489214247, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.335277795791626, "grad_norm": 1.1692370176315308, "tokens_seen": 94707712, "tokens_per_sec": 24571.515317195648, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.206217288970947, "grad_norm": 1.080479621887207, "tokens_seen": 94871552, "tokens_per_sec": 24571.31448481123, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.341522932052612, "grad_norm": 1.0546525716781616, "tokens_seen": 95035392, "tokens_per_sec": 24571.383734313415, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.2916014194488525, "grad_norm": 1.2240389585494995, "tokens_seen": 95199232, "tokens_per_sec": 24571.61209354949, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.357154369354248, "grad_norm": 1.0608469247817993, "tokens_seen": 95363072, "tokens_per_sec": 24571.778259695926, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.43749475479126, "grad_norm": 1.2489664554595947, "tokens_seen": 95526912, "tokens_per_sec": 24571.65640380298, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.227008581161499, "grad_norm": 1.1480997800827026, "tokens_seen": 95690752, "tokens_per_sec": 24571.79871076992, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.3744847774505615, "grad_norm": 1.0507954359054565, "tokens_seen": 95854592, "tokens_per_sec": 24571.835854829482, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.287149667739868, "grad_norm": 1.1813939809799194, "tokens_seen": 96018432, "tokens_per_sec": 24572.0176141671, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.330986499786377, "grad_norm": 1.2213317155838013, "tokens_seen": 96182272, "tokens_per_sec": 24572.148058032602, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.33454704284668, "grad_norm": 1.1018885374069214, "tokens_seen": 96346112, "tokens_per_sec": 24572.314898166464, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.362759590148926, "grad_norm": 1.0808581113815308, "tokens_seen": 96509952, "tokens_per_sec": 24572.513310164497, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.331709623336792, "grad_norm": 1.0917547941207886, "tokens_seen": 96673792, "tokens_per_sec": 24572.57320328592, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.370408535003662, "grad_norm": 1.1933610439300537, "tokens_seen": 96837632, "tokens_per_sec": 24572.620907437755, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.260551452636719, "grad_norm": 1.2492669820785522, "tokens_seen": 97001472, "tokens_per_sec": 24572.815345851133, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.28389310836792, "grad_norm": 1.0801804065704346, "tokens_seen": 97165312, "tokens_per_sec": 24572.881551856448, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.477040767669678, "grad_norm": 1.1938647031784058, "tokens_seen": 97329152, "tokens_per_sec": 24572.826404040647, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.29252552986145, "grad_norm": 1.1950846910476685, "tokens_seen": 97492992, "tokens_per_sec": 24572.58838250502, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.286885976791382, "grad_norm": 1.1478155851364136, "tokens_seen": 97656832, "tokens_per_sec": 24572.705802692453, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.418955326080322, "grad_norm": 1.3432420492172241, "tokens_seen": 97820672, "tokens_per_sec": 24572.779808505158, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.269880533218384, "grad_norm": 1.0985333919525146, "tokens_seen": 97984512, "tokens_per_sec": 24572.73203937674, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.305428981781006, "grad_norm": 1.1749076843261719, "tokens_seen": 98148352, "tokens_per_sec": 24572.79203416852, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.374352741241455, "val_ppl": 79.38843805716219, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4001.651510953903, "nan_failures": 0} diff --git a/baseline_s4/config.json b/baseline_s4/config.json new file mode 100644 index 0000000000000000000000000000000000000000..5e59d8c40c4133b7271ba77c74a41c1b021064f1 --- /dev/null +++ b/baseline_s4/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 4, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s4", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 4001349, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/baseline_s4/final.pt b/baseline_s4/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..fe71f0576873250949a7fe3ef599ab200d3b24e2 --- /dev/null +++ b/baseline_s4/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1653ae95a80f8c998ce1f4d3e1a8c1e6466df0324a40c29b930f3c62f5b7887 +size 253572039 diff --git a/baseline_s4/metrics.jsonl b/baseline_s4/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..8d696a81feb57743fcc55bf785691cea38055a11 --- /dev/null +++ b/baseline_s4/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 4, "config": {"variant": "baseline", "seed": 4, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s4", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 4001349, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.919853210449219, "grad_norm": 14.713665962219238, "tokens_seen": 8192, "tokens_per_sec": 7714.034861147437, "vram_gb": 4.843584512, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.813322067260742, "grad_norm": 1.9636000394821167, "tokens_seen": 172032, "tokens_per_sec": 22784.157953176615, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.06106185913086, "grad_norm": 1.734129548072815, "tokens_seen": 335872, "tokens_per_sec": 23908.01087195187, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.178411483764648, "grad_norm": 1.4424699544906616, "tokens_seen": 499712, "tokens_per_sec": 24304.33692610345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.553223609924316, "grad_norm": 0.7029538154602051, "tokens_seen": 663552, "tokens_per_sec": 24488.61111395072, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.001829147338867, "grad_norm": 0.7597211003303528, "tokens_seen": 827392, "tokens_per_sec": 24586.591060564082, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.185313701629639, "grad_norm": 0.6737990379333496, "tokens_seen": 991232, "tokens_per_sec": 24651.195038967035, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.8237340450286865, "grad_norm": 0.8770545125007629, "tokens_seen": 1155072, "tokens_per_sec": 24689.07686387385, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.733876705169678, "grad_norm": 0.6634705066680908, "tokens_seen": 1318912, "tokens_per_sec": 24720.228292677493, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.758383750915527, "grad_norm": 0.6049861311912537, "tokens_seen": 1482752, "tokens_per_sec": 24741.081138483332, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.706985950469971, "grad_norm": 0.7008812427520752, "tokens_seen": 1646592, "tokens_per_sec": 24755.633991231563, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.746486663818359, "grad_norm": 0.6257993578910828, "tokens_seen": 1810432, "tokens_per_sec": 24761.60002147494, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.547783851623535, "grad_norm": 0.7400040626525879, "tokens_seen": 1974272, "tokens_per_sec": 24769.04301869308, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.622644424438477, "grad_norm": 0.5063258409500122, "tokens_seen": 2138112, "tokens_per_sec": 24772.368969444356, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.680788040161133, "grad_norm": 0.5846331119537354, "tokens_seen": 2301952, "tokens_per_sec": 24774.62867502793, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.356199741363525, "grad_norm": 0.5277341604232788, "tokens_seen": 2465792, "tokens_per_sec": 24774.94323380025, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.40461540222168, "grad_norm": 0.505806565284729, "tokens_seen": 2629632, "tokens_per_sec": 24777.75118970363, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.427757740020752, "grad_norm": 0.5032873749732971, "tokens_seen": 2793472, "tokens_per_sec": 24772.925629380446, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.481728553771973, "grad_norm": 0.5751906037330627, "tokens_seen": 2957312, "tokens_per_sec": 24774.829137608092, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.396223783493042, "grad_norm": 0.5539876818656921, "tokens_seen": 3121152, "tokens_per_sec": 24771.272888156156, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.291515827178955, "grad_norm": 0.5380405187606812, "tokens_seen": 3284992, "tokens_per_sec": 24771.196463706594, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.184622287750244, "grad_norm": 0.5782186985015869, "tokens_seen": 3448832, "tokens_per_sec": 24771.53113558764, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.2904746532440186, "grad_norm": 0.5852439999580383, "tokens_seen": 3612672, "tokens_per_sec": 24771.4498486476, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.295798301696777, "grad_norm": 0.4886191785335541, "tokens_seen": 3776512, "tokens_per_sec": 24772.696805104682, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.308142185211182, "grad_norm": 0.5627264380455017, "tokens_seen": 3940352, "tokens_per_sec": 24772.93955523164, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.231135368347168, "grad_norm": 0.5496675372123718, "tokens_seen": 4104192, "tokens_per_sec": 24772.103916134773, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.274666786193848, "grad_norm": 0.49551817774772644, "tokens_seen": 4268032, "tokens_per_sec": 24769.231681209243, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.088371753692627, "grad_norm": 0.5603948831558228, "tokens_seen": 4431872, "tokens_per_sec": 24759.64718448602, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.225209951400757, "grad_norm": 0.5953545570373535, "tokens_seen": 4595712, "tokens_per_sec": 24759.2715834542, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.078108310699463, "grad_norm": 0.4977032542228699, "tokens_seen": 4759552, "tokens_per_sec": 24758.709453062977, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.0251100063323975, "grad_norm": 0.5519063472747803, "tokens_seen": 4923392, "tokens_per_sec": 24756.48838101712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.134277820587158, "grad_norm": 0.5021079778671265, "tokens_seen": 5087232, "tokens_per_sec": 24755.485144610724, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.239884376525879, "grad_norm": 0.5641618371009827, "tokens_seen": 5251072, "tokens_per_sec": 24753.29356634549, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.07848596572876, "grad_norm": 0.556637704372406, "tokens_seen": 5414912, "tokens_per_sec": 24751.804906502173, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.080238103866577, "grad_norm": 0.5914859175682068, "tokens_seen": 5578752, "tokens_per_sec": 24751.321017350343, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.9757325649261475, "grad_norm": 0.5206144452095032, "tokens_seen": 5742592, "tokens_per_sec": 24749.378947031142, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.937537431716919, "grad_norm": 0.5344303250312805, "tokens_seen": 5906432, "tokens_per_sec": 24747.716296760933, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.043330430984497, "grad_norm": 0.5067342519760132, "tokens_seen": 6070272, "tokens_per_sec": 24744.3567346037, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.995749473571777, "grad_norm": 0.5241780281066895, "tokens_seen": 6234112, "tokens_per_sec": 24743.901938206105, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.14500093460083, "grad_norm": 0.6176931858062744, "tokens_seen": 6397952, "tokens_per_sec": 24742.10778580024, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.8629138469696045, "grad_norm": 0.49610278010368347, "tokens_seen": 6561792, "tokens_per_sec": 24740.332970006362, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.9215381145477295, "grad_norm": 0.5441201329231262, "tokens_seen": 6725632, "tokens_per_sec": 24739.503859726687, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.878593444824219, "grad_norm": 0.551176905632019, "tokens_seen": 6889472, "tokens_per_sec": 24739.246055349944, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.939048767089844, "grad_norm": 0.6078978776931763, "tokens_seen": 7053312, "tokens_per_sec": 24736.955303061608, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.782316207885742, "grad_norm": 0.5764644145965576, "tokens_seen": 7217152, "tokens_per_sec": 24734.521289082124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.779170989990234, "grad_norm": 0.5532328486442566, "tokens_seen": 7380992, "tokens_per_sec": 24732.045756708554, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.856397867202759, "grad_norm": 0.5551220178604126, "tokens_seen": 7544832, "tokens_per_sec": 24731.97005445273, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.762040615081787, "grad_norm": 0.5556927919387817, "tokens_seen": 7708672, "tokens_per_sec": 24730.79973163424, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.816256999969482, "grad_norm": 0.5469992756843567, "tokens_seen": 7872512, "tokens_per_sec": 24728.23754472765, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.819634914398193, "grad_norm": 0.6106510162353516, "tokens_seen": 8036352, "tokens_per_sec": 24726.629400059723, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.7347681522369385, "grad_norm": 0.5847804546356201, "tokens_seen": 8200192, "tokens_per_sec": 24726.27625583902, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.791070437431335, "val_ppl": 327.3632587763605, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.789685487747192, "grad_norm": 0.6921414732933044, "tokens_seen": 8364032, "tokens_per_sec": 24639.4537752746, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.731751203536987, "grad_norm": 0.5499544143676758, "tokens_seen": 8527872, "tokens_per_sec": 24638.87027539907, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.614630699157715, "grad_norm": 0.5113490223884583, "tokens_seen": 8691712, "tokens_per_sec": 24640.18919448201, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.669239282608032, "grad_norm": 0.5981350541114807, "tokens_seen": 8855552, "tokens_per_sec": 24640.415047864393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.837506055831909, "grad_norm": 0.5439339876174927, "tokens_seen": 9019392, "tokens_per_sec": 24641.1898451243, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.786453485488892, "grad_norm": 0.5292195081710815, "tokens_seen": 9183232, "tokens_per_sec": 24643.14876854124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.591692209243774, "grad_norm": 0.5566762685775757, "tokens_seen": 9347072, "tokens_per_sec": 24643.27381330393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.641799449920654, "grad_norm": 0.5523586273193359, "tokens_seen": 9510912, "tokens_per_sec": 24642.662756408485, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.718124866485596, "grad_norm": 0.5742684006690979, "tokens_seen": 9674752, "tokens_per_sec": 24641.588759821683, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.678699970245361, "grad_norm": 0.5357720255851746, "tokens_seen": 9838592, "tokens_per_sec": 24640.75759501457, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.907982587814331, "grad_norm": 0.5434350371360779, "tokens_seen": 10002432, "tokens_per_sec": 24640.388777639128, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.565268278121948, "grad_norm": 0.5526307225227356, "tokens_seen": 10166272, "tokens_per_sec": 24641.222131559553, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.574530124664307, "grad_norm": 0.5464513897895813, "tokens_seen": 10330112, "tokens_per_sec": 24640.317115108333, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.46287727355957, "grad_norm": 0.5332947969436646, "tokens_seen": 10493952, "tokens_per_sec": 24639.274960509996, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.775139570236206, "grad_norm": 0.5717770457267761, "tokens_seen": 10657792, "tokens_per_sec": 24639.322584816924, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.534456968307495, "grad_norm": 0.5673647522926331, "tokens_seen": 10821632, "tokens_per_sec": 24638.712026283807, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.500269889831543, "grad_norm": 0.5290024876594543, "tokens_seen": 10985472, "tokens_per_sec": 24638.541895080598, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.612955570220947, "grad_norm": 0.5719244480133057, "tokens_seen": 11149312, "tokens_per_sec": 24637.698567061234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.565749645233154, "grad_norm": 0.556207001209259, "tokens_seen": 11313152, "tokens_per_sec": 24636.738066531383, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.432851076126099, "grad_norm": 0.6072396039962769, "tokens_seen": 11476992, "tokens_per_sec": 24634.255814115793, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.553359270095825, "grad_norm": 0.551673412322998, "tokens_seen": 11640832, "tokens_per_sec": 24633.945755633926, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.504666090011597, "grad_norm": 0.6032030582427979, "tokens_seen": 11804672, "tokens_per_sec": 24632.77470361167, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.425481081008911, "grad_norm": 0.5566093325614929, "tokens_seen": 11968512, "tokens_per_sec": 24632.534986738992, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.453951358795166, "grad_norm": 0.5844210386276245, "tokens_seen": 12132352, "tokens_per_sec": 24632.36274976817, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.503625154495239, "grad_norm": 0.5656510591506958, "tokens_seen": 12296192, "tokens_per_sec": 24631.03160971293, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.34906005859375, "grad_norm": 0.5690260529518127, "tokens_seen": 12460032, "tokens_per_sec": 24631.269578040712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.431041240692139, "grad_norm": 0.5849127769470215, "tokens_seen": 12623872, "tokens_per_sec": 24631.019826620868, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.505039930343628, "grad_norm": 0.5752646327018738, "tokens_seen": 12787712, "tokens_per_sec": 24630.51650910472, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.579098701477051, "grad_norm": 0.6178063750267029, "tokens_seen": 12951552, "tokens_per_sec": 24629.94816557028, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.27016282081604, "grad_norm": 0.5935547351837158, "tokens_seen": 13115392, "tokens_per_sec": 24629.519614665063, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.402882814407349, "grad_norm": 0.579407811164856, "tokens_seen": 13279232, "tokens_per_sec": 24628.6425334508, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.429823875427246, "grad_norm": 0.628602921962738, "tokens_seen": 13443072, "tokens_per_sec": 24626.99355401265, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.379249095916748, "grad_norm": 0.5623547434806824, "tokens_seen": 13606912, "tokens_per_sec": 24626.460114850604, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.261050701141357, "grad_norm": 0.5780271887779236, "tokens_seen": 13770752, "tokens_per_sec": 24626.76514064375, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.325093746185303, "grad_norm": 0.5946807861328125, "tokens_seen": 13934592, "tokens_per_sec": 24626.75893828591, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.407270193099976, "grad_norm": 0.6017515063285828, "tokens_seen": 14098432, "tokens_per_sec": 24626.625694456605, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.502148389816284, "grad_norm": 0.6552979946136475, "tokens_seen": 14262272, "tokens_per_sec": 24627.583511421584, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.302916765213013, "grad_norm": 0.5928848385810852, "tokens_seen": 14426112, "tokens_per_sec": 24627.78802989108, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.473259210586548, "grad_norm": 0.612054169178009, "tokens_seen": 14589952, "tokens_per_sec": 24627.75354110503, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.209400415420532, "grad_norm": 0.5784200429916382, "tokens_seen": 14753792, "tokens_per_sec": 24628.564584487453, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.309912204742432, "grad_norm": 0.5729336142539978, "tokens_seen": 14917632, "tokens_per_sec": 24628.621756971057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.359092473983765, "grad_norm": 0.6461144685745239, "tokens_seen": 15081472, "tokens_per_sec": 24628.862241008082, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.556570768356323, "grad_norm": 0.5864977240562439, "tokens_seen": 15245312, "tokens_per_sec": 24629.04961657064, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.426908254623413, "grad_norm": 0.5912821888923645, "tokens_seen": 15409152, "tokens_per_sec": 24629.218115323805, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.338169097900391, "grad_norm": 0.6115378141403198, "tokens_seen": 15572992, "tokens_per_sec": 24628.943348869747, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.326625823974609, "grad_norm": 0.5650570392608643, "tokens_seen": 15736832, "tokens_per_sec": 24628.564768051692, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.341748476028442, "grad_norm": 0.5919605493545532, "tokens_seen": 15900672, "tokens_per_sec": 24628.63418392487, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.4522552490234375, "grad_norm": 0.5970034599304199, "tokens_seen": 16064512, "tokens_per_sec": 24628.287579266012, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.229365348815918, "grad_norm": 0.5779790878295898, "tokens_seen": 16228352, "tokens_per_sec": 24628.065580509792, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.281999111175537, "grad_norm": 0.6129874587059021, "tokens_seen": 16392192, "tokens_per_sec": 24627.530637650325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.336907768249512, "val_ppl": 207.86893652776732, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.235679864883423, "grad_norm": 0.6609030365943909, "tokens_seen": 16556032, "tokens_per_sec": 24584.591831690424, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.426510334014893, "grad_norm": 0.7887150049209595, "tokens_seen": 16719872, "tokens_per_sec": 24584.81554047177, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.3380255699157715, "grad_norm": 0.650937557220459, "tokens_seen": 16883712, "tokens_per_sec": 24585.375513420542, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.241187810897827, "grad_norm": 0.6095457673072815, "tokens_seen": 17047552, "tokens_per_sec": 24585.673004944645, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.143876075744629, "grad_norm": 0.6331862807273865, "tokens_seen": 17211392, "tokens_per_sec": 24585.507077801754, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.25458025932312, "grad_norm": 0.6338463425636292, "tokens_seen": 17375232, "tokens_per_sec": 24585.621378179392, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.212348699569702, "grad_norm": 0.6172888875007629, "tokens_seen": 17539072, "tokens_per_sec": 24585.85531690104, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.1908278465271, "grad_norm": 0.608995258808136, "tokens_seen": 17702912, "tokens_per_sec": 24585.46689385975, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.332590103149414, "grad_norm": 0.6553593277931213, "tokens_seen": 17866752, "tokens_per_sec": 24585.320657158412, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.223516464233398, "grad_norm": 0.6436971426010132, "tokens_seen": 18030592, "tokens_per_sec": 24585.24130682269, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.181731700897217, "grad_norm": 0.6065406799316406, "tokens_seen": 18194432, "tokens_per_sec": 24586.33899780772, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.106683969497681, "grad_norm": 0.6264036893844604, "tokens_seen": 18358272, "tokens_per_sec": 24586.718449634256, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.284675598144531, "grad_norm": 0.6721166968345642, "tokens_seen": 18522112, "tokens_per_sec": 24587.40709434034, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.175126552581787, "grad_norm": 0.6231140494346619, "tokens_seen": 18685952, "tokens_per_sec": 24587.53230750199, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.153388977050781, "grad_norm": 0.6181606650352478, "tokens_seen": 18849792, "tokens_per_sec": 24587.161071987233, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.033239841461182, "grad_norm": 0.6253473162651062, "tokens_seen": 19013632, "tokens_per_sec": 24587.47158864641, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.204245328903198, "grad_norm": 0.656116783618927, "tokens_seen": 19177472, "tokens_per_sec": 24587.585774335515, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 4.937016487121582, "grad_norm": 0.6626937389373779, "tokens_seen": 19341312, "tokens_per_sec": 24588.63187153467, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.117950916290283, "grad_norm": 0.6947903633117676, "tokens_seen": 19505152, "tokens_per_sec": 24589.102763926578, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.102142810821533, "grad_norm": 0.6428354978561401, "tokens_seen": 19668992, "tokens_per_sec": 24589.355144067464, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.0647406578063965, "grad_norm": 0.6427130103111267, "tokens_seen": 19832832, "tokens_per_sec": 24589.560008795495, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.1100664138793945, "grad_norm": 0.6488584876060486, "tokens_seen": 19996672, "tokens_per_sec": 24589.935689664715, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.079358816146851, "grad_norm": 0.6568612456321716, "tokens_seen": 20160512, "tokens_per_sec": 24590.50551933373, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.207922458648682, "grad_norm": 0.7456318736076355, "tokens_seen": 20324352, "tokens_per_sec": 24590.80873558809, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.130889654159546, "grad_norm": 0.6406971216201782, "tokens_seen": 20488192, "tokens_per_sec": 24591.01469900559, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.01992654800415, "grad_norm": 0.685389518737793, "tokens_seen": 20652032, "tokens_per_sec": 24591.117977292975, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.0462870597839355, "grad_norm": 0.6435150504112244, "tokens_seen": 20815872, "tokens_per_sec": 24590.858356539065, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.218005895614624, "grad_norm": 0.6381978392601013, "tokens_seen": 20979712, "tokens_per_sec": 24590.784477639172, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.087067127227783, "grad_norm": 0.6905254125595093, "tokens_seen": 21143552, "tokens_per_sec": 24590.813827969097, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.043694019317627, "grad_norm": 0.6337220072746277, "tokens_seen": 21307392, "tokens_per_sec": 24590.772478602248, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.092136383056641, "grad_norm": 0.6827443242073059, "tokens_seen": 21471232, "tokens_per_sec": 24590.786646425706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.089622974395752, "grad_norm": 0.6947107315063477, "tokens_seen": 21635072, "tokens_per_sec": 24589.913123011498, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.217426776885986, "grad_norm": 0.6615592241287231, "tokens_seen": 21798912, "tokens_per_sec": 24589.935651829757, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 4.931478261947632, "grad_norm": 0.7145560383796692, "tokens_seen": 21962752, "tokens_per_sec": 24589.725106838712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.19002890586853, "grad_norm": 0.6212631464004517, "tokens_seen": 22126592, "tokens_per_sec": 24589.80222360806, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.1509809494018555, "grad_norm": 0.6488533020019531, "tokens_seen": 22290432, "tokens_per_sec": 24589.626935653614, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.12502908706665, "grad_norm": 0.7277303338050842, "tokens_seen": 22454272, "tokens_per_sec": 24589.693794185798, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.044553518295288, "grad_norm": 0.6902080178260803, "tokens_seen": 22618112, "tokens_per_sec": 24589.70977841073, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.098550319671631, "grad_norm": 0.7357662916183472, "tokens_seen": 22781952, "tokens_per_sec": 24589.567799317738, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.016386985778809, "grad_norm": 0.736953854560852, "tokens_seen": 22945792, "tokens_per_sec": 24589.582456888438, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.968082666397095, "grad_norm": 0.705011785030365, "tokens_seen": 23109632, "tokens_per_sec": 24589.43156160633, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 4.946936845779419, "grad_norm": 0.7243330478668213, "tokens_seen": 23273472, "tokens_per_sec": 24588.98491481928, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.046282052993774, "grad_norm": 0.6479573845863342, "tokens_seen": 23437312, "tokens_per_sec": 24589.461114624944, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.02330756187439, "grad_norm": 0.6639623641967773, "tokens_seen": 23601152, "tokens_per_sec": 24589.426453099393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.037958145141602, "grad_norm": 0.7983595728874207, "tokens_seen": 23764992, "tokens_per_sec": 24589.338804527768, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.070118188858032, "grad_norm": 0.6730234622955322, "tokens_seen": 23928832, "tokens_per_sec": 24589.089959848254, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 4.998603582382202, "grad_norm": 0.6750176548957825, "tokens_seen": 24092672, "tokens_per_sec": 24589.70545910366, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.992451429367065, "grad_norm": 0.7067858576774597, "tokens_seen": 24256512, "tokens_per_sec": 24589.879687979006, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.0011279582977295, "grad_norm": 0.743629515171051, "tokens_seen": 24420352, "tokens_per_sec": 24589.792593193277, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.978675603866577, "grad_norm": 0.7236683964729309, "tokens_seen": 24584192, "tokens_per_sec": 24589.879067327467, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.006447863578797, "val_ppl": 149.37319868175624, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.906506061553955, "grad_norm": 0.7205712199211121, "tokens_seen": 24748032, "tokens_per_sec": 24562.17462961069, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.867074966430664, "grad_norm": 0.6897066831588745, "tokens_seen": 24911872, "tokens_per_sec": 24562.392559057123, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.146523475646973, "grad_norm": 0.7204432487487793, "tokens_seen": 25075712, "tokens_per_sec": 24563.30038583327, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 5.002178907394409, "grad_norm": 0.7138057351112366, "tokens_seen": 25239552, "tokens_per_sec": 24563.524805894554, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.035497665405273, "grad_norm": 0.7503541111946106, "tokens_seen": 25403392, "tokens_per_sec": 24563.42408171029, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.915507555007935, "grad_norm": 0.7041272521018982, "tokens_seen": 25567232, "tokens_per_sec": 24563.7766770044, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.916794061660767, "grad_norm": 0.7009994387626648, "tokens_seen": 25731072, "tokens_per_sec": 24564.018268287506, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 5.022599935531616, "grad_norm": 0.7136244177818298, "tokens_seen": 25894912, "tokens_per_sec": 24564.466389264067, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.915451526641846, "grad_norm": 0.7306773066520691, "tokens_seen": 26058752, "tokens_per_sec": 24564.6075484473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.950073003768921, "grad_norm": 0.7756032943725586, "tokens_seen": 26222592, "tokens_per_sec": 24564.902804497986, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.09894061088562, "grad_norm": 0.7513917684555054, "tokens_seen": 26386432, "tokens_per_sec": 24565.07894907729, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.961409568786621, "grad_norm": 0.7206085920333862, "tokens_seen": 26550272, "tokens_per_sec": 24564.962227082906, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.95234751701355, "grad_norm": 0.7028285264968872, "tokens_seen": 26714112, "tokens_per_sec": 24564.517763307314, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.877238512039185, "grad_norm": 0.7850756645202637, "tokens_seen": 26877952, "tokens_per_sec": 24565.02802680012, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.949707984924316, "grad_norm": 0.732530951499939, "tokens_seen": 27041792, "tokens_per_sec": 24565.420003094714, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.856924295425415, "grad_norm": 0.7209001183509827, "tokens_seen": 27205632, "tokens_per_sec": 24565.432756428592, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.970666408538818, "grad_norm": 0.7466726899147034, "tokens_seen": 27369472, "tokens_per_sec": 24565.669704367094, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.914111375808716, "grad_norm": 0.7447790503501892, "tokens_seen": 27533312, "tokens_per_sec": 24565.60228443609, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.837092399597168, "grad_norm": 0.6996043920516968, "tokens_seen": 27697152, "tokens_per_sec": 24565.405303101445, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.836347818374634, "grad_norm": 0.7649754285812378, "tokens_seen": 27860992, "tokens_per_sec": 24565.621639288536, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.838958263397217, "grad_norm": 0.6609510183334351, "tokens_seen": 28024832, "tokens_per_sec": 24565.895323156637, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.78399658203125, "grad_norm": 0.6956945657730103, "tokens_seen": 28188672, "tokens_per_sec": 24565.971689246526, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.795168876647949, "grad_norm": 0.6812328100204468, "tokens_seen": 28352512, "tokens_per_sec": 24566.119687275124, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 5.034050703048706, "grad_norm": 0.6940694451332092, "tokens_seen": 28516352, "tokens_per_sec": 24566.330738504737, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.9443440437316895, "grad_norm": 0.7381407618522644, "tokens_seen": 28680192, "tokens_per_sec": 24566.35638745876, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.7855095863342285, "grad_norm": 0.7312657237052917, "tokens_seen": 28844032, "tokens_per_sec": 24566.352772139653, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 5.0520007610321045, "grad_norm": 0.7594785690307617, "tokens_seen": 29007872, "tokens_per_sec": 24566.434107885736, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.925135850906372, "grad_norm": 0.7178319692611694, "tokens_seen": 29171712, "tokens_per_sec": 24566.15055841833, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 5.156211614608765, "grad_norm": 0.7625604867935181, "tokens_seen": 29335552, "tokens_per_sec": 24566.639436173213, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.914743185043335, "grad_norm": 0.7473546862602234, "tokens_seen": 29499392, "tokens_per_sec": 24566.23187355519, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.749342918395996, "grad_norm": 0.7892408967018127, "tokens_seen": 29663232, "tokens_per_sec": 24566.81418221164, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.894644737243652, "grad_norm": 0.7163547873497009, "tokens_seen": 29827072, "tokens_per_sec": 24566.9912516498, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.819216728210449, "grad_norm": 1.0548133850097656, "tokens_seen": 29990912, "tokens_per_sec": 24567.172784682578, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.90562891960144, "grad_norm": 0.8112969994544983, "tokens_seen": 30154752, "tokens_per_sec": 24567.28344489423, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.890588760375977, "grad_norm": 0.8119442462921143, "tokens_seen": 30318592, "tokens_per_sec": 24568.048202216425, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.809032201766968, "grad_norm": 0.760934591293335, "tokens_seen": 30482432, "tokens_per_sec": 24568.20987770078, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.654073715209961, "grad_norm": 0.7287163138389587, "tokens_seen": 30646272, "tokens_per_sec": 24568.31180938042, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.799718856811523, "grad_norm": 0.7323764562606812, "tokens_seen": 30810112, "tokens_per_sec": 24568.25918217051, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.773254871368408, "grad_norm": 0.7515669465065002, "tokens_seen": 30973952, "tokens_per_sec": 24568.643162665092, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.849020957946777, "grad_norm": 0.7382162809371948, "tokens_seen": 31137792, "tokens_per_sec": 24569.043779026924, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.811038017272949, "grad_norm": 0.7446809411048889, "tokens_seen": 31301632, "tokens_per_sec": 24569.13403196862, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.625561237335205, "grad_norm": 0.74357670545578, "tokens_seen": 31465472, "tokens_per_sec": 24568.79104452069, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.936185836791992, "grad_norm": 0.7639886736869812, "tokens_seen": 31629312, "tokens_per_sec": 24568.842083939016, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.85159969329834, "grad_norm": 0.7452654242515564, "tokens_seen": 31793152, "tokens_per_sec": 24568.01898361528, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.889380216598511, "grad_norm": 0.7816011905670166, "tokens_seen": 31956992, "tokens_per_sec": 24568.46976077582, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.873472213745117, "grad_norm": 0.7863430976867676, "tokens_seen": 32120832, "tokens_per_sec": 24569.011748834997, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.761213302612305, "grad_norm": 0.7522119879722595, "tokens_seen": 32284672, "tokens_per_sec": 24569.415604046477, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.965019226074219, "grad_norm": 0.9483680725097656, "tokens_seen": 32448512, "tokens_per_sec": 24566.5737952186, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.816012859344482, "grad_norm": 0.7347808480262756, "tokens_seen": 32612352, "tokens_per_sec": 24566.871800845845, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.741724252700806, "grad_norm": 0.7007746696472168, "tokens_seen": 32776192, "tokens_per_sec": 24567.04050915277, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.901801753044128, "val_ppl": 134.5319548086466, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.678696155548096, "grad_norm": 0.7509511709213257, "tokens_seen": 32940032, "tokens_per_sec": 24545.94528077425, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.796050548553467, "grad_norm": 0.6926636695861816, "tokens_seen": 33103872, "tokens_per_sec": 24546.535854135018, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.796229839324951, "grad_norm": 0.7874542474746704, "tokens_seen": 33267712, "tokens_per_sec": 24547.135312249113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.817756652832031, "grad_norm": 0.8183422684669495, "tokens_seen": 33431552, "tokens_per_sec": 24547.821717811355, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.864771127700806, "grad_norm": 0.8068675398826599, "tokens_seen": 33595392, "tokens_per_sec": 24548.039965954784, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.823627233505249, "grad_norm": 0.7682071328163147, "tokens_seen": 33759232, "tokens_per_sec": 24548.60710780354, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.826829671859741, "grad_norm": 0.7950896620750427, "tokens_seen": 33923072, "tokens_per_sec": 24549.303020936324, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.92429256439209, "grad_norm": 0.8725032210350037, "tokens_seen": 34086912, "tokens_per_sec": 24550.109670107653, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.82353401184082, "grad_norm": 0.7858254313468933, "tokens_seen": 34250752, "tokens_per_sec": 24550.174616323384, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.805941581726074, "grad_norm": 0.7772108316421509, "tokens_seen": 34414592, "tokens_per_sec": 24550.809168752687, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.765462398529053, "grad_norm": 0.7578731775283813, "tokens_seen": 34578432, "tokens_per_sec": 24551.37008284931, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.686642646789551, "grad_norm": 0.7510023713111877, "tokens_seen": 34742272, "tokens_per_sec": 24551.91634153756, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.764525890350342, "grad_norm": 0.7744021415710449, "tokens_seen": 34906112, "tokens_per_sec": 24552.47702941561, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.884298801422119, "grad_norm": 0.8091252446174622, "tokens_seen": 35069952, "tokens_per_sec": 24553.09226682792, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.9123663902282715, "grad_norm": 0.8625580072402954, "tokens_seen": 35233792, "tokens_per_sec": 24553.358521604692, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.710234880447388, "grad_norm": 0.8606763482093811, "tokens_seen": 35397632, "tokens_per_sec": 24553.58590946753, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.9458982944488525, "grad_norm": 0.7447866797447205, "tokens_seen": 35561472, "tokens_per_sec": 24554.376172012486, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.7484307289123535, "grad_norm": 0.8201411962509155, "tokens_seen": 35725312, "tokens_per_sec": 24554.57783755057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.468214511871338, "grad_norm": 0.7764523029327393, "tokens_seen": 35889152, "tokens_per_sec": 24555.1197413948, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.914767265319824, "grad_norm": 0.7854021787643433, "tokens_seen": 36052992, "tokens_per_sec": 24556.033234162325, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.7499823570251465, "grad_norm": 0.7490805387496948, "tokens_seen": 36216832, "tokens_per_sec": 24556.061921978053, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.925877332687378, "grad_norm": 0.8139446377754211, "tokens_seen": 36380672, "tokens_per_sec": 24556.539841901806, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.6151204109191895, "grad_norm": 0.786750078201294, "tokens_seen": 36544512, "tokens_per_sec": 24556.883069109772, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.758065700531006, "grad_norm": 0.755559504032135, "tokens_seen": 36708352, "tokens_per_sec": 24557.5884165083, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.8500916957855225, "grad_norm": 0.7495033144950867, "tokens_seen": 36872192, "tokens_per_sec": 24558.055065546017, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.547818422317505, "grad_norm": 0.8038188815116882, "tokens_seen": 37036032, "tokens_per_sec": 24558.72842515243, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.669687747955322, "grad_norm": 0.8113996386528015, "tokens_seen": 37199872, "tokens_per_sec": 24558.97497847534, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.669902324676514, "grad_norm": 0.7531301975250244, "tokens_seen": 37363712, "tokens_per_sec": 24559.497818909855, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.7232818603515625, "grad_norm": 0.7895333766937256, "tokens_seen": 37527552, "tokens_per_sec": 24560.253717997923, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.612389087677002, "grad_norm": 0.8065343499183655, "tokens_seen": 37691392, "tokens_per_sec": 24560.632563701078, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.714954853057861, "grad_norm": 0.7983185648918152, "tokens_seen": 37855232, "tokens_per_sec": 24561.34693597439, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.754674434661865, "grad_norm": 0.8203844428062439, "tokens_seen": 38019072, "tokens_per_sec": 24561.858100266487, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.689387321472168, "grad_norm": 0.8099186420440674, "tokens_seen": 38182912, "tokens_per_sec": 24562.31109669556, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.627206802368164, "grad_norm": 0.8281697630882263, "tokens_seen": 38346752, "tokens_per_sec": 24563.020947226636, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.6689019203186035, "grad_norm": 0.8451581597328186, "tokens_seen": 38510592, "tokens_per_sec": 24563.569058932742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.695230007171631, "grad_norm": 0.8383651971817017, "tokens_seen": 38674432, "tokens_per_sec": 24563.381730675563, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.7366251945495605, "grad_norm": 0.8082486987113953, "tokens_seen": 38838272, "tokens_per_sec": 24563.922870926956, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.799737930297852, "grad_norm": 0.8247146010398865, "tokens_seen": 39002112, "tokens_per_sec": 24564.51970180443, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.658817768096924, "grad_norm": 0.8192147016525269, "tokens_seen": 39165952, "tokens_per_sec": 24564.65741118141, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.854035377502441, "grad_norm": 0.7920680642127991, "tokens_seen": 39329792, "tokens_per_sec": 24565.25158914499, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.658473968505859, "grad_norm": 0.8198215365409851, "tokens_seen": 39493632, "tokens_per_sec": 24565.677402460173, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.572418689727783, "grad_norm": 0.7783718109130859, "tokens_seen": 39657472, "tokens_per_sec": 24566.180917907364, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.660287857055664, "grad_norm": 0.8018028140068054, "tokens_seen": 39821312, "tokens_per_sec": 24566.82817936901, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.628945350646973, "grad_norm": 0.7618391513824463, "tokens_seen": 39985152, "tokens_per_sec": 24567.461025543904, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.6974194049835205, "grad_norm": 0.7868585586547852, "tokens_seen": 40148992, "tokens_per_sec": 24568.0542720036, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.683935880661011, "grad_norm": 0.8862113952636719, "tokens_seen": 40312832, "tokens_per_sec": 24568.514686334816, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.6362714767456055, "grad_norm": 0.8537544012069702, "tokens_seen": 40476672, "tokens_per_sec": 24569.07784464773, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.701441287994385, "grad_norm": 0.7981436252593994, "tokens_seen": 40640512, "tokens_per_sec": 24569.301781381473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.65000319480896, "grad_norm": 0.8470927476882935, "tokens_seen": 40804352, "tokens_per_sec": 24569.7815196895, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.802793741226196, "grad_norm": 0.8339836001396179, "tokens_seen": 40968192, "tokens_per_sec": 24570.475229500742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.7747427225112915, "val_ppl": 118.47982850645523, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.7613770961761475, "grad_norm": 0.824299693107605, "tokens_seen": 41132032, "tokens_per_sec": 24554.04231172888, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.814504623413086, "grad_norm": 0.8301552534103394, "tokens_seen": 41295872, "tokens_per_sec": 24554.50739752102, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.665094614028931, "grad_norm": 0.9262716770172119, "tokens_seen": 41459712, "tokens_per_sec": 24554.99117171175, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.759365081787109, "grad_norm": 0.8249866366386414, "tokens_seen": 41623552, "tokens_per_sec": 24555.216181922908, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.712787389755249, "grad_norm": 0.9455620050430298, "tokens_seen": 41787392, "tokens_per_sec": 24555.940672948407, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.734677314758301, "grad_norm": 0.8419005870819092, "tokens_seen": 41951232, "tokens_per_sec": 24556.6657709643, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.658986330032349, "grad_norm": 0.8342955708503723, "tokens_seen": 42115072, "tokens_per_sec": 24557.222435322892, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.763835668563843, "grad_norm": 0.904941201210022, "tokens_seen": 42278912, "tokens_per_sec": 24557.682391028797, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.571490287780762, "grad_norm": 0.856993556022644, "tokens_seen": 42442752, "tokens_per_sec": 24558.250108276705, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.532677173614502, "grad_norm": 0.8296653628349304, "tokens_seen": 42606592, "tokens_per_sec": 24558.353601707713, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.717053174972534, "grad_norm": 0.8214353919029236, "tokens_seen": 42770432, "tokens_per_sec": 24559.15998456106, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.749372482299805, "grad_norm": 0.858715295791626, "tokens_seen": 42934272, "tokens_per_sec": 24559.64822845068, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.633222579956055, "grad_norm": 0.8227642178535461, "tokens_seen": 43098112, "tokens_per_sec": 24559.736265196778, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.6150970458984375, "grad_norm": 0.7931081056594849, "tokens_seen": 43261952, "tokens_per_sec": 24560.103278887455, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.637022495269775, "grad_norm": 0.7991732358932495, "tokens_seen": 43425792, "tokens_per_sec": 24560.692533940914, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.630323886871338, "grad_norm": 0.7655116319656372, "tokens_seen": 43589632, "tokens_per_sec": 24561.063817174192, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.747725009918213, "grad_norm": 0.8301108479499817, "tokens_seen": 43753472, "tokens_per_sec": 24561.507573360304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.649711847305298, "grad_norm": 0.7642079591751099, "tokens_seen": 43917312, "tokens_per_sec": 24561.904796075876, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.620263338088989, "grad_norm": 0.85990309715271, "tokens_seen": 44081152, "tokens_per_sec": 24562.155436349676, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.71837854385376, "grad_norm": 0.8542933464050293, "tokens_seen": 44244992, "tokens_per_sec": 24562.61889446088, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.601494312286377, "grad_norm": 0.9325602054595947, "tokens_seen": 44408832, "tokens_per_sec": 24563.03206620599, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.762969493865967, "grad_norm": 0.8739762306213379, "tokens_seen": 44572672, "tokens_per_sec": 24563.1540449983, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.584115982055664, "grad_norm": 0.8727326989173889, "tokens_seen": 44736512, "tokens_per_sec": 24563.366265995173, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.702790975570679, "grad_norm": 0.8686047792434692, "tokens_seen": 44900352, "tokens_per_sec": 24564.03443277634, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.719768762588501, "grad_norm": 0.8446729183197021, "tokens_seen": 45064192, "tokens_per_sec": 24564.58509172945, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.827042102813721, "grad_norm": 0.8870123028755188, "tokens_seen": 45228032, "tokens_per_sec": 24565.166573565024, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.736997127532959, "grad_norm": 0.8345959186553955, "tokens_seen": 45391872, "tokens_per_sec": 24565.579583272644, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.5307440757751465, "grad_norm": 0.8422975540161133, "tokens_seen": 45555712, "tokens_per_sec": 24566.068425415135, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.562878608703613, "grad_norm": 0.8575479388237, "tokens_seen": 45719552, "tokens_per_sec": 24566.592138373922, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.688007593154907, "grad_norm": 0.8514198660850525, "tokens_seen": 45883392, "tokens_per_sec": 24567.332510476965, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.4338719844818115, "grad_norm": 0.8039156198501587, "tokens_seen": 46047232, "tokens_per_sec": 24568.025508544113, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.690595388412476, "grad_norm": 0.8178121447563171, "tokens_seen": 46211072, "tokens_per_sec": 24568.705260079656, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.715451240539551, "grad_norm": 0.8380759954452515, "tokens_seen": 46374912, "tokens_per_sec": 24569.18875481587, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.524723529815674, "grad_norm": 0.8830233812332153, "tokens_seen": 46538752, "tokens_per_sec": 24569.778317930093, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.479012727737427, "grad_norm": 0.8107718825340271, "tokens_seen": 46702592, "tokens_per_sec": 24570.351885498018, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.6227028369903564, "grad_norm": 0.8842249512672424, "tokens_seen": 46866432, "tokens_per_sec": 24571.092648950234, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.665932893753052, "grad_norm": 0.8493221402168274, "tokens_seen": 47030272, "tokens_per_sec": 24571.810040681878, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.577295541763306, "grad_norm": 0.9078730940818787, "tokens_seen": 47194112, "tokens_per_sec": 24572.45321621173, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.542585134506226, "grad_norm": 0.8333434462547302, "tokens_seen": 47357952, "tokens_per_sec": 24572.994146471414, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.515857219696045, "grad_norm": 0.8371649384498596, "tokens_seen": 47521792, "tokens_per_sec": 24573.47179687381, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.558531999588013, "grad_norm": 0.9083607792854309, "tokens_seen": 47685632, "tokens_per_sec": 24574.15394131763, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.750376224517822, "grad_norm": 0.8615137338638306, "tokens_seen": 47849472, "tokens_per_sec": 24574.8149646943, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.567577600479126, "grad_norm": 0.8555772304534912, "tokens_seen": 48013312, "tokens_per_sec": 24575.504678355228, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.673708915710449, "grad_norm": 0.9112712740898132, "tokens_seen": 48177152, "tokens_per_sec": 24576.20460657505, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.500024080276489, "grad_norm": 0.8985008001327515, "tokens_seen": 48340992, "tokens_per_sec": 24576.65576407368, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.454681873321533, "grad_norm": 0.8552420139312744, "tokens_seen": 48504832, "tokens_per_sec": 24577.191913658615, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.620147705078125, "grad_norm": 0.8478307127952576, "tokens_seen": 48668672, "tokens_per_sec": 24577.90311574382, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.54154109954834, "grad_norm": 0.8573920726776123, "tokens_seen": 48832512, "tokens_per_sec": 24578.13222487672, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.640336990356445, "grad_norm": 0.8499054312705994, "tokens_seen": 48996352, "tokens_per_sec": 24578.919810572326, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.705247163772583, "grad_norm": 0.8848538994789124, "tokens_seen": 49160192, "tokens_per_sec": 24579.52415433824, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.655383157730102, "val_ppl": 105.14950112556697, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.68977427482605, "grad_norm": 0.8767391443252563, "tokens_seen": 49324032, "tokens_per_sec": 24566.082843756776, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.485123157501221, "grad_norm": 0.9317498207092285, "tokens_seen": 49487872, "tokens_per_sec": 24566.80351198776, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.556593656539917, "grad_norm": 0.8397045135498047, "tokens_seen": 49651712, "tokens_per_sec": 24567.483986259995, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.74583625793457, "grad_norm": 0.8443084955215454, "tokens_seen": 49815552, "tokens_per_sec": 24567.717838142722, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.401811838150024, "grad_norm": 0.8229142427444458, "tokens_seen": 49979392, "tokens_per_sec": 24568.4754806581, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.664807081222534, "grad_norm": 0.8435518741607666, "tokens_seen": 50143232, "tokens_per_sec": 24568.995382045337, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.487149477005005, "grad_norm": 0.8604838252067566, "tokens_seen": 50307072, "tokens_per_sec": 24569.519980839414, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.57841944694519, "grad_norm": 0.8683661818504333, "tokens_seen": 50470912, "tokens_per_sec": 24570.17251374874, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.495023012161255, "grad_norm": 0.902439534664154, "tokens_seen": 50634752, "tokens_per_sec": 24570.62564309768, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.638481378555298, "grad_norm": 0.9043551683425903, "tokens_seen": 50798592, "tokens_per_sec": 24571.121311230127, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.680959224700928, "grad_norm": 0.8724486231803894, "tokens_seen": 50962432, "tokens_per_sec": 24571.481443479195, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.622512102127075, "grad_norm": 0.89349365234375, "tokens_seen": 51126272, "tokens_per_sec": 24572.013804357786, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.4987828731536865, "grad_norm": 0.9147058129310608, "tokens_seen": 51290112, "tokens_per_sec": 24572.652675259316, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.484899044036865, "grad_norm": 0.8747223615646362, "tokens_seen": 51453952, "tokens_per_sec": 24573.38462108095, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.60084867477417, "grad_norm": 0.9525841474533081, "tokens_seen": 51617792, "tokens_per_sec": 24573.894243328337, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.359783411026001, "grad_norm": 0.8360088467597961, "tokens_seen": 51781632, "tokens_per_sec": 24574.14080273943, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.586611747741699, "grad_norm": 0.8365687727928162, "tokens_seen": 51945472, "tokens_per_sec": 24574.67697611728, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.458004713058472, "grad_norm": 0.8626038432121277, "tokens_seen": 52109312, "tokens_per_sec": 24575.277185694238, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.630993366241455, "grad_norm": 0.8890843987464905, "tokens_seen": 52273152, "tokens_per_sec": 24575.8260574062, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.589706659317017, "grad_norm": 0.9243417382240295, "tokens_seen": 52436992, "tokens_per_sec": 24576.053041143223, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.6369218826293945, "grad_norm": 0.8947737216949463, "tokens_seen": 52600832, "tokens_per_sec": 24576.643118139666, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.653875827789307, "grad_norm": 0.9136117100715637, "tokens_seen": 52764672, "tokens_per_sec": 24577.224361980803, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.531580209732056, "grad_norm": 0.9494286179542542, "tokens_seen": 52928512, "tokens_per_sec": 24577.77191260023, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.725345134735107, "grad_norm": 0.9259796142578125, "tokens_seen": 53092352, "tokens_per_sec": 24577.989248394628, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.469691276550293, "grad_norm": 0.9052245616912842, "tokens_seen": 53256192, "tokens_per_sec": 24578.482422036737, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.535799503326416, "grad_norm": 0.8535586595535278, "tokens_seen": 53420032, "tokens_per_sec": 24578.644943768348, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.6180055141448975, "grad_norm": 0.8821328282356262, "tokens_seen": 53583872, "tokens_per_sec": 24579.260994330005, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.385958671569824, "grad_norm": 0.8823183178901672, "tokens_seen": 53747712, "tokens_per_sec": 24579.791110063314, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.698070526123047, "grad_norm": 0.9530470967292786, "tokens_seen": 53911552, "tokens_per_sec": 24580.2780028292, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.511190891265869, "grad_norm": 0.9232984185218811, "tokens_seen": 54075392, "tokens_per_sec": 24580.82058000245, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.77936053276062, "grad_norm": 1.134818434715271, "tokens_seen": 54239232, "tokens_per_sec": 24580.91841216271, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.4545207023620605, "grad_norm": 1.0432909727096558, "tokens_seen": 54403072, "tokens_per_sec": 24581.6196294969, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.47189998626709, "grad_norm": 0.8906442523002625, "tokens_seen": 54566912, "tokens_per_sec": 24582.161676669948, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.4524009227752686, "grad_norm": 0.9048233032226562, "tokens_seen": 54730752, "tokens_per_sec": 24582.599088338637, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.611374139785767, "grad_norm": 0.8945034146308899, "tokens_seen": 54894592, "tokens_per_sec": 24583.219102791547, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.497286796569824, "grad_norm": 0.9053565859794617, "tokens_seen": 55058432, "tokens_per_sec": 24583.74922865275, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.49887228012085, "grad_norm": 0.8616023063659668, "tokens_seen": 55222272, "tokens_per_sec": 24584.337388157193, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.612370491027832, "grad_norm": 0.9628223180770874, "tokens_seen": 55386112, "tokens_per_sec": 24584.661027002883, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.467108249664307, "grad_norm": 0.8975920081138611, "tokens_seen": 55549952, "tokens_per_sec": 24585.23831046296, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.415887355804443, "grad_norm": 0.9878414869308472, "tokens_seen": 55713792, "tokens_per_sec": 24585.76203582224, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.600670337677002, "grad_norm": 0.9212672114372253, "tokens_seen": 55877632, "tokens_per_sec": 24586.272214638902, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.430617570877075, "grad_norm": 0.8744909763336182, "tokens_seen": 56041472, "tokens_per_sec": 24586.763830969438, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.619937181472778, "grad_norm": 0.8251757621765137, "tokens_seen": 56205312, "tokens_per_sec": 24587.33198903704, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.545548915863037, "grad_norm": 0.9175305962562561, "tokens_seen": 56369152, "tokens_per_sec": 24587.965972624337, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.462454795837402, "grad_norm": 0.9305545091629028, "tokens_seen": 56532992, "tokens_per_sec": 24588.477111458866, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.497774839401245, "grad_norm": 0.9555764198303223, "tokens_seen": 56696832, "tokens_per_sec": 24588.93803427739, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.545768976211548, "grad_norm": 0.9082730412483215, "tokens_seen": 56860672, "tokens_per_sec": 24589.439922142898, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.525605201721191, "grad_norm": 0.9333004951477051, "tokens_seen": 57024512, "tokens_per_sec": 24589.887449131333, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.48468279838562, "grad_norm": 0.8897904753684998, "tokens_seen": 57188352, "tokens_per_sec": 24590.445903296473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.3344292640686035, "grad_norm": 0.9336578845977783, "tokens_seen": 57352192, "tokens_per_sec": 24590.95883329478, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.624353694915771, "val_ppl": 101.9368694710646, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.6908745765686035, "grad_norm": 0.9207393527030945, "tokens_seen": 57516032, "tokens_per_sec": 24579.387865869983, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.6819751262664795, "grad_norm": 1.0225471258163452, "tokens_seen": 57679872, "tokens_per_sec": 24579.91509053506, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.575076580047607, "grad_norm": 0.9317463636398315, "tokens_seen": 57843712, "tokens_per_sec": 24580.44945934183, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.535609483718872, "grad_norm": 0.9359048008918762, "tokens_seen": 58007552, "tokens_per_sec": 24580.844311096807, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.561814308166504, "grad_norm": 0.9100548028945923, "tokens_seen": 58171392, "tokens_per_sec": 24581.07027656455, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.746997356414795, "grad_norm": 1.0341936349868774, "tokens_seen": 58335232, "tokens_per_sec": 24581.410192014897, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.467188358306885, "grad_norm": 0.9600183963775635, "tokens_seen": 58499072, "tokens_per_sec": 24581.719462899444, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.481711149215698, "grad_norm": 0.904186487197876, "tokens_seen": 58662912, "tokens_per_sec": 24582.06104060637, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.55223274230957, "grad_norm": 0.9789466261863708, "tokens_seen": 58826752, "tokens_per_sec": 24582.37257955452, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.554083347320557, "grad_norm": 0.9634771347045898, "tokens_seen": 58990592, "tokens_per_sec": 24582.66459808343, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.351544380187988, "grad_norm": 0.9170109629631042, "tokens_seen": 59154432, "tokens_per_sec": 24583.045586879176, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.43619966506958, "grad_norm": 0.8520582914352417, "tokens_seen": 59318272, "tokens_per_sec": 24583.345155124473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.551273822784424, "grad_norm": 0.9296157956123352, "tokens_seen": 59482112, "tokens_per_sec": 24583.704108046888, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.435048341751099, "grad_norm": 0.910316526889801, "tokens_seen": 59645952, "tokens_per_sec": 24584.08477210896, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.585331916809082, "grad_norm": 0.917659342288971, "tokens_seen": 59809792, "tokens_per_sec": 24584.416706485623, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.4152305126190186, "grad_norm": 0.9216893911361694, "tokens_seen": 59973632, "tokens_per_sec": 24584.902166253414, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.50570821762085, "grad_norm": 0.9658368229866028, "tokens_seen": 60137472, "tokens_per_sec": 24585.26962605886, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.565312385559082, "grad_norm": 0.9599086046218872, "tokens_seen": 60301312, "tokens_per_sec": 24585.736053716035, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.541912317276001, "grad_norm": 1.04285728931427, "tokens_seen": 60465152, "tokens_per_sec": 24586.25690634567, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.400683403015137, "grad_norm": 0.9385383725166321, "tokens_seen": 60628992, "tokens_per_sec": 24586.694158461334, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.484048128128052, "grad_norm": 0.965962290763855, "tokens_seen": 60792832, "tokens_per_sec": 24587.00838183964, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.393276214599609, "grad_norm": 0.9431990385055542, "tokens_seen": 60956672, "tokens_per_sec": 24587.457680999873, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.582413673400879, "grad_norm": 0.9123157858848572, "tokens_seen": 61120512, "tokens_per_sec": 24587.948185339737, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.420116424560547, "grad_norm": 0.9640544056892395, "tokens_seen": 61284352, "tokens_per_sec": 24588.315721612402, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.478687047958374, "grad_norm": 0.944177508354187, "tokens_seen": 61448192, "tokens_per_sec": 24588.848867495057, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.521983861923218, "grad_norm": 0.9047247171401978, "tokens_seen": 61612032, "tokens_per_sec": 24589.23623581839, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.406179428100586, "grad_norm": 0.9490012526512146, "tokens_seen": 61775872, "tokens_per_sec": 24589.69601032462, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.315804958343506, "grad_norm": 0.9697878360748291, "tokens_seen": 61939712, "tokens_per_sec": 24590.17325123169, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.589089632034302, "grad_norm": 0.9630635976791382, "tokens_seen": 62103552, "tokens_per_sec": 24590.6228439388, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.3540263175964355, "grad_norm": 0.9055280685424805, "tokens_seen": 62267392, "tokens_per_sec": 24591.075975166164, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.4447548389434814, "grad_norm": 0.9009047746658325, "tokens_seen": 62431232, "tokens_per_sec": 24591.34651622219, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.41488790512085, "grad_norm": 0.9408949017524719, "tokens_seen": 62595072, "tokens_per_sec": 24591.68823036399, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.442930221557617, "grad_norm": 0.9318659901618958, "tokens_seen": 62758912, "tokens_per_sec": 24592.02906117891, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.6260926723480225, "grad_norm": 1.0028938055038452, "tokens_seen": 62922752, "tokens_per_sec": 24592.27798099633, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.582983016967773, "grad_norm": 0.9260270595550537, "tokens_seen": 63086592, "tokens_per_sec": 24592.629230881612, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.580858945846558, "grad_norm": 0.9544792771339417, "tokens_seen": 63250432, "tokens_per_sec": 24593.030358997177, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.4935712814331055, "grad_norm": 0.9404527544975281, "tokens_seen": 63414272, "tokens_per_sec": 24593.35552698372, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.487290382385254, "grad_norm": 0.8793418407440186, "tokens_seen": 63578112, "tokens_per_sec": 24593.81316909517, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.564378023147583, "grad_norm": 1.0287652015686035, "tokens_seen": 63741952, "tokens_per_sec": 24593.75082000554, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.4831702709198, "grad_norm": 0.9749252796173096, "tokens_seen": 63905792, "tokens_per_sec": 24593.6449572939, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.347405433654785, "grad_norm": 0.9236550331115723, "tokens_seen": 64069632, "tokens_per_sec": 24594.04527692305, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.3891236782073975, "grad_norm": 0.9360880851745605, "tokens_seen": 64233472, "tokens_per_sec": 24594.334031802373, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.387739658355713, "grad_norm": 0.9698083996772766, "tokens_seen": 64397312, "tokens_per_sec": 24594.70920047901, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.390562057495117, "grad_norm": 0.9456549882888794, "tokens_seen": 64561152, "tokens_per_sec": 24594.959303713906, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.352295637130737, "grad_norm": 0.9172099828720093, "tokens_seen": 64724992, "tokens_per_sec": 24595.35916595548, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.512453079223633, "grad_norm": 1.0812885761260986, "tokens_seen": 64888832, "tokens_per_sec": 24595.577251627135, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.499207973480225, "grad_norm": 0.9634764194488525, "tokens_seen": 65052672, "tokens_per_sec": 24596.01897136887, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.500421524047852, "grad_norm": 0.9952672123908997, "tokens_seen": 65216512, "tokens_per_sec": 24596.3110116683, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.5052924156188965, "grad_norm": 0.9662765860557556, "tokens_seen": 65380352, "tokens_per_sec": 24596.640371242393, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.553324222564697, "grad_norm": 1.0116239786148071, "tokens_seen": 65544192, "tokens_per_sec": 24596.761883026884, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.546896195411682, "val_ppl": 94.33914316608319, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.520293474197388, "grad_norm": 1.0312329530715942, "tokens_seen": 65708032, "tokens_per_sec": 24586.696875844184, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.419244289398193, "grad_norm": 1.0183335542678833, "tokens_seen": 65871872, "tokens_per_sec": 24587.145529181784, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.4575982093811035, "grad_norm": 0.9341976642608643, "tokens_seen": 66035712, "tokens_per_sec": 24587.317276727055, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.547530174255371, "grad_norm": 0.9493760466575623, "tokens_seen": 66199552, "tokens_per_sec": 24587.80645912187, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.527514696121216, "grad_norm": 0.9607624411582947, "tokens_seen": 66363392, "tokens_per_sec": 24588.179139878117, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.664694786071777, "grad_norm": 0.9921653270721436, "tokens_seen": 66527232, "tokens_per_sec": 24588.583619701392, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.321754217147827, "grad_norm": 0.9465335607528687, "tokens_seen": 66691072, "tokens_per_sec": 24589.013507174473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.401129722595215, "grad_norm": 0.9734531044960022, "tokens_seen": 66854912, "tokens_per_sec": 24589.259890011064, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.486019611358643, "grad_norm": 1.0003795623779297, "tokens_seen": 67018752, "tokens_per_sec": 24589.63288273306, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.34604549407959, "grad_norm": 0.9468969702720642, "tokens_seen": 67182592, "tokens_per_sec": 24590.061853321065, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.453733205795288, "grad_norm": 0.9827287793159485, "tokens_seen": 67346432, "tokens_per_sec": 24590.43632759771, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.377308368682861, "grad_norm": 0.9485863447189331, "tokens_seen": 67510272, "tokens_per_sec": 24590.863999706173, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.352003574371338, "grad_norm": 1.0525423288345337, "tokens_seen": 67674112, "tokens_per_sec": 24591.29005892491, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.34055495262146, "grad_norm": 0.90416419506073, "tokens_seen": 67837952, "tokens_per_sec": 24591.636826608486, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.520720958709717, "grad_norm": 0.9182345271110535, "tokens_seen": 68001792, "tokens_per_sec": 24591.844492952994, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.398280143737793, "grad_norm": 0.9649325609207153, "tokens_seen": 68165632, "tokens_per_sec": 24592.139629303118, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.45125675201416, "grad_norm": 0.9695706367492676, "tokens_seen": 68329472, "tokens_per_sec": 24592.478069508605, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.537782907485962, "grad_norm": 1.0275256633758545, "tokens_seen": 68493312, "tokens_per_sec": 24592.66626003236, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.38018798828125, "grad_norm": 1.0472893714904785, "tokens_seen": 68657152, "tokens_per_sec": 24592.938510872696, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.459697008132935, "grad_norm": 1.0041474103927612, "tokens_seen": 68820992, "tokens_per_sec": 24593.26165957275, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.3395514488220215, "grad_norm": 0.9784644842147827, "tokens_seen": 68984832, "tokens_per_sec": 24593.4726978055, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.30684494972229, "grad_norm": 0.940044105052948, "tokens_seen": 69148672, "tokens_per_sec": 24593.808337481205, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.374234437942505, "grad_norm": 1.111238956451416, "tokens_seen": 69312512, "tokens_per_sec": 24594.163471983735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.368047714233398, "grad_norm": 0.9492058753967285, "tokens_seen": 69476352, "tokens_per_sec": 24594.56585737322, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.390593528747559, "grad_norm": 0.9539051651954651, "tokens_seen": 69640192, "tokens_per_sec": 24594.90980043599, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.485187292098999, "grad_norm": 1.0214414596557617, "tokens_seen": 69804032, "tokens_per_sec": 24595.284110152887, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.551527500152588, "grad_norm": 1.0230647325515747, "tokens_seen": 69967872, "tokens_per_sec": 24595.68463878547, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.369680404663086, "grad_norm": 0.9599302411079407, "tokens_seen": 70131712, "tokens_per_sec": 24596.054606574275, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.470537185668945, "grad_norm": 1.027823567390442, "tokens_seen": 70295552, "tokens_per_sec": 24596.372525773837, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.295091152191162, "grad_norm": 1.0569815635681152, "tokens_seen": 70459392, "tokens_per_sec": 24596.773654160155, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.4203104972839355, "grad_norm": 1.0442801713943481, "tokens_seen": 70623232, "tokens_per_sec": 24597.029432968397, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.486906290054321, "grad_norm": 0.9895553588867188, "tokens_seen": 70787072, "tokens_per_sec": 24597.253665854063, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.304336786270142, "grad_norm": 1.0473852157592773, "tokens_seen": 70950912, "tokens_per_sec": 24597.561648641284, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.406091213226318, "grad_norm": 0.923879086971283, "tokens_seen": 71114752, "tokens_per_sec": 24597.908783987572, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.2666542530059814, "grad_norm": 1.0815314054489136, "tokens_seen": 71278592, "tokens_per_sec": 24598.33202096105, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.512030124664307, "grad_norm": 1.0916693210601807, "tokens_seen": 71442432, "tokens_per_sec": 24598.744567223115, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.429418563842773, "grad_norm": 1.034367561340332, "tokens_seen": 71606272, "tokens_per_sec": 24598.82466942938, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.350789308547974, "grad_norm": 0.9873936176300049, "tokens_seen": 71770112, "tokens_per_sec": 24599.21768995359, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.410492420196533, "grad_norm": 1.0059930086135864, "tokens_seen": 71933952, "tokens_per_sec": 24599.594245011107, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.3472185134887695, "grad_norm": 1.0125160217285156, "tokens_seen": 72097792, "tokens_per_sec": 24600.018271315836, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.298893690109253, "grad_norm": 0.9686195850372314, "tokens_seen": 72261632, "tokens_per_sec": 24600.435137952067, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.502504825592041, "grad_norm": 1.0658583641052246, "tokens_seen": 72425472, "tokens_per_sec": 24600.89042237613, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.3545982837677, "grad_norm": 1.0416911840438843, "tokens_seen": 72589312, "tokens_per_sec": 24601.307968443998, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.392042398452759, "grad_norm": 0.9450470805168152, "tokens_seen": 72753152, "tokens_per_sec": 24601.677533040074, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.201383590698242, "grad_norm": 0.9572449326515198, "tokens_seen": 72916992, "tokens_per_sec": 24602.14719452755, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.384069919586182, "grad_norm": 0.9787265658378601, "tokens_seen": 73080832, "tokens_per_sec": 24602.61709611886, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.388242244720459, "grad_norm": 0.9490050077438354, "tokens_seen": 73244672, "tokens_per_sec": 24603.044050304805, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.408102750778198, "grad_norm": 0.922942042350769, "tokens_seen": 73408512, "tokens_per_sec": 24603.473623412377, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.454636573791504, "grad_norm": 1.0355608463287354, "tokens_seen": 73572352, "tokens_per_sec": 24603.896896105794, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.393136024475098, "grad_norm": 1.0509861707687378, "tokens_seen": 73736192, "tokens_per_sec": 24604.339154582263, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.46153678894043, "val_ppl": 86.62052432578201, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.469821214675903, "grad_norm": 1.0443923473358154, "tokens_seen": 73900032, "tokens_per_sec": 24595.32841229167, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.417250871658325, "grad_norm": 1.0406875610351562, "tokens_seen": 74063872, "tokens_per_sec": 24595.70076541366, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.149109601974487, "grad_norm": 1.0414003133773804, "tokens_seen": 74227712, "tokens_per_sec": 24595.83326242304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.488269329071045, "grad_norm": 1.0120686292648315, "tokens_seen": 74391552, "tokens_per_sec": 24596.015226803753, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.431069612503052, "grad_norm": 1.023634910583496, "tokens_seen": 74555392, "tokens_per_sec": 24595.94680284272, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.2795093059539795, "grad_norm": 1.0399128198623657, "tokens_seen": 74719232, "tokens_per_sec": 24596.270244093965, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.233463764190674, "grad_norm": 1.0032613277435303, "tokens_seen": 74883072, "tokens_per_sec": 24596.650498649426, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.465545177459717, "grad_norm": 1.0719804763793945, "tokens_seen": 75046912, "tokens_per_sec": 24597.040294950224, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.469727993011475, "grad_norm": 0.9422900676727295, "tokens_seen": 75210752, "tokens_per_sec": 24597.4789627843, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.264620542526245, "grad_norm": 1.0919371843338013, "tokens_seen": 75374592, "tokens_per_sec": 24597.7873690781, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.460926532745361, "grad_norm": 1.0884660482406616, "tokens_seen": 75538432, "tokens_per_sec": 24598.12125613163, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.481432199478149, "grad_norm": 1.0961025953292847, "tokens_seen": 75702272, "tokens_per_sec": 24598.50112191706, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.376585960388184, "grad_norm": 1.0608859062194824, "tokens_seen": 75866112, "tokens_per_sec": 24598.749176579742, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.3604161739349365, "grad_norm": 1.1299569606781006, "tokens_seen": 76029952, "tokens_per_sec": 24599.029284876986, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.508991479873657, "grad_norm": 1.1179170608520508, "tokens_seen": 76193792, "tokens_per_sec": 24599.417853878203, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.305900573730469, "grad_norm": 1.0392998456954956, "tokens_seen": 76357632, "tokens_per_sec": 24599.732142289344, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.497642993927002, "grad_norm": 1.0410501956939697, "tokens_seen": 76521472, "tokens_per_sec": 24600.02975800081, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.304457902908325, "grad_norm": 0.9475066661834717, "tokens_seen": 76685312, "tokens_per_sec": 24598.91984295153, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.441699981689453, "grad_norm": 1.0219961404800415, "tokens_seen": 76849152, "tokens_per_sec": 24599.293778647327, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.252110958099365, "grad_norm": 1.1149816513061523, "tokens_seen": 77012992, "tokens_per_sec": 24599.590218338286, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.268855333328247, "grad_norm": 1.009492039680481, "tokens_seen": 77176832, "tokens_per_sec": 24599.863036395178, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.33047342300415, "grad_norm": 1.235129952430725, "tokens_seen": 77340672, "tokens_per_sec": 24600.07931836003, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.397671222686768, "grad_norm": 0.9438460469245911, "tokens_seen": 77504512, "tokens_per_sec": 24600.327162170946, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.388841152191162, "grad_norm": 1.060976505279541, "tokens_seen": 77668352, "tokens_per_sec": 24600.637942545436, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.53842830657959, "grad_norm": 1.1314245462417603, "tokens_seen": 77832192, "tokens_per_sec": 24600.89838139648, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.223340749740601, "grad_norm": 1.0806119441986084, "tokens_seen": 77996032, "tokens_per_sec": 24601.139560583953, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.389642953872681, "grad_norm": 1.0642023086547852, "tokens_seen": 78159872, "tokens_per_sec": 24601.355132954504, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.560466766357422, "grad_norm": 1.0483723878860474, "tokens_seen": 78323712, "tokens_per_sec": 24601.609406762465, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.28456449508667, "grad_norm": 1.0742146968841553, "tokens_seen": 78487552, "tokens_per_sec": 24601.903127527297, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.41940975189209, "grad_norm": 0.9951014518737793, "tokens_seen": 78651392, "tokens_per_sec": 24602.15313837396, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.365836143493652, "grad_norm": 1.0172523260116577, "tokens_seen": 78815232, "tokens_per_sec": 24602.46563372263, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.525678634643555, "grad_norm": 1.1278876066207886, "tokens_seen": 78979072, "tokens_per_sec": 24602.62765973251, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.344626426696777, "grad_norm": 1.0100573301315308, "tokens_seen": 79142912, "tokens_per_sec": 24602.86069570028, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.524908542633057, "grad_norm": 1.0981086492538452, "tokens_seen": 79306752, "tokens_per_sec": 24602.961799927205, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.282565116882324, "grad_norm": 1.071353554725647, "tokens_seen": 79470592, "tokens_per_sec": 24603.171807180985, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.325547695159912, "grad_norm": 1.1664046049118042, "tokens_seen": 79634432, "tokens_per_sec": 24603.383266347504, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.379688739776611, "grad_norm": 1.0380260944366455, "tokens_seen": 79798272, "tokens_per_sec": 24603.585130667434, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.405158519744873, "grad_norm": 0.995862603187561, "tokens_seen": 79962112, "tokens_per_sec": 24603.783247069547, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.133334398269653, "grad_norm": 1.0740985870361328, "tokens_seen": 80125952, "tokens_per_sec": 24603.948641659277, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.303662061691284, "grad_norm": 1.0636543035507202, "tokens_seen": 80289792, "tokens_per_sec": 24604.106726677877, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.1839048862457275, "grad_norm": 0.9765245914459229, "tokens_seen": 80453632, "tokens_per_sec": 24604.206802407807, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.50187873840332, "grad_norm": 0.9987130761146545, "tokens_seen": 80617472, "tokens_per_sec": 24604.415800975752, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.322329998016357, "grad_norm": 1.1164215803146362, "tokens_seen": 80781312, "tokens_per_sec": 24604.571702038073, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.350412368774414, "grad_norm": 1.007084846496582, "tokens_seen": 80945152, "tokens_per_sec": 24604.729359796143, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.269128799438477, "grad_norm": 1.0673171281814575, "tokens_seen": 81108992, "tokens_per_sec": 24604.975629906272, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.317590713500977, "grad_norm": 1.0897092819213867, "tokens_seen": 81272832, "tokens_per_sec": 24605.14825322025, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.521036624908447, "grad_norm": 1.0350210666656494, "tokens_seen": 81436672, "tokens_per_sec": 24605.276936235543, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.410877704620361, "grad_norm": 1.0838862657546997, "tokens_seen": 81600512, "tokens_per_sec": 24605.392804545096, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.378622770309448, "grad_norm": 1.0311148166656494, "tokens_seen": 81764352, "tokens_per_sec": 24605.512887869463, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.356709957122803, "grad_norm": 0.9636567234992981, "tokens_seen": 81928192, "tokens_per_sec": 24605.714202953106, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.426478838920593, "val_ppl": 83.63640057653609, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.509706497192383, "grad_norm": 1.0697921514511108, "tokens_seen": 82092032, "tokens_per_sec": 24597.244912160233, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.327465534210205, "grad_norm": 1.0260210037231445, "tokens_seen": 82255872, "tokens_per_sec": 24597.282408257826, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.318425893783569, "grad_norm": 1.095786690711975, "tokens_seen": 82419712, "tokens_per_sec": 24597.30270507927, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.354230165481567, "grad_norm": 1.553205966949463, "tokens_seen": 82583552, "tokens_per_sec": 24597.2322514683, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.385586261749268, "grad_norm": 1.0649734735488892, "tokens_seen": 82747392, "tokens_per_sec": 24597.380910902117, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.3742759227752686, "grad_norm": 0.9509404301643372, "tokens_seen": 82911232, "tokens_per_sec": 24597.573559413915, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.158597707748413, "grad_norm": 1.0065778493881226, "tokens_seen": 83075072, "tokens_per_sec": 24597.65404653856, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.287349224090576, "grad_norm": 1.1831307411193848, "tokens_seen": 83238912, "tokens_per_sec": 24597.817574215293, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.451792478561401, "grad_norm": 1.0433036088943481, "tokens_seen": 83402752, "tokens_per_sec": 24598.06093041375, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.32798433303833, "grad_norm": 1.0451014041900635, "tokens_seen": 83566592, "tokens_per_sec": 24598.180602609304, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.41112756729126, "grad_norm": 1.1396420001983643, "tokens_seen": 83730432, "tokens_per_sec": 24598.417366959133, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.241602659225464, "grad_norm": 1.1436249017715454, "tokens_seen": 83894272, "tokens_per_sec": 24598.454171040452, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.409156560897827, "grad_norm": 1.2107537984848022, "tokens_seen": 84058112, "tokens_per_sec": 24598.545233176523, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.41068172454834, "grad_norm": 1.0818127393722534, "tokens_seen": 84221952, "tokens_per_sec": 24598.75423644934, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.289965391159058, "grad_norm": 1.0936791896820068, "tokens_seen": 84385792, "tokens_per_sec": 24598.982651582406, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.4488325119018555, "grad_norm": 1.1162813901901245, "tokens_seen": 84549632, "tokens_per_sec": 24598.945269302807, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.352364778518677, "grad_norm": 1.1608208417892456, "tokens_seen": 84713472, "tokens_per_sec": 24599.130648872953, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.440574645996094, "grad_norm": 1.1375916004180908, "tokens_seen": 84877312, "tokens_per_sec": 24599.200716429692, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.402225017547607, "grad_norm": 1.1402888298034668, "tokens_seen": 85041152, "tokens_per_sec": 24599.281075154257, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.450283527374268, "grad_norm": 1.110572099685669, "tokens_seen": 85204992, "tokens_per_sec": 24599.506463489033, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.41303563117981, "grad_norm": 1.0712367296218872, "tokens_seen": 85368832, "tokens_per_sec": 24599.649011454712, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.272220134735107, "grad_norm": 1.1804946660995483, "tokens_seen": 85532672, "tokens_per_sec": 24599.75185192307, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.410445928573608, "grad_norm": 1.1124979257583618, "tokens_seen": 85696512, "tokens_per_sec": 24599.80406618311, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.268516540527344, "grad_norm": 1.0603928565979004, "tokens_seen": 85860352, "tokens_per_sec": 24599.92654242795, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.350883483886719, "grad_norm": 1.1324810981750488, "tokens_seen": 86024192, "tokens_per_sec": 24599.972808412145, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.426238059997559, "grad_norm": 1.109974980354309, "tokens_seen": 86188032, "tokens_per_sec": 24599.947688860564, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.345098972320557, "grad_norm": 1.171645998954773, "tokens_seen": 86351872, "tokens_per_sec": 24600.13960058443, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.348527431488037, "grad_norm": 1.0553429126739502, "tokens_seen": 86515712, "tokens_per_sec": 24600.23002929268, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.382145166397095, "grad_norm": 1.251644253730774, "tokens_seen": 86679552, "tokens_per_sec": 24600.29701577428, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.238852024078369, "grad_norm": 0.9996647834777832, "tokens_seen": 86843392, "tokens_per_sec": 24600.51028367624, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.267632007598877, "grad_norm": 1.1501504182815552, "tokens_seen": 87007232, "tokens_per_sec": 24600.742529754796, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.28058385848999, "grad_norm": 1.0250529050827026, "tokens_seen": 87171072, "tokens_per_sec": 24600.713990726046, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.438117504119873, "grad_norm": 1.1347999572753906, "tokens_seen": 87334912, "tokens_per_sec": 24600.910567385334, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.331837892532349, "grad_norm": 1.2426601648330688, "tokens_seen": 87498752, "tokens_per_sec": 24600.98376789177, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.317903995513916, "grad_norm": 1.1009423732757568, "tokens_seen": 87662592, "tokens_per_sec": 24601.0348937848, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.443336009979248, "grad_norm": 1.1772034168243408, "tokens_seen": 87826432, "tokens_per_sec": 24600.938106138696, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.4193291664123535, "grad_norm": 1.2099355459213257, "tokens_seen": 87990272, "tokens_per_sec": 24600.951491411663, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.209381818771362, "grad_norm": 1.1978665590286255, "tokens_seen": 88154112, "tokens_per_sec": 24601.123542540317, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.263108968734741, "grad_norm": 1.3736793994903564, "tokens_seen": 88317952, "tokens_per_sec": 24601.136305609598, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.332732677459717, "grad_norm": 1.1657583713531494, "tokens_seen": 88481792, "tokens_per_sec": 24601.310747534615, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.285773038864136, "grad_norm": 1.1420011520385742, "tokens_seen": 88645632, "tokens_per_sec": 24601.34712435509, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.297527074813843, "grad_norm": 1.141848087310791, "tokens_seen": 88809472, "tokens_per_sec": 24601.218775941576, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.27739143371582, "grad_norm": 1.2246205806732178, "tokens_seen": 88973312, "tokens_per_sec": 24601.21067790783, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.259083271026611, "grad_norm": 1.1956831216812134, "tokens_seen": 89137152, "tokens_per_sec": 24601.21138518336, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.182358503341675, "grad_norm": 1.0511200428009033, "tokens_seen": 89300992, "tokens_per_sec": 24601.40931209999, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.35050630569458, "grad_norm": 1.0676591396331787, "tokens_seen": 89464832, "tokens_per_sec": 24601.576816843557, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.324424743652344, "grad_norm": 1.176755428314209, "tokens_seen": 89628672, "tokens_per_sec": 24601.529190968817, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.312631845474243, "grad_norm": 1.1225353479385376, "tokens_seen": 89792512, "tokens_per_sec": 24601.55034636101, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.4738006591796875, "grad_norm": 1.1914727687835693, "tokens_seen": 89956352, "tokens_per_sec": 24601.654124860663, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.393742799758911, "grad_norm": 1.0436522960662842, "tokens_seen": 90120192, "tokens_per_sec": 24601.87021306775, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.378033256530761, "val_ppl": 79.6811667830548, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.213124990463257, "grad_norm": 1.0985002517700195, "tokens_seen": 90284032, "tokens_per_sec": 24594.253578351476, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.298722743988037, "grad_norm": 1.1033021211624146, "tokens_seen": 90447872, "tokens_per_sec": 24594.22045792972, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.4623823165893555, "grad_norm": 1.2067465782165527, "tokens_seen": 90611712, "tokens_per_sec": 24594.444234616487, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.3255650997161865, "grad_norm": 1.053234338760376, "tokens_seen": 90775552, "tokens_per_sec": 24594.273009119657, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.360315799713135, "grad_norm": 1.2325764894485474, "tokens_seen": 90939392, "tokens_per_sec": 24594.346286460343, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.40711236000061, "grad_norm": 1.1042463779449463, "tokens_seen": 91103232, "tokens_per_sec": 24594.15596282725, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.370296239852905, "grad_norm": 1.1750739812850952, "tokens_seen": 91267072, "tokens_per_sec": 24594.162348825626, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.257128000259399, "grad_norm": 1.0535023212432861, "tokens_seen": 91430912, "tokens_per_sec": 24594.23223269662, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.369032859802246, "grad_norm": 1.2785930633544922, "tokens_seen": 91594752, "tokens_per_sec": 24594.414233731215, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.360669136047363, "grad_norm": 1.1448826789855957, "tokens_seen": 91758592, "tokens_per_sec": 24594.27857588079, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.213503122329712, "grad_norm": 1.113585352897644, "tokens_seen": 91922432, "tokens_per_sec": 24594.458073356334, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.411959886550903, "grad_norm": 1.0655301809310913, "tokens_seen": 92086272, "tokens_per_sec": 24594.547651897214, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.39774489402771, "grad_norm": 1.063378930091858, "tokens_seen": 92250112, "tokens_per_sec": 24594.498406256473, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.427378177642822, "grad_norm": 1.2209258079528809, "tokens_seen": 92413952, "tokens_per_sec": 24594.424393234345, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.273033142089844, "grad_norm": 1.2082258462905884, "tokens_seen": 92577792, "tokens_per_sec": 24594.41797284703, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.3536834716796875, "grad_norm": 1.0310345888137817, "tokens_seen": 92741632, "tokens_per_sec": 24594.412117852735, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.585066556930542, "grad_norm": 1.1981810331344604, "tokens_seen": 92905472, "tokens_per_sec": 24594.382870418132, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.272357940673828, "grad_norm": 1.079086422920227, "tokens_seen": 93069312, "tokens_per_sec": 24594.45099760137, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.352266073226929, "grad_norm": 1.253174901008606, "tokens_seen": 93233152, "tokens_per_sec": 24594.57908128237, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.329866886138916, "grad_norm": 1.2881574630737305, "tokens_seen": 93396992, "tokens_per_sec": 24594.633334320166, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.284115791320801, "grad_norm": 1.1256723403930664, "tokens_seen": 93560832, "tokens_per_sec": 24594.657259247142, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.3702147006988525, "grad_norm": 1.127892255783081, "tokens_seen": 93724672, "tokens_per_sec": 24594.69962408157, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.291997671127319, "grad_norm": 1.1788818836212158, "tokens_seen": 93888512, "tokens_per_sec": 24594.694485665532, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.503677845001221, "grad_norm": 1.119734764099121, "tokens_seen": 94052352, "tokens_per_sec": 24594.649572169412, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.273355960845947, "grad_norm": 1.1257693767547607, "tokens_seen": 94216192, "tokens_per_sec": 24594.649482909797, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.450467348098755, "grad_norm": 1.195599913597107, "tokens_seen": 94380032, "tokens_per_sec": 24594.69269475625, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.37795352935791, "grad_norm": 1.2785358428955078, "tokens_seen": 94543872, "tokens_per_sec": 24594.809667129262, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.351700305938721, "grad_norm": 1.120737910270691, "tokens_seen": 94707712, "tokens_per_sec": 24594.869905749492, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.455416202545166, "grad_norm": 1.1372952461242676, "tokens_seen": 94871552, "tokens_per_sec": 24594.914927866306, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.312630653381348, "grad_norm": 1.294201135635376, "tokens_seen": 95035392, "tokens_per_sec": 24595.001327668073, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.220583558082581, "grad_norm": 1.2488622665405273, "tokens_seen": 95199232, "tokens_per_sec": 24595.11013249468, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.3181312084198, "grad_norm": 1.1959495544433594, "tokens_seen": 95363072, "tokens_per_sec": 24595.23566945326, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.212463617324829, "grad_norm": 1.1940419673919678, "tokens_seen": 95526912, "tokens_per_sec": 24595.338359506568, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.354119062423706, "grad_norm": 1.1249003410339355, "tokens_seen": 95690752, "tokens_per_sec": 24595.376833101032, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.253894329071045, "grad_norm": 1.1879644393920898, "tokens_seen": 95854592, "tokens_per_sec": 24595.495588565464, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.309677362442017, "grad_norm": 1.2800830602645874, "tokens_seen": 96018432, "tokens_per_sec": 24595.61311674214, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.2739222049713135, "grad_norm": 1.0987046957015991, "tokens_seen": 96182272, "tokens_per_sec": 24595.645457675808, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.340238571166992, "grad_norm": 1.1227855682373047, "tokens_seen": 96346112, "tokens_per_sec": 24595.70998364919, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.587297677993774, "grad_norm": 1.5597435235977173, "tokens_seen": 96509952, "tokens_per_sec": 24595.544233365174, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.42743706703186, "grad_norm": 1.1768147945404053, "tokens_seen": 96673792, "tokens_per_sec": 24595.622881942887, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.444751024246216, "grad_norm": 1.2787437438964844, "tokens_seen": 96837632, "tokens_per_sec": 24595.794037628148, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.433004379272461, "grad_norm": 1.1582187414169312, "tokens_seen": 97001472, "tokens_per_sec": 24595.771053914177, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.27366042137146, "grad_norm": 1.228344202041626, "tokens_seen": 97165312, "tokens_per_sec": 24595.864259012265, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.317997932434082, "grad_norm": 1.1356804370880127, "tokens_seen": 97329152, "tokens_per_sec": 24595.929593273664, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.316471099853516, "grad_norm": 1.241527795791626, "tokens_seen": 97492992, "tokens_per_sec": 24595.922080147982, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.301726818084717, "grad_norm": 1.2281150817871094, "tokens_seen": 97656832, "tokens_per_sec": 24595.897877696265, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.471074104309082, "grad_norm": 1.2661408185958862, "tokens_seen": 97820672, "tokens_per_sec": 24595.909226105417, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.387374639511108, "grad_norm": 1.2859017848968506, "tokens_seen": 97984512, "tokens_per_sec": 24596.009933139245, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.286897659301758, "grad_norm": 1.127110481262207, "tokens_seen": 98148352, "tokens_per_sec": 24595.83412591376, "vram_gb": 5.350175744, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.353432512283325, "val_ppl": 77.74486565469935, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 3997.937122106552, "nan_failures": 0} diff --git a/figures_4ksteps/ablation_results.csv b/figures_4ksteps/ablation_results.csv new file mode 100644 index 0000000000000000000000000000000000000000..d2833cd55e295fae40f2e1347d2921fa44c3096a --- /dev/null +++ b/figures_4ksteps/ablation_results.csv @@ -0,0 +1,5 @@ +variant,seed,final_val_ppl,delta_pct_vs_baseline,final_val_loss,final_step,tokens_seen,lambda_start,lambda_end,G_start,G_end +baseline,0,148.44493951411144,+0.00,5.000214111804962,3999,32768000,None,None,None,None +full,0,141.5651204177534,-4.63,4.9527598261833194,3999,32768000,0.12692801654338837,0.16720068734139204,0.5060585426787535,0.4214151619623105 +g_only,0,144.973346966276,-2.34,4.976549911499023,3999,32768000,None,None,0.4969075520833333,0.4358113606770833 +s_only,0,147.89029507035235,-0.37,4.996470749378204,3999,32768000,0.12692801654338837,0.16468850150704384,None,None diff --git a/figures_4ksteps/ablation_table.png b/figures_4ksteps/ablation_table.png new file mode 100644 index 0000000000000000000000000000000000000000..8277f1c98ad7485074c050c86ab38c684e1749c5 Binary files /dev/null and b/figures_4ksteps/ablation_table.png differ diff --git a/figures_4ksteps/gate_activity.png b/figures_4ksteps/gate_activity.png new file mode 100644 index 0000000000000000000000000000000000000000..1aa2131d6b7c3b91d40f0d20fdc1ea4c48e45c15 Binary files /dev/null and b/figures_4ksteps/gate_activity.png differ diff --git a/figures_4ksteps/lambda_trajectory.png b/figures_4ksteps/lambda_trajectory.png new file mode 100644 index 0000000000000000000000000000000000000000..92b21989bd33ef1cd6ab3d20642fe5128d8251e2 Binary files /dev/null and b/figures_4ksteps/lambda_trajectory.png differ diff --git a/figures_4ksteps/loss_curves.png b/figures_4ksteps/loss_curves.png new file mode 100644 index 0000000000000000000000000000000000000000..773eaf7b9b3e8c002dfcb5b56761fc156326a39a Binary files /dev/null and b/figures_4ksteps/loss_curves.png differ diff --git a/figures_seed0/ablation_results.csv b/figures_seed0/ablation_results.csv new file mode 100644 index 0000000000000000000000000000000000000000..6c0972016e185baafd2e8871b4ae207cede97bbc --- /dev/null +++ b/figures_seed0/ablation_results.csv @@ -0,0 +1,5 @@ +variant,seed,final_val_ppl,delta_pct_vs_baseline,final_val_loss,final_step,tokens_seen,lambda_start,lambda_end,G_start,G_end +baseline,0,78.32438343443285,+0.00,4.360858964920044,11999,98304000,None,None,None,None +full,0,77.86344065821494,-0.59,4.354956531524659,11999,98304000,0.12692801654338837,0.3069125845407446,0.5030857535700003,0.3505655957075457 +g_only,0,78.21752726763506,-0.14,4.359493756294251,11999,98304000,None,None,0.4967447916666667,0.3484141031901042 +s_only,0,78.7377125064686,+0.53,4.366122233867645,11999,98304000,0.12692801654338837,0.31397200624148053,None,None diff --git a/figures_seed0/ablation_table.png b/figures_seed0/ablation_table.png new file mode 100644 index 0000000000000000000000000000000000000000..3dccdc0353909fc31d7ab678f7d9b4e8934e0075 Binary files /dev/null and b/figures_seed0/ablation_table.png differ diff --git a/figures_seed0/gate_activity.png b/figures_seed0/gate_activity.png new file mode 100644 index 0000000000000000000000000000000000000000..b628ac37d77fd43d52de3826a0657c06b619c1d5 Binary files /dev/null and b/figures_seed0/gate_activity.png differ diff --git a/figures_seed0/lambda_per_layer.png b/figures_seed0/lambda_per_layer.png new file mode 100644 index 0000000000000000000000000000000000000000..7c7feb200743cd959b0fb19177d6151e41fb1a86 --- /dev/null +++ b/figures_seed0/lambda_per_layer.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:15dbebcafa80d99a9227ce377ab7bfd8c9b131cf086e71bf78a7fcc2b6cc15bc +size 112693 diff --git a/figures_seed0/lambda_trajectory.png b/figures_seed0/lambda_trajectory.png new file mode 100644 index 0000000000000000000000000000000000000000..40e92aeb5633796c2deb8659075253f3d4bd88a7 Binary files /dev/null and b/figures_seed0/lambda_trajectory.png differ diff --git a/figures_seed0/loss_curves.png b/figures_seed0/loss_curves.png new file mode 100644 index 0000000000000000000000000000000000000000..41e1ba2315971694901829b403f7c974a14d6557 Binary files /dev/null and b/figures_seed0/loss_curves.png differ diff --git a/full_s0/config.json b/full_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..5a92dc4c108672f0b19f7e9483c3e3248d7e45f2 --- /dev/null +++ b/full_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s0", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/full_s0/final.pt b/full_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..963fb204a9dec0800c9265a340e77497396c6edf --- /dev/null +++ b/full_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5a8d83d8725d3cfe8ed58aa8720b6a40900aed797f617addfea94d49a315bda +size 256003111 diff --git a/full_s0/metrics.jsonl b/full_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..bb1bfdb3c2d1adbfd3b646e8e72619480ddbf16a --- /dev/null +++ b/full_s0/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 0, "config": {"variant": "full", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s0", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.902755737304688, "grad_norm": 9.849164962768555, "tokens_seen": 8192, "tokens_per_sec": 7412.056843747, "vram_gb": 5.920787456, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.17699432373046875, "G_mean": 0.5030857535700003, "G_std": 0.08675276922682922, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.783544540405273, "grad_norm": 1.9391835927963257, "tokens_seen": 172032, "tokens_per_sec": 14665.843335665328, "vram_gb": 6.442267648, "lambda": 0.12692161525289217, "lambda_min": 0.12687550485134125, "lambda_max": 0.12695592641830444, "S_mean": 0.4437510172526042, "G_mean": 0.47064803664882976, "G_std": 0.06774007955876489, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.060419082641602, "grad_norm": 1.7936872243881226, "tokens_seen": 335872, "tokens_per_sec": 14895.267401701838, "vram_gb": 6.442267648, "lambda": 0.12684216164052486, "lambda_min": 0.12677119672298431, "lambda_max": 0.1269581913948059, "S_mean": 0.4613393147786458, "G_mean": 0.45263293012976646, "G_std": 0.07626633695326746, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.113292694091797, "grad_norm": 1.5507972240447998, "tokens_seen": 499712, "tokens_per_sec": 15032.11411084941, "vram_gb": 6.442267648, "lambda": 0.12675164019068083, "lambda_min": 0.12664151191711426, "lambda_max": 0.12703245878219604, "S_mean": 0.4600982666015625, "G_mean": 0.45841004078586894, "G_std": 0.08083176364501317, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.5704545974731445, "grad_norm": 0.8774337768554688, "tokens_seen": 663552, "tokens_per_sec": 15121.140059319616, "vram_gb": 6.442267648, "lambda": 0.12665745243430138, "lambda_min": 0.12651881575584412, "lambda_max": 0.12702292203903198, "S_mean": 0.4110209147135417, "G_mean": 0.45741988345980644, "G_std": 0.0923708884511143, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.303678035736084, "grad_norm": 0.6080539226531982, "tokens_seen": 827392, "tokens_per_sec": 15173.403438713887, "vram_gb": 6.442267648, "lambda": 0.12657030237217745, "lambda_min": 0.12637200951576233, "lambda_max": 0.1270727515220642, "S_mean": 0.3252766927083333, "G_mean": 0.45109697928031284, "G_std": 0.10429403589417537, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.07873797416687, "grad_norm": 0.5846303105354309, "tokens_seen": 991232, "tokens_per_sec": 15203.895696537678, "vram_gb": 6.442267648, "lambda": 0.1265490793933471, "lambda_min": 0.1262207180261612, "lambda_max": 0.12718799710273743, "S_mean": 0.2832845052083333, "G_mean": 0.4301280876000722, "G_std": 0.11022276307145755, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.972445726394653, "grad_norm": 0.5821143388748169, "tokens_seen": 1155072, "tokens_per_sec": 15228.157172088344, "vram_gb": 6.442267648, "lambda": 0.12653516170879206, "lambda_min": 0.12610271573066711, "lambda_max": 0.1272432953119278, "S_mean": 0.31231689453125, "G_mean": 0.420130950709184, "G_std": 0.12349979610492785, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.80590033531189, "grad_norm": 0.8342288732528687, "tokens_seen": 1318912, "tokens_per_sec": 15239.404522744999, "vram_gb": 6.442267648, "lambda": 0.12643708661198616, "lambda_min": 0.1257990449666977, "lambda_max": 0.12717370688915253, "S_mean": 0.29998779296875, "G_mean": 0.4219328748683135, "G_std": 0.1326294271275401, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.857113361358643, "grad_norm": 0.5344687104225159, "tokens_seen": 1482752, "tokens_per_sec": 15260.276326890978, "vram_gb": 6.442267648, "lambda": 0.12642721955974898, "lambda_min": 0.1256375014781952, "lambda_max": 0.1272270530462265, "S_mean": 0.3313395182291667, "G_mean": 0.40475064516067505, "G_std": 0.13966316217556596, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.686572551727295, "grad_norm": 0.5693752765655518, "tokens_seen": 1646592, "tokens_per_sec": 15277.689414068003, "vram_gb": 6.442267648, "lambda": 0.12640887995560965, "lambda_min": 0.12529566884040833, "lambda_max": 0.12729617953300476, "S_mean": 0.2992757161458333, "G_mean": 0.4089492931962013, "G_std": 0.15125936393936476, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.827714920043945, "grad_norm": 0.5897318720817566, "tokens_seen": 1810432, "tokens_per_sec": 15292.185940369289, "vram_gb": 6.442267648, "lambda": 0.12637591734528542, "lambda_min": 0.1250058263540268, "lambda_max": 0.12731200456619263, "S_mean": 0.2816670735677083, "G_mean": 0.4014202232162158, "G_std": 0.16228430749227604, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.546076774597168, "grad_norm": 0.4790916442871094, "tokens_seen": 1974272, "tokens_per_sec": 15299.540146518459, "vram_gb": 6.442267648, "lambda": 0.1263762547944983, "lambda_min": 0.12484916299581528, "lambda_max": 0.1275469958782196, "S_mean": 0.2994588216145833, "G_mean": 0.4008398416141669, "G_std": 0.15552720396469036, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.64783239364624, "grad_norm": 0.5151584148406982, "tokens_seen": 2138112, "tokens_per_sec": 15306.803262113743, "vram_gb": 6.442267648, "lambda": 0.12640171311795712, "lambda_min": 0.12481871247291565, "lambda_max": 0.12770189344882965, "S_mean": 0.30419921875, "G_mean": 0.3963535465300083, "G_std": 0.1636720011010766, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.413406610488892, "grad_norm": 0.5548398494720459, "tokens_seen": 2301952, "tokens_per_sec": 15309.444988622465, "vram_gb": 6.442267648, "lambda": 0.12637430553634962, "lambda_min": 0.12462185323238373, "lambda_max": 0.12789011001586914, "S_mean": 0.3267720540364583, "G_mean": 0.39529188101490337, "G_std": 0.16774355433881283, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.546427011489868, "grad_norm": 0.47361627221107483, "tokens_seen": 2465792, "tokens_per_sec": 15313.583029545447, "vram_gb": 6.442267648, "lambda": 0.12641033654411635, "lambda_min": 0.12460367381572723, "lambda_max": 0.1279575079679489, "S_mean": 0.28765869140625, "G_mean": 0.4036916022499402, "G_std": 0.17650944025566181, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.346770286560059, "grad_norm": 0.4397766888141632, "tokens_seen": 2629632, "tokens_per_sec": 15318.12815381789, "vram_gb": 6.442267648, "lambda": 0.12642817137142023, "lambda_min": 0.12433713674545288, "lambda_max": 0.1279725283384323, "S_mean": 0.307281494140625, "G_mean": 0.4039401262998581, "G_std": 0.1848672085131208, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.38763952255249, "grad_norm": 0.4749566912651062, "tokens_seen": 2793472, "tokens_per_sec": 15323.616899582401, "vram_gb": 6.442267648, "lambda": 0.12647280345360437, "lambda_min": 0.12393172085285187, "lambda_max": 0.12806692719459534, "S_mean": 0.2986653645833333, "G_mean": 0.4167020283639431, "G_std": 0.1954538207501173, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.5535430908203125, "grad_norm": 0.4799998700618744, "tokens_seen": 2957312, "tokens_per_sec": 15326.879056011388, "vram_gb": 6.442267648, "lambda": 0.12646443210542202, "lambda_min": 0.12376353144645691, "lambda_max": 0.12831445038318634, "S_mean": 0.3111368815104167, "G_mean": 0.4114844873547554, "G_std": 0.1959647492816051, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.393575668334961, "grad_norm": 0.5101221799850464, "tokens_seen": 3121152, "tokens_per_sec": 15328.535209966734, "vram_gb": 6.442267648, "lambda": 0.12644707473615804, "lambda_min": 0.12365531176328659, "lambda_max": 0.12829282879829407, "S_mean": 0.3017374674479167, "G_mean": 0.39927731330196065, "G_std": 0.19815452148516974, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.2887046337127686, "grad_norm": 0.4521481692790985, "tokens_seen": 3284992, "tokens_per_sec": 15331.316666673565, "vram_gb": 6.442267648, "lambda": 0.1265264286970099, "lambda_min": 0.12368454039096832, "lambda_max": 0.12844030559062958, "S_mean": 0.2938028971354167, "G_mean": 0.41610954081018764, "G_std": 0.20868099698175988, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.414016485214233, "grad_norm": 0.46566760540008545, "tokens_seen": 3448832, "tokens_per_sec": 15334.452152319385, "vram_gb": 6.442267648, "lambda": 0.1266352403908968, "lambda_min": 0.12336984276771545, "lambda_max": 0.12867051362991333, "S_mean": 0.3051554361979167, "G_mean": 0.4286409330864747, "G_std": 0.21975880892326435, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.359542369842529, "grad_norm": 0.4962195158004761, "tokens_seen": 3612672, "tokens_per_sec": 15335.73385210737, "vram_gb": 6.442267648, "lambda": 0.12672430432091156, "lambda_min": 0.12343012541532516, "lambda_max": 0.1291559338569641, "S_mean": 0.3478190104166667, "G_mean": 0.4180796990791957, "G_std": 0.21676430075118938, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.394898891448975, "grad_norm": 0.4405446946620941, "tokens_seen": 3776512, "tokens_per_sec": 15334.229110817596, "vram_gb": 6.442267648, "lambda": 0.12681444454938173, "lambda_min": 0.1234099343419075, "lambda_max": 0.12934213876724243, "S_mean": 0.3491414388020833, "G_mean": 0.43334127217531204, "G_std": 0.21924616986264786, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.180383920669556, "grad_norm": 0.4425899386405945, "tokens_seen": 3940352, "tokens_per_sec": 15333.265797651718, "vram_gb": 6.442267648, "lambda": 0.12698069928834835, "lambda_min": 0.12335556000471115, "lambda_max": 0.12993434071540833, "S_mean": 0.3723347981770833, "G_mean": 0.4337771063049634, "G_std": 0.22069154679775238, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.058091640472412, "grad_norm": 0.495559960603714, "tokens_seen": 4104192, "tokens_per_sec": 15334.809649205243, "vram_gb": 6.442267648, "lambda": 0.12712886463850737, "lambda_min": 0.12347199767827988, "lambda_max": 0.13010095059871674, "S_mean": 0.3714803059895833, "G_mean": 0.447833130757014, "G_std": 0.22643144025156894, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.28065505027771, "val_ppl": 534.1384367849688, "lambda": 0.12713577567289272, "lambda_min": 0.12343979626893997, "lambda_max": 0.1301042139530182, "lambda_per_layer": [0.1301042139530182, 0.12925443053245544, 0.1286480575799942, 0.12851613759994507, 0.12583625316619873, 0.12552809715270996, 0.126484677195549, 0.12750190496444702, 0.1257655918598175, 0.12862782180309296, 0.12908685207366943, 0.12831386923789978, 0.12762537598609924, 0.1278669685125351, 0.12699414789676666, 0.1269710212945938, 0.12623539566993713, 0.12811879813671112, 0.12680819630622864, 0.1258712112903595, 0.12343979626893997, 0.12650291621685028, 0.1259949505329132, 0.12516193091869354], "S_mean": 0.3823140462239583, "G_mean": 0.44824398929874104, "G_std": 0.2276501009861628, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.24564790725708, "grad_norm": 0.5270732641220093, "tokens_seen": 4268032, "tokens_per_sec": 15112.19465123212, "vram_gb": 6.442267648, "lambda": 0.1272899117320776, "lambda_min": 0.12335407733917236, "lambda_max": 0.13049176335334778, "S_mean": 0.3538818359375, "G_mean": 0.4641081790129344, "G_std": 0.22841936443001032, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.178030967712402, "grad_norm": 0.5092683434486389, "tokens_seen": 4431872, "tokens_per_sec": 15118.132249703924, "vram_gb": 6.442267648, "lambda": 0.1274485563238462, "lambda_min": 0.12339864671230316, "lambda_max": 0.13086485862731934, "S_mean": 0.382293701171875, "G_mean": 0.44570186113317806, "G_std": 0.22503506081799665, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.140071868896484, "grad_norm": 0.46432241797447205, "tokens_seen": 4595712, "tokens_per_sec": 15127.13966880326, "vram_gb": 6.442267648, "lambda": 0.12760758120566607, "lambda_min": 0.12334910780191422, "lambda_max": 0.13128460943698883, "S_mean": 0.3924153645833333, "G_mean": 0.4752626543243726, "G_std": 0.224220823807021, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.197661876678467, "grad_norm": 0.5055051445960999, "tokens_seen": 4759552, "tokens_per_sec": 15131.016852872528, "vram_gb": 6.442267648, "lambda": 0.12774930521845818, "lambda_min": 0.12331797182559967, "lambda_max": 0.13148798048496246, "S_mean": 0.3858846028645833, "G_mean": 0.47633228326837224, "G_std": 0.2311538482705752, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.1416544914245605, "grad_norm": 0.49198853969573975, "tokens_seen": 4923392, "tokens_per_sec": 15139.447635123757, "vram_gb": 6.442267648, "lambda": 0.12792953196913004, "lambda_min": 0.1231345683336258, "lambda_max": 0.13173213601112366, "S_mean": 0.3989359537760417, "G_mean": 0.5081705699364344, "G_std": 0.22872159102310738, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.037497520446777, "grad_norm": 0.46052685379981995, "tokens_seen": 5087232, "tokens_per_sec": 15146.662675917805, "vram_gb": 6.442267648, "lambda": 0.12815746354560056, "lambda_min": 0.12284742295742035, "lambda_max": 0.13245783746242523, "S_mean": 0.375274658203125, "G_mean": 0.49799548586209613, "G_std": 0.22769542100528875, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.139193534851074, "grad_norm": 0.5044844746589661, "tokens_seen": 5251072, "tokens_per_sec": 15152.436848230114, "vram_gb": 6.442267648, "lambda": 0.12842637362579504, "lambda_min": 0.12313351035118103, "lambda_max": 0.1333184838294983, "S_mean": 0.4140828450520833, "G_mean": 0.5167213914295038, "G_std": 0.22566450014710426, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.009713649749756, "grad_norm": 0.49754461646080017, "tokens_seen": 5414912, "tokens_per_sec": 15159.102014893018, "vram_gb": 6.442267648, "lambda": 0.12869772935907045, "lambda_min": 0.12310034781694412, "lambda_max": 0.1340305656194687, "S_mean": 0.4039713541666667, "G_mean": 0.539153183499972, "G_std": 0.2214450885852178, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 5.99549126625061, "grad_norm": 0.46074870228767395, "tokens_seen": 5578752, "tokens_per_sec": 15163.623992072096, "vram_gb": 6.442267648, "lambda": 0.12900689585755268, "lambda_min": 0.12321944534778595, "lambda_max": 0.135030135512352, "S_mean": 0.41583251953125, "G_mean": 0.5239760689437389, "G_std": 0.22155350322524706, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.987174034118652, "grad_norm": 0.5105113983154297, "tokens_seen": 5742592, "tokens_per_sec": 15168.961733588789, "vram_gb": 6.442267648, "lambda": 0.12928377681722245, "lambda_min": 0.12340271472930908, "lambda_max": 0.13600987195968628, "S_mean": 0.41204833984375, "G_mean": 0.5265582477053007, "G_std": 0.21969810935358206, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.076775789260864, "grad_norm": 0.49765875935554504, "tokens_seen": 5906432, "tokens_per_sec": 15173.092741987157, "vram_gb": 6.442267648, "lambda": 0.12954649205009142, "lambda_min": 0.12347757816314697, "lambda_max": 0.1371350884437561, "S_mean": 0.39337158203125, "G_mean": 0.5424879031876723, "G_std": 0.21298224665224552, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.917410612106323, "grad_norm": 0.4788220524787903, "tokens_seen": 6070272, "tokens_per_sec": 15176.64007725079, "vram_gb": 6.442267648, "lambda": 0.12987229755769172, "lambda_min": 0.12372133135795593, "lambda_max": 0.13835208117961884, "S_mean": 0.4108072916666667, "G_mean": 0.5502202585339546, "G_std": 0.2103519228597482, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.903132915496826, "grad_norm": 0.5018932819366455, "tokens_seen": 6234112, "tokens_per_sec": 15180.43920381672, "vram_gb": 6.442267648, "lambda": 0.13020805486788353, "lambda_min": 0.12364349514245987, "lambda_max": 0.13931812345981598, "S_mean": 0.4168701171875, "G_mean": 0.5586358122527599, "G_std": 0.20994325447827578, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.933964252471924, "grad_norm": 0.4858669638633728, "tokens_seen": 6397952, "tokens_per_sec": 15185.622801375877, "vram_gb": 6.442267648, "lambda": 0.13058105887224278, "lambda_min": 0.12398134917020798, "lambda_max": 0.14046718180179596, "S_mean": 0.4191487630208333, "G_mean": 0.5582266089816889, "G_std": 0.20568901052077612, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.820725440979004, "grad_norm": 0.5501937866210938, "tokens_seen": 6561792, "tokens_per_sec": 15189.81866093683, "vram_gb": 6.442267648, "lambda": 0.13096831801036993, "lambda_min": 0.12380237132310867, "lambda_max": 0.14179790019989014, "S_mean": 0.4142049153645833, "G_mean": 0.5530718813339869, "G_std": 0.20588501232365766, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.78459095954895, "grad_norm": 0.465146005153656, "tokens_seen": 6725632, "tokens_per_sec": 15194.373894276892, "vram_gb": 6.442267648, "lambda": 0.13132955630620322, "lambda_min": 0.12342477589845657, "lambda_max": 0.14303795993328094, "S_mean": 0.4069417317708333, "G_mean": 0.569093340386947, "G_std": 0.1965185391406218, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.768403768539429, "grad_norm": 0.49010032415390015, "tokens_seen": 6889472, "tokens_per_sec": 15198.185406067169, "vram_gb": 6.442267648, "lambda": 0.13163264064739147, "lambda_min": 0.12319299578666687, "lambda_max": 0.14424677193164825, "S_mean": 0.4060872395833333, "G_mean": 0.5557899003227552, "G_std": 0.19782377934704223, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.736021518707275, "grad_norm": 0.5032194256782532, "tokens_seen": 7053312, "tokens_per_sec": 15201.83086670034, "vram_gb": 6.442267648, "lambda": 0.13197322034587464, "lambda_min": 0.12282545119524002, "lambda_max": 0.14563269913196564, "S_mean": 0.3851114908854167, "G_mean": 0.5638702635963758, "G_std": 0.19727550291766724, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.888177156448364, "grad_norm": 0.533523678779602, "tokens_seen": 7217152, "tokens_per_sec": 15203.84155788634, "vram_gb": 6.442267648, "lambda": 0.13233029997597137, "lambda_min": 0.12267904728651047, "lambda_max": 0.1467026174068451, "S_mean": 0.4186197916666667, "G_mean": 0.5668986402451992, "G_std": 0.1940642468010386, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.729521751403809, "grad_norm": 0.4932403862476349, "tokens_seen": 7380992, "tokens_per_sec": 15205.262329288844, "vram_gb": 6.442267648, "lambda": 0.13276872225105762, "lambda_min": 0.1224440261721611, "lambda_max": 0.14793919026851654, "S_mean": 0.3778483072916667, "G_mean": 0.5559453802804152, "G_std": 0.19952705533554158, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.770722150802612, "grad_norm": 0.5197076201438904, "tokens_seen": 7544832, "tokens_per_sec": 15207.900986331695, "vram_gb": 6.442267648, "lambda": 0.13312861137092113, "lambda_min": 0.121967613697052, "lambda_max": 0.1491268277168274, "S_mean": 0.4109293619791667, "G_mean": 0.5541894050935904, "G_std": 0.19882339083900055, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.721481800079346, "grad_norm": 0.4719458818435669, "tokens_seen": 7708672, "tokens_per_sec": 15211.066003328384, "vram_gb": 6.442267648, "lambda": 0.1335356617346406, "lambda_min": 0.12167910486459732, "lambda_max": 0.1503293216228485, "S_mean": 0.4002482096354167, "G_mean": 0.5472374310096105, "G_std": 0.19676211290061474, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.761938571929932, "grad_norm": 0.5439544916152954, "tokens_seen": 7872512, "tokens_per_sec": 15214.406196479256, "vram_gb": 6.442267648, "lambda": 0.1339366969962915, "lambda_min": 0.12133496254682541, "lambda_max": 0.15159325301647186, "S_mean": 0.38690185546875, "G_mean": 0.549694117779533, "G_std": 0.19479672921200594, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.8843605518341064, "grad_norm": 0.5474268198013306, "tokens_seen": 8036352, "tokens_per_sec": 15217.13717459948, "vram_gb": 6.442267648, "lambda": 0.1343954810872674, "lambda_min": 0.12110814452171326, "lambda_max": 0.15291646122932434, "S_mean": 0.4065958658854167, "G_mean": 0.5470301850388447, "G_std": 0.19582459734131893, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.806299209594727, "grad_norm": 0.5273318290710449, "tokens_seen": 8200192, "tokens_per_sec": 15220.886529651027, "vram_gb": 6.442267648, "lambda": 0.13477198841671148, "lambda_min": 0.1208634078502655, "lambda_max": 0.15392015874385834, "S_mean": 0.406494140625, "G_mean": 0.5581433965514103, "G_std": 0.19189688842743635, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.797925102710724, "val_ppl": 329.61493275927353, "lambda": 0.13479192027201256, "lambda_min": 0.1208595559000969, "lambda_max": 0.1539555788040161, "lambda_per_layer": [0.13745225965976715, 0.13257698714733124, 0.13239198923110962, 0.1304393708705902, 0.12371046841144562, 0.1208595559000969, 0.127107635140419, 0.12613138556480408, 0.1254536211490631, 0.1455443799495697, 0.150276318192482, 0.14045293629169464, 0.1382441222667694, 0.1539555788040161, 0.1353747546672821, 0.14283594489097595, 0.13989311456680298, 0.13823379576206207, 0.14191703498363495, 0.13699078559875488, 0.12497501075267792, 0.13300468027591705, 0.1295442432165146, 0.12764011323451996], "S_mean": 0.4151407877604167, "G_mean": 0.5622756040344635, "G_std": 0.19070991904785237, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.611230850219727, "grad_norm": 0.5194713473320007, "tokens_seen": 8364032, "tokens_per_sec": 15111.89894214041, "vram_gb": 6.442267648, "lambda": 0.13516638800501823, "lambda_min": 0.12054195255041122, "lambda_max": 0.15479956567287445, "S_mean": 0.39154052734375, "G_mean": 0.5565496012568474, "G_std": 0.18924140060941377, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.67523455619812, "grad_norm": 0.46954721212387085, "tokens_seen": 8527872, "tokens_per_sec": 15114.972409435877, "vram_gb": 6.442267648, "lambda": 0.1355804561947783, "lambda_min": 0.12026644498109818, "lambda_max": 0.1559748500585556, "S_mean": 0.3931477864583333, "G_mean": 0.5382961761206388, "G_std": 0.1929577412083745, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.7272162437438965, "grad_norm": 0.47673937678337097, "tokens_seen": 8691712, "tokens_per_sec": 15120.429466728001, "vram_gb": 6.442267648, "lambda": 0.13602483086287975, "lambda_min": 0.1199847012758255, "lambda_max": 0.15725773572921753, "S_mean": 0.4099527994791667, "G_mean": 0.5388183612376451, "G_std": 0.1873296626533071, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.70568323135376, "grad_norm": 0.48693904280662537, "tokens_seen": 8855552, "tokens_per_sec": 15125.372839833994, "vram_gb": 6.442267648, "lambda": 0.13646685549368462, "lambda_min": 0.11957202851772308, "lambda_max": 0.158342644572258, "S_mean": 0.40716552734375, "G_mean": 0.5334495715796947, "G_std": 0.19195827934890985, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.605070352554321, "grad_norm": 0.496533066034317, "tokens_seen": 9019392, "tokens_per_sec": 15130.165225076758, "vram_gb": 6.442267648, "lambda": 0.13685248543818793, "lambda_min": 0.1192212849855423, "lambda_max": 0.15963657200336456, "S_mean": 0.4101155598958333, "G_mean": 0.5363288577646017, "G_std": 0.18727843587597212, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.698260068893433, "grad_norm": 0.5189411640167236, "tokens_seen": 9183232, "tokens_per_sec": 15135.037286825107, "vram_gb": 6.442267648, "lambda": 0.1372672182818254, "lambda_min": 0.11905691772699356, "lambda_max": 0.1608155518770218, "S_mean": 0.41357421875, "G_mean": 0.5416510012000799, "G_std": 0.19096309877932072, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.5237085819244385, "grad_norm": 0.5040441751480103, "tokens_seen": 9347072, "tokens_per_sec": 15139.344751249182, "vram_gb": 6.442267648, "lambda": 0.13769453205168247, "lambda_min": 0.11887098848819733, "lambda_max": 0.1616327166557312, "S_mean": 0.410888671875, "G_mean": 0.5288531885792812, "G_std": 0.1885130883504947, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.57066011428833, "grad_norm": 0.4897090196609497, "tokens_seen": 9510912, "tokens_per_sec": 15143.487891164903, "vram_gb": 6.442267648, "lambda": 0.13812631585945687, "lambda_min": 0.11852136999368668, "lambda_max": 0.16287972033023834, "S_mean": 0.3995768229166667, "G_mean": 0.5419504096110662, "G_std": 0.1876325746998191, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.629234790802002, "grad_norm": 0.47788530588150024, "tokens_seen": 9674752, "tokens_per_sec": 15147.263618567575, "vram_gb": 6.442267648, "lambda": 0.13858075874547163, "lambda_min": 0.11822119355201721, "lambda_max": 0.1641693115234375, "S_mean": 0.4010213216145833, "G_mean": 0.5418329760432243, "G_std": 0.19044733482102552, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.510490655899048, "grad_norm": 0.5578616857528687, "tokens_seen": 9838592, "tokens_per_sec": 15150.28257592323, "vram_gb": 6.442267648, "lambda": 0.13903489398459593, "lambda_min": 0.11786401271820068, "lambda_max": 0.165374293923378, "S_mean": 0.4109090169270833, "G_mean": 0.5176569943626722, "G_std": 0.1903478732953469, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.68370246887207, "grad_norm": 0.5080989003181458, "tokens_seen": 10002432, "tokens_per_sec": 15153.822392044684, "vram_gb": 6.442267648, "lambda": 0.13948220138748488, "lambda_min": 0.11759413778781891, "lambda_max": 0.1665840595960617, "S_mean": 0.4295654296875, "G_mean": 0.5235080905258656, "G_std": 0.19356819211194912, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.649246454238892, "grad_norm": 0.5167788863182068, "tokens_seen": 10166272, "tokens_per_sec": 15157.467459034242, "vram_gb": 6.442267648, "lambda": 0.13990986657639345, "lambda_min": 0.1173185333609581, "lambda_max": 0.167526513338089, "S_mean": 0.4183553059895833, "G_mean": 0.5182733138402303, "G_std": 0.1944649058083693, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.61809229850769, "grad_norm": 0.5522627830505371, "tokens_seen": 10330112, "tokens_per_sec": 15161.465931534767, "vram_gb": 6.442267648, "lambda": 0.14034730475395918, "lambda_min": 0.11698949337005615, "lambda_max": 0.16863538324832916, "S_mean": 0.4228719075520833, "G_mean": 0.5297393829872211, "G_std": 0.19292587538560232, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.475043535232544, "grad_norm": 0.553426206111908, "tokens_seen": 10493952, "tokens_per_sec": 15165.259740169648, "vram_gb": 6.442267648, "lambda": 0.14076858821014562, "lambda_min": 0.11644826084375381, "lambda_max": 0.16979378461837769, "S_mean": 0.3942667643229167, "G_mean": 0.5189812332391739, "G_std": 0.18947422287116447, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.563565969467163, "grad_norm": 0.6384329795837402, "tokens_seen": 10657792, "tokens_per_sec": 15168.774993387053, "vram_gb": 6.442267648, "lambda": 0.14123849043001732, "lambda_min": 0.11606378853321075, "lambda_max": 0.17101825773715973, "S_mean": 0.4079182942708333, "G_mean": 0.5294869635254145, "G_std": 0.1870218412950635, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.528751373291016, "grad_norm": 0.5330633521080017, "tokens_seen": 10821632, "tokens_per_sec": 15171.648831898043, "vram_gb": 6.442267648, "lambda": 0.14167054556310177, "lambda_min": 0.11581023782491684, "lambda_max": 0.1719743013381958, "S_mean": 0.4227091471354167, "G_mean": 0.5145816070338091, "G_std": 0.1956677936638395, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.614763259887695, "grad_norm": 0.5420897603034973, "tokens_seen": 10985472, "tokens_per_sec": 15174.102888873973, "vram_gb": 6.442267648, "lambda": 0.1420488211636742, "lambda_min": 0.11527657508850098, "lambda_max": 0.17308670282363892, "S_mean": 0.4125162760416667, "G_mean": 0.5206595876564583, "G_std": 0.19184663550307354, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.436861515045166, "grad_norm": 0.5380321741104126, "tokens_seen": 11149312, "tokens_per_sec": 15176.82079493464, "vram_gb": 6.442267648, "lambda": 0.1424604132771492, "lambda_min": 0.11501054465770721, "lambda_max": 0.17412075400352478, "S_mean": 0.4291788736979167, "G_mean": 0.5144363678991795, "G_std": 0.18912744785969457, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.270485877990723, "grad_norm": 0.5237792134284973, "tokens_seen": 11313152, "tokens_per_sec": 15179.670294226366, "vram_gb": 6.442267648, "lambda": 0.14290577142188945, "lambda_min": 0.11438137292861938, "lambda_max": 0.1751495748758316, "S_mean": 0.4056396484375, "G_mean": 0.5145300648485621, "G_std": 0.18405937248220047, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.554769039154053, "grad_norm": 0.5467957854270935, "tokens_seen": 11476992, "tokens_per_sec": 15181.67472266628, "vram_gb": 6.442267648, "lambda": 0.14335099204132953, "lambda_min": 0.11419346183538437, "lambda_max": 0.17654404044151306, "S_mean": 0.4232177734375, "G_mean": 0.51185775299867, "G_std": 0.18728972241903344, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.501059532165527, "grad_norm": 0.5506683588027954, "tokens_seen": 11640832, "tokens_per_sec": 15184.321379768067, "vram_gb": 6.442267648, "lambda": 0.14381762593984604, "lambda_min": 0.11388105154037476, "lambda_max": 0.1779271960258484, "S_mean": 0.4161783854166667, "G_mean": 0.4992476760720213, "G_std": 0.18700562293330827, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.398487329483032, "grad_norm": 0.5276551842689514, "tokens_seen": 11804672, "tokens_per_sec": 15187.707396219052, "vram_gb": 6.442267648, "lambda": 0.14428733258197704, "lambda_min": 0.11329394578933716, "lambda_max": 0.1791129857301712, "S_mean": 0.4308675130208333, "G_mean": 0.4918586763863762, "G_std": 0.19126568331072727, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.51033878326416, "grad_norm": 0.5695117115974426, "tokens_seen": 11968512, "tokens_per_sec": 15190.518175170919, "vram_gb": 6.442267648, "lambda": 0.14477369282394648, "lambda_min": 0.1127861961722374, "lambda_max": 0.18038217723369598, "S_mean": 0.4054972330729167, "G_mean": 0.5062761837616563, "G_std": 0.19476227089762688, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.401941299438477, "grad_norm": 0.5249546766281128, "tokens_seen": 12132352, "tokens_per_sec": 15193.084991307678, "vram_gb": 6.442267648, "lambda": 0.14526124050219855, "lambda_min": 0.11224128305912018, "lambda_max": 0.18177863955497742, "S_mean": 0.4105021158854167, "G_mean": 0.5015654700497786, "G_std": 0.1925058802589774, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.636854648590088, "grad_norm": 0.5683583617210388, "tokens_seen": 12296192, "tokens_per_sec": 15196.007495080845, "vram_gb": 6.442267648, "lambda": 0.1456847603743275, "lambda_min": 0.11186347901821136, "lambda_max": 0.18279320001602173, "S_mean": 0.4200642903645833, "G_mean": 0.4963273545727134, "G_std": 0.18818704100946584, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.520173788070679, "val_ppl": 249.67842455117454, "lambda": 0.1457106824964285, "lambda_min": 0.11183574795722961, "lambda_max": 0.18285900354385376, "lambda_per_layer": [0.14258375763893127, 0.13455361127853394, 0.13355693221092224, 0.13099317252635956, 0.11987367272377014, 0.11183574795722961, 0.127712681889534, 0.1219986230134964, 0.12510354816913605, 0.17411793768405914, 0.17199838161468506, 0.16643166542053223, 0.15653342008590698, 0.18285900354385376, 0.15932954847812653, 0.16386321187019348, 0.16531625390052795, 0.1572493612766266, 0.15568089485168457, 0.1559867113828659, 0.12719042599201202, 0.14576798677444458, 0.13595253229141235, 0.1305672973394394], "S_mean": 0.4105122884114583, "G_mean": 0.5030220905318856, "G_std": 0.188745248131454, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.420814275741577, "grad_norm": 0.5586658120155334, "tokens_seen": 12460032, "tokens_per_sec": 15123.872678177886, "vram_gb": 6.442267648, "lambda": 0.146172607007126, "lambda_min": 0.11133387684822083, "lambda_max": 0.18414749205112457, "S_mean": 0.39959716796875, "G_mean": 0.49396850261837244, "G_std": 0.19093746164192757, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.272874593734741, "grad_norm": 0.512881875038147, "tokens_seen": 12623872, "tokens_per_sec": 15125.874747080527, "vram_gb": 6.442267648, "lambda": 0.14661350504805645, "lambda_min": 0.11092178523540497, "lambda_max": 0.18539805710315704, "S_mean": 0.4044392903645833, "G_mean": 0.4788104897985856, "G_std": 0.18876336235553026, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.508932113647461, "grad_norm": 0.5615019798278809, "tokens_seen": 12787712, "tokens_per_sec": 15128.737876825946, "vram_gb": 6.442267648, "lambda": 0.1470441666121284, "lambda_min": 0.11045798659324646, "lambda_max": 0.18651004135608673, "S_mean": 0.4318033854166667, "G_mean": 0.5041911782075962, "G_std": 0.19469511260588965, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.42390251159668, "grad_norm": 0.636780321598053, "tokens_seen": 12951552, "tokens_per_sec": 15131.448842353037, "vram_gb": 6.442267648, "lambda": 0.1475201128050685, "lambda_min": 0.11028259247541428, "lambda_max": 0.18773153424263, "S_mean": 0.4269205729166667, "G_mean": 0.485867432008187, "G_std": 0.1908353129401803, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.53483772277832, "grad_norm": 0.5333805084228516, "tokens_seen": 13115392, "tokens_per_sec": 15134.754686457456, "vram_gb": 6.442267648, "lambda": 0.1480239707355698, "lambda_min": 0.10977776348590851, "lambda_max": 0.18928159773349762, "S_mean": 0.4053751627604167, "G_mean": 0.48696788617720205, "G_std": 0.18969256073857346, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.331751346588135, "grad_norm": 0.5438115000724792, "tokens_seen": 13279232, "tokens_per_sec": 15136.959735927752, "vram_gb": 6.442267648, "lambda": 0.14851641623924175, "lambda_min": 0.10934562236070633, "lambda_max": 0.19053971767425537, "S_mean": 0.4058634440104167, "G_mean": 0.49762762772540253, "G_std": 0.1869133934378624, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.383272647857666, "grad_norm": 0.5693897604942322, "tokens_seen": 13443072, "tokens_per_sec": 15140.030919998999, "vram_gb": 6.442267648, "lambda": 0.1490536853671074, "lambda_min": 0.10913889110088348, "lambda_max": 0.1917176991701126, "S_mean": 0.40130615234375, "G_mean": 0.4872809685766697, "G_std": 0.18993867437044779, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.366852760314941, "grad_norm": 0.5484638810157776, "tokens_seen": 13606912, "tokens_per_sec": 15142.78330998289, "vram_gb": 6.442267648, "lambda": 0.14951497440536818, "lambda_min": 0.10892757773399353, "lambda_max": 0.19324201345443726, "S_mean": 0.3912353515625, "G_mean": 0.4886004456008474, "G_std": 0.1903441515751183, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.462980031967163, "grad_norm": 0.5320667028427124, "tokens_seen": 13770752, "tokens_per_sec": 15144.506184795728, "vram_gb": 6.442267648, "lambda": 0.14999949683745703, "lambda_min": 0.10853202641010284, "lambda_max": 0.19459427893161774, "S_mean": 0.3935343424479167, "G_mean": 0.47575207458188135, "G_std": 0.19118478382006288, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.3841447830200195, "grad_norm": 0.5432111620903015, "tokens_seen": 13934592, "tokens_per_sec": 15144.897399472275, "vram_gb": 6.442267648, "lambda": 0.15052947433044514, "lambda_min": 0.10827235877513885, "lambda_max": 0.1957901269197464, "S_mean": 0.4136149088541667, "G_mean": 0.47670603916049004, "G_std": 0.18794546338419119, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.335934162139893, "grad_norm": 0.6095689535140991, "tokens_seen": 14098432, "tokens_per_sec": 15147.286327661024, "vram_gb": 6.442267648, "lambda": 0.15104490083952746, "lambda_min": 0.10797078907489777, "lambda_max": 0.19715753197669983, "S_mean": 0.39013671875, "G_mean": 0.47610893038411933, "G_std": 0.1889628836264213, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.40704345703125, "grad_norm": 0.6716081500053406, "tokens_seen": 14262272, "tokens_per_sec": 15149.919354907339, "vram_gb": 6.442267648, "lambda": 0.15157404635101557, "lambda_min": 0.10787495970726013, "lambda_max": 0.19860045611858368, "S_mean": 0.4019368489583333, "G_mean": 0.4794648407648007, "G_std": 0.18912616558372974, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.372819900512695, "grad_norm": 0.5275672078132629, "tokens_seen": 14426112, "tokens_per_sec": 15151.836649192366, "vram_gb": 6.442267648, "lambda": 0.152010602876544, "lambda_min": 0.10744009166955948, "lambda_max": 0.1998690962791443, "S_mean": 0.4032999674479167, "G_mean": 0.47699382280309993, "G_std": 0.18823397050922117, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.320407867431641, "grad_norm": 0.5222680568695068, "tokens_seen": 14589952, "tokens_per_sec": 15153.497422766472, "vram_gb": 6.442267648, "lambda": 0.15248788303385177, "lambda_min": 0.10707636177539825, "lambda_max": 0.2013341337442398, "S_mean": 0.4029744466145833, "G_mean": 0.4691106112052997, "G_std": 0.18937886382142702, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.358256816864014, "grad_norm": 0.6309812068939209, "tokens_seen": 14753792, "tokens_per_sec": 15155.46807710727, "vram_gb": 6.442267648, "lambda": 0.1530351536348462, "lambda_min": 0.1066112071275711, "lambda_max": 0.20288431644439697, "S_mean": 0.4043986002604167, "G_mean": 0.45547197572886944, "G_std": 0.18726818837846318, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.269862174987793, "grad_norm": 0.596153199672699, "tokens_seen": 14917632, "tokens_per_sec": 15157.016213896117, "vram_gb": 6.442267648, "lambda": 0.15351038333028555, "lambda_min": 0.10619012266397476, "lambda_max": 0.20438365638256073, "S_mean": 0.38751220703125, "G_mean": 0.47635356554140645, "G_std": 0.19311670182893673, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.392306566238403, "grad_norm": 0.6145326495170593, "tokens_seen": 15081472, "tokens_per_sec": 15159.269624246615, "vram_gb": 6.442267648, "lambda": 0.15402550778041282, "lambda_min": 0.10571306943893433, "lambda_max": 0.20562340319156647, "S_mean": 0.3792826334635417, "G_mean": 0.47356026091923314, "G_std": 0.19257123194014034, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.3603129386901855, "grad_norm": 0.5657926201820374, "tokens_seen": 15245312, "tokens_per_sec": 15161.395363454114, "vram_gb": 6.442267648, "lambda": 0.15452732456227145, "lambda_min": 0.10552497953176498, "lambda_max": 0.20716597139835358, "S_mean": 0.3956095377604167, "G_mean": 0.4700409785533945, "G_std": 0.18971710186451674, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.297930479049683, "grad_norm": 0.5660515427589417, "tokens_seen": 15409152, "tokens_per_sec": 15163.687893345472, "vram_gb": 6.442267648, "lambda": 0.15499083107958236, "lambda_min": 0.10521059483289719, "lambda_max": 0.20858794450759888, "S_mean": 0.39471435546875, "G_mean": 0.46564727276563644, "G_std": 0.18668479550008973, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.2457427978515625, "grad_norm": 0.598924994468689, "tokens_seen": 15572992, "tokens_per_sec": 15165.915852041937, "vram_gb": 6.442267648, "lambda": 0.1555120162665844, "lambda_min": 0.10448846966028214, "lambda_max": 0.2098764330148697, "S_mean": 0.4033406575520833, "G_mean": 0.4621336354563634, "G_std": 0.18577480611080924, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.195011615753174, "grad_norm": 0.5875924229621887, "tokens_seen": 15736832, "tokens_per_sec": 15168.373770819042, "vram_gb": 6.442267648, "lambda": 0.15611659766485295, "lambda_min": 0.10427553206682205, "lambda_max": 0.21142354607582092, "S_mean": 0.3983357747395833, "G_mean": 0.46651650965213776, "G_std": 0.18990339990705252, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.1755547523498535, "grad_norm": 0.5782762169837952, "tokens_seen": 15900672, "tokens_per_sec": 15169.61011780546, "vram_gb": 6.442267648, "lambda": 0.15667632004866996, "lambda_min": 0.10385008156299591, "lambda_max": 0.21283365786075592, "S_mean": 0.3858846028645833, "G_mean": 0.46511891204863787, "G_std": 0.1879985399233798, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.170973062515259, "grad_norm": 0.6520832777023315, "tokens_seen": 16064512, "tokens_per_sec": 15171.275583968316, "vram_gb": 6.442267648, "lambda": 0.15717450808733702, "lambda_min": 0.1032554879784584, "lambda_max": 0.21432064473628998, "S_mean": 0.3882039388020833, "G_mean": 0.46314529422670603, "G_std": 0.18835986390088996, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.04676079750061, "grad_norm": 0.5699921250343323, "tokens_seen": 16228352, "tokens_per_sec": 15173.347722426664, "vram_gb": 6.442267648, "lambda": 0.1577411104614536, "lambda_min": 0.10290923714637756, "lambda_max": 0.21573372185230255, "S_mean": 0.40814208984375, "G_mean": 0.45130133753021556, "G_std": 0.18726783075059453, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.2173848152160645, "grad_norm": 0.5988911390304565, "tokens_seen": 16392192, "tokens_per_sec": 15175.281202916523, "vram_gb": 6.442267648, "lambda": 0.15821693216760954, "lambda_min": 0.10274924337863922, "lambda_max": 0.21713878214359283, "S_mean": 0.4062093098958333, "G_mean": 0.46244405892988044, "G_std": 0.18716078608607253, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.321232378482819, "val_ppl": 204.63591555984533, "lambda": 0.15824061166495085, "lambda_min": 0.10272547602653503, "lambda_max": 0.21718505024909973, "lambda_per_layer": [0.14680205285549164, 0.1362430900335312, 0.13444647192955017, 0.13141366839408875, 0.11891476809978485, 0.10272547602653503, 0.12764941155910492, 0.11737734824419022, 0.12664490938186646, 0.19712617993354797, 0.19024741649627686, 0.19127853214740753, 0.17588889598846436, 0.21718505024909973, 0.1929720640182495, 0.19149664044380188, 0.19674357771873474, 0.18329031765460968, 0.17203977704048157, 0.17939893901348114, 0.1323760598897934, 0.15787054598331451, 0.14328061044216156, 0.13436287641525269], "S_mean": 0.38311767578125, "G_mean": 0.4668790688738227, "G_std": 0.18870671931654215, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.110081434249878, "grad_norm": 0.590739905834198, "tokens_seen": 16556032, "tokens_per_sec": 15119.565570619541, "vram_gb": 6.442267648, "lambda": 0.1586812355866035, "lambda_min": 0.1022791862487793, "lambda_max": 0.21825213730335236, "S_mean": 0.3940022786458333, "G_mean": 0.4563881183664004, "G_std": 0.18837924037749568, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.311071395874023, "grad_norm": 1.0005970001220703, "tokens_seen": 16719872, "tokens_per_sec": 15121.72874220955, "vram_gb": 6.442267648, "lambda": 0.15919070908178887, "lambda_min": 0.10217814147472382, "lambda_max": 0.2196637988090515, "S_mean": 0.3788045247395833, "G_mean": 0.4711304021378358, "G_std": 0.18935805341849724, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.292270660400391, "grad_norm": 0.5508734583854675, "tokens_seen": 16883712, "tokens_per_sec": 15124.476404309691, "vram_gb": 6.442267648, "lambda": 0.15968584828078747, "lambda_min": 0.10186714679002762, "lambda_max": 0.22111846506595612, "S_mean": 0.3821004231770833, "G_mean": 0.4572604785983761, "G_std": 0.1903058604026834, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.221492528915405, "grad_norm": 0.6742516160011292, "tokens_seen": 17047552, "tokens_per_sec": 15126.519093667583, "vram_gb": 6.442267648, "lambda": 0.1601607445627451, "lambda_min": 0.10174977779388428, "lambda_max": 0.22233018279075623, "S_mean": 0.37603759765625, "G_mean": 0.4637175689761837, "G_std": 0.1875401120632887, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.238934516906738, "grad_norm": 0.6048938632011414, "tokens_seen": 17211392, "tokens_per_sec": 15128.731392927164, "vram_gb": 6.442267648, "lambda": 0.1607115234558781, "lambda_min": 0.10136749595403671, "lambda_max": 0.2237185686826706, "S_mean": 0.3934326171875, "G_mean": 0.45194955278808874, "G_std": 0.1848094385738174, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.2036519050598145, "grad_norm": 0.6747453212738037, "tokens_seen": 17375232, "tokens_per_sec": 15130.965643569974, "vram_gb": 6.442267648, "lambda": 0.1612076967333754, "lambda_min": 0.10076595842838287, "lambda_max": 0.2252144068479538, "S_mean": 0.4121500651041667, "G_mean": 0.44715134520083666, "G_std": 0.1869461052119732, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.25266432762146, "grad_norm": 0.6136128902435303, "tokens_seen": 17539072, "tokens_per_sec": 15133.145096043843, "vram_gb": 6.442267648, "lambda": 0.1617035949602723, "lambda_min": 0.10018187761306763, "lambda_max": 0.2264362871646881, "S_mean": 0.3856201171875, "G_mean": 0.4465855322778225, "G_std": 0.1860598170508941, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.3042237758636475, "grad_norm": 0.5841037631034851, "tokens_seen": 17702912, "tokens_per_sec": 15135.760624693541, "vram_gb": 6.442267648, "lambda": 0.16223273985087872, "lambda_min": 0.0998542383313179, "lambda_max": 0.2279052883386612, "S_mean": 0.3851318359375, "G_mean": 0.4485787176527083, "G_std": 0.18411510453249016, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.264498710632324, "grad_norm": 0.5919615626335144, "tokens_seen": 17866752, "tokens_per_sec": 15137.584889267959, "vram_gb": 6.442267648, "lambda": 0.16272384083519378, "lambda_min": 0.09950583428144455, "lambda_max": 0.22955814003944397, "S_mean": 0.3669026692708333, "G_mean": 0.4528226237744093, "G_std": 0.1859256892154614, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.195349454879761, "grad_norm": 0.5674337148666382, "tokens_seen": 18030592, "tokens_per_sec": 15139.782049814976, "vram_gb": 6.442267648, "lambda": 0.16315928722421327, "lambda_min": 0.09915702790021896, "lambda_max": 0.23066022992134094, "S_mean": 0.3698527018229167, "G_mean": 0.43795497932781774, "G_std": 0.18450645310804248, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.191408634185791, "grad_norm": 0.7105959057807922, "tokens_seen": 18194432, "tokens_per_sec": 15141.728215546858, "vram_gb": 6.442267648, "lambda": 0.16367512413611016, "lambda_min": 0.09860504418611526, "lambda_max": 0.23216651380062103, "S_mean": 0.4000447591145833, "G_mean": 0.43534837616607547, "G_std": 0.1831607964510719, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.095575332641602, "grad_norm": 0.6075640916824341, "tokens_seen": 18358272, "tokens_per_sec": 15143.717340834253, "vram_gb": 6.442267648, "lambda": 0.16415310899416605, "lambda_min": 0.09824074059724808, "lambda_max": 0.23362120985984802, "S_mean": 0.3633931477864583, "G_mean": 0.4575654047851761, "G_std": 0.18733184598386288, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.203378677368164, "grad_norm": 0.6398047804832458, "tokens_seen": 18522112, "tokens_per_sec": 15144.45823097173, "vram_gb": 6.442267648, "lambda": 0.16468209338684878, "lambda_min": 0.09796350449323654, "lambda_max": 0.2348775863647461, "S_mean": 0.38031005859375, "G_mean": 0.4340684798856576, "G_std": 0.1859391435670356, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.117295265197754, "grad_norm": 0.7466011643409729, "tokens_seen": 18685952, "tokens_per_sec": 15146.287322782713, "vram_gb": 6.442267648, "lambda": 0.1652182461693883, "lambda_min": 0.09784963726997375, "lambda_max": 0.2362150400876999, "S_mean": 0.3649393717447917, "G_mean": 0.44931269545728963, "G_std": 0.18655739755680165, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.144132137298584, "grad_norm": 0.6881024241447449, "tokens_seen": 18849792, "tokens_per_sec": 15147.824317986953, "vram_gb": 6.442267648, "lambda": 0.16571071526656547, "lambda_min": 0.09763892740011215, "lambda_max": 0.23746849596500397, "S_mean": 0.3658243815104167, "G_mean": 0.44450906788309413, "G_std": 0.18249442397306362, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.128937244415283, "grad_norm": 0.6426095366477966, "tokens_seen": 19013632, "tokens_per_sec": 15149.73404969448, "vram_gb": 6.442267648, "lambda": 0.16620047638813654, "lambda_min": 0.09740006923675537, "lambda_max": 0.23854303359985352, "S_mean": 0.372344970703125, "G_mean": 0.4379417252105971, "G_std": 0.18421842313061157, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.0598249435424805, "grad_norm": 0.6162910461425781, "tokens_seen": 19177472, "tokens_per_sec": 15151.229553399062, "vram_gb": 6.442267648, "lambda": 0.16663923393934965, "lambda_min": 0.09717592597007751, "lambda_max": 0.23956777155399323, "S_mean": 0.3954264322916667, "G_mean": 0.4344549171316127, "G_std": 0.18274323906128606, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.161998748779297, "grad_norm": 0.6075576543807983, "tokens_seen": 19341312, "tokens_per_sec": 15153.060207789937, "vram_gb": 6.442267648, "lambda": 0.16714676345388094, "lambda_min": 0.096796415746212, "lambda_max": 0.24102097749710083, "S_mean": 0.3683878580729167, "G_mean": 0.4417942327757676, "G_std": 0.18165398466711244, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.221909999847412, "grad_norm": 0.638297975063324, "tokens_seen": 19505152, "tokens_per_sec": 15155.086639877458, "vram_gb": 6.442267648, "lambda": 0.16766617447137833, "lambda_min": 0.09646152704954147, "lambda_max": 0.24241207540035248, "S_mean": 0.37554931640625, "G_mean": 0.4409579737111926, "G_std": 0.18368477343271175, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.124197244644165, "grad_norm": 0.6500988602638245, "tokens_seen": 19668992, "tokens_per_sec": 15157.119390139824, "vram_gb": 6.442267648, "lambda": 0.1681990623474121, "lambda_min": 0.09610951691865921, "lambda_max": 0.24338647723197937, "S_mean": 0.3855794270833333, "G_mean": 0.44166227321450907, "G_std": 0.18325812051383158, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.225209951400757, "grad_norm": 0.6363241076469421, "tokens_seen": 19832832, "tokens_per_sec": 15159.009521301281, "vram_gb": 6.442267648, "lambda": 0.16869600613911948, "lambda_min": 0.09604218602180481, "lambda_max": 0.24445873498916626, "S_mean": 0.3742879231770833, "G_mean": 0.43691842444241047, "G_std": 0.18514870759099722, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.023004531860352, "grad_norm": 0.6528570652008057, "tokens_seen": 19996672, "tokens_per_sec": 15160.177009857856, "vram_gb": 6.442267648, "lambda": 0.16922400705516338, "lambda_min": 0.09586206823587418, "lambda_max": 0.24574866890907288, "S_mean": 0.3674519856770833, "G_mean": 0.43753723163778585, "G_std": 0.18052420718595386, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.077590227127075, "grad_norm": 0.6661504507064819, "tokens_seen": 20160512, "tokens_per_sec": 15161.870658395095, "vram_gb": 6.442267648, "lambda": 0.16961592870453993, "lambda_min": 0.09546596556901932, "lambda_max": 0.24727913737297058, "S_mean": 0.3777669270833333, "G_mean": 0.42989698130016524, "G_std": 0.1842666487209499, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.04698634147644, "grad_norm": 0.6663743257522583, "tokens_seen": 20324352, "tokens_per_sec": 15163.634219186566, "vram_gb": 6.442267648, "lambda": 0.17004887852817774, "lambda_min": 0.09518568962812424, "lambda_max": 0.24836772680282593, "S_mean": 0.3648274739583333, "G_mean": 0.43911324922616285, "G_std": 0.18301153159700334, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.18242073059082, "grad_norm": 0.613558292388916, "tokens_seen": 20488192, "tokens_per_sec": 15165.281575564552, "vram_gb": 6.442267648, "lambda": 0.17053523318221173, "lambda_min": 0.09511884301900864, "lambda_max": 0.2495531588792801, "S_mean": 0.3745320638020833, "G_mean": 0.43972835317254066, "G_std": 0.18282496572161713, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.158579969406128, "val_ppl": 173.9173122668673, "lambda": 0.17056251193086305, "lambda_min": 0.09512325376272202, "lambda_max": 0.24962511658668518, "lambda_per_layer": [0.15037015080451965, 0.13736246526241302, 0.13509057462215424, 0.13184989988803864, 0.11974503099918365, 0.09512325376272202, 0.1274152249097824, 0.11142650991678238, 0.12884587049484253, 0.2149696797132492, 0.2057400345802307, 0.21776814758777618, 0.1912866085767746, 0.24962511658668518, 0.22759009897708893, 0.2194753885269165, 0.23386496305465698, 0.20893551409244537, 0.18814152479171753, 0.20404349267482758, 0.136674165725708, 0.16931454837322235, 0.15172986686229706, 0.13711215555667877], "S_mean": 0.36468505859375, "G_mean": 0.4387037434304754, "G_std": 0.18381889133403698, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.092965364456177, "grad_norm": 0.6018621921539307, "tokens_seen": 20652032, "tokens_per_sec": 15120.943150203639, "vram_gb": 6.442267648, "lambda": 0.1710941499720017, "lambda_min": 0.09503213316202164, "lambda_max": 0.2508680522441864, "S_mean": 0.3550821940104167, "G_mean": 0.432137871781985, "G_std": 0.18301269280103347, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 4.958301544189453, "grad_norm": 0.6295021772384644, "tokens_seen": 20815872, "tokens_per_sec": 15122.485285102504, "vram_gb": 6.442267648, "lambda": 0.171570540095369, "lambda_min": 0.09462989121675491, "lambda_max": 0.25228509306907654, "S_mean": 0.3770345052083333, "G_mean": 0.4367065044740836, "G_std": 0.18282733655845126, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.2204179763793945, "grad_norm": 0.6682658195495605, "tokens_seen": 20979712, "tokens_per_sec": 15124.265545895669, "vram_gb": 6.442267648, "lambda": 0.17201251784960428, "lambda_min": 0.09451556205749512, "lambda_max": 0.2533802390098572, "S_mean": 0.3668212890625, "G_mean": 0.4352534149462978, "G_std": 0.1826080436973522, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.081791162490845, "grad_norm": 0.6534812450408936, "tokens_seen": 21143552, "tokens_per_sec": 15126.31108965579, "vram_gb": 6.442267648, "lambda": 0.17251587938517332, "lambda_min": 0.09432699531316757, "lambda_max": 0.25461241602897644, "S_mean": 0.3603922526041667, "G_mean": 0.42544189825033146, "G_std": 0.18202056254570684, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 4.93201756477356, "grad_norm": 0.6044642329216003, "tokens_seen": 21307392, "tokens_per_sec": 15128.455191066607, "vram_gb": 6.442267648, "lambda": 0.17299400735646486, "lambda_min": 0.09415146708488464, "lambda_max": 0.2558308243751526, "S_mean": 0.353515625, "G_mean": 0.41622836142778397, "G_std": 0.18143059026139477, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.981318473815918, "grad_norm": 0.6839517951011658, "tokens_seen": 21471232, "tokens_per_sec": 15130.292884100907, "vram_gb": 6.442267648, "lambda": 0.17351485136896372, "lambda_min": 0.09419016540050507, "lambda_max": 0.2572252154350281, "S_mean": 0.3703206380208333, "G_mean": 0.4240449314626555, "G_std": 0.18296705512329936, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.0928826332092285, "grad_norm": 0.6456636786460876, "tokens_seen": 21635072, "tokens_per_sec": 15132.089104938084, "vram_gb": 6.442267648, "lambda": 0.17392214015126228, "lambda_min": 0.09406910836696625, "lambda_max": 0.25811782479286194, "S_mean": 0.3693745930989583, "G_mean": 0.42283995635807514, "G_std": 0.18012028994659582, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.05021858215332, "grad_norm": 0.7765688300132751, "tokens_seen": 21798912, "tokens_per_sec": 15133.255200700256, "vram_gb": 6.442267648, "lambda": 0.17444972228258848, "lambda_min": 0.09374641627073288, "lambda_max": 0.25927063822746277, "S_mean": 0.3749593098958333, "G_mean": 0.4279710765307148, "G_std": 0.17855992393257716, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.0415520668029785, "grad_norm": 0.6908031105995178, "tokens_seen": 21962752, "tokens_per_sec": 15134.763975131744, "vram_gb": 6.442267648, "lambda": 0.17501085375746092, "lambda_min": 0.09348268061876297, "lambda_max": 0.2606469988822937, "S_mean": 0.3779093424479167, "G_mean": 0.4265733954186241, "G_std": 0.18261173747790357, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.044876575469971, "grad_norm": 0.6257720589637756, "tokens_seen": 22126592, "tokens_per_sec": 15136.366214514082, "vram_gb": 6.442267648, "lambda": 0.17550581693649292, "lambda_min": 0.09311141073703766, "lambda_max": 0.26170310378074646, "S_mean": 0.3421834309895833, "G_mean": 0.433244073142608, "G_std": 0.18336600523131588, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 4.994304418563843, "grad_norm": 0.6488397121429443, "tokens_seen": 22290432, "tokens_per_sec": 15137.53709845554, "vram_gb": 6.442267648, "lambda": 0.17601683735847473, "lambda_min": 0.0931088924407959, "lambda_max": 0.26294079422950745, "S_mean": 0.3592325846354167, "G_mean": 0.4283275920897722, "G_std": 0.18110783956944942, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.075089693069458, "grad_norm": 0.663480281829834, "tokens_seen": 22454272, "tokens_per_sec": 15138.906608943897, "vram_gb": 6.442267648, "lambda": 0.17651626529792944, "lambda_min": 0.09293684363365173, "lambda_max": 0.2639027237892151, "S_mean": 0.3591817220052083, "G_mean": 0.42746219877153635, "G_std": 0.18077512954672179, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.239394426345825, "grad_norm": 0.6682804226875305, "tokens_seen": 22618112, "tokens_per_sec": 15140.513464765381, "vram_gb": 6.442267648, "lambda": 0.17702119580159584, "lambda_min": 0.09263633191585541, "lambda_max": 0.26514875888824463, "S_mean": 0.343597412109375, "G_mean": 0.42482205961520475, "G_std": 0.18194128301305076, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.2076029777526855, "grad_norm": 0.6458008885383606, "tokens_seen": 22781952, "tokens_per_sec": 15142.22502434207, "vram_gb": 6.442267648, "lambda": 0.1775593419248859, "lambda_min": 0.09237660467624664, "lambda_max": 0.2667318284511566, "S_mean": 0.3751220703125, "G_mean": 0.427764184307307, "G_std": 0.17876985288845995, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.9842822551727295, "grad_norm": 0.6261724233627319, "tokens_seen": 22945792, "tokens_per_sec": 15143.862133443456, "vram_gb": 6.442267648, "lambda": 0.1780392462387681, "lambda_min": 0.09224000573158264, "lambda_max": 0.268187552690506, "S_mean": 0.3548177083333333, "G_mean": 0.42343819607049227, "G_std": 0.17949689265030125, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.266672372817993, "grad_norm": 0.7087611556053162, "tokens_seen": 23109632, "tokens_per_sec": 15145.448599936926, "vram_gb": 6.442267648, "lambda": 0.1785952222223083, "lambda_min": 0.09207642823457718, "lambda_max": 0.2693071961402893, "S_mean": 0.3782958984375, "G_mean": 0.42792007777219015, "G_std": 0.17996401991695166, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 4.999699831008911, "grad_norm": 0.6557350754737854, "tokens_seen": 23273472, "tokens_per_sec": 15146.923725835695, "vram_gb": 6.442267648, "lambda": 0.17908049188554287, "lambda_min": 0.09186270087957382, "lambda_max": 0.2706882357597351, "S_mean": 0.3495686848958333, "G_mean": 0.4321679063141346, "G_std": 0.17748420941643417, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.019970655441284, "grad_norm": 0.6294455528259277, "tokens_seen": 23437312, "tokens_per_sec": 15147.957697272826, "vram_gb": 6.442267648, "lambda": 0.17951612391819557, "lambda_min": 0.09170574694871902, "lambda_max": 0.2718335688114166, "S_mean": 0.3558451334635417, "G_mean": 0.41727534495294094, "G_std": 0.17826066608540714, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.066039085388184, "grad_norm": 0.6675633788108826, "tokens_seen": 23601152, "tokens_per_sec": 15149.467344481978, "vram_gb": 6.442267648, "lambda": 0.1800707345828414, "lambda_min": 0.09164772182703018, "lambda_max": 0.27331024408340454, "S_mean": 0.3506368001302083, "G_mean": 0.42635943042114377, "G_std": 0.17938278747412065, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.891045093536377, "grad_norm": 0.6492661833763123, "tokens_seen": 23764992, "tokens_per_sec": 15150.669781073053, "vram_gb": 6.442267648, "lambda": 0.1804889620592197, "lambda_min": 0.09147043526172638, "lambda_max": 0.2743312418460846, "S_mean": 0.3605143229166667, "G_mean": 0.40841255864749354, "G_std": 0.17940826838215193, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.0241193771362305, "grad_norm": 0.6576728820800781, "tokens_seen": 23928832, "tokens_per_sec": 15152.062870331412, "vram_gb": 6.442267648, "lambda": 0.18106321121255556, "lambda_min": 0.09154079109430313, "lambda_max": 0.2757236659526825, "S_mean": 0.3464457194010417, "G_mean": 0.41869119290883344, "G_std": 0.17837051884271204, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.069908380508423, "grad_norm": 0.7231227159500122, "tokens_seen": 24092672, "tokens_per_sec": 15153.748126614188, "vram_gb": 6.442267648, "lambda": 0.18157555597523847, "lambda_min": 0.09133758395910263, "lambda_max": 0.2770819664001465, "S_mean": 0.3540445963541667, "G_mean": 0.42433939781039953, "G_std": 0.17965221925017735, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.917970180511475, "grad_norm": 0.7647451758384705, "tokens_seen": 24256512, "tokens_per_sec": 15155.26434993241, "vram_gb": 6.442267648, "lambda": 0.18208395907034478, "lambda_min": 0.09111279994249344, "lambda_max": 0.2784770131111145, "S_mean": 0.3309224446614583, "G_mean": 0.42504422754670185, "G_std": 0.17826489405706525, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.069119930267334, "grad_norm": 0.7213230133056641, "tokens_seen": 24420352, "tokens_per_sec": 15156.695144303838, "vram_gb": 6.442267648, "lambda": 0.18259849213063717, "lambda_min": 0.09065183252096176, "lambda_max": 0.2798512876033783, "S_mean": 0.3598836263020833, "G_mean": 0.4200627530614535, "G_std": 0.17980881485467157, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.05009126663208, "grad_norm": 0.6907325387001038, "tokens_seen": 24584192, "tokens_per_sec": 15157.912836027812, "vram_gb": 6.442267648, "lambda": 0.18303113151341677, "lambda_min": 0.09036873281002045, "lambda_max": 0.2802252769470215, "S_mean": 0.343170166015625, "G_mean": 0.4237479887281855, "G_std": 0.17942660014765957, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.017487037181854, "val_ppl": 151.03129049699567, "lambda": 0.1830528595795234, "lambda_min": 0.09036466479301453, "lambda_max": 0.280264675617218, "lambda_per_layer": [0.15237390995025635, 0.13933713734149933, 0.13598231971263885, 0.13207441568374634, 0.120505191385746, 0.09036466479301453, 0.1266225427389145, 0.10624933987855911, 0.1332426220178604, 0.2294865995645523, 0.22135215997695923, 0.24611243605613708, 0.2071406990289688, 0.280264675617218, 0.2588062584400177, 0.2453925609588623, 0.27286309003829956, 0.23383420705795288, 0.20519447326660156, 0.23236723244190216, 0.14116595685482025, 0.18169014155864716, 0.1598198115825653, 0.14102618396282196], "S_mean": 0.3470458984375, "G_mean": 0.42215216159820557, "G_std": 0.18071834929287434, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.9993908405303955, "grad_norm": 0.6691079139709473, "tokens_seen": 24748032, "tokens_per_sec": 15121.307390507718, "vram_gb": 6.442267648, "lambda": 0.18354421481490135, "lambda_min": 0.09035434573888779, "lambda_max": 0.2811647355556488, "S_mean": 0.368896484375, "G_mean": 0.4150480581447482, "G_std": 0.17836242238990963, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.150953054428101, "grad_norm": 0.6587517857551575, "tokens_seen": 24911872, "tokens_per_sec": 15123.014798199665, "vram_gb": 6.442267648, "lambda": 0.18405207029233375, "lambda_min": 0.09026478976011276, "lambda_max": 0.28249138593673706, "S_mean": 0.3580525716145833, "G_mean": 0.41946570575237274, "G_std": 0.17912870272994041, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.888821601867676, "grad_norm": 0.6093928217887878, "tokens_seen": 25075712, "tokens_per_sec": 15124.522817777464, "vram_gb": 6.442267648, "lambda": 0.18454770868023238, "lambda_min": 0.09034460783004761, "lambda_max": 0.28353646397590637, "S_mean": 0.343231201171875, "G_mean": 0.42124345370878774, "G_std": 0.176960394019261, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.9621124267578125, "grad_norm": 0.670688807964325, "tokens_seen": 25239552, "tokens_per_sec": 15126.114381024969, "vram_gb": 6.442267648, "lambda": 0.18507327201465765, "lambda_min": 0.09003700315952301, "lambda_max": 0.28473034501075745, "S_mean": 0.3806966145833333, "G_mean": 0.41507322403291863, "G_std": 0.17706565690847734, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.877901315689087, "grad_norm": 0.7154895663261414, "tokens_seen": 25403392, "tokens_per_sec": 15127.673666081522, "vram_gb": 6.442267648, "lambda": 0.18557003823419413, "lambda_min": 0.08974770456552505, "lambda_max": 0.28576353192329407, "S_mean": 0.3644002278645833, "G_mean": 0.4135791154888769, "G_std": 0.17564708561015627, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.858312129974365, "grad_norm": 0.6837120652198792, "tokens_seen": 25567232, "tokens_per_sec": 15129.016593658302, "vram_gb": 6.442267648, "lambda": 0.18606651636461416, "lambda_min": 0.08968837559223175, "lambda_max": 0.28701403737068176, "S_mean": 0.3533732096354167, "G_mean": 0.4133033159499367, "G_std": 0.1792609135930737, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 5.0407445430755615, "grad_norm": 0.7610180377960205, "tokens_seen": 25731072, "tokens_per_sec": 15130.41949588984, "vram_gb": 6.442267648, "lambda": 0.18653381088127693, "lambda_min": 0.08953718096017838, "lambda_max": 0.28797921538352966, "S_mean": 0.3583780924479167, "G_mean": 0.4283616701141, "G_std": 0.17705774097703397, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.828386545181274, "grad_norm": 0.6549761295318604, "tokens_seen": 25894912, "tokens_per_sec": 15131.990213926772, "vram_gb": 6.442267648, "lambda": 0.18695409906407198, "lambda_min": 0.08927541226148605, "lambda_max": 0.2894221842288971, "S_mean": 0.3484090169270833, "G_mean": 0.4077521103123824, "G_std": 0.177245944350337, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.863428354263306, "grad_norm": 0.7298623323440552, "tokens_seen": 26058752, "tokens_per_sec": 15133.313191135627, "vram_gb": 6.442267648, "lambda": 0.18743332754820585, "lambda_min": 0.08910258859395981, "lambda_max": 0.2907598912715912, "S_mean": 0.3412679036458333, "G_mean": 0.4180694765721758, "G_std": 0.18019567453302443, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.805732011795044, "grad_norm": 0.6855307221412659, "tokens_seen": 26222592, "tokens_per_sec": 15135.108660943753, "vram_gb": 6.442267648, "lambda": 0.1878522913902998, "lambda_min": 0.08878268301486969, "lambda_max": 0.2917047441005707, "S_mean": 0.3475341796875, "G_mean": 0.4036712725646794, "G_std": 0.17733402899466455, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.7788472175598145, "grad_norm": 0.6942004561424255, "tokens_seen": 26386432, "tokens_per_sec": 15136.851104362677, "vram_gb": 6.442267648, "lambda": 0.18832485750317574, "lambda_min": 0.08866181969642639, "lambda_max": 0.29313549399375916, "S_mean": 0.3510538736979167, "G_mean": 0.42147670841465396, "G_std": 0.17576724686659873, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.8344690799713135, "grad_norm": 0.6773051023483276, "tokens_seen": 26550272, "tokens_per_sec": 15138.319257971847, "vram_gb": 6.442267648, "lambda": 0.18879811962445578, "lambda_min": 0.0884915441274643, "lambda_max": 0.29461154341697693, "S_mean": 0.34051513671875, "G_mean": 0.41517681054150063, "G_std": 0.17872804620613655, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.979574203491211, "grad_norm": 0.6639286875724792, "tokens_seen": 26714112, "tokens_per_sec": 15140.091869834763, "vram_gb": 6.442267648, "lambda": 0.18929200495282808, "lambda_min": 0.08828606456518173, "lambda_max": 0.2959243655204773, "S_mean": 0.3412679036458333, "G_mean": 0.40772299530605477, "G_std": 0.17727748032969734, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.824740171432495, "grad_norm": 0.6363000273704529, "tokens_seen": 26877952, "tokens_per_sec": 15141.55710038814, "vram_gb": 6.442267648, "lambda": 0.18980584914485613, "lambda_min": 0.08823190629482269, "lambda_max": 0.29727447032928467, "S_mean": 0.336181640625, "G_mean": 0.4084823538238804, "G_std": 0.17445372270109752, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.011361837387085, "grad_norm": 0.6854971647262573, "tokens_seen": 27041792, "tokens_per_sec": 15142.439058855502, "vram_gb": 6.442267648, "lambda": 0.1903259518245856, "lambda_min": 0.08835545927286148, "lambda_max": 0.29863420128822327, "S_mean": 0.3460693359375, "G_mean": 0.4178314986638725, "G_std": 0.17627248781112334, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.8809614181518555, "grad_norm": 0.7010342478752136, "tokens_seen": 27205632, "tokens_per_sec": 15144.00926488599, "vram_gb": 6.442267648, "lambda": 0.19079325441271067, "lambda_min": 0.08834723383188248, "lambda_max": 0.30008602142333984, "S_mean": 0.3551228841145833, "G_mean": 0.41009761827687424, "G_std": 0.1763911540620029, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.971625804901123, "grad_norm": 0.6868041753768921, "tokens_seen": 27369472, "tokens_per_sec": 15145.486631952877, "vram_gb": 6.442267648, "lambda": 0.1912977018704017, "lambda_min": 0.08817427605390549, "lambda_max": 0.30106037855148315, "S_mean": 0.3509114583333333, "G_mean": 0.4167470975468556, "G_std": 0.1741140567076703, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.766345262527466, "grad_norm": 0.666496217250824, "tokens_seen": 27533312, "tokens_per_sec": 15146.544946398366, "vram_gb": 6.442267648, "lambda": 0.19171746416638294, "lambda_min": 0.08823277801275253, "lambda_max": 0.30194249749183655, "S_mean": 0.3560587565104167, "G_mean": 0.408527256300052, "G_std": 0.1780582258167366, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.9366559982299805, "grad_norm": 1.1206133365631104, "tokens_seen": 27697152, "tokens_per_sec": 15148.009984548813, "vram_gb": 6.442267648, "lambda": 0.1921984221165379, "lambda_min": 0.08845355361700058, "lambda_max": 0.3030601441860199, "S_mean": 0.3828328450520833, "G_mean": 0.40314778049166006, "G_std": 0.17439409407476583, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.951912879943848, "grad_norm": 0.675046443939209, "tokens_seen": 27860992, "tokens_per_sec": 15149.536514329386, "vram_gb": 6.442267648, "lambda": 0.19265196782847246, "lambda_min": 0.0884571522474289, "lambda_max": 0.303859680891037, "S_mean": 0.3372599283854167, "G_mean": 0.40912951560070115, "G_std": 0.17679609482487044, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.811490058898926, "grad_norm": 0.700502872467041, "tokens_seen": 28024832, "tokens_per_sec": 15151.025308345199, "vram_gb": 6.442267648, "lambda": 0.19314223621040583, "lambda_min": 0.08820530027151108, "lambda_max": 0.30501803755760193, "S_mean": 0.3543701171875, "G_mean": 0.40629174855227274, "G_std": 0.17406459152698517, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.896039962768555, "grad_norm": 0.689314603805542, "tokens_seen": 28188672, "tokens_per_sec": 15152.310483558405, "vram_gb": 6.442267648, "lambda": 0.19360573372493187, "lambda_min": 0.08813285827636719, "lambda_max": 0.3062872290611267, "S_mean": 0.3364054361979167, "G_mean": 0.41025739163160324, "G_std": 0.17474315230113766, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.795178651809692, "grad_norm": 0.6704794764518738, "tokens_seen": 28352512, "tokens_per_sec": 15153.58418278408, "vram_gb": 6.442267648, "lambda": 0.19409532907108465, "lambda_min": 0.08802706748247147, "lambda_max": 0.307652086019516, "S_mean": 0.3506978352864583, "G_mean": 0.3998872651718557, "G_std": 0.17405501349518696, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.783068656921387, "grad_norm": 0.6662055850028992, "tokens_seen": 28516352, "tokens_per_sec": 15154.981590053318, "vram_gb": 6.442267648, "lambda": 0.19469708142181238, "lambda_min": 0.08809603750705719, "lambda_max": 0.3089054822921753, "S_mean": 0.3545430501302083, "G_mean": 0.39722216641530395, "G_std": 0.17522675481935343, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.940044641494751, "grad_norm": 0.7208013534545898, "tokens_seen": 28680192, "tokens_per_sec": 15156.384317502943, "vram_gb": 6.442267648, "lambda": 0.19519802369177341, "lambda_min": 0.08813879638910294, "lambda_max": 0.31006377935409546, "S_mean": 0.3472696940104167, "G_mean": 0.4075687707712253, "G_std": 0.17419126249539355, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.91127177476883, "val_ppl": 135.81202693222954, "lambda": 0.19522119654963413, "lambda_min": 0.08815238624811172, "lambda_max": 0.31011706590652466, "lambda_per_layer": [0.15528999269008636, 0.14129871129989624, 0.136890709400177, 0.13262568414211273, 0.12116773426532745, 0.08815238624811172, 0.12541057169437408, 0.10259810090065002, 0.13520577549934387, 0.24271897971630096, 0.23627398908138275, 0.27323922514915466, 0.22191232442855835, 0.3094252049922943, 0.28781232237815857, 0.27117422223091125, 0.31011706590652466, 0.25806501507759094, 0.22100205719470978, 0.26384294033050537, 0.14417648315429688, 0.19498856365680695, 0.16770538687705994, 0.14421527087688446], "S_mean": 0.35076904296875, "G_mean": 0.40206975194935995, "G_std": 0.1745601613074541, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.9210498332977295, "grad_norm": 0.6722676753997803, "tokens_seen": 28844032, "tokens_per_sec": 15124.854449608181, "vram_gb": 6.442267648, "lambda": 0.19566717185080051, "lambda_min": 0.08808556199073792, "lambda_max": 0.3116641640663147, "S_mean": 0.3421630859375, "G_mean": 0.40233205414066714, "G_std": 0.17304504057392478, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.788249254226685, "grad_norm": 0.7274940609931946, "tokens_seen": 29007872, "tokens_per_sec": 15125.892023216571, "vram_gb": 6.442267648, "lambda": 0.196157518774271, "lambda_min": 0.08803801238536835, "lambda_max": 0.31301435828208923, "S_mean": 0.3622945149739583, "G_mean": 0.39936579174051684, "G_std": 0.17557229893282056, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.838567018508911, "grad_norm": 0.678704023361206, "tokens_seen": 29171712, "tokens_per_sec": 15127.290273791632, "vram_gb": 6.442267648, "lambda": 0.19666343523810306, "lambda_min": 0.08784354478120804, "lambda_max": 0.31474488973617554, "S_mean": 0.35638427734375, "G_mean": 0.402777759047846, "G_std": 0.17165876839620373, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.906018257141113, "grad_norm": 0.7224381566047668, "tokens_seen": 29335552, "tokens_per_sec": 15128.729080014373, "vram_gb": 6.442267648, "lambda": 0.19719347295661768, "lambda_min": 0.0877063050866127, "lambda_max": 0.3162841200828552, "S_mean": 0.3442586263020833, "G_mean": 0.40737946378067136, "G_std": 0.17252414960724613, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.769292593002319, "grad_norm": 0.68942791223526, "tokens_seen": 29499392, "tokens_per_sec": 15130.013866144918, "vram_gb": 6.442267648, "lambda": 0.19768324059744677, "lambda_min": 0.08773098886013031, "lambda_max": 0.317655086517334, "S_mean": 0.3428446451822917, "G_mean": 0.4009760059416294, "G_std": 0.17338037180403867, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.84837532043457, "grad_norm": 0.6665964126586914, "tokens_seen": 29663232, "tokens_per_sec": 15131.189501634095, "vram_gb": 6.442267648, "lambda": 0.19816474057734013, "lambda_min": 0.0876784399151802, "lambda_max": 0.31943827867507935, "S_mean": 0.3444620768229167, "G_mean": 0.4116689378085236, "G_std": 0.17272776861985525, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.903280973434448, "grad_norm": 0.6800984144210815, "tokens_seen": 29827072, "tokens_per_sec": 15132.695364597921, "vram_gb": 6.442267648, "lambda": 0.19863907744487128, "lambda_min": 0.08782119303941727, "lambda_max": 0.32070088386535645, "S_mean": 0.3503011067708333, "G_mean": 0.3990051302437981, "G_std": 0.17325976281426847, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.7990500926971436, "grad_norm": 0.7223203182220459, "tokens_seen": 29990912, "tokens_per_sec": 15134.315767616908, "vram_gb": 6.442267648, "lambda": 0.19915147932867208, "lambda_min": 0.08766405284404755, "lambda_max": 0.32252198457717896, "S_mean": 0.3403828938802083, "G_mean": 0.4010490065750976, "G_std": 0.171585588172699, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.915369510650635, "grad_norm": 0.6552968621253967, "tokens_seen": 30154752, "tokens_per_sec": 15135.79152178614, "vram_gb": 6.442267648, "lambda": 0.1995557484527429, "lambda_min": 0.08735284954309464, "lambda_max": 0.32399529218673706, "S_mean": 0.3514811197916667, "G_mean": 0.4044267051698019, "G_std": 0.1746417851342509, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.777500867843628, "grad_norm": 0.7757702469825745, "tokens_seen": 30318592, "tokens_per_sec": 15137.056063179982, "vram_gb": 6.442267648, "lambda": 0.1999612689639131, "lambda_min": 0.08736222237348557, "lambda_max": 0.325221449136734, "S_mean": 0.3375447591145833, "G_mean": 0.3995776614174247, "G_std": 0.17435884421380857, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.906266927719116, "grad_norm": 0.7836076617240906, "tokens_seen": 30482432, "tokens_per_sec": 15138.44324857238, "vram_gb": 6.442267648, "lambda": 0.2005290056889256, "lambda_min": 0.087323859333992, "lambda_max": 0.32666465640068054, "S_mean": 0.3456827799479167, "G_mean": 0.4103377099769811, "G_std": 0.17459022362406054, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.857895374298096, "grad_norm": 0.676888108253479, "tokens_seen": 30646272, "tokens_per_sec": 15139.563508078632, "vram_gb": 6.442267648, "lambda": 0.20101405618091425, "lambda_min": 0.08726129680871964, "lambda_max": 0.32824403047561646, "S_mean": 0.3336588541666667, "G_mean": 0.3964865958938996, "G_std": 0.17456573892074326, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.918368101119995, "grad_norm": 0.674628496170044, "tokens_seen": 30810112, "tokens_per_sec": 15140.963202273395, "vram_gb": 6.442267648, "lambda": 0.20150733211388192, "lambda_min": 0.08712423592805862, "lambda_max": 0.330121785402298, "S_mean": 0.3451639811197917, "G_mean": 0.40290993157153326, "G_std": 0.17348118918016553, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.90958309173584, "grad_norm": 0.7190079092979431, "tokens_seen": 30973952, "tokens_per_sec": 15142.12628205751, "vram_gb": 6.442267648, "lambda": 0.2019740374137958, "lambda_min": 0.08714301139116287, "lambda_max": 0.33133983612060547, "S_mean": 0.342041015625, "G_mean": 0.4141649509159227, "G_std": 0.17342077599217495, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.8835906982421875, "grad_norm": 0.7260870337486267, "tokens_seen": 31137792, "tokens_per_sec": 15143.318051829914, "vram_gb": 6.442267648, "lambda": 0.202532474262019, "lambda_min": 0.08710211515426636, "lambda_max": 0.33282890915870667, "S_mean": 0.3370564778645833, "G_mean": 0.4054215303622186, "G_std": 0.17059240117669106, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.6448094844818115, "grad_norm": 0.697502076625824, "tokens_seen": 31301632, "tokens_per_sec": 15144.747061360402, "vram_gb": 6.442267648, "lambda": 0.2030342317496737, "lambda_min": 0.08712024241685867, "lambda_max": 0.3342474400997162, "S_mean": 0.3461710611979167, "G_mean": 0.3970128642395139, "G_std": 0.17180636144864062, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.947157621383667, "grad_norm": 0.6963329911231995, "tokens_seen": 31465472, "tokens_per_sec": 15145.96996516586, "vram_gb": 6.442267648, "lambda": 0.2035044301301241, "lambda_min": 0.08715686947107315, "lambda_max": 0.3353951573371887, "S_mean": 0.3344624837239583, "G_mean": 0.3974209391356756, "G_std": 0.1724968027944366, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.832875967025757, "grad_norm": 0.7518036365509033, "tokens_seen": 31629312, "tokens_per_sec": 15147.24173188597, "vram_gb": 6.442267648, "lambda": 0.20403934394319853, "lambda_min": 0.08712377399206161, "lambda_max": 0.3368316888809204, "S_mean": 0.3289998372395833, "G_mean": 0.40113243569309515, "G_std": 0.17097979954754314, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.746967554092407, "grad_norm": 0.663368284702301, "tokens_seen": 31793152, "tokens_per_sec": 15148.59610451596, "vram_gb": 6.442267648, "lambda": 0.20445159325997034, "lambda_min": 0.08711981773376465, "lambda_max": 0.33820489048957825, "S_mean": 0.3279012044270833, "G_mean": 0.4012607814123233, "G_std": 0.17287413768159846, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.720385551452637, "grad_norm": 0.6950237154960632, "tokens_seen": 31956992, "tokens_per_sec": 15149.675358314676, "vram_gb": 6.442267648, "lambda": 0.2049207870538036, "lambda_min": 0.08706225454807281, "lambda_max": 0.3394259810447693, "S_mean": 0.3375040690104167, "G_mean": 0.39952827865878743, "G_std": 0.17360502341762185, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.832756519317627, "grad_norm": 0.7756239175796509, "tokens_seen": 32120832, "tokens_per_sec": 15150.903150104772, "vram_gb": 6.442267648, "lambda": 0.20536515768617392, "lambda_min": 0.08683563023805618, "lambda_max": 0.34091711044311523, "S_mean": 0.3519287109375, "G_mean": 0.39485114300623536, "G_std": 0.16921259090304375, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.652648448944092, "grad_norm": 0.6989971995353699, "tokens_seen": 32284672, "tokens_per_sec": 15151.871582215104, "vram_gb": 6.442267648, "lambda": 0.20595746984084448, "lambda_min": 0.08691012114286423, "lambda_max": 0.342558890581131, "S_mean": 0.3191324869791667, "G_mean": 0.3957008776875834, "G_std": 0.17268686958899102, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.76901912689209, "grad_norm": 0.7294602990150452, "tokens_seen": 32448512, "tokens_per_sec": 15152.768454912934, "vram_gb": 6.442267648, "lambda": 0.206428123327593, "lambda_min": 0.08693830668926239, "lambda_max": 0.3443395793437958, "S_mean": 0.3408101399739583, "G_mean": 0.4038659512686233, "G_std": 0.17059279846337935, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.671482086181641, "grad_norm": 0.6927920579910278, "tokens_seen": 32612352, "tokens_per_sec": 15154.124038107071, "vram_gb": 6.442267648, "lambda": 0.20686180361857018, "lambda_min": 0.08694569021463394, "lambda_max": 0.34592512249946594, "S_mean": 0.3269449869791667, "G_mean": 0.400312392351528, "G_std": 0.16897470581655702, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.762777090072632, "grad_norm": 0.7171890735626221, "tokens_seen": 32776192, "tokens_per_sec": 15154.82815547702, "vram_gb": 6.442267648, "lambda": 0.20733655306200185, "lambda_min": 0.08684340119361877, "lambda_max": 0.34781110286712646, "S_mean": 0.3353678385416667, "G_mean": 0.41114364052191377, "G_std": 0.1686321182642132, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.871827435493469, "val_ppl": 130.55928770283597, "lambda": 0.20736115022252002, "lambda_min": 0.08683854341506958, "lambda_max": 0.3478964865207672, "lambda_per_layer": [0.15934740006923676, 0.14268076419830322, 0.13821910321712494, 0.13332968950271606, 0.1232023760676384, 0.08683854341506958, 0.12567606568336487, 0.09951969981193542, 0.1385420709848404, 0.2551914155483246, 0.25112777948379517, 0.30077916383743286, 0.23506617546081543, 0.33764326572418213, 0.3149283230304718, 0.2941807508468628, 0.3478964865207672, 0.2817477583885193, 0.23505154252052307, 0.2949596643447876, 0.1502821147441864, 0.20716600120067596, 0.17541150748729706, 0.1478799432516098], "S_mean": 0.3364054361979167, "G_mean": 0.40309428120963275, "G_std": 0.1698011391951392, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.910485506057739, "grad_norm": 0.6672772169113159, "tokens_seen": 32940032, "tokens_per_sec": 15126.871575593872, "vram_gb": 6.442267648, "lambda": 0.20781329025824866, "lambda_min": 0.08686188608407974, "lambda_max": 0.3496861457824707, "S_mean": 0.3305155436197917, "G_mean": 0.39211222203448415, "G_std": 0.17262990687352917, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.905571460723877, "grad_norm": 0.7121490240097046, "tokens_seen": 33103872, "tokens_per_sec": 15128.08326635403, "vram_gb": 6.442267648, "lambda": 0.2082832089314858, "lambda_min": 0.08679251372814178, "lambda_max": 0.3513028025627136, "S_mean": 0.33673095703125, "G_mean": 0.3922128643995772, "G_std": 0.1721228484530002, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.761224269866943, "grad_norm": 0.7028594613075256, "tokens_seen": 33267712, "tokens_per_sec": 15129.013563510913, "vram_gb": 6.442267648, "lambda": 0.20876962722589573, "lambda_min": 0.086681067943573, "lambda_max": 0.35315603017807007, "S_mean": 0.3514404296875, "G_mean": 0.3957918072895457, "G_std": 0.17296502188158533, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.793102741241455, "grad_norm": 0.6853398084640503, "tokens_seen": 33431552, "tokens_per_sec": 15130.262415259474, "vram_gb": 6.442267648, "lambda": 0.2092914922783772, "lambda_min": 0.08670155704021454, "lambda_max": 0.3548862636089325, "S_mean": 0.3370768229166667, "G_mean": 0.3895923637319356, "G_std": 0.17161936250825724, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.689381122589111, "grad_norm": 0.725368320941925, "tokens_seen": 33595392, "tokens_per_sec": 15131.428920407781, "vram_gb": 6.442267648, "lambda": 0.20976976677775383, "lambda_min": 0.08677265048027039, "lambda_max": 0.3563803732395172, "S_mean": 0.3399251302083333, "G_mean": 0.39156696464245516, "G_std": 0.17140334961004555, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.625999450683594, "grad_norm": 0.8083025813102722, "tokens_seen": 33759232, "tokens_per_sec": 15132.648950644118, "vram_gb": 6.442267648, "lambda": 0.2101848175128301, "lambda_min": 0.0867898166179657, "lambda_max": 0.3576781451702118, "S_mean": 0.375244140625, "G_mean": 0.38494530017487705, "G_std": 0.16746300435625017, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.734194993972778, "grad_norm": 0.7358510494232178, "tokens_seen": 33923072, "tokens_per_sec": 15134.052431386704, "vram_gb": 6.442267648, "lambda": 0.21068514324724674, "lambda_min": 0.08672134578227997, "lambda_max": 0.35924533009529114, "S_mean": 0.3291829427083333, "G_mean": 0.40033878044535715, "G_std": 0.17066613375209272, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.740932464599609, "grad_norm": 0.7917338013648987, "tokens_seen": 34086912, "tokens_per_sec": 15135.115112320007, "vram_gb": 6.442267648, "lambda": 0.21110688957075277, "lambda_min": 0.08684659004211426, "lambda_max": 0.3610548675060272, "S_mean": 0.33685302734375, "G_mean": 0.39206655983192223, "G_std": 0.1711883437819779, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.908297061920166, "grad_norm": 0.7114264369010925, "tokens_seen": 34250752, "tokens_per_sec": 15136.281535180695, "vram_gb": 6.442267648, "lambda": 0.21161337538311878, "lambda_min": 0.08702154457569122, "lambda_max": 0.3627699315547943, "S_mean": 0.3259073893229167, "G_mean": 0.39505979341144365, "G_std": 0.17321887705475092, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.728142738342285, "grad_norm": 0.7611643671989441, "tokens_seen": 34414592, "tokens_per_sec": 15137.320929718751, "vram_gb": 6.442267648, "lambda": 0.21214932451645532, "lambda_min": 0.08684763312339783, "lambda_max": 0.36399951577186584, "S_mean": 0.3454996744791667, "G_mean": 0.3897402674580614, "G_std": 0.17119742829042175, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.804657697677612, "grad_norm": 0.6671916246414185, "tokens_seen": 34578432, "tokens_per_sec": 15138.602646493804, "vram_gb": 6.442267648, "lambda": 0.21262734942138195, "lambda_min": 0.0867600068449974, "lambda_max": 0.3655344545841217, "S_mean": 0.325103759765625, "G_mean": 0.39211626444011927, "G_std": 0.17000000248663127, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.802039623260498, "grad_norm": 0.6786661744117737, "tokens_seen": 34742272, "tokens_per_sec": 15139.738105863833, "vram_gb": 6.442267648, "lambda": 0.21304297198851904, "lambda_min": 0.0866876170039177, "lambda_max": 0.3671918213367462, "S_mean": 0.336639404296875, "G_mean": 0.38949206626663607, "G_std": 0.17115074830750623, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.85455322265625, "grad_norm": 0.7236613631248474, "tokens_seen": 34906112, "tokens_per_sec": 15140.847524023284, "vram_gb": 6.442267648, "lambda": 0.21359918235490719, "lambda_min": 0.08680537343025208, "lambda_max": 0.36885401606559753, "S_mean": 0.3184814453125, "G_mean": 0.39041900425218046, "G_std": 0.17017586824173728, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.864190340042114, "grad_norm": 0.8020175099372864, "tokens_seen": 35069952, "tokens_per_sec": 15141.903606859907, "vram_gb": 6.442267648, "lambda": 0.21408760144064823, "lambda_min": 0.08668509125709534, "lambda_max": 0.3702875077724457, "S_mean": 0.33447265625, "G_mean": 0.3972075354928772, "G_std": 0.17347731906920671, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.840441942214966, "grad_norm": 0.7451948523521423, "tokens_seen": 35233792, "tokens_per_sec": 15143.023217121288, "vram_gb": 6.442267648, "lambda": 0.21456307700524727, "lambda_min": 0.0867348313331604, "lambda_max": 0.3716469407081604, "S_mean": 0.3367716471354167, "G_mean": 0.393576090844969, "G_std": 0.17067822076690695, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.685955762863159, "grad_norm": 0.7394644618034363, "tokens_seen": 35397632, "tokens_per_sec": 15144.258094020777, "vram_gb": 6.442267648, "lambda": 0.21499274546901384, "lambda_min": 0.08654128015041351, "lambda_max": 0.37315893173217773, "S_mean": 0.3474222819010417, "G_mean": 0.39231724400694173, "G_std": 0.1712601133622229, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.690598964691162, "grad_norm": 0.7619850039482117, "tokens_seen": 35561472, "tokens_per_sec": 15145.146045140684, "vram_gb": 6.442267648, "lambda": 0.21551166443775097, "lambda_min": 0.08633224666118622, "lambda_max": 0.37446510791778564, "S_mean": 0.3406168619791667, "G_mean": 0.3928618535088996, "G_std": 0.16980860483211777, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.852670192718506, "grad_norm": 0.6829832792282104, "tokens_seen": 35725312, "tokens_per_sec": 15145.88101762992, "vram_gb": 6.442267648, "lambda": 0.2159822260340055, "lambda_min": 0.086310476064682, "lambda_max": 0.3762771189212799, "S_mean": 0.3202921549479167, "G_mean": 0.40105347249967355, "G_std": 0.1669642589210222, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.832034349441528, "grad_norm": 0.7130746841430664, "tokens_seen": 35889152, "tokens_per_sec": 15147.134779483225, "vram_gb": 6.442267648, "lambda": 0.21636081269631782, "lambda_min": 0.08626055717468262, "lambda_max": 0.37795472145080566, "S_mean": 0.3265889485677083, "G_mean": 0.38982328516431153, "G_std": 0.16840084898285568, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.589189529418945, "grad_norm": 0.744275689125061, "tokens_seen": 36052992, "tokens_per_sec": 15148.323969663554, "vram_gb": 6.442267648, "lambda": 0.2168724319587151, "lambda_min": 0.08619779348373413, "lambda_max": 0.37947607040405273, "S_mean": 0.3307698567708333, "G_mean": 0.3918915918717782, "G_std": 0.1679907562987258, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.854618787765503, "grad_norm": 0.693608283996582, "tokens_seen": 36216832, "tokens_per_sec": 15149.193215097835, "vram_gb": 6.442267648, "lambda": 0.21732835285365582, "lambda_min": 0.08603919297456741, "lambda_max": 0.38064101338386536, "S_mean": 0.3324991861979167, "G_mean": 0.39792427280917764, "G_std": 0.1692142648777614, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.714554071426392, "grad_norm": 0.8198394179344177, "tokens_seen": 36380672, "tokens_per_sec": 15149.85919422524, "vram_gb": 6.442267648, "lambda": 0.21781370198975006, "lambda_min": 0.08599364757537842, "lambda_max": 0.38212850689888, "S_mean": 0.3260091145833333, "G_mean": 0.3909165835939348, "G_std": 0.16855445931044719, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.726042032241821, "grad_norm": 0.7672957181930542, "tokens_seen": 36544512, "tokens_per_sec": 15150.89465248384, "vram_gb": 6.442267648, "lambda": 0.21830343548208475, "lambda_min": 0.08599266409873962, "lambda_max": 0.38370653986930847, "S_mean": 0.3232421875, "G_mean": 0.3847840049614509, "G_std": 0.17049745679832995, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.694061517715454, "grad_norm": 0.694099485874176, "tokens_seen": 36708352, "tokens_per_sec": 15151.597260751087, "vram_gb": 6.442267648, "lambda": 0.21881148715813956, "lambda_min": 0.08599462360143661, "lambda_max": 0.3852402865886688, "S_mean": 0.316192626953125, "G_mean": 0.38561362369606894, "G_std": 0.16962529649026692, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.7189085483551025, "grad_norm": 0.7944169640541077, "tokens_seen": 36872192, "tokens_per_sec": 15152.509839938371, "vram_gb": 6.442267648, "lambda": 0.21923744088659683, "lambda_min": 0.08578726649284363, "lambda_max": 0.38701143860816956, "S_mean": 0.3173319498697917, "G_mean": 0.3949823022509615, "G_std": 0.17244007841994366, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.809940433502197, "val_ppl": 122.72430704237183, "lambda": 0.2192558767274022, "lambda_min": 0.08577536791563034, "lambda_max": 0.38710832595825195, "lambda_per_layer": [0.1619368940591812, 0.1438300907611847, 0.1393493264913559, 0.1337980031967163, 0.12183650583028793, 0.08577536791563034, 0.12497870624065399, 0.09803550690412521, 0.14126718044281006, 0.26603153347969055, 0.263078510761261, 0.32406291365623474, 0.24771209061145782, 0.367214560508728, 0.3411346673965454, 0.31683990359306335, 0.38710832595825195, 0.3050299286842346, 0.24978666007518768, 0.3297310471534729, 0.15541137754917145, 0.22279496490955353, 0.1851089745759964, 0.15028800070285797], "S_mean": 0.3382161458333333, "G_mean": 0.39415635222879547, "G_std": 0.17191247354882458, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.705840826034546, "grad_norm": 0.6894204616546631, "tokens_seen": 37036032, "tokens_per_sec": 15125.986084519782, "vram_gb": 6.442267648, "lambda": 0.21971843546877304, "lambda_min": 0.08566261827945709, "lambda_max": 0.3886084258556366, "S_mean": 0.335052490234375, "G_mean": 0.3846162559154133, "G_std": 0.16952644917182624, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.745904445648193, "grad_norm": 0.7414753437042236, "tokens_seen": 37199872, "tokens_per_sec": 15125.130056667169, "vram_gb": 6.442267648, "lambda": 0.220151098134617, "lambda_min": 0.08571228384971619, "lambda_max": 0.3901638984680176, "S_mean": 0.3290608723958333, "G_mean": 0.38585143520807225, "G_std": 0.1709164067481955, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.762210845947266, "grad_norm": 0.7413001656532288, "tokens_seen": 37363712, "tokens_per_sec": 15124.218655327708, "vram_gb": 6.442267648, "lambda": 0.22062792349606752, "lambda_min": 0.0858224406838417, "lambda_max": 0.39149197936058044, "S_mean": 0.3248494466145833, "G_mean": 0.3876468523715933, "G_std": 0.17045870896739265, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.721010208129883, "grad_norm": 0.7377870678901672, "tokens_seen": 37527552, "tokens_per_sec": 15122.261368124482, "vram_gb": 6.442267648, "lambda": 0.22103441537668309, "lambda_min": 0.08565087616443634, "lambda_max": 0.3929823935031891, "S_mean": 0.3339640299479167, "G_mean": 0.388102900236845, "G_std": 0.16868512254829207, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.716953754425049, "grad_norm": 0.7286366820335388, "tokens_seen": 37691392, "tokens_per_sec": 15121.44827509561, "vram_gb": 6.442267648, "lambda": 0.22153022326529026, "lambda_min": 0.08549220114946365, "lambda_max": 0.3945577144622803, "S_mean": 0.3462727864583333, "G_mean": 0.3885766194822888, "G_std": 0.16907477557348707, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.739979982376099, "grad_norm": 0.7686018347740173, "tokens_seen": 37855232, "tokens_per_sec": 15119.493699215991, "vram_gb": 6.442267648, "lambda": 0.22199827153235674, "lambda_min": 0.08550515025854111, "lambda_max": 0.39592111110687256, "S_mean": 0.3178304036458333, "G_mean": 0.3859506705775857, "G_std": 0.16782513579043248, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.7983222007751465, "grad_norm": 0.7703383564949036, "tokens_seen": 38019072, "tokens_per_sec": 15115.17457450793, "vram_gb": 6.442267648, "lambda": 0.2224260043973724, "lambda_min": 0.08537621051073074, "lambda_max": 0.39738258719444275, "S_mean": 0.3327229817708333, "G_mean": 0.3819904289363573, "G_std": 0.16790888264464834, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.6769349575042725, "grad_norm": 0.763344943523407, "tokens_seen": 38182912, "tokens_per_sec": 15111.783542992232, "vram_gb": 6.442267648, "lambda": 0.22282474146535, "lambda_min": 0.08526663482189178, "lambda_max": 0.398739218711853, "S_mean": 0.3236897786458333, "G_mean": 0.3854291300910215, "G_std": 0.16702316678129137, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.721460580825806, "grad_norm": 0.7485673427581787, "tokens_seen": 38346752, "tokens_per_sec": 15109.55142691681, "vram_gb": 6.442267648, "lambda": 0.22328661661595106, "lambda_min": 0.08531200885772705, "lambda_max": 0.4002477824687958, "S_mean": 0.3104146321614583, "G_mean": 0.3910363026273747, "G_std": 0.16981158684939146, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.688882350921631, "grad_norm": 0.7918930053710938, "tokens_seen": 38510592, "tokens_per_sec": 15107.922105096044, "vram_gb": 6.442267648, "lambda": 0.22377826614926258, "lambda_min": 0.08524604141712189, "lambda_max": 0.4020656645298004, "S_mean": 0.3224894205729167, "G_mean": 0.3876813513537248, "G_std": 0.16889543241510788, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.724027156829834, "grad_norm": 0.7591776251792908, "tokens_seen": 38674432, "tokens_per_sec": 15105.465208343236, "vram_gb": 6.442267648, "lambda": 0.22424745808045068, "lambda_min": 0.08520499616861343, "lambda_max": 0.4036899507045746, "S_mean": 0.3266499837239583, "G_mean": 0.39499434852041304, "G_std": 0.16623009811155498, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.601147651672363, "grad_norm": 0.7940176725387573, "tokens_seen": 38838272, "tokens_per_sec": 15106.53650814863, "vram_gb": 6.442267648, "lambda": 0.2247697881733378, "lambda_min": 0.08525688946247101, "lambda_max": 0.40517640113830566, "S_mean": 0.3295491536458333, "G_mean": 0.3901003492064774, "G_std": 0.16871467682843408, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.689515829086304, "grad_norm": 0.7889002561569214, "tokens_seen": 39002112, "tokens_per_sec": 15107.623564927473, "vram_gb": 6.442267648, "lambda": 0.2252369367827972, "lambda_min": 0.0851568877696991, "lambda_max": 0.4065721929073334, "S_mean": 0.3352254231770833, "G_mean": 0.3910440008621663, "G_std": 0.16778496156136194, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.734548091888428, "grad_norm": 0.7333645224571228, "tokens_seen": 39165952, "tokens_per_sec": 15108.802952390897, "vram_gb": 6.442267648, "lambda": 0.22570862962553898, "lambda_min": 0.08507712930440903, "lambda_max": 0.40810948610305786, "S_mean": 0.3194783528645833, "G_mean": 0.3882999330914269, "G_std": 0.16861501328336695, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.7938573360443115, "grad_norm": 0.7420093417167664, "tokens_seen": 39329792, "tokens_per_sec": 15110.076250821343, "vram_gb": 6.442267648, "lambda": 0.2262173273290197, "lambda_min": 0.08517862111330032, "lambda_max": 0.40983113646507263, "S_mean": 0.326568603515625, "G_mean": 0.38989774106691283, "G_std": 0.1684829193012168, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.690460681915283, "grad_norm": 0.7247873544692993, "tokens_seen": 39493632, "tokens_per_sec": 15111.270005328728, "vram_gb": 6.442267648, "lambda": 0.2266270158191522, "lambda_min": 0.08519163727760315, "lambda_max": 0.41180387139320374, "S_mean": 0.3279317220052083, "G_mean": 0.38377432866642874, "G_std": 0.16878403125641248, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.464458465576172, "grad_norm": 0.7421135306358337, "tokens_seen": 39657472, "tokens_per_sec": 15112.363942513897, "vram_gb": 6.442267648, "lambda": 0.2270338467011849, "lambda_min": 0.08520269393920898, "lambda_max": 0.4130954444408417, "S_mean": 0.3352864583333333, "G_mean": 0.3813349664366494, "G_std": 0.16882330703083426, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.720158576965332, "grad_norm": 0.7347882390022278, "tokens_seen": 39821312, "tokens_per_sec": 15112.697046844149, "vram_gb": 6.442267648, "lambda": 0.22748661134392023, "lambda_min": 0.08515466004610062, "lambda_max": 0.41440844535827637, "S_mean": 0.3394063313802083, "G_mean": 0.3844110432546586, "G_std": 0.1667984836579611, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.718455791473389, "grad_norm": 0.7486737370491028, "tokens_seen": 39985152, "tokens_per_sec": 15113.01564866524, "vram_gb": 6.442267648, "lambda": 0.22796694530795017, "lambda_min": 0.08508045226335526, "lambda_max": 0.41599491238594055, "S_mean": 0.3218892415364583, "G_mean": 0.3827349589361499, "G_std": 0.1674762835415701, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.531691551208496, "grad_norm": 0.6755222082138062, "tokens_seen": 40148992, "tokens_per_sec": 15114.161392185953, "vram_gb": 6.442267648, "lambda": 0.22844892212500176, "lambda_min": 0.08492882549762726, "lambda_max": 0.41753455996513367, "S_mean": 0.3281758626302083, "G_mean": 0.3793092481791973, "G_std": 0.1681562274073561, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.766132593154907, "grad_norm": 0.796536922454834, "tokens_seen": 40312832, "tokens_per_sec": 15115.271406070477, "vram_gb": 6.442267648, "lambda": 0.2289095583061377, "lambda_min": 0.0851123183965683, "lambda_max": 0.41926097869873047, "S_mean": 0.3419596354166667, "G_mean": 0.3826396669416378, "G_std": 0.16725953031952182, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.624762535095215, "grad_norm": 0.7225190997123718, "tokens_seen": 40476672, "tokens_per_sec": 15116.346432263783, "vram_gb": 6.442267648, "lambda": 0.2292969487607479, "lambda_min": 0.08518889546394348, "lambda_max": 0.42068251967430115, "S_mean": 0.3234354654947917, "G_mean": 0.37789292509357136, "G_std": 0.1672609394105772, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.704333782196045, "grad_norm": 0.7274549007415771, "tokens_seen": 40640512, "tokens_per_sec": 15116.709437902939, "vram_gb": 6.442267648, "lambda": 0.22976280159006515, "lambda_min": 0.0851554274559021, "lambda_max": 0.42222774028778076, "S_mean": 0.3197733561197917, "G_mean": 0.3848522547632456, "G_std": 0.16765603826691708, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.565037965774536, "grad_norm": 0.7115857601165771, "tokens_seen": 40804352, "tokens_per_sec": 15117.341136268325, "vram_gb": 6.442267648, "lambda": 0.23024764439711967, "lambda_min": 0.08505105972290039, "lambda_max": 0.42364826798439026, "S_mean": 0.3410746256510417, "G_mean": 0.37631673847014707, "G_std": 0.1673910537889848, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.754896640777588, "grad_norm": 0.8730747699737549, "tokens_seen": 40968192, "tokens_per_sec": 15118.307665853563, "vram_gb": 6.442267648, "lambda": 0.2307071409498652, "lambda_min": 0.08493191003799438, "lambda_max": 0.42518243193626404, "S_mean": 0.3389078776041667, "G_mean": 0.3863384883540372, "G_std": 0.16790895358038446, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.766323745250702, "val_ppl": 117.48653664219897, "lambda": 0.23072556561479965, "lambda_min": 0.08492577821016312, "lambda_max": 0.42526382207870483, "lambda_per_layer": [0.1646893322467804, 0.1456385850906372, 0.14088314771652222, 0.1343381702899933, 0.1207364946603775, 0.08492577821016312, 0.12476026266813278, 0.09488589316606522, 0.14351090788841248, 0.27500203251838684, 0.2748093008995056, 0.3446230888366699, 0.25822797417640686, 0.3942036032676697, 0.36966463923454285, 0.33915820717811584, 0.42526382207870483, 0.3264607787132263, 0.26584392786026, 0.3622555136680603, 0.16085216403007507, 0.2378326803445816, 0.19531594216823578, 0.15353132784366608], "S_mean": 0.3246663411458333, "G_mean": 0.38415876488822204, "G_std": 0.1658375619444996, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.6418843269348145, "grad_norm": 0.811730146408081, "tokens_seen": 41132032, "tokens_per_sec": 15095.614453851244, "vram_gb": 6.442267648, "lambda": 0.23115596547722816, "lambda_min": 0.08486603200435638, "lambda_max": 0.426493763923645, "S_mean": 0.3387247721354167, "G_mean": 0.3886050156628092, "G_std": 0.16561864914062122, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.654191493988037, "grad_norm": 0.7939016819000244, "tokens_seen": 41295872, "tokens_per_sec": 15096.516804478624, "vram_gb": 6.442267648, "lambda": 0.2317021976535519, "lambda_min": 0.08482182770967484, "lambda_max": 0.4282301664352417, "S_mean": 0.322967529296875, "G_mean": 0.3856355821869026, "G_std": 0.16805686304966608, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.551573276519775, "grad_norm": 0.8194537162780762, "tokens_seen": 41459712, "tokens_per_sec": 15094.110680832551, "vram_gb": 6.442267648, "lambda": 0.2321969916423162, "lambda_min": 0.08473768085241318, "lambda_max": 0.4296332597732544, "S_mean": 0.33135986328125, "G_mean": 0.3778656650489817, "G_std": 0.16769541155857345, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.665900707244873, "grad_norm": 0.7936710715293884, "tokens_seen": 41623552, "tokens_per_sec": 15088.90160011249, "vram_gb": 6.442267648, "lambda": 0.23262640802810589, "lambda_min": 0.08473373204469681, "lambda_max": 0.43102625012397766, "S_mean": 0.344329833984375, "G_mean": 0.3746630970854312, "G_std": 0.1666259493213147, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.678974151611328, "grad_norm": 0.7271668314933777, "tokens_seen": 41787392, "tokens_per_sec": 15083.983539868032, "vram_gb": 6.442267648, "lambda": 0.2330037116383513, "lambda_min": 0.08456632494926453, "lambda_max": 0.431708961725235, "S_mean": 0.3234659830729167, "G_mean": 0.3910587728799631, "G_std": 0.1656228166539222, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.575775384902954, "grad_norm": 0.7750360369682312, "tokens_seen": 41951232, "tokens_per_sec": 15078.683456127994, "vram_gb": 6.442267648, "lambda": 0.23348270170390606, "lambda_min": 0.08450201153755188, "lambda_max": 0.4331348240375519, "S_mean": 0.337890625, "G_mean": 0.37818511319346726, "G_std": 0.16683251348634562, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.705313682556152, "grad_norm": 0.8216353058815002, "tokens_seen": 42115072, "tokens_per_sec": 15073.451598350115, "vram_gb": 6.442267648, "lambda": 0.23393933412929377, "lambda_min": 0.0844583585858345, "lambda_max": 0.4341926872730255, "S_mean": 0.327392578125, "G_mean": 0.3811164586804807, "G_std": 0.16608820320107043, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.540383338928223, "grad_norm": 0.760969340801239, "tokens_seen": 42278912, "tokens_per_sec": 15068.611603477126, "vram_gb": 6.442267648, "lambda": 0.23435907593617836, "lambda_min": 0.0844801813364029, "lambda_max": 0.43553850054740906, "S_mean": 0.3359476725260417, "G_mean": 0.3838896327652037, "G_std": 0.16695426303582886, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.568725824356079, "grad_norm": 0.7398916482925415, "tokens_seen": 42442752, "tokens_per_sec": 15067.591663353154, "vram_gb": 6.442267648, "lambda": 0.23470297579964003, "lambda_min": 0.08447623252868652, "lambda_max": 0.4370118975639343, "S_mean": 0.3265380859375, "G_mean": 0.3831328412828346, "G_std": 0.1683037173934281, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.503219842910767, "grad_norm": 0.7827414870262146, "tokens_seen": 42606592, "tokens_per_sec": 15066.47981383946, "vram_gb": 6.442267648, "lambda": 0.23531275211522976, "lambda_min": 0.08446245640516281, "lambda_max": 0.43881556391716003, "S_mean": 0.3273111979166667, "G_mean": 0.38004589344685274, "G_std": 0.1656403822513918, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.603505373001099, "grad_norm": 0.744313657283783, "tokens_seen": 42770432, "tokens_per_sec": 15066.739186266923, "vram_gb": 6.442267648, "lambda": 0.23572298822303614, "lambda_min": 0.08440351486206055, "lambda_max": 0.44055265188217163, "S_mean": 0.3225504557291667, "G_mean": 0.38306743279099464, "G_std": 0.16837096159967282, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.593130350112915, "grad_norm": 0.8030393719673157, "tokens_seen": 42934272, "tokens_per_sec": 15067.442876063886, "vram_gb": 6.442267648, "lambda": 0.23611155586938062, "lambda_min": 0.08435078710317612, "lambda_max": 0.4418851435184479, "S_mean": 0.3332316080729167, "G_mean": 0.388134543163081, "G_std": 0.16690844856202602, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.543735980987549, "grad_norm": 0.7231416702270508, "tokens_seen": 43098112, "tokens_per_sec": 15068.286924125796, "vram_gb": 6.442267648, "lambda": 0.23657434557874998, "lambda_min": 0.0844152495265007, "lambda_max": 0.4433090090751648, "S_mean": 0.3115946451822917, "G_mean": 0.3789380687909822, "G_std": 0.1678592620883137, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.591943740844727, "grad_norm": 0.7877677083015442, "tokens_seen": 43261952, "tokens_per_sec": 15068.6455940384, "vram_gb": 6.442267648, "lambda": 0.23702862082670131, "lambda_min": 0.08448653668165207, "lambda_max": 0.44509434700012207, "S_mean": 0.3150533040364583, "G_mean": 0.38504793277631205, "G_std": 0.16653761081397533, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.717881917953491, "grad_norm": 0.7487972974777222, "tokens_seen": 43425792, "tokens_per_sec": 15069.34636413507, "vram_gb": 6.442267648, "lambda": 0.23744185672452053, "lambda_min": 0.08446111530065536, "lambda_max": 0.4463496804237366, "S_mean": 0.3250935872395833, "G_mean": 0.38119179964996874, "G_std": 0.16805494010138014, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.581394910812378, "grad_norm": 0.7214634418487549, "tokens_seen": 43589632, "tokens_per_sec": 15069.36735705645, "vram_gb": 6.442267648, "lambda": 0.2378607395415505, "lambda_min": 0.08441280573606491, "lambda_max": 0.447504460811615, "S_mean": 0.32073974609375, "G_mean": 0.374936076036344, "G_std": 0.16703185276128352, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.663760185241699, "grad_norm": 0.748325526714325, "tokens_seen": 43753472, "tokens_per_sec": 15069.54926269774, "vram_gb": 6.442267648, "lambda": 0.23828900636484227, "lambda_min": 0.08440359681844711, "lambda_max": 0.44877004623413086, "S_mean": 0.3216145833333333, "G_mean": 0.3817270491272211, "G_std": 0.16675248524794975, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.514021396636963, "grad_norm": 0.7519374489784241, "tokens_seen": 43917312, "tokens_per_sec": 15070.263961723835, "vram_gb": 6.442267648, "lambda": 0.23877200484275818, "lambda_min": 0.08440537005662918, "lambda_max": 0.4506228268146515, "S_mean": 0.3273213704427083, "G_mean": 0.38166411872953176, "G_std": 0.1677941185577462, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.771535158157349, "grad_norm": 0.7349217534065247, "tokens_seen": 44081152, "tokens_per_sec": 15070.288186341304, "vram_gb": 6.442267648, "lambda": 0.23924288774530092, "lambda_min": 0.08436357229948044, "lambda_max": 0.45268383622169495, "S_mean": 0.324249267578125, "G_mean": 0.3784514629902939, "G_std": 0.1678110487603893, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.473650693893433, "grad_norm": 0.7484071254730225, "tokens_seen": 44244992, "tokens_per_sec": 15070.857191458435, "vram_gb": 6.442267648, "lambda": 0.23971633551021418, "lambda_min": 0.08432240784168243, "lambda_max": 0.45474863052368164, "S_mean": 0.3162841796875, "G_mean": 0.3741914709874739, "G_std": 0.1672816463978961, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.637436866760254, "grad_norm": 0.7221217751502991, "tokens_seen": 44408832, "tokens_per_sec": 15071.685053127045, "vram_gb": 6.442267648, "lambda": 0.24014966345081726, "lambda_min": 0.0842386782169342, "lambda_max": 0.4560372829437256, "S_mean": 0.33013916015625, "G_mean": 0.37872939463704824, "G_std": 0.1650985838690152, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.512876510620117, "grad_norm": 0.75877845287323, "tokens_seen": 44572672, "tokens_per_sec": 15072.405039442876, "vram_gb": 6.442267648, "lambda": 0.2405006717890501, "lambda_min": 0.08424516022205353, "lambda_max": 0.4575059413909912, "S_mean": 0.3149515787760417, "G_mean": 0.3832904367397229, "G_std": 0.16483547321210304, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.632986783981323, "grad_norm": 0.7686243057250977, "tokens_seen": 44736512, "tokens_per_sec": 15073.310568719291, "vram_gb": 6.442267648, "lambda": 0.24091452515373626, "lambda_min": 0.0841912105679512, "lambda_max": 0.4589681029319763, "S_mean": 0.3222859700520833, "G_mean": 0.38085759912307066, "G_std": 0.16590716410428286, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.541865348815918, "grad_norm": 0.7752079367637634, "tokens_seen": 44900352, "tokens_per_sec": 15074.828710217162, "vram_gb": 6.442267648, "lambda": 0.24141437280923128, "lambda_min": 0.08420788496732712, "lambda_max": 0.46056222915649414, "S_mean": 0.311065673828125, "G_mean": 0.38053329785664874, "G_std": 0.1650619093173494, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.614858865737915, "grad_norm": 0.7892166376113892, "tokens_seen": 45064192, "tokens_per_sec": 15075.62988474858, "vram_gb": 6.442267648, "lambda": 0.2419064131875833, "lambda_min": 0.08431477099657059, "lambda_max": 0.4621172249317169, "S_mean": 0.3325602213541667, "G_mean": 0.3722052489562581, "G_std": 0.16702739514100054, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.69539510011673, "val_ppl": 109.44204066380331, "lambda": 0.2419262264544765, "lambda_min": 0.0843234658241272, "lambda_max": 0.46216434240341187, "lambda_per_layer": [0.16659192740917206, 0.14628979563713074, 0.1427544206380844, 0.1347232609987259, 0.12227965891361237, 0.0843234658241272, 0.12414877116680145, 0.09375777095556259, 0.14559884369373322, 0.28283634781837463, 0.28441452980041504, 0.36535072326660156, 0.26992499828338623, 0.41904836893081665, 0.3944537341594696, 0.3616788685321808, 0.46216434240341187, 0.3484961986541748, 0.28059419989585876, 0.3956335783004761, 0.16470082104206085, 0.25349676609039307, 0.20545315742492676, 0.15751488506793976], "S_mean": 0.3233540852864583, "G_mean": 0.37840679039557773, "G_std": 0.16792312753386796, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.444223642349243, "grad_norm": 0.7677919268608093, "tokens_seen": 45228032, "tokens_per_sec": 15055.493273122322, "vram_gb": 6.442267648, "lambda": 0.24222877342253923, "lambda_min": 0.0843486338853836, "lambda_max": 0.4632396399974823, "S_mean": 0.3151448567708333, "G_mean": 0.3731184909120202, "G_std": 0.16630379758620015, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.661320447921753, "grad_norm": 0.8227590322494507, "tokens_seen": 45391872, "tokens_per_sec": 15055.665913375222, "vram_gb": 6.442267648, "lambda": 0.24264594167470932, "lambda_min": 0.08418302983045578, "lambda_max": 0.4647335410118103, "S_mean": 0.323638916015625, "G_mean": 0.3793259069013099, "G_std": 0.16514883105022213, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.68831729888916, "grad_norm": 0.7659909129142761, "tokens_seen": 45555712, "tokens_per_sec": 15055.269653657757, "vram_gb": 6.442267648, "lambda": 0.24303429325421652, "lambda_min": 0.08424050360918045, "lambda_max": 0.4659297466278076, "S_mean": 0.314208984375, "G_mean": 0.3756120529336234, "G_std": 0.16642628121189773, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.724480390548706, "grad_norm": 0.7912384271621704, "tokens_seen": 45719552, "tokens_per_sec": 15054.88901596851, "vram_gb": 6.442267648, "lambda": 0.2433958643426498, "lambda_min": 0.08427970856428146, "lambda_max": 0.4670187830924988, "S_mean": 0.3109944661458333, "G_mean": 0.3789449748583138, "G_std": 0.16588370168271163, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.5524210929870605, "grad_norm": 0.7272500991821289, "tokens_seen": 45883392, "tokens_per_sec": 15054.998523786373, "vram_gb": 6.442267648, "lambda": 0.24383553645263115, "lambda_min": 0.08426893502473831, "lambda_max": 0.46890902519226074, "S_mean": 0.3284098307291667, "G_mean": 0.37716613709926605, "G_std": 0.1662051713016505, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.628749847412109, "grad_norm": 0.8222120404243469, "tokens_seen": 46047232, "tokens_per_sec": 15055.078284699439, "vram_gb": 6.442267648, "lambda": 0.24424950095514456, "lambda_min": 0.08423496037721634, "lambda_max": 0.4702416658401489, "S_mean": 0.3238525390625, "G_mean": 0.3736687171428154, "G_std": 0.167443872972702, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.729532957077026, "grad_norm": 0.7823415398597717, "tokens_seen": 46211072, "tokens_per_sec": 15055.126266026553, "vram_gb": 6.442267648, "lambda": 0.2446556119248271, "lambda_min": 0.08412139117717743, "lambda_max": 0.47142493724823, "S_mean": 0.3215128580729167, "G_mean": 0.3797593166430791, "G_std": 0.1647566962055862, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.494379997253418, "grad_norm": 0.7369957566261292, "tokens_seen": 46374912, "tokens_per_sec": 15055.07740696809, "vram_gb": 6.442267648, "lambda": 0.24506564593563476, "lambda_min": 0.08408067375421524, "lambda_max": 0.47256505489349365, "S_mean": 0.3235880533854167, "G_mean": 0.3771767970950653, "G_std": 0.16589758304568628, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.558949947357178, "grad_norm": 0.8003881573677063, "tokens_seen": 46538752, "tokens_per_sec": 15055.149219755156, "vram_gb": 6.442267648, "lambda": 0.24549279424051443, "lambda_min": 0.08411237597465515, "lambda_max": 0.4739839732646942, "S_mean": 0.3144327799479167, "G_mean": 0.37535428046248853, "G_std": 0.16710779583081603, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.691119194030762, "grad_norm": 0.8554272055625916, "tokens_seen": 46702592, "tokens_per_sec": 15055.408459694718, "vram_gb": 6.442267648, "lambda": 0.2459304928779602, "lambda_min": 0.08406063169240952, "lambda_max": 0.47546714544296265, "S_mean": 0.32366943359375, "G_mean": 0.3709365854350229, "G_std": 0.16618428829436502, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.564256191253662, "grad_norm": 0.7528659701347351, "tokens_seen": 46866432, "tokens_per_sec": 15055.27561777298, "vram_gb": 6.442267648, "lambda": 0.2463596553231279, "lambda_min": 0.08402448147535324, "lambda_max": 0.47697192430496216, "S_mean": 0.3144327799479167, "G_mean": 0.3763433409233888, "G_std": 0.1656126215433081, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.730515480041504, "grad_norm": 0.84363853931427, "tokens_seen": 47030272, "tokens_per_sec": 15055.256969346055, "vram_gb": 6.442267648, "lambda": 0.24675203238924345, "lambda_min": 0.0841665118932724, "lambda_max": 0.47822242975234985, "S_mean": 0.33319091796875, "G_mean": 0.3767194544586043, "G_std": 0.16529810354889682, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.588881254196167, "grad_norm": 0.7319701910018921, "tokens_seen": 47194112, "tokens_per_sec": 15055.29189279039, "vram_gb": 6.442267648, "lambda": 0.24713907421876988, "lambda_min": 0.08415640890598297, "lambda_max": 0.4793516993522644, "S_mean": 0.3106689453125, "G_mean": 0.372845664465179, "G_std": 0.16536885919049382, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.6359922885894775, "grad_norm": 0.775590717792511, "tokens_seen": 47357952, "tokens_per_sec": 15055.2517171617, "vram_gb": 6.442267648, "lambda": 0.2474533642331759, "lambda_min": 0.0841573104262352, "lambda_max": 0.4808984398841858, "S_mean": 0.3264567057291667, "G_mean": 0.37353368538121384, "G_std": 0.16436450889644524, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.484380722045898, "grad_norm": 0.7924505472183228, "tokens_seen": 47521792, "tokens_per_sec": 15055.013828422541, "vram_gb": 6.442267648, "lambda": 0.24780782560507456, "lambda_min": 0.084195077419281, "lambda_max": 0.48243796825408936, "S_mean": 0.3330078125, "G_mean": 0.37253940012305975, "G_std": 0.16530987403045097, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.71575140953064, "grad_norm": 0.7518150806427002, "tokens_seen": 47685632, "tokens_per_sec": 15055.15585337524, "vram_gb": 6.442267648, "lambda": 0.24827480999131998, "lambda_min": 0.08413553237915039, "lambda_max": 0.48378869891166687, "S_mean": 0.3320414225260417, "G_mean": 0.3712646110604207, "G_std": 0.16335011444364986, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.531414031982422, "grad_norm": 0.7620919942855835, "tokens_seen": 47849472, "tokens_per_sec": 15054.973385235098, "vram_gb": 6.442267648, "lambda": 0.24856825297077498, "lambda_min": 0.08398014307022095, "lambda_max": 0.48498544096946716, "S_mean": 0.3190104166666667, "G_mean": 0.37440690437021357, "G_std": 0.16350510041229427, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.632373809814453, "grad_norm": 0.7723290324211121, "tokens_seen": 48013312, "tokens_per_sec": 15055.43434701876, "vram_gb": 6.442267648, "lambda": 0.24896474741399288, "lambda_min": 0.08389545232057571, "lambda_max": 0.4866437017917633, "S_mean": 0.316558837890625, "G_mean": 0.3694636778285106, "G_std": 0.16450217342935503, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.652785301208496, "grad_norm": 0.7895969152450562, "tokens_seen": 48177152, "tokens_per_sec": 15055.604654912744, "vram_gb": 6.442267648, "lambda": 0.24932707908252874, "lambda_min": 0.08379505574703217, "lambda_max": 0.48797354102134705, "S_mean": 0.3383280436197917, "G_mean": 0.3732403563335538, "G_std": 0.16564135563870272, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.560433626174927, "grad_norm": 0.7697533369064331, "tokens_seen": 48340992, "tokens_per_sec": 15055.404019802043, "vram_gb": 6.442267648, "lambda": 0.2497617918998003, "lambda_min": 0.08370952308177948, "lambda_max": 0.4891660511493683, "S_mean": 0.3190714518229167, "G_mean": 0.36842549964785576, "G_std": 0.1656548990479981, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.571462154388428, "grad_norm": 0.8215556144714355, "tokens_seen": 48504832, "tokens_per_sec": 15055.131463996513, "vram_gb": 6.442267648, "lambda": 0.2501053201655547, "lambda_min": 0.08358646184206009, "lambda_max": 0.4902827739715576, "S_mean": 0.3332621256510417, "G_mean": 0.3678687899373472, "G_std": 0.16479263136473796, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.487907648086548, "grad_norm": 0.8136134147644043, "tokens_seen": 48668672, "tokens_per_sec": 15054.998678072869, "vram_gb": 6.442267648, "lambda": 0.2504966442162792, "lambda_min": 0.08349984139204025, "lambda_max": 0.49149850010871887, "S_mean": 0.3227945963541667, "G_mean": 0.3693230397378405, "G_std": 0.16336353639295945, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.5497777462005615, "grad_norm": 0.8131731152534485, "tokens_seen": 48832512, "tokens_per_sec": 15054.966009223179, "vram_gb": 6.442267648, "lambda": 0.25092022276173037, "lambda_min": 0.08346932381391525, "lambda_max": 0.49266383051872253, "S_mean": 0.31475830078125, "G_mean": 0.3673605048873772, "G_std": 0.16514000801059106, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.797592878341675, "grad_norm": 0.754355788230896, "tokens_seen": 48996352, "tokens_per_sec": 15055.105459226135, "vram_gb": 6.442267648, "lambda": 0.2513288144643108, "lambda_min": 0.08345735818147659, "lambda_max": 0.49416112899780273, "S_mean": 0.3226725260416667, "G_mean": 0.37645416404120624, "G_std": 0.16696271665083864, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.544891119003296, "grad_norm": 0.770806074142456, "tokens_seen": 49160192, "tokens_per_sec": 15055.077178461523, "vram_gb": 6.442267648, "lambda": 0.2517046018814047, "lambda_min": 0.08351025730371475, "lambda_max": 0.4953291416168213, "S_mean": 0.32965087890625, "G_mean": 0.37760728869276744, "G_std": 0.1630134052441766, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.643554997444153, "val_ppl": 103.91310254741305, "lambda": 0.25172906555235386, "lambda_min": 0.08350756019353867, "lambda_max": 0.49537748098373413, "lambda_per_layer": [0.16894666850566864, 0.14690813422203064, 0.14424560964107513, 0.1350403130054474, 0.12246081978082657, 0.08350756019353867, 0.12390469014644623, 0.09300543367862701, 0.1483447104692459, 0.289438933134079, 0.29362547397613525, 0.38322943449020386, 0.27857258915901184, 0.4408146142959595, 0.41729092597961426, 0.38293203711509705, 0.49537748098373413, 0.3654923439025879, 0.2953893542289734, 0.42214876413345337, 0.169486865401268, 0.26611778140068054, 0.21453098952770233, 0.16068604588508606], "S_mean": 0.3280843098958333, "G_mean": 0.3755603265017271, "G_std": 0.16419646865688264, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.485884189605713, "grad_norm": 0.8069875240325928, "tokens_seen": 49324032, "tokens_per_sec": 15036.74459064036, "vram_gb": 6.442267648, "lambda": 0.2521008277932803, "lambda_min": 0.08356239646673203, "lambda_max": 0.4964972734451294, "S_mean": 0.311676025390625, "G_mean": 0.3718633310248454, "G_std": 0.16459524181361, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.508800506591797, "grad_norm": 0.8073917031288147, "tokens_seen": 49487872, "tokens_per_sec": 15036.394560251501, "vram_gb": 6.442267648, "lambda": 0.25253972349067527, "lambda_min": 0.08352027088403702, "lambda_max": 0.4981268048286438, "S_mean": 0.330657958984375, "G_mean": 0.37357604519153637, "G_std": 0.16461957773814598, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.6636598110198975, "grad_norm": 0.8078941702842712, "tokens_seen": 49651712, "tokens_per_sec": 15036.264243594855, "vram_gb": 6.442267648, "lambda": 0.2529549704243739, "lambda_min": 0.0835358202457428, "lambda_max": 0.49930649995803833, "S_mean": 0.3294779459635417, "G_mean": 0.37255659233778715, "G_std": 0.16465328362149498, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.589503288269043, "grad_norm": 0.7886185646057129, "tokens_seen": 49815552, "tokens_per_sec": 15036.682022005522, "vram_gb": 6.442267648, "lambda": 0.2533588030685981, "lambda_min": 0.08341632783412933, "lambda_max": 0.5004243850708008, "S_mean": 0.3268534342447917, "G_mean": 0.37531870991612476, "G_std": 0.16373151044050852, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.5301737785339355, "grad_norm": 0.8052544593811035, "tokens_seen": 49979392, "tokens_per_sec": 15037.163195631343, "vram_gb": 6.442267648, "lambda": 0.25378550589084625, "lambda_min": 0.08337666094303131, "lambda_max": 0.5020231008529663, "S_mean": 0.3323567708333333, "G_mean": 0.3715431815944612, "G_std": 0.16483108807976046, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.6563873291015625, "grad_norm": 0.8082514405250549, "tokens_seen": 50143232, "tokens_per_sec": 15037.930623821894, "vram_gb": 6.442267648, "lambda": 0.25418926992764074, "lambda_min": 0.08338864147663116, "lambda_max": 0.5031930208206177, "S_mean": 0.315704345703125, "G_mean": 0.37067493811870617, "G_std": 0.16387716471217573, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.580044507980347, "grad_norm": 0.7833402752876282, "tokens_seen": 50307072, "tokens_per_sec": 15038.398438718048, "vram_gb": 6.442267648, "lambda": 0.25454604625701904, "lambda_min": 0.08339256793260574, "lambda_max": 0.5044144988059998, "S_mean": 0.3204752604166667, "G_mean": 0.37490262168770033, "G_std": 0.16500474815256894, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.456403017044067, "grad_norm": 0.748447060585022, "tokens_seen": 50470912, "tokens_per_sec": 15038.852426684593, "vram_gb": 6.442267648, "lambda": 0.25490698404610157, "lambda_min": 0.08332754671573639, "lambda_max": 0.505975067615509, "S_mean": 0.31878662109375, "G_mean": 0.37562513157414895, "G_std": 0.16379411068434516, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.625851154327393, "grad_norm": 0.8012614846229553, "tokens_seen": 50634752, "tokens_per_sec": 15039.16565983084, "vram_gb": 6.442267648, "lambda": 0.255305342686673, "lambda_min": 0.08338414132595062, "lambda_max": 0.5072713494300842, "S_mean": 0.315887451171875, "G_mean": 0.3725527743032823, "G_std": 0.16383479931391776, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.5426671504974365, "grad_norm": 0.798578143119812, "tokens_seen": 50798592, "tokens_per_sec": 15039.528140882856, "vram_gb": 6.442267648, "lambda": 0.2555528149629633, "lambda_min": 0.08342555910348892, "lambda_max": 0.5084196329116821, "S_mean": 0.3244120279947917, "G_mean": 0.3727540757972747, "G_std": 0.1636531153538575, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.52784538269043, "grad_norm": 0.7863597273826599, "tokens_seen": 50962432, "tokens_per_sec": 15039.984814252164, "vram_gb": 6.442267648, "lambda": 0.25589611660689116, "lambda_min": 0.0834682434797287, "lambda_max": 0.5097701549530029, "S_mean": 0.3275146484375, "G_mean": 0.37571656331419945, "G_std": 0.16253517696168274, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.484514951705933, "grad_norm": 0.7905145883560181, "tokens_seen": 51126272, "tokens_per_sec": 15040.35304885735, "vram_gb": 6.442267648, "lambda": 0.25627377939720947, "lambda_min": 0.08349350839853287, "lambda_max": 0.5107301473617554, "S_mean": 0.3236185709635417, "G_mean": 0.36686310668786365, "G_std": 0.16291753094022474, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.601443767547607, "grad_norm": 0.850358784198761, "tokens_seen": 51290112, "tokens_per_sec": 15040.576340708829, "vram_gb": 6.442267648, "lambda": 0.25662251748144627, "lambda_min": 0.08338402956724167, "lambda_max": 0.5119013786315918, "S_mean": 0.3289082845052083, "G_mean": 0.37630698752279085, "G_std": 0.1658638600880901, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.570384740829468, "grad_norm": 0.7624081373214722, "tokens_seen": 51453952, "tokens_per_sec": 15041.088250303088, "vram_gb": 6.442267648, "lambda": 0.2569559992601474, "lambda_min": 0.08333027362823486, "lambda_max": 0.5131102800369263, "S_mean": 0.313232421875, "G_mean": 0.3718844644414882, "G_std": 0.1640199867542833, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.506402254104614, "grad_norm": 0.8182016611099243, "tokens_seen": 51617792, "tokens_per_sec": 15041.708738620338, "vram_gb": 6.442267648, "lambda": 0.2573394713302453, "lambda_min": 0.0834803655743599, "lambda_max": 0.5142818689346313, "S_mean": 0.3281656901041667, "G_mean": 0.3704437300718079, "G_std": 0.16326955799013376, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.457780361175537, "grad_norm": 0.8242130279541016, "tokens_seen": 51781632, "tokens_per_sec": 15042.308566322497, "vram_gb": 6.442267648, "lambda": 0.25775218692918617, "lambda_min": 0.08353109657764435, "lambda_max": 0.5157109498977661, "S_mean": 0.33355712890625, "G_mean": 0.3724597793382903, "G_std": 0.16415477361685285, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.5503387451171875, "grad_norm": 0.7843629717826843, "tokens_seen": 51945472, "tokens_per_sec": 15042.914265869502, "vram_gb": 6.442267648, "lambda": 0.2580646686255932, "lambda_min": 0.08354268968105316, "lambda_max": 0.517014741897583, "S_mean": 0.3281351725260417, "G_mean": 0.3718521043968697, "G_std": 0.16346870277387401, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.57109808921814, "grad_norm": 0.7965813279151917, "tokens_seen": 52109312, "tokens_per_sec": 15043.482385554615, "vram_gb": 6.442267648, "lambda": 0.25841206808884937, "lambda_min": 0.0835936889052391, "lambda_max": 0.518217921257019, "S_mean": 0.3303120930989583, "G_mean": 0.3716808574584623, "G_std": 0.16454859248672923, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.57478666305542, "grad_norm": 0.7857851982116699, "tokens_seen": 52273152, "tokens_per_sec": 15044.182165708306, "vram_gb": 6.442267648, "lambda": 0.25880969657252234, "lambda_min": 0.08363077789545059, "lambda_max": 0.5194813013076782, "S_mean": 0.3134562174479167, "G_mean": 0.37009835592471063, "G_std": 0.1633212622643138, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.422714710235596, "grad_norm": 0.8351292610168457, "tokens_seen": 52436992, "tokens_per_sec": 15044.589272414374, "vram_gb": 6.442267648, "lambda": 0.25912587872395915, "lambda_min": 0.08362427353858948, "lambda_max": 0.5202838182449341, "S_mean": 0.3231201171875, "G_mean": 0.37573495600372553, "G_std": 0.1658962236251682, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.604879379272461, "grad_norm": 0.808135449886322, "tokens_seen": 52600832, "tokens_per_sec": 15045.032164133236, "vram_gb": 6.442267648, "lambda": 0.25949678383767605, "lambda_min": 0.08361022919416428, "lambda_max": 0.5215332508087158, "S_mean": 0.3111775716145833, "G_mean": 0.3704458430875093, "G_std": 0.16441410554883382, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.654132604598999, "grad_norm": 0.8379269242286682, "tokens_seen": 52764672, "tokens_per_sec": 15045.516207187115, "vram_gb": 6.442267648, "lambda": 0.25990564345071715, "lambda_min": 0.08365119248628616, "lambda_max": 0.5231422185897827, "S_mean": 0.325653076171875, "G_mean": 0.36193965445272624, "G_std": 0.16318930444928506, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.4922168254852295, "grad_norm": 0.8756210803985596, "tokens_seen": 52928512, "tokens_per_sec": 15045.779514488233, "vram_gb": 6.442267648, "lambda": 0.2602034506077568, "lambda_min": 0.08368821442127228, "lambda_max": 0.5241965055465698, "S_mean": 0.3164265950520833, "G_mean": 0.3694150006243338, "G_std": 0.1648700364942973, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.499673843383789, "grad_norm": 0.8039416074752808, "tokens_seen": 53092352, "tokens_per_sec": 15046.059240427781, "vram_gb": 6.442267648, "lambda": 0.2605235294128458, "lambda_min": 0.08372669667005539, "lambda_max": 0.525371789932251, "S_mean": 0.3217264811197917, "G_mean": 0.3696099272929132, "G_std": 0.16459266387391835, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.512223482131958, "grad_norm": 0.8418415784835815, "tokens_seen": 53256192, "tokens_per_sec": 15046.728184757543, "vram_gb": 6.442267648, "lambda": 0.2608110023041566, "lambda_min": 0.083684042096138, "lambda_max": 0.5263059139251709, "S_mean": 0.3388163248697917, "G_mean": 0.3736143534382184, "G_std": 0.16411837462025383, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.621581423282623, "val_ppl": 101.65466413512232, "lambda": 0.2608221285045147, "lambda_min": 0.08368142694234848, "lambda_max": 0.5263550281524658, "lambda_per_layer": [0.17121239006519318, 0.14754141867160797, 0.14589360356330872, 0.1351906657218933, 0.12241898477077484, 0.08368142694234848, 0.12337280809879303, 0.09280324727296829, 0.15037177503108978, 0.29414138197898865, 0.3007512390613556, 0.3975358307361603, 0.28581321239471436, 0.460012286901474, 0.43964844942092896, 0.40104639530181885, 0.5263550281524658, 0.38275766372680664, 0.3087354302406311, 0.44982999563217163, 0.17342455685138702, 0.2802571952342987, 0.22394870221614838, 0.16298739612102509], "S_mean": 0.3385416666666667, "G_mean": 0.37017030036076903, "G_std": 0.16642059021008512, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.6771087646484375, "grad_norm": 0.8392704129219055, "tokens_seen": 53420032, "tokens_per_sec": 15030.000843907528, "vram_gb": 6.442267648, "lambda": 0.2610552596549193, "lambda_min": 0.08360910415649414, "lambda_max": 0.5270882248878479, "S_mean": 0.3357137044270833, "G_mean": 0.37577600435664255, "G_std": 0.1636643917299807, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.473106384277344, "grad_norm": 0.80524742603302, "tokens_seen": 53583872, "tokens_per_sec": 15030.199009555568, "vram_gb": 6.442267648, "lambda": 0.2614286780978243, "lambda_min": 0.08355504274368286, "lambda_max": 0.5283796787261963, "S_mean": 0.3486226399739583, "G_mean": 0.3714661021561672, "G_std": 0.16357102532250187, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.675312042236328, "grad_norm": 0.7974771857261658, "tokens_seen": 53747712, "tokens_per_sec": 15030.722102153119, "vram_gb": 6.442267648, "lambda": 0.2618023694182436, "lambda_min": 0.08341163396835327, "lambda_max": 0.5294920802116394, "S_mean": 0.3239542643229167, "G_mean": 0.3748769968127211, "G_std": 0.16642195840055743, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.620777368545532, "grad_norm": 0.8295574188232422, "tokens_seen": 53911552, "tokens_per_sec": 15031.33749063994, "vram_gb": 6.442267648, "lambda": 0.26221567175040644, "lambda_min": 0.0833931639790535, "lambda_max": 0.5304390788078308, "S_mean": 0.3218994140625, "G_mean": 0.3707975223660469, "G_std": 0.16437350248452276, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.377859830856323, "grad_norm": 0.8416187763214111, "tokens_seen": 54075392, "tokens_per_sec": 15032.035210274855, "vram_gb": 6.442267648, "lambda": 0.26258288603276014, "lambda_min": 0.08336712419986725, "lambda_max": 0.5316397547721863, "S_mean": 0.3302408854166667, "G_mean": 0.3646160075441003, "G_std": 0.16270653631848594, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.362560033798218, "grad_norm": 0.8290915489196777, "tokens_seen": 54239232, "tokens_per_sec": 15032.352468684257, "vram_gb": 6.442267648, "lambda": 0.2629324250544111, "lambda_min": 0.08342371135950089, "lambda_max": 0.5329336524009705, "S_mean": 0.32220458984375, "G_mean": 0.3692897035119434, "G_std": 0.16339140195244303, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.680741310119629, "grad_norm": 0.8377299904823303, "tokens_seen": 54403072, "tokens_per_sec": 15032.926402458228, "vram_gb": 6.442267648, "lambda": 0.263213450089097, "lambda_min": 0.0835222527384758, "lambda_max": 0.5339121222496033, "S_mean": 0.317535400390625, "G_mean": 0.3726840321129809, "G_std": 0.16261423386943838, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.656807899475098, "grad_norm": 0.7739217877388, "tokens_seen": 54566912, "tokens_per_sec": 15033.419225335238, "vram_gb": 6.442267648, "lambda": 0.26349103543907404, "lambda_min": 0.08372244238853455, "lambda_max": 0.5350528359413147, "S_mean": 0.3086446126302083, "G_mean": 0.37219421852690476, "G_std": 0.16409313073381782, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.505241870880127, "grad_norm": 0.778660237789154, "tokens_seen": 54730752, "tokens_per_sec": 15033.983240587082, "vram_gb": 6.442267648, "lambda": 0.2638085301344593, "lambda_min": 0.08370926976203918, "lambda_max": 0.5360340476036072, "S_mean": 0.3106892903645833, "G_mean": 0.36736786101634306, "G_std": 0.16506888546670476, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.482115030288696, "grad_norm": 0.81717848777771, "tokens_seen": 54894592, "tokens_per_sec": 15034.497278492505, "vram_gb": 6.442267648, "lambda": 0.2640962302684784, "lambda_min": 0.08364139497280121, "lambda_max": 0.5372470617294312, "S_mean": 0.304290771484375, "G_mean": 0.36576455164079863, "G_std": 0.16243171296082437, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.5648252964019775, "grad_norm": 0.8019782900810242, "tokens_seen": 55058432, "tokens_per_sec": 15034.671585944534, "vram_gb": 6.442267648, "lambda": 0.26443634430567425, "lambda_min": 0.0837244763970375, "lambda_max": 0.5384655594825745, "S_mean": 0.3054097493489583, "G_mean": 0.3736119088716805, "G_std": 0.16499800002202392, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.462529182434082, "grad_norm": 0.8307566046714783, "tokens_seen": 55222272, "tokens_per_sec": 15035.189640827592, "vram_gb": 6.442267648, "lambda": 0.2648094616209467, "lambda_min": 0.08375553786754608, "lambda_max": 0.5395667552947998, "S_mean": 0.3251444498697917, "G_mean": 0.3644495272698502, "G_std": 0.16471649295029542, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.489695072174072, "grad_norm": 0.8285270929336548, "tokens_seen": 55386112, "tokens_per_sec": 15035.921180130812, "vram_gb": 6.442267648, "lambda": 0.2651348930473129, "lambda_min": 0.08371260017156601, "lambda_max": 0.5407963991165161, "S_mean": 0.315185546875, "G_mean": 0.3687275207291047, "G_std": 0.1640526863435904, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.476245164871216, "grad_norm": 0.829259991645813, "tokens_seen": 55549952, "tokens_per_sec": 15036.617730069493, "vram_gb": 6.442267648, "lambda": 0.26548257749527693, "lambda_min": 0.08375410735607147, "lambda_max": 0.541858434677124, "S_mean": 0.3212687174479167, "G_mean": 0.36658488120883703, "G_std": 0.16440083420214555, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.424849033355713, "grad_norm": 0.7642423510551453, "tokens_seen": 55713792, "tokens_per_sec": 15036.98423508022, "vram_gb": 6.442267648, "lambda": 0.2657978093872468, "lambda_min": 0.08378876000642776, "lambda_max": 0.5431962609291077, "S_mean": 0.317138671875, "G_mean": 0.3642652676595996, "G_std": 0.16203304772110036, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.427365303039551, "grad_norm": 0.7902004718780518, "tokens_seen": 55877632, "tokens_per_sec": 15037.27520951904, "vram_gb": 6.442267648, "lambda": 0.2661310142527024, "lambda_min": 0.08372265845537186, "lambda_max": 0.5445653200149536, "S_mean": 0.3184611002604167, "G_mean": 0.3702369128198673, "G_std": 0.1630828749233236, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.352040767669678, "grad_norm": 0.8203119039535522, "tokens_seen": 56041472, "tokens_per_sec": 15037.645785168303, "vram_gb": 6.442267648, "lambda": 0.266487338890632, "lambda_min": 0.08372507244348526, "lambda_max": 0.5457755923271179, "S_mean": 0.3131815592447917, "G_mean": 0.3702721279114485, "G_std": 0.16322765592485666, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.560091495513916, "grad_norm": 0.8623738884925842, "tokens_seen": 56205312, "tokens_per_sec": 15037.868395474736, "vram_gb": 6.442267648, "lambda": 0.2667763453597824, "lambda_min": 0.08373159915208817, "lambda_max": 0.5463380813598633, "S_mean": 0.3202718098958333, "G_mean": 0.36677354760468006, "G_std": 0.16497779559964934, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.499534368515015, "grad_norm": 0.8131430149078369, "tokens_seen": 56369152, "tokens_per_sec": 15038.37280386704, "vram_gb": 6.442267648, "lambda": 0.26709318937112886, "lambda_min": 0.0837288424372673, "lambda_max": 0.5474377870559692, "S_mean": 0.34283447265625, "G_mean": 0.3704655942662309, "G_std": 0.16334903864966085, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.648316144943237, "grad_norm": 0.8539161682128906, "tokens_seen": 56532992, "tokens_per_sec": 15038.804480505218, "vram_gb": 6.442267648, "lambda": 0.26744314935058355, "lambda_min": 0.08372070640325546, "lambda_max": 0.5485871434211731, "S_mean": 0.31683349609375, "G_mean": 0.37264287021632, "G_std": 0.16306157507157573, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.5642125606536865, "grad_norm": 0.8225578665733337, "tokens_seen": 56696832, "tokens_per_sec": 15039.253506178275, "vram_gb": 6.442267648, "lambda": 0.2678127496813734, "lambda_min": 0.0836348906159401, "lambda_max": 0.5499107837677002, "S_mean": 0.3130594889322917, "G_mean": 0.3645773116343965, "G_std": 0.16243819190034023, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.399421215057373, "grad_norm": 0.8205400705337524, "tokens_seen": 56860672, "tokens_per_sec": 15039.864901199386, "vram_gb": 6.442267648, "lambda": 0.268150607123971, "lambda_min": 0.083549365401268, "lambda_max": 0.5512586832046509, "S_mean": 0.32037353515625, "G_mean": 0.36496744444593787, "G_std": 0.15981560270301998, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.530365705490112, "grad_norm": 0.8220804929733276, "tokens_seen": 57024512, "tokens_per_sec": 15040.363123476436, "vram_gb": 6.442267648, "lambda": 0.2684732728327314, "lambda_min": 0.0835326611995697, "lambda_max": 0.5522651672363281, "S_mean": 0.31158447265625, "G_mean": 0.3607904207116614, "G_std": 0.1634663537455102, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.352999925613403, "grad_norm": 0.7779445648193359, "tokens_seen": 57188352, "tokens_per_sec": 15041.073468274746, "vram_gb": 6.442267648, "lambda": 0.2687076237052679, "lambda_min": 0.08345817774534225, "lambda_max": 0.5526066422462463, "S_mean": 0.3176472981770833, "G_mean": 0.3735479221989711, "G_std": 0.16103072534315288, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.6609203815460205, "grad_norm": 0.8847330808639526, "tokens_seen": 57352192, "tokens_per_sec": 15041.790594299118, "vram_gb": 6.442267648, "lambda": 0.26895450334995985, "lambda_min": 0.08344447612762451, "lambda_max": 0.5529806613922119, "S_mean": 0.3119710286458333, "G_mean": 0.3690581746244182, "G_std": 0.16397573166371635, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.570192670822143, "val_ppl": 96.56271279792018, "lambda": 0.2689694402118524, "lambda_min": 0.08345174789428711, "lambda_max": 0.5530297160148621, "lambda_per_layer": [0.17302240431308746, 0.14826534688472748, 0.14758725464344025, 0.13564659655094147, 0.12217714637517929, 0.08345174789428711, 0.12331108003854752, 0.09247538447380066, 0.15123166143894196, 0.2998961806297302, 0.307574987411499, 0.4095461964607239, 0.2935909032821655, 0.47995856404304504, 0.4589062035083771, 0.4182579219341278, 0.5530297160148621, 0.3976220190525055, 0.3208165764808655, 0.4749206602573395, 0.17642880976200104, 0.29023969173431396, 0.2318698763847351, 0.165439635515213], "S_mean": 0.3164164225260417, "G_mean": 0.3663938433552782, "G_std": 0.16305002003597716, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.445780038833618, "grad_norm": 0.8445504307746887, "tokens_seen": 57516032, "tokens_per_sec": 15026.475651591138, "vram_gb": 6.442267648, "lambda": 0.2691964255645871, "lambda_min": 0.08341070264577866, "lambda_max": 0.5535774230957031, "S_mean": 0.3338623046875, "G_mean": 0.36305108907011646, "G_std": 0.16223656867320338, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.4551472663879395, "grad_norm": 0.8411733508110046, "tokens_seen": 57679872, "tokens_per_sec": 15026.875124629052, "vram_gb": 6.442267648, "lambda": 0.2695753965526819, "lambda_min": 0.0834183320403099, "lambda_max": 0.5544001460075378, "S_mean": 0.3226521809895833, "G_mean": 0.36654868923748535, "G_std": 0.16143464106911173, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.5455098152160645, "grad_norm": 0.8391540050506592, "tokens_seen": 57843712, "tokens_per_sec": 15027.49724807624, "vram_gb": 6.442267648, "lambda": 0.26991186446199816, "lambda_min": 0.08337970077991486, "lambda_max": 0.5558116436004639, "S_mean": 0.3192138671875, "G_mean": 0.36299153285411495, "G_std": 0.16426502641600868, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.558484077453613, "grad_norm": 0.8299166560173035, "tokens_seen": 58007552, "tokens_per_sec": 15028.134123335274, "vram_gb": 6.442267648, "lambda": 0.27025072649121284, "lambda_min": 0.08342066407203674, "lambda_max": 0.5569592714309692, "S_mean": 0.3071390787760417, "G_mean": 0.36864552561504144, "G_std": 0.16297625296283513, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.517008543014526, "grad_norm": 0.820134699344635, "tokens_seen": 58171392, "tokens_per_sec": 15028.653186859443, "vram_gb": 6.442267648, "lambda": 0.2705916417762637, "lambda_min": 0.08342186361551285, "lambda_max": 0.557492733001709, "S_mean": 0.3223368326822917, "G_mean": 0.35914174071513116, "G_std": 0.16251011778755733, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.7401697635650635, "grad_norm": 0.8671605587005615, "tokens_seen": 58335232, "tokens_per_sec": 15029.283710601472, "vram_gb": 6.442267648, "lambda": 0.27088877093046904, "lambda_min": 0.08336957544088364, "lambda_max": 0.5584279298782349, "S_mean": 0.32025146484375, "G_mean": 0.3725667907856405, "G_std": 0.16461037922029695, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.513646364212036, "grad_norm": 0.8532999753952026, "tokens_seen": 58499072, "tokens_per_sec": 15029.952702634604, "vram_gb": 6.442267648, "lambda": 0.27114191092550755, "lambda_min": 0.0833418145775795, "lambda_max": 0.5593063831329346, "S_mean": 0.3143819173177083, "G_mean": 0.3694961511840423, "G_std": 0.161925310579439, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.43347954750061, "grad_norm": 0.839658796787262, "tokens_seen": 58662912, "tokens_per_sec": 15030.630368299266, "vram_gb": 6.442267648, "lambda": 0.27142058095584315, "lambda_min": 0.08340289443731308, "lambda_max": 0.5602743029594421, "S_mean": 0.3122355143229167, "G_mean": 0.36239044996909797, "G_std": 0.16357348878712705, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.599181652069092, "grad_norm": 0.850709855556488, "tokens_seen": 58826752, "tokens_per_sec": 15031.228048759072, "vram_gb": 6.442267648, "lambda": 0.2716836885859569, "lambda_min": 0.08349298685789108, "lambda_max": 0.5611594915390015, "S_mean": 0.3205057779947917, "G_mean": 0.36760792788118124, "G_std": 0.165458797942847, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.61430549621582, "grad_norm": 0.8592169880867004, "tokens_seen": 58990592, "tokens_per_sec": 15031.937092386288, "vram_gb": 6.442267648, "lambda": 0.27200183769067127, "lambda_min": 0.08352624624967575, "lambda_max": 0.5623522996902466, "S_mean": 0.3113810221354167, "G_mean": 0.37039985810406506, "G_std": 0.164359765049691, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.582726240158081, "grad_norm": 0.8449804186820984, "tokens_seen": 59154432, "tokens_per_sec": 15032.69938939942, "vram_gb": 6.442267648, "lambda": 0.27228064090013504, "lambda_min": 0.08352430164813995, "lambda_max": 0.5631845593452454, "S_mean": 0.3134053548177083, "G_mean": 0.3716807649470866, "G_std": 0.16254460291626552, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.382264614105225, "grad_norm": 0.8762969970703125, "tokens_seen": 59318272, "tokens_per_sec": 15033.168569887628, "vram_gb": 6.442267648, "lambda": 0.2725673370684187, "lambda_min": 0.08349646627902985, "lambda_max": 0.563802182674408, "S_mean": 0.3090006510416667, "G_mean": 0.3643574430607259, "G_std": 0.16320190958989164, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.430804252624512, "grad_norm": 0.8472523093223572, "tokens_seen": 59482112, "tokens_per_sec": 15033.473573427073, "vram_gb": 6.442267648, "lambda": 0.27285871220131713, "lambda_min": 0.08344840258359909, "lambda_max": 0.5643019676208496, "S_mean": 0.333465576171875, "G_mean": 0.36993917970297235, "G_std": 0.16374024162845066, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.496800422668457, "grad_norm": 0.8285976648330688, "tokens_seen": 59645952, "tokens_per_sec": 15033.692243582578, "vram_gb": 6.442267648, "lambda": 0.2731536269808809, "lambda_min": 0.08343274146318436, "lambda_max": 0.5652234554290771, "S_mean": 0.314361572265625, "G_mean": 0.3596188734130313, "G_std": 0.16444934202202907, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.365819454193115, "grad_norm": 0.813615620136261, "tokens_seen": 59809792, "tokens_per_sec": 15034.328349963926, "vram_gb": 6.442267648, "lambda": 0.2734677130356431, "lambda_min": 0.08342788368463516, "lambda_max": 0.5663188099861145, "S_mean": 0.320831298828125, "G_mean": 0.3683599379534523, "G_std": 0.16167292930185795, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.592329263687134, "grad_norm": 0.8064789772033691, "tokens_seen": 59973632, "tokens_per_sec": 15034.691561139998, "vram_gb": 6.442267648, "lambda": 0.27378449340661365, "lambda_min": 0.08338417857885361, "lambda_max": 0.5673589706420898, "S_mean": 0.3206990559895833, "G_mean": 0.3656483479620268, "G_std": 0.1621618098579347, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.426693439483643, "grad_norm": 0.8542269468307495, "tokens_seen": 60137472, "tokens_per_sec": 15035.206756040956, "vram_gb": 6.442267648, "lambda": 0.2741472941512863, "lambda_min": 0.08330761641263962, "lambda_max": 0.5684332251548767, "S_mean": 0.33514404296875, "G_mean": 0.367054678266868, "G_std": 0.16148502942329893, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.378757953643799, "grad_norm": 0.7974082827568054, "tokens_seen": 60301312, "tokens_per_sec": 15035.768246056126, "vram_gb": 6.442267648, "lambda": 0.2744293864816427, "lambda_min": 0.08321532607078552, "lambda_max": 0.5693394541740417, "S_mean": 0.3248189290364583, "G_mean": 0.3591368881364663, "G_std": 0.1626494712739562, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.356425523757935, "grad_norm": 0.8954523205757141, "tokens_seen": 60465152, "tokens_per_sec": 15036.213399098904, "vram_gb": 6.442267648, "lambda": 0.2746526872118314, "lambda_min": 0.08324920386075974, "lambda_max": 0.5698679685592651, "S_mean": 0.3046976725260417, "G_mean": 0.3658777526579797, "G_std": 0.16340783758399388, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.474182844161987, "grad_norm": 0.8073301911354065, "tokens_seen": 60628992, "tokens_per_sec": 15036.835163154748, "vram_gb": 6.442267648, "lambda": 0.2749244002625346, "lambda_min": 0.08335226029157639, "lambda_max": 0.5705981254577637, "S_mean": 0.308746337890625, "G_mean": 0.3663593408030768, "G_std": 0.16263163198406497, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.571926832199097, "grad_norm": 0.8302000761032104, "tokens_seen": 60792832, "tokens_per_sec": 15037.364522184658, "vram_gb": 6.442267648, "lambda": 0.2752235348646839, "lambda_min": 0.08346319943666458, "lambda_max": 0.5717819929122925, "S_mean": 0.316650390625, "G_mean": 0.3729113922454417, "G_std": 0.16320323622009406, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.616137742996216, "grad_norm": 0.8797479867935181, "tokens_seen": 60956672, "tokens_per_sec": 15038.139483783834, "vram_gb": 6.442267648, "lambda": 0.2755170976743102, "lambda_min": 0.083470918238163, "lambda_max": 0.5725180506706238, "S_mean": 0.311614990234375, "G_mean": 0.3705325702515741, "G_std": 0.16299152413072684, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.421215534210205, "grad_norm": 0.9114567041397095, "tokens_seen": 61120512, "tokens_per_sec": 15038.60129026705, "vram_gb": 6.442267648, "lambda": 0.27575831456730765, "lambda_min": 0.08345483243465424, "lambda_max": 0.5733803510665894, "S_mean": 0.3242696126302083, "G_mean": 0.3638699528140326, "G_std": 0.1633931101144602, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.455841302871704, "grad_norm": 0.8490456938743591, "tokens_seen": 61284352, "tokens_per_sec": 15039.143437724957, "vram_gb": 6.442267648, "lambda": 0.27598631009459496, "lambda_min": 0.0834132507443428, "lambda_max": 0.5744341611862183, "S_mean": 0.3210652669270833, "G_mean": 0.36698949166263145, "G_std": 0.1635752572134758, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.51230525970459, "grad_norm": 0.830319881439209, "tokens_seen": 61448192, "tokens_per_sec": 15039.5362843777, "vram_gb": 6.442267648, "lambda": 0.27628016161421937, "lambda_min": 0.08339940011501312, "lambda_max": 0.5756415128707886, "S_mean": 0.3074849446614583, "G_mean": 0.35959672601893544, "G_std": 0.16108678087281683, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.587097716331482, "val_ppl": 98.20898580645702, "lambda": 0.2762928356726964, "lambda_min": 0.0833968073129654, "lambda_max": 0.5757046341896057, "lambda_per_layer": [0.1752496361732483, 0.14904114603996277, 0.148906871676445, 0.1359405666589737, 0.12231580168008804, 0.0833968073129654, 0.12284538149833679, 0.09220752865076065, 0.15335705876350403, 0.3041898012161255, 0.312542200088501, 0.4201996326446533, 0.3003445565700531, 0.4955580234527588, 0.4760502874851227, 0.43255814909935, 0.5757046341896057, 0.41126567125320435, 0.3345693051815033, 0.49442529678344727, 0.18025153875350952, 0.3013157248497009, 0.24028204381465912, 0.16851039230823517], "S_mean": 0.3341878255208333, "G_mean": 0.3725674299057573, "G_std": 0.16075387853197753, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.49789571762085, "grad_norm": 0.8681749105453491, "tokens_seen": 61612032, "tokens_per_sec": 15024.806030275335, "vram_gb": 6.442267648, "lambda": 0.27653876536836225, "lambda_min": 0.0834311917424202, "lambda_max": 0.5766323208808899, "S_mean": 0.3138020833333333, "G_mean": 0.3611405238043517, "G_std": 0.1622154062303404, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.582602024078369, "grad_norm": 0.8284175992012024, "tokens_seen": 61775872, "tokens_per_sec": 15025.32344233003, "vram_gb": 6.442267648, "lambda": 0.2767543140798807, "lambda_min": 0.08341719955205917, "lambda_max": 0.5773206949234009, "S_mean": 0.32733154296875, "G_mean": 0.3669085471580426, "G_std": 0.16251811183368167, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.277085065841675, "grad_norm": 0.8189244866371155, "tokens_seen": 61939712, "tokens_per_sec": 15025.747532955587, "vram_gb": 6.442267648, "lambda": 0.27702591630319756, "lambda_min": 0.08342504501342773, "lambda_max": 0.578158438205719, "S_mean": 0.3108113606770833, "G_mean": 0.3615619655077656, "G_std": 0.16168518837851784, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.501033306121826, "grad_norm": 0.8695755004882812, "tokens_seen": 62103552, "tokens_per_sec": 15026.505017149771, "vram_gb": 6.442267648, "lambda": 0.2772821479787429, "lambda_min": 0.08347801864147186, "lambda_max": 0.5789004564285278, "S_mean": 0.3054911295572917, "G_mean": 0.3618208991829306, "G_std": 0.16382739677404365, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.355515956878662, "grad_norm": 0.8726622462272644, "tokens_seen": 62267392, "tokens_per_sec": 15027.415168018879, "vram_gb": 6.442267648, "lambda": 0.2775251657391588, "lambda_min": 0.08358744531869888, "lambda_max": 0.5793215036392212, "S_mean": 0.3205159505208333, "G_mean": 0.36202836005638045, "G_std": 0.16289732861332595, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.543731689453125, "grad_norm": 0.8829922676086426, "tokens_seen": 62431232, "tokens_per_sec": 15028.092369912121, "vram_gb": 6.442267648, "lambda": 0.2777461313332121, "lambda_min": 0.0835748091340065, "lambda_max": 0.580226719379425, "S_mean": 0.3178507486979167, "G_mean": 0.3631243551305185, "G_std": 0.16371763951610774, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.358264207839966, "grad_norm": 0.8436792492866516, "tokens_seen": 62595072, "tokens_per_sec": 15028.725785211593, "vram_gb": 6.442267648, "lambda": 0.2780178893978397, "lambda_min": 0.08357062190771103, "lambda_max": 0.5811855792999268, "S_mean": 0.3087666829427083, "G_mean": 0.36482729964579147, "G_std": 0.1625317318054537, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.5074427127838135, "grad_norm": 0.8565487265586853, "tokens_seen": 62758912, "tokens_per_sec": 15029.28224486235, "vram_gb": 6.442267648, "lambda": 0.27825140797843534, "lambda_min": 0.08355475217103958, "lambda_max": 0.5819776058197021, "S_mean": 0.320648193359375, "G_mean": 0.36049557306493324, "G_std": 0.16158619499765337, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.4794697761535645, "grad_norm": 0.865778923034668, "tokens_seen": 62922752, "tokens_per_sec": 15030.03160674871, "vram_gb": 6.442267648, "lambda": 0.27846843966593343, "lambda_min": 0.08345148712396622, "lambda_max": 0.5828225612640381, "S_mean": 0.3051961263020833, "G_mean": 0.36374363225574297, "G_std": 0.16241999311993519, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.53349232673645, "grad_norm": 0.8247413039207458, "tokens_seen": 63086592, "tokens_per_sec": 15030.40628024309, "vram_gb": 6.442267648, "lambda": 0.27875682059675455, "lambda_min": 0.08342477679252625, "lambda_max": 0.5837637186050415, "S_mean": 0.3129679361979167, "G_mean": 0.3610304932420452, "G_std": 0.16206310510945818, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.490085601806641, "grad_norm": 0.9001249074935913, "tokens_seen": 63250432, "tokens_per_sec": 15031.118757975168, "vram_gb": 6.442267648, "lambda": 0.2790155593926708, "lambda_min": 0.08336017280817032, "lambda_max": 0.5848901271820068, "S_mean": 0.3170878092447917, "G_mean": 0.36043354294573265, "G_std": 0.16055257696037492, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.328122138977051, "grad_norm": 0.8567500114440918, "tokens_seen": 63414272, "tokens_per_sec": 15031.588819800805, "vram_gb": 6.442267648, "lambda": 0.2792706002170841, "lambda_min": 0.08329647779464722, "lambda_max": 0.5858986377716064, "S_mean": 0.3068644205729167, "G_mean": 0.3593555052454273, "G_std": 0.1619631281355396, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.462610721588135, "grad_norm": 0.8491324782371521, "tokens_seen": 63578112, "tokens_per_sec": 15032.153979502138, "vram_gb": 6.442267648, "lambda": 0.27950433890024823, "lambda_min": 0.08330845087766647, "lambda_max": 0.5866391062736511, "S_mean": 0.3185017903645833, "G_mean": 0.36844494519755244, "G_std": 0.16148792152913907, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.402057886123657, "grad_norm": 0.8568081855773926, "tokens_seen": 63741952, "tokens_per_sec": 15032.829935461752, "vram_gb": 6.442267648, "lambda": 0.279755847218136, "lambda_min": 0.08330172300338745, "lambda_max": 0.5875825881958008, "S_mean": 0.3299051920572917, "G_mean": 0.3665320348615448, "G_std": 0.1612806744330252, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.493343114852905, "grad_norm": 0.8577147126197815, "tokens_seen": 63905792, "tokens_per_sec": 15033.413226420183, "vram_gb": 6.442267648, "lambda": 0.28003670182079077, "lambda_min": 0.08327417075634003, "lambda_max": 0.588642954826355, "S_mean": 0.308441162109375, "G_mean": 0.3566385632536064, "G_std": 0.1617414242743204, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.550835132598877, "grad_norm": 0.932670533657074, "tokens_seen": 64069632, "tokens_per_sec": 15033.94556291118, "vram_gb": 6.442267648, "lambda": 0.2803347948938608, "lambda_min": 0.08333679288625717, "lambda_max": 0.5894215703010559, "S_mean": 0.3188985188802083, "G_mean": 0.368739877982686, "G_std": 0.16215398837812245, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.301982402801514, "grad_norm": 0.8328716158866882, "tokens_seen": 64233472, "tokens_per_sec": 15034.485044865194, "vram_gb": 6.442267648, "lambda": 0.28050737734884024, "lambda_min": 0.08333014696836472, "lambda_max": 0.5900723934173584, "S_mean": 0.3231913248697917, "G_mean": 0.36179986541780335, "G_std": 0.1623318128210182, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.478484869003296, "grad_norm": 0.8937939405441284, "tokens_seen": 64397312, "tokens_per_sec": 15035.266922568442, "vram_gb": 6.442267648, "lambda": 0.28076241600016755, "lambda_min": 0.08331157267093658, "lambda_max": 0.5908446311950684, "S_mean": 0.3211568196614583, "G_mean": 0.36224934404405457, "G_std": 0.1622962801872442, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.4995269775390625, "grad_norm": 0.8932244181632996, "tokens_seen": 64561152, "tokens_per_sec": 15036.090743036466, "vram_gb": 6.442267648, "lambda": 0.2809906577070554, "lambda_min": 0.08332459628582001, "lambda_max": 0.5915075540542603, "S_mean": 0.3127543131510417, "G_mean": 0.3594335032782207, "G_std": 0.1623876889158661, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.287459135055542, "grad_norm": 0.8624368906021118, "tokens_seen": 64724992, "tokens_per_sec": 15036.675614835362, "vram_gb": 6.442267648, "lambda": 0.28123165263483924, "lambda_min": 0.08329498022794724, "lambda_max": 0.5921773314476013, "S_mean": 0.3155110677083333, "G_mean": 0.36183837909872335, "G_std": 0.1629049228116249, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.3578925132751465, "grad_norm": 0.8347675800323486, "tokens_seen": 64888832, "tokens_per_sec": 15037.206178046077, "vram_gb": 6.442267648, "lambda": 0.281511386235555, "lambda_min": 0.08326619118452072, "lambda_max": 0.5930154323577881, "S_mean": 0.3208211263020833, "G_mean": 0.3657500286741803, "G_std": 0.1610984094440937, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.456830978393555, "grad_norm": 0.8350610733032227, "tokens_seen": 65052672, "tokens_per_sec": 15037.773236885429, "vram_gb": 6.442267648, "lambda": 0.2817483637481928, "lambda_min": 0.08325374871492386, "lambda_max": 0.5936582684516907, "S_mean": 0.3163859049479167, "G_mean": 0.364165772140647, "G_std": 0.1632341651323562, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.515106678009033, "grad_norm": 0.9190705418586731, "tokens_seen": 65216512, "tokens_per_sec": 15038.269601125214, "vram_gb": 6.442267648, "lambda": 0.28194482903927565, "lambda_min": 0.08330433815717697, "lambda_max": 0.5944864153862, "S_mean": 0.3132527669270833, "G_mean": 0.36385091316575807, "G_std": 0.16351200287075093, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.423585891723633, "grad_norm": 0.8670142292976379, "tokens_seen": 65380352, "tokens_per_sec": 15038.939848253183, "vram_gb": 6.442267648, "lambda": 0.2822293716793259, "lambda_min": 0.08334632217884064, "lambda_max": 0.5955480337142944, "S_mean": 0.3203938802083333, "G_mean": 0.3599027187253038, "G_std": 0.16053614191090068, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.37782096862793, "grad_norm": 0.9120305180549622, "tokens_seen": 65544192, "tokens_per_sec": 15039.580016965063, "vram_gb": 6.442267648, "lambda": 0.2825062225262324, "lambda_min": 0.08336383104324341, "lambda_max": 0.5963359475135803, "S_mean": 0.3234049479166667, "G_mean": 0.36515758908353746, "G_std": 0.16250208637211472, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.515785109996796, "val_ppl": 91.44933563278805, "lambda": 0.28252046648412943, "lambda_min": 0.08336623758077621, "lambda_max": 0.5963478088378906, "lambda_per_layer": [0.17665472626686096, 0.14913977682590485, 0.1502225548028946, 0.13629823923110962, 0.12180334329605103, 0.08336623758077621, 0.12240651249885559, 0.09234598278999329, 0.1552419662475586, 0.30829739570617676, 0.3157050907611847, 0.42820775508880615, 0.30544307827949524, 0.5088703036308289, 0.4906017482280731, 0.4454387426376343, 0.5963478088378906, 0.423342227935791, 0.3455665111541748, 0.5123099684715271, 0.1833106130361557, 0.3113873302936554, 0.24725864827632904, 0.17092463374137878], "S_mean": 0.32781982421875, "G_mean": 0.3650288959033787, "G_std": 0.16052371733045825, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.422759532928467, "grad_norm": 0.8216822147369385, "tokens_seen": 65708032, "tokens_per_sec": 15025.901949813442, "vram_gb": 6.442267648, "lambda": 0.28274272723744315, "lambda_min": 0.08339300751686096, "lambda_max": 0.5969182848930359, "S_mean": 0.329864501953125, "G_mean": 0.3592962365752707, "G_std": 0.16145663991725692, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.57402491569519, "grad_norm": 0.8591815233230591, "tokens_seen": 65871872, "tokens_per_sec": 15026.458703395363, "vram_gb": 6.442267648, "lambda": 0.2829401275763909, "lambda_min": 0.08338446170091629, "lambda_max": 0.5975747108459473, "S_mean": 0.3206278483072917, "G_mean": 0.3604607152907799, "G_std": 0.16262534792379787, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.496614933013916, "grad_norm": 0.8461804986000061, "tokens_seen": 66035712, "tokens_per_sec": 15027.145547884098, "vram_gb": 6.442267648, "lambda": 0.28316085909803707, "lambda_min": 0.08337131887674332, "lambda_max": 0.5982909202575684, "S_mean": 0.3181050618489583, "G_mean": 0.3568619737246384, "G_std": 0.16139518504496664, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.381086349487305, "grad_norm": 0.8672437071800232, "tokens_seen": 66199552, "tokens_per_sec": 15027.804246949525, "vram_gb": 6.442267648, "lambda": 0.28331817282984656, "lambda_min": 0.08333887159824371, "lambda_max": 0.5988892316818237, "S_mean": 0.3118896484375, "G_mean": 0.3570008672152956, "G_std": 0.16163894666048387, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.514524698257446, "grad_norm": 0.8753014206886292, "tokens_seen": 66363392, "tokens_per_sec": 15028.38449622996, "vram_gb": 6.442267648, "lambda": 0.2835194831714034, "lambda_min": 0.08335801213979721, "lambda_max": 0.5997358560562134, "S_mean": 0.3257598876953125, "G_mean": 0.35911922878585756, "G_std": 0.1608381833260258, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.345003128051758, "grad_norm": 0.8186098337173462, "tokens_seen": 66527232, "tokens_per_sec": 15029.069520486315, "vram_gb": 6.442267648, "lambda": 0.28374885333081085, "lambda_min": 0.08335307240486145, "lambda_max": 0.6006152033805847, "S_mean": 0.305938720703125, "G_mean": 0.35938746194976073, "G_std": 0.16173357776521394, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.503472328186035, "grad_norm": 0.9417242407798767, "tokens_seen": 66691072, "tokens_per_sec": 15029.724366122111, "vram_gb": 6.442267648, "lambda": 0.283959973603487, "lambda_min": 0.08334904909133911, "lambda_max": 0.6013565063476562, "S_mean": 0.3170267740885417, "G_mean": 0.3583690511683623, "G_std": 0.1613943789852783, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.474133491516113, "grad_norm": 0.8868807554244995, "tokens_seen": 66854912, "tokens_per_sec": 15030.262072232512, "vram_gb": 6.442267648, "lambda": 0.2841803689176838, "lambda_min": 0.08330076932907104, "lambda_max": 0.6024545431137085, "S_mean": 0.3225911458333333, "G_mean": 0.3603085884824395, "G_std": 0.16108194482512772, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.556605100631714, "grad_norm": 0.8760786056518555, "tokens_seen": 67018752, "tokens_per_sec": 15030.824186286423, "vram_gb": 6.442267648, "lambda": 0.28441902343183756, "lambda_min": 0.08332514762878418, "lambda_max": 0.6033056974411011, "S_mean": 0.3135884602864583, "G_mean": 0.3599741633515805, "G_std": 0.16281039077633372, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.218118190765381, "grad_norm": 0.8578190803527832, "tokens_seen": 67182592, "tokens_per_sec": 15031.53146856694, "vram_gb": 6.442267648, "lambda": 0.28463245648890734, "lambda_min": 0.08330921828746796, "lambda_max": 0.6038739681243896, "S_mean": 0.324493408203125, "G_mean": 0.3628523667963843, "G_std": 0.16174315296423933, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.3862481117248535, "grad_norm": 0.8628191947937012, "tokens_seen": 67346432, "tokens_per_sec": 15032.122432144715, "vram_gb": 6.442267648, "lambda": 0.28481648800273734, "lambda_min": 0.08328356593847275, "lambda_max": 0.6046847105026245, "S_mean": 0.3219706217447917, "G_mean": 0.3613696300114195, "G_std": 0.1616214942963173, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.260618686676025, "grad_norm": 0.8508287668228149, "tokens_seen": 67510272, "tokens_per_sec": 15032.53515735703, "vram_gb": 6.442267648, "lambda": 0.28503039944916964, "lambda_min": 0.08323688060045242, "lambda_max": 0.6056838035583496, "S_mean": 0.333160400390625, "G_mean": 0.3557949614090224, "G_std": 0.16037071651468673, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.401531219482422, "grad_norm": 0.8466490507125854, "tokens_seen": 67674112, "tokens_per_sec": 15033.377694683932, "vram_gb": 6.442267648, "lambda": 0.28520850899318856, "lambda_min": 0.08321631699800491, "lambda_max": 0.6066819429397583, "S_mean": 0.3214213053385417, "G_mean": 0.3636594043734173, "G_std": 0.16187918065891913, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.3350608348846436, "grad_norm": 0.8823560476303101, "tokens_seen": 67837952, "tokens_per_sec": 15034.067951300856, "vram_gb": 6.442267648, "lambda": 0.2853433412189285, "lambda_min": 0.08322928845882416, "lambda_max": 0.6070160269737244, "S_mean": 0.3231404622395833, "G_mean": 0.35658857855014503, "G_std": 0.16140388732310385, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.366464376449585, "grad_norm": 0.8515457510948181, "tokens_seen": 68001792, "tokens_per_sec": 15033.99537494792, "vram_gb": 6.442267648, "lambda": 0.28553036755571765, "lambda_min": 0.08317667245864868, "lambda_max": 0.6074436902999878, "S_mean": 0.329925537109375, "G_mean": 0.36088929045945406, "G_std": 0.16065946112697324, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.410477876663208, "grad_norm": 0.9127928614616394, "tokens_seen": 68165632, "tokens_per_sec": 15034.62224943147, "vram_gb": 6.442267648, "lambda": 0.2856839296097557, "lambda_min": 0.08318038284778595, "lambda_max": 0.6078928112983704, "S_mean": 0.323699951171875, "G_mean": 0.35626769554801285, "G_std": 0.1597408209539329, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.438594341278076, "grad_norm": 0.8600230813026428, "tokens_seen": 68329472, "tokens_per_sec": 15035.316681667375, "vram_gb": 6.442267648, "lambda": 0.2859235570455591, "lambda_min": 0.08318443596363068, "lambda_max": 0.6086218953132629, "S_mean": 0.3279520670572917, "G_mean": 0.35951902306017774, "G_std": 0.16010741447098553, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.475869655609131, "grad_norm": 0.903073787689209, "tokens_seen": 68493312, "tokens_per_sec": 15035.814437657227, "vram_gb": 6.442267648, "lambda": 0.2861397524053852, "lambda_min": 0.08321806788444519, "lambda_max": 0.609459400177002, "S_mean": 0.33795166015625, "G_mean": 0.3596027287033697, "G_std": 0.16101573745254427, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.428096771240234, "grad_norm": 0.8283807039260864, "tokens_seen": 68657152, "tokens_per_sec": 15036.291333663048, "vram_gb": 6.442267648, "lambda": 0.2863622574756543, "lambda_min": 0.08321823924779892, "lambda_max": 0.6105285882949829, "S_mean": 0.3200887044270833, "G_mean": 0.35720649687573314, "G_std": 0.16111338693493357, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.3725433349609375, "grad_norm": 0.8465951085090637, "tokens_seen": 68820992, "tokens_per_sec": 15036.67942080722, "vram_gb": 6.442267648, "lambda": 0.2865758442009489, "lambda_min": 0.08320586383342743, "lambda_max": 0.6113477349281311, "S_mean": 0.3202107747395833, "G_mean": 0.3613565871492028, "G_std": 0.16021850072623542, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.512987375259399, "grad_norm": 0.8768965005874634, "tokens_seen": 68984832, "tokens_per_sec": 15037.295329329692, "vram_gb": 6.442267648, "lambda": 0.2867719891170661, "lambda_min": 0.08319195359945297, "lambda_max": 0.6118518114089966, "S_mean": 0.3389994303385417, "G_mean": 0.36507285906312364, "G_std": 0.16010016160241017, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.181423902511597, "grad_norm": 0.8668331503868103, "tokens_seen": 69148672, "tokens_per_sec": 15037.797540304597, "vram_gb": 6.442267648, "lambda": 0.2869762113938729, "lambda_min": 0.08319602906703949, "lambda_max": 0.6122511029243469, "S_mean": 0.31268310546875, "G_mean": 0.3540540363173932, "G_std": 0.1597229837983226, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.355834484100342, "grad_norm": 0.9238988161087036, "tokens_seen": 69312512, "tokens_per_sec": 15038.427956157282, "vram_gb": 6.442267648, "lambda": 0.28715901263058186, "lambda_min": 0.08322402834892273, "lambda_max": 0.6128765940666199, "S_mean": 0.3254801432291667, "G_mean": 0.356534885475412, "G_std": 0.160540372133255, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.3043296337127686, "grad_norm": 0.8533751368522644, "tokens_seen": 69476352, "tokens_per_sec": 15039.072413907115, "vram_gb": 6.442267648, "lambda": 0.2873095801721017, "lambda_min": 0.08325865119695663, "lambda_max": 0.6135265231132507, "S_mean": 0.335540771484375, "G_mean": 0.36014279816299677, "G_std": 0.16093513536422202, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.597026109695435, "grad_norm": 0.9324711561203003, "tokens_seen": 69640192, "tokens_per_sec": 15039.737691137336, "vram_gb": 6.442267648, "lambda": 0.28749642614275217, "lambda_min": 0.08323846757411957, "lambda_max": 0.6144307851791382, "S_mean": 0.3299458821614583, "G_mean": 0.361760484908397, "G_std": 0.1589583995907257, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.506905710697174, "val_ppl": 90.6409149227312, "lambda": 0.2875045680751403, "lambda_min": 0.0832400992512703, "lambda_max": 0.6144542694091797, "lambda_per_layer": [0.1782669723033905, 0.1493101418018341, 0.15123960375785828, 0.1365068107843399, 0.12174863368272781, 0.0832400992512703, 0.122359499335289, 0.09264014661312103, 0.15613973140716553, 0.31016620993614197, 0.31902214884757996, 0.4345742464065552, 0.30917325615882874, 0.5170332193374634, 0.5038571357727051, 0.45575541257858276, 0.6144542694091797, 0.4329192340373993, 0.35282543301582336, 0.5270718932151794, 0.18603138625621796, 0.3191983103752136, 0.2531963884830475, 0.17337945103645325], "S_mean": 0.32342529296875, "G_mean": 0.3641279428265989, "G_std": 0.16000883106607944, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.41344690322876, "grad_norm": 0.8914705514907837, "tokens_seen": 69804032, "tokens_per_sec": 15026.781472127834, "vram_gb": 6.442267648, "lambda": 0.28769484131286543, "lambda_min": 0.0832318589091301, "lambda_max": 0.6151778101921082, "S_mean": 0.3283589680989583, "G_mean": 0.36184446707678336, "G_std": 0.15867410452725986, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.463765621185303, "grad_norm": 0.9559510946273804, "tokens_seen": 69967872, "tokens_per_sec": 15027.22890055051, "vram_gb": 6.442267648, "lambda": 0.2878760118037462, "lambda_min": 0.08322200924158096, "lambda_max": 0.6159460544586182, "S_mean": 0.3078104654947917, "G_mean": 0.3581450345615546, "G_std": 0.162279352468128, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.5067925453186035, "grad_norm": 0.8794608116149902, "tokens_seen": 70131712, "tokens_per_sec": 15027.441090878827, "vram_gb": 6.442267648, "lambda": 0.2880452141786615, "lambda_min": 0.08324740827083588, "lambda_max": 0.6166125535964966, "S_mean": 0.321990966796875, "G_mean": 0.3618041845038533, "G_std": 0.16011833709975085, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.361429214477539, "grad_norm": 0.847067654132843, "tokens_seen": 70295552, "tokens_per_sec": 15027.879538140802, "vram_gb": 6.442267648, "lambda": 0.2882512103145321, "lambda_min": 0.08325956761837006, "lambda_max": 0.6173343062400818, "S_mean": 0.3272298177083333, "G_mean": 0.3550414452329278, "G_std": 0.16094791186818233, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.487356424331665, "grad_norm": 1.0575543642044067, "tokens_seen": 70459392, "tokens_per_sec": 15028.846653783303, "vram_gb": 6.442267648, "lambda": 0.288453777320683, "lambda_min": 0.08317254483699799, "lambda_max": 0.6178171038627625, "S_mean": 0.3245951334635417, "G_mean": 0.36168521526269615, "G_std": 0.16011715094403675, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.438117027282715, "grad_norm": 0.8856242299079895, "tokens_seen": 70623232, "tokens_per_sec": 15029.575913598908, "vram_gb": 6.442267648, "lambda": 0.2886202170824011, "lambda_min": 0.083182193338871, "lambda_max": 0.6181145906448364, "S_mean": 0.33013916015625, "G_mean": 0.3659763087828954, "G_std": 0.16082894010469317, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.481652498245239, "grad_norm": 0.9192745685577393, "tokens_seen": 70787072, "tokens_per_sec": 15029.753311737843, "vram_gb": 6.442267648, "lambda": 0.2888142690062523, "lambda_min": 0.08309570699930191, "lambda_max": 0.6187923550605774, "S_mean": 0.3334248860677083, "G_mean": 0.3601409282224874, "G_std": 0.16026760179859897, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.326096534729004, "grad_norm": 0.9345465898513794, "tokens_seen": 70950912, "tokens_per_sec": 15030.065907542037, "vram_gb": 6.442267648, "lambda": 0.2889861998458703, "lambda_min": 0.0830821543931961, "lambda_max": 0.6195676922798157, "S_mean": 0.33782958984375, "G_mean": 0.3637861223736157, "G_std": 0.16037863733557364, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.396530866622925, "grad_norm": 0.8444190621376038, "tokens_seen": 71114752, "tokens_per_sec": 15030.519541734644, "vram_gb": 6.442267648, "lambda": 0.2891568535317977, "lambda_min": 0.08311132341623306, "lambda_max": 0.6202855706214905, "S_mean": 0.3288370768229167, "G_mean": 0.3605387373827398, "G_std": 0.16084357684788606, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.366966724395752, "grad_norm": 0.8414785861968994, "tokens_seen": 71278592, "tokens_per_sec": 15030.823510117392, "vram_gb": 6.442267648, "lambda": 0.2893655312558015, "lambda_min": 0.08318085968494415, "lambda_max": 0.620886504650116, "S_mean": 0.3274434407552083, "G_mean": 0.35853359134246904, "G_std": 0.1600297458547478, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.353685140609741, "grad_norm": 0.9020000100135803, "tokens_seen": 71442432, "tokens_per_sec": 15031.273479328856, "vram_gb": 6.442267648, "lambda": 0.28954537988950807, "lambda_min": 0.08320107311010361, "lambda_max": 0.6216831207275391, "S_mean": 0.32000732421875, "G_mean": 0.3610219975622992, "G_std": 0.15987403426940242, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.424300670623779, "grad_norm": 0.8953785300254822, "tokens_seen": 71606272, "tokens_per_sec": 15031.726548551771, "vram_gb": 6.442267648, "lambda": 0.2897492793078224, "lambda_min": 0.08322843164205551, "lambda_max": 0.6222857236862183, "S_mean": 0.3389078776041667, "G_mean": 0.3538465742021799, "G_std": 0.1613372804907461, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.307567358016968, "grad_norm": 0.9571755528450012, "tokens_seen": 71770112, "tokens_per_sec": 15032.089801273638, "vram_gb": 6.442267648, "lambda": 0.28994202831139165, "lambda_min": 0.08322722464799881, "lambda_max": 0.622878909111023, "S_mean": 0.318634033203125, "G_mean": 0.3604841170211633, "G_std": 0.16089117290296903, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.411248207092285, "grad_norm": 0.8567363619804382, "tokens_seen": 71933952, "tokens_per_sec": 15032.467744326495, "vram_gb": 6.442267648, "lambda": 0.29014147880176705, "lambda_min": 0.08326829224824905, "lambda_max": 0.6234750151634216, "S_mean": 0.3272806803385417, "G_mean": 0.3596707660083969, "G_std": 0.16006163996644318, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.435391902923584, "grad_norm": 0.9381030797958374, "tokens_seen": 72097792, "tokens_per_sec": 15032.886816202572, "vram_gb": 6.442267648, "lambda": 0.29028414624432725, "lambda_min": 0.08329799026250839, "lambda_max": 0.6238822340965271, "S_mean": 0.34619140625, "G_mean": 0.36567126167938113, "G_std": 0.15770226682070643, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.426706075668335, "grad_norm": 0.9453678131103516, "tokens_seen": 72261632, "tokens_per_sec": 15033.239695012639, "vram_gb": 6.442267648, "lambda": 0.29045009923477966, "lambda_min": 0.08333776891231537, "lambda_max": 0.6245200634002686, "S_mean": 0.3350626627604167, "G_mean": 0.36128956149332225, "G_std": 0.16112740566798797, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.424821615219116, "grad_norm": 0.9007453322410583, "tokens_seen": 72425472, "tokens_per_sec": 15033.621803659975, "vram_gb": 6.442267648, "lambda": 0.2906138049438596, "lambda_min": 0.08338978886604309, "lambda_max": 0.6249906420707703, "S_mean": 0.332427978515625, "G_mean": 0.35865468842287856, "G_std": 0.1600054963103806, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.4539384841918945, "grad_norm": 0.864361047744751, "tokens_seen": 72589312, "tokens_per_sec": 15033.907661358988, "vram_gb": 6.442267648, "lambda": 0.29082162771373987, "lambda_min": 0.08345407247543335, "lambda_max": 0.6256545782089233, "S_mean": 0.3328501383463542, "G_mean": 0.3615119362560411, "G_std": 0.1599021477935215, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.256659030914307, "grad_norm": 0.9376378059387207, "tokens_seen": 72753152, "tokens_per_sec": 15034.144859254407, "vram_gb": 6.442267648, "lambda": 0.2910139604161183, "lambda_min": 0.08345888555049896, "lambda_max": 0.626297116279602, "S_mean": 0.3367207845052083, "G_mean": 0.3585327711577217, "G_std": 0.16017796044858792, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.386265754699707, "grad_norm": 0.9183561205863953, "tokens_seen": 72916992, "tokens_per_sec": 15034.46511133021, "vram_gb": 6.442267648, "lambda": 0.29117806411037844, "lambda_min": 0.08348435908555984, "lambda_max": 0.6268623471260071, "S_mean": 0.3260091145833333, "G_mean": 0.3592370899083714, "G_std": 0.16068305699930838, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.4433300495147705, "grad_norm": 0.9564658999443054, "tokens_seen": 73080832, "tokens_per_sec": 15034.70990609091, "vram_gb": 6.442267648, "lambda": 0.2913525427381198, "lambda_min": 0.0835120752453804, "lambda_max": 0.6276564598083496, "S_mean": 0.3289286295572917, "G_mean": 0.362059460952878, "G_std": 0.16182155348360538, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.263902187347412, "grad_norm": 0.9164770841598511, "tokens_seen": 73244672, "tokens_per_sec": 15035.086282844197, "vram_gb": 6.442267648, "lambda": 0.29155348520725965, "lambda_min": 0.08348407596349716, "lambda_max": 0.6284030079841614, "S_mean": 0.3200785319010417, "G_mean": 0.3615978870075196, "G_std": 0.15996954306804886, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.5148351192474365, "grad_norm": 0.9518908262252808, "tokens_seen": 73408512, "tokens_per_sec": 15035.443211018399, "vram_gb": 6.442267648, "lambda": 0.2917197570204735, "lambda_min": 0.08347588032484055, "lambda_max": 0.6288831233978271, "S_mean": 0.3300069173177083, "G_mean": 0.359662250848487, "G_std": 0.16173726735481372, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.3732757568359375, "grad_norm": 0.8953475952148438, "tokens_seen": 73572352, "tokens_per_sec": 15035.95595001658, "vram_gb": 6.442267648, "lambda": 0.2919054962694645, "lambda_min": 0.08348429948091507, "lambda_max": 0.6294445395469666, "S_mean": 0.3356119791666667, "G_mean": 0.36268803601463634, "G_std": 0.16054287529550493, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.3175671100616455, "grad_norm": 0.8897601366043091, "tokens_seen": 73736192, "tokens_per_sec": 15036.468816715029, "vram_gb": 6.442267648, "lambda": 0.29205132741481066, "lambda_min": 0.08351761102676392, "lambda_max": 0.6299769282341003, "S_mean": 0.334320068359375, "G_mean": 0.35789333873738843, "G_std": 0.1608728205319494, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.470931196212769, "val_ppl": 87.4381071494898, "lambda": 0.29206064995378256, "lambda_min": 0.08351646363735199, "lambda_max": 0.6300214529037476, "lambda_per_layer": [0.17946669459342957, 0.14923644065856934, 0.1522018164396286, 0.1366986483335495, 0.12079542875289917, 0.08351646363735199, 0.122066430747509, 0.09238368272781372, 0.15746335685253143, 0.3129218518733978, 0.32227519154548645, 0.4406062960624695, 0.3123517334461212, 0.5254519581794739, 0.514661967754364, 0.46446913480758667, 0.6300214529037476, 0.4415658414363861, 0.3621276021003723, 0.5405350923538208, 0.18818682432174683, 0.327008992433548, 0.2582673728466034, 0.17517532408237457], "S_mean": 0.3223673502604167, "G_mean": 0.3587429206818342, "G_std": 0.1606678436510265, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.3404340744018555, "grad_norm": 0.8773359656333923, "tokens_seen": 73900032, "tokens_per_sec": 15024.410112129133, "vram_gb": 6.442267648, "lambda": 0.2922489109138648, "lambda_min": 0.08345653861761093, "lambda_max": 0.6306234002113342, "S_mean": 0.33453369140625, "G_mean": 0.3656360963359475, "G_std": 0.16110113177758953, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.304530382156372, "grad_norm": 0.8805994391441345, "tokens_seen": 74063872, "tokens_per_sec": 15024.858166190299, "vram_gb": 6.442267648, "lambda": 0.292378350161016, "lambda_min": 0.08346934616565704, "lambda_max": 0.6311929821968079, "S_mean": 0.326904296875, "G_mean": 0.35682266919563216, "G_std": 0.160179762984626, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.329747200012207, "grad_norm": 0.8773090839385986, "tokens_seen": 74227712, "tokens_per_sec": 15025.272628980745, "vram_gb": 6.442267648, "lambda": 0.29250552225857973, "lambda_min": 0.08344960957765579, "lambda_max": 0.6315774917602539, "S_mean": 0.3399658203125, "G_mean": 0.35918089048936963, "G_std": 0.16113508368531862, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.30877423286438, "grad_norm": 0.8827403783798218, "tokens_seen": 74391552, "tokens_per_sec": 15025.760191806054, "vram_gb": 6.442267648, "lambda": 0.2926728830983241, "lambda_min": 0.08341409265995026, "lambda_max": 0.6321069598197937, "S_mean": 0.3258768717447917, "G_mean": 0.360056731539468, "G_std": 0.15900014302072427, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.264007329940796, "grad_norm": 0.8698505759239197, "tokens_seen": 74555392, "tokens_per_sec": 15026.246859808212, "vram_gb": 6.442267648, "lambda": 0.2928345936040084, "lambda_min": 0.08339881151914597, "lambda_max": 0.6324180960655212, "S_mean": 0.3248799641927083, "G_mean": 0.36117946713541943, "G_std": 0.1604284212614099, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.351485967636108, "grad_norm": 0.9102940559387207, "tokens_seen": 74719232, "tokens_per_sec": 15026.551749733117, "vram_gb": 6.442267648, "lambda": 0.29298064950853586, "lambda_min": 0.08338922262191772, "lambda_max": 0.6330140233039856, "S_mean": 0.321044921875, "G_mean": 0.36127024757054943, "G_std": 0.1602627943502739, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.472531080245972, "grad_norm": 0.8882570266723633, "tokens_seen": 74883072, "tokens_per_sec": 15026.936643281539, "vram_gb": 6.442267648, "lambda": 0.29312839824706316, "lambda_min": 0.08340510725975037, "lambda_max": 0.6334938406944275, "S_mean": 0.311065673828125, "G_mean": 0.3602388611373802, "G_std": 0.16121582609290877, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.308815002441406, "grad_norm": 0.9632748961448669, "tokens_seen": 75046912, "tokens_per_sec": 15027.218588319052, "vram_gb": 6.442267648, "lambda": 0.29322462684164446, "lambda_min": 0.08338591456413269, "lambda_max": 0.6337292790412903, "S_mean": 0.3309122721354167, "G_mean": 0.3603007663041353, "G_std": 0.1598144945843766, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.470222234725952, "grad_norm": 0.9445265531539917, "tokens_seen": 75210752, "tokens_per_sec": 15027.776391271917, "vram_gb": 6.442267648, "lambda": 0.29340921094020206, "lambda_min": 0.08341006189584732, "lambda_max": 0.6343522071838379, "S_mean": 0.3497416178385417, "G_mean": 0.3631959219152729, "G_std": 0.15993464027997106, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.414315938949585, "grad_norm": 0.9758800864219666, "tokens_seen": 75374592, "tokens_per_sec": 15028.182792534193, "vram_gb": 6.442267648, "lambda": 0.29357157461345196, "lambda_min": 0.08340535312891006, "lambda_max": 0.6349592208862305, "S_mean": 0.3331298828125, "G_mean": 0.3681852699567874, "G_std": 0.15855601154423007, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.460771083831787, "grad_norm": 0.911716103553772, "tokens_seen": 75538432, "tokens_per_sec": 15028.512334009278, "vram_gb": 6.442267648, "lambda": 0.2937112453704079, "lambda_min": 0.08341401815414429, "lambda_max": 0.6356031894683838, "S_mean": 0.3188374837239583, "G_mean": 0.3586834479744236, "G_std": 0.16191815103714666, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.4569079875946045, "grad_norm": 0.9145289659500122, "tokens_seen": 75702272, "tokens_per_sec": 15028.901753407186, "vram_gb": 6.442267648, "lambda": 0.29386542613307637, "lambda_min": 0.08346547186374664, "lambda_max": 0.6362529397010803, "S_mean": 0.3141377766927083, "G_mean": 0.3578078344774743, "G_std": 0.16046219035827866, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.392730712890625, "grad_norm": 0.9115044474601746, "tokens_seen": 75866112, "tokens_per_sec": 15029.317032689116, "vram_gb": 6.442267648, "lambda": 0.2940109533568223, "lambda_min": 0.08348213136196136, "lambda_max": 0.6366685032844543, "S_mean": 0.3175760904947917, "G_mean": 0.3576989093174537, "G_std": 0.16201383077229062, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.336359977722168, "grad_norm": 0.8598942756652832, "tokens_seen": 76029952, "tokens_per_sec": 15029.745660808889, "vram_gb": 6.442267648, "lambda": 0.2941441560784976, "lambda_min": 0.08344463258981705, "lambda_max": 0.6370500922203064, "S_mean": 0.3208109537760417, "G_mean": 0.35499814540768665, "G_std": 0.1611814902862534, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.440007448196411, "grad_norm": 0.8954524397850037, "tokens_seen": 76193792, "tokens_per_sec": 15030.034391141986, "vram_gb": 6.442267648, "lambda": 0.2942757671698928, "lambda_min": 0.08346660435199738, "lambda_max": 0.6374651193618774, "S_mean": 0.3208109537760417, "G_mean": 0.35635567060671747, "G_std": 0.16099909378681332, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.323129177093506, "grad_norm": 0.8778164386749268, "tokens_seen": 76357632, "tokens_per_sec": 15030.348788287058, "vram_gb": 6.442267648, "lambda": 0.29442432802170515, "lambda_min": 0.08341047167778015, "lambda_max": 0.6380891799926758, "S_mean": 0.3222757975260417, "G_mean": 0.35952125846718747, "G_std": 0.1584441929589957, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.470181465148926, "grad_norm": 0.9024012684822083, "tokens_seen": 76521472, "tokens_per_sec": 15030.77790914001, "vram_gb": 6.442267648, "lambda": 0.29459185991436243, "lambda_min": 0.08340691775083542, "lambda_max": 0.638654351234436, "S_mean": 0.32733154296875, "G_mean": 0.35549192999800044, "G_std": 0.15891083795577288, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.535730600357056, "grad_norm": 1.0082979202270508, "tokens_seen": 76685312, "tokens_per_sec": 15031.354396589524, "vram_gb": 6.442267648, "lambda": 0.29475019592791796, "lambda_min": 0.08341487497091293, "lambda_max": 0.6390455961227417, "S_mean": 0.3338724772135417, "G_mean": 0.36132036180545885, "G_std": 0.16202109109144658, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.423816204071045, "grad_norm": 0.9365132451057434, "tokens_seen": 76849152, "tokens_per_sec": 15031.819706703525, "vram_gb": 6.442267648, "lambda": 0.2949247695505619, "lambda_min": 0.08343426883220673, "lambda_max": 0.6396556496620178, "S_mean": 0.323272705078125, "G_mean": 0.35978238695922, "G_std": 0.16126297099981457, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.29046106338501, "grad_norm": 0.8906809091567993, "tokens_seen": 77012992, "tokens_per_sec": 15032.370332934472, "vram_gb": 6.442267648, "lambda": 0.2950837602838874, "lambda_min": 0.08341208845376968, "lambda_max": 0.6402890682220459, "S_mean": 0.3189188639322917, "G_mean": 0.3607391412369907, "G_std": 0.1611899237226074, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.52546238899231, "grad_norm": 0.9756705164909363, "tokens_seen": 77176832, "tokens_per_sec": 15032.842731843788, "vram_gb": 6.442267648, "lambda": 0.2952130176126957, "lambda_min": 0.08335141837596893, "lambda_max": 0.6408507227897644, "S_mean": 0.3383890787760417, "G_mean": 0.36994058825075626, "G_std": 0.1591544710487748, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.416954755783081, "grad_norm": 0.8888912796974182, "tokens_seen": 77340672, "tokens_per_sec": 15033.283807151409, "vram_gb": 6.442267648, "lambda": 0.2953497739508748, "lambda_min": 0.08333075046539307, "lambda_max": 0.6415463089942932, "S_mean": 0.3268534342447917, "G_mean": 0.35707308794371784, "G_std": 0.1601166280452162, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.339210033416748, "grad_norm": 0.9007087349891663, "tokens_seen": 77504512, "tokens_per_sec": 15033.713468967768, "vram_gb": 6.442267648, "lambda": 0.2954567385216554, "lambda_min": 0.08333027362823486, "lambda_max": 0.6421370506286621, "S_mean": 0.3344014485677083, "G_mean": 0.3551462067601581, "G_std": 0.15804186556488276, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.4074859619140625, "grad_norm": 0.8757627010345459, "tokens_seen": 77668352, "tokens_per_sec": 15034.163579755916, "vram_gb": 6.442267648, "lambda": 0.2956008951490124, "lambda_min": 0.08333376795053482, "lambda_max": 0.6427275538444519, "S_mean": 0.3214111328125, "G_mean": 0.35745541223635274, "G_std": 0.1584426808015754, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.3463921546936035, "grad_norm": 0.9302581548690796, "tokens_seen": 77832192, "tokens_per_sec": 15034.625329238144, "vram_gb": 6.442267648, "lambda": 0.29575240239501, "lambda_min": 0.08330319076776505, "lambda_max": 0.6434556841850281, "S_mean": 0.323028564453125, "G_mean": 0.3559929217832784, "G_std": 0.15987981157377362, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.439501941204071, "val_ppl": 84.73272928145225, "lambda": 0.2957618832588196, "lambda_min": 0.08330722898244858, "lambda_max": 0.6434731483459473, "lambda_per_layer": [0.1802998036146164, 0.14920593798160553, 0.1530390977859497, 0.1366344690322876, 0.12029222398996353, 0.08330722898244858, 0.1218167096376419, 0.09221872687339783, 0.1578769087791443, 0.31458887457847595, 0.32463401556015015, 0.44473835825920105, 0.31532812118530273, 0.5319607853889465, 0.5231014490127563, 0.47221532464027405, 0.6434731483459473, 0.4484391212463379, 0.369595468044281, 0.5523155927658081, 0.1902160942554474, 0.33313876390457153, 0.26280665397644043, 0.17704232037067413], "S_mean": 0.3291880289713542, "G_mean": 0.35662542656064034, "G_std": 0.1612039509539803, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.210537672042847, "grad_norm": 0.9386403560638428, "tokens_seen": 77996032, "tokens_per_sec": 15023.248280045193, "vram_gb": 6.442267648, "lambda": 0.2959083517392476, "lambda_min": 0.08333349227905273, "lambda_max": 0.6440401673316956, "S_mean": 0.3102823893229167, "G_mean": 0.35803080404487747, "G_std": 0.15941094409208745, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.518207788467407, "grad_norm": 1.003307580947876, "tokens_seen": 78159872, "tokens_per_sec": 15023.616732870207, "vram_gb": 6.442267648, "lambda": 0.2960332526514928, "lambda_min": 0.08329566568136215, "lambda_max": 0.6444005966186523, "S_mean": 0.3316446940104167, "G_mean": 0.3582208335089187, "G_std": 0.15897587527676174, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.470003604888916, "grad_norm": 0.9022462368011475, "tokens_seen": 78323712, "tokens_per_sec": 15023.763113972202, "vram_gb": 6.442267648, "lambda": 0.29619730326036614, "lambda_min": 0.08326034247875214, "lambda_max": 0.6448423266410828, "S_mean": 0.331787109375, "G_mean": 0.3564637800057729, "G_std": 0.15943935707521936, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.281353950500488, "grad_norm": 0.9561741352081299, "tokens_seen": 78487552, "tokens_per_sec": 15024.058788070257, "vram_gb": 6.442267648, "lambda": 0.2963089033340414, "lambda_min": 0.0833120346069336, "lambda_max": 0.6453133225440979, "S_mean": 0.31610107421875, "G_mean": 0.3531519764413436, "G_std": 0.15928697035027048, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.35053825378418, "grad_norm": 0.9347555041313171, "tokens_seen": 78651392, "tokens_per_sec": 15024.252976307136, "vram_gb": 6.442267648, "lambda": 0.29642096534371376, "lambda_min": 0.08334644138813019, "lambda_max": 0.6457836031913757, "S_mean": 0.3186848958333333, "G_mean": 0.3597104190848768, "G_std": 0.16079998757534972, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.4456398487091064, "grad_norm": 0.9778335690498352, "tokens_seen": 78815232, "tokens_per_sec": 15024.704910968385, "vram_gb": 6.442267648, "lambda": 0.2965585341056188, "lambda_min": 0.0833730399608612, "lambda_max": 0.6463192105293274, "S_mean": 0.3229878743489583, "G_mean": 0.35844819985019666, "G_std": 0.1598481476927797, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.299177885055542, "grad_norm": 0.9912563562393188, "tokens_seen": 78979072, "tokens_per_sec": 15025.401192898504, "vram_gb": 6.442267648, "lambda": 0.29669858049601316, "lambda_min": 0.0833708718419075, "lambda_max": 0.6469904184341431, "S_mean": 0.3328653971354167, "G_mean": 0.3540510962096353, "G_std": 0.1606871895104026, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.433836936950684, "grad_norm": 0.8698013424873352, "tokens_seen": 79142912, "tokens_per_sec": 15025.923082704881, "vram_gb": 6.442267648, "lambda": 0.2968192957341671, "lambda_min": 0.08338235318660736, "lambda_max": 0.6475606560707092, "S_mean": 0.3166453043619792, "G_mean": 0.3594433416922887, "G_std": 0.16064016140686968, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.5978758335113525, "grad_norm": 0.9166929721832275, "tokens_seen": 79306752, "tokens_per_sec": 15026.332929367676, "vram_gb": 6.442267648, "lambda": 0.2969347055380543, "lambda_min": 0.08336251229047775, "lambda_max": 0.6480069160461426, "S_mean": 0.3289388020833333, "G_mean": 0.3582506523622821, "G_std": 0.16032182118700197, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.267582893371582, "grad_norm": 0.9350684881210327, "tokens_seen": 79470592, "tokens_per_sec": 15026.599447246204, "vram_gb": 6.442267648, "lambda": 0.29709628007064265, "lambda_min": 0.08338499814271927, "lambda_max": 0.6485828757286072, "S_mean": 0.3200785319010417, "G_mean": 0.34946609071145457, "G_std": 0.16076669605293623, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.260303974151611, "grad_norm": 0.910900890827179, "tokens_seen": 79634432, "tokens_per_sec": 15027.02637402971, "vram_gb": 6.442267648, "lambda": 0.29720732538650435, "lambda_min": 0.08336780965328217, "lambda_max": 0.6490685939788818, "S_mean": 0.3301493326822917, "G_mean": 0.35389701354627806, "G_std": 0.15983734210021794, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.431431531906128, "grad_norm": 1.1961629390716553, "tokens_seen": 79798272, "tokens_per_sec": 15027.48888337839, "vram_gb": 6.442267648, "lambda": 0.29730000377943117, "lambda_min": 0.08339663594961166, "lambda_max": 0.6493677496910095, "S_mean": 0.3286336263020833, "G_mean": 0.3581996407204618, "G_std": 0.1620942660762618, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.280236721038818, "grad_norm": 0.9126841425895691, "tokens_seen": 79962112, "tokens_per_sec": 15027.928800661612, "vram_gb": 6.442267648, "lambda": 0.29742907267063856, "lambda_min": 0.08344203233718872, "lambda_max": 0.6497378349304199, "S_mean": 0.3280131022135417, "G_mean": 0.356913054206719, "G_std": 0.1589979592245072, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.364770412445068, "grad_norm": 0.9236781001091003, "tokens_seen": 80125952, "tokens_per_sec": 15028.303469228973, "vram_gb": 6.442267648, "lambda": 0.29753911557296914, "lambda_min": 0.08345035463571548, "lambda_max": 0.6501826643943787, "S_mean": 0.3370157877604167, "G_mean": 0.35329023050144315, "G_std": 0.1594473945830638, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.3228819370269775, "grad_norm": 0.9365066289901733, "tokens_seen": 80289792, "tokens_per_sec": 15028.789303202546, "vram_gb": 6.442267648, "lambda": 0.29766348532090586, "lambda_min": 0.08341460675001144, "lambda_max": 0.6504487991333008, "S_mean": 0.3271993001302083, "G_mean": 0.359042527154088, "G_std": 0.1592622751292462, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.466001033782959, "grad_norm": 0.9404023289680481, "tokens_seen": 80453632, "tokens_per_sec": 15029.349418613247, "vram_gb": 6.442267648, "lambda": 0.29777155971775454, "lambda_min": 0.08339753746986389, "lambda_max": 0.6506605744361877, "S_mean": 0.3231964111328125, "G_mean": 0.35748573547850054, "G_std": 0.16025818116031587, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.12202787399292, "grad_norm": 0.9568600654602051, "tokens_seen": 80617472, "tokens_per_sec": 15029.838648723982, "vram_gb": 6.442267648, "lambda": 0.29788480264445144, "lambda_min": 0.08339852839708328, "lambda_max": 0.6511450409889221, "S_mean": 0.3486735026041667, "G_mean": 0.3607317948093017, "G_std": 0.1558278031491985, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.456521987915039, "grad_norm": 0.9778349995613098, "tokens_seen": 80781312, "tokens_per_sec": 15030.213243979635, "vram_gb": 6.442267648, "lambda": 0.29804216232150793, "lambda_min": 0.08337825536727905, "lambda_max": 0.6516739726066589, "S_mean": 0.326690673828125, "G_mean": 0.3568486788620551, "G_std": 0.1602212773092712, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.498927116394043, "grad_norm": 1.051926851272583, "tokens_seen": 80945152, "tokens_per_sec": 15030.669835084782, "vram_gb": 6.442267648, "lambda": 0.29817223610977334, "lambda_min": 0.08339892327785492, "lambda_max": 0.6520279049873352, "S_mean": 0.3214925130208333, "G_mean": 0.36374200973659754, "G_std": 0.15901914611458778, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.311722755432129, "grad_norm": 0.9539486765861511, "tokens_seen": 81108992, "tokens_per_sec": 15031.007359638423, "vram_gb": 6.442267648, "lambda": 0.29827761308600503, "lambda_min": 0.08341635018587112, "lambda_max": 0.6523930430412292, "S_mean": 0.3274078369140625, "G_mean": 0.35590190161019564, "G_std": 0.16036168524685004, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.428753137588501, "grad_norm": 0.9083033204078674, "tokens_seen": 81272832, "tokens_per_sec": 15031.432046857344, "vram_gb": 6.442267648, "lambda": 0.29838708105186623, "lambda_min": 0.0833972841501236, "lambda_max": 0.6528512239456177, "S_mean": 0.3313140869140625, "G_mean": 0.35463118460029364, "G_std": 0.1600125739738966, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.571376800537109, "grad_norm": 1.0062150955200195, "tokens_seen": 81436672, "tokens_per_sec": 15031.929593899486, "vram_gb": 6.442267648, "lambda": 0.29851334790388745, "lambda_min": 0.0834338515996933, "lambda_max": 0.6532022953033447, "S_mean": 0.329803466796875, "G_mean": 0.35825271997600794, "G_std": 0.15880972077138722, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.472095727920532, "grad_norm": 0.8982868790626526, "tokens_seen": 81600512, "tokens_per_sec": 15032.396394344654, "vram_gb": 6.442267648, "lambda": 0.2986322057743867, "lambda_min": 0.08343323320150375, "lambda_max": 0.6536717414855957, "S_mean": 0.3205464680989583, "G_mean": 0.3540138112536321, "G_std": 0.15921082790009677, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.346381187438965, "grad_norm": 0.8663289546966553, "tokens_seen": 81764352, "tokens_per_sec": 15032.953853028663, "vram_gb": 6.442267648, "lambda": 0.2987455374871691, "lambda_min": 0.08339844644069672, "lambda_max": 0.6540229320526123, "S_mean": 0.3185984293619792, "G_mean": 0.35476406291127205, "G_std": 0.15976982986709723, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.468996286392212, "grad_norm": 0.9729671478271484, "tokens_seen": 81928192, "tokens_per_sec": 15033.418070529186, "vram_gb": 6.442267648, "lambda": 0.2988184057176113, "lambda_min": 0.08340504765510559, "lambda_max": 0.6542119979858398, "S_mean": 0.3230183919270833, "G_mean": 0.3558954269780467, "G_std": 0.1594198088084037, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.4305883049964905, "val_ppl": 83.98080870949624, "lambda": 0.2988224485889077, "lambda_min": 0.08340254426002502, "lambda_max": 0.6542307734489441, "lambda_per_layer": [0.18085353076457977, 0.1490868479013443, 0.15380962193012238, 0.13651242852210999, 0.12054121494293213, 0.08340254426002502, 0.12160029262304306, 0.09219364821910858, 0.15870746970176697, 0.31609123945236206, 0.32674136757850647, 0.4482461214065552, 0.31791791319847107, 0.5372613668441772, 0.5301061868667603, 0.4780256152153015, 0.6542307734489441, 0.4541510045528412, 0.3752278983592987, 0.5611750483512878, 0.19172050058841705, 0.3384925425052643, 0.26685938239097595, 0.17878420650959015], "S_mean": 0.3146769205729167, "G_mean": 0.3585035325959325, "G_std": 0.16067228718505552, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.244568347930908, "grad_norm": 0.9043054580688477, "tokens_seen": 82092032, "tokens_per_sec": 15022.742117995916, "vram_gb": 6.442267648, "lambda": 0.29891528841108084, "lambda_min": 0.08335435390472412, "lambda_max": 0.6545467376708984, "S_mean": 0.3235575358072917, "G_mean": 0.3568601736333221, "G_std": 0.158766221255064, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.239662408828735, "grad_norm": 0.8920642733573914, "tokens_seen": 82255872, "tokens_per_sec": 15023.174414969268, "vram_gb": 6.442267648, "lambda": 0.2990020752574007, "lambda_min": 0.08332470059394836, "lambda_max": 0.6547581553459167, "S_mean": 0.326263427734375, "G_mean": 0.35231720842421055, "G_std": 0.16099667052427927, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.255927562713623, "grad_norm": 0.9103569984436035, "tokens_seen": 82419712, "tokens_per_sec": 15023.633792034661, "vram_gb": 6.442267648, "lambda": 0.2991238233322899, "lambda_min": 0.0833735391497612, "lambda_max": 0.6551440954208374, "S_mean": 0.3255106608072917, "G_mean": 0.3519400929411252, "G_std": 0.15990602093127868, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.595212697982788, "grad_norm": 0.9742167592048645, "tokens_seen": 82583552, "tokens_per_sec": 15024.041431007954, "vram_gb": 6.442267648, "lambda": 0.2992284058903654, "lambda_min": 0.08335297554731369, "lambda_max": 0.6555110812187195, "S_mean": 0.3348592122395833, "G_mean": 0.35400338113928836, "G_std": 0.16144572004365423, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.573580980300903, "grad_norm": 0.9622275233268738, "tokens_seen": 82747392, "tokens_per_sec": 15024.479817147027, "vram_gb": 6.442267648, "lambda": 0.2993472069501877, "lambda_min": 0.08340118080377579, "lambda_max": 0.6559958457946777, "S_mean": 0.3243865966796875, "G_mean": 0.3586220351668696, "G_std": 0.16018636066777012, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.326800346374512, "grad_norm": 0.9642325639724731, "tokens_seen": 82911232, "tokens_per_sec": 15024.924454827848, "vram_gb": 6.442267648, "lambda": 0.29943193929890793, "lambda_min": 0.08340207487344742, "lambda_max": 0.6563695669174194, "S_mean": 0.3296000162760417, "G_mean": 0.35663331719115376, "G_std": 0.16050552642749003, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.349306106567383, "grad_norm": 1.0133044719696045, "tokens_seen": 83075072, "tokens_per_sec": 15025.38519620232, "vram_gb": 6.442267648, "lambda": 0.2995311391229431, "lambda_min": 0.08337903022766113, "lambda_max": 0.656629204750061, "S_mean": 0.3289337158203125, "G_mean": 0.3545027143942813, "G_std": 0.15993598301429302, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.297933578491211, "grad_norm": 0.9418309330940247, "tokens_seen": 83238912, "tokens_per_sec": 15025.633978334783, "vram_gb": 6.442267648, "lambda": 0.2996478605394562, "lambda_min": 0.08333633840084076, "lambda_max": 0.6570072770118713, "S_mean": 0.3416646321614583, "G_mean": 0.35680585998731357, "G_std": 0.15939132830438515, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.292533874511719, "grad_norm": 0.9395480751991272, "tokens_seen": 83402752, "tokens_per_sec": 15026.043270201433, "vram_gb": 6.442267648, "lambda": 0.29974141499648493, "lambda_min": 0.0833623856306076, "lambda_max": 0.6571882963180542, "S_mean": 0.3340861002604167, "G_mean": 0.35732950766881305, "G_std": 0.1590344865884011, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.332656621932983, "grad_norm": 0.9303736090660095, "tokens_seen": 83566592, "tokens_per_sec": 15026.47822900617, "vram_gb": 6.442267648, "lambda": 0.29983059347917634, "lambda_min": 0.08332693576812744, "lambda_max": 0.6576169729232788, "S_mean": 0.3248392740885417, "G_mean": 0.36047388481286663, "G_std": 0.1585056394493828, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.351492881774902, "grad_norm": 1.0100356340408325, "tokens_seen": 83730432, "tokens_per_sec": 15026.905544437286, "vram_gb": 6.442267648, "lambda": 0.2999269347637892, "lambda_min": 0.08330915868282318, "lambda_max": 0.6579291820526123, "S_mean": 0.327606201171875, "G_mean": 0.3570485101081431, "G_std": 0.15956676580632725, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.333796262741089, "grad_norm": 0.9490490555763245, "tokens_seen": 83894272, "tokens_per_sec": 15027.229133758146, "vram_gb": 6.442267648, "lambda": 0.3000488355755806, "lambda_min": 0.08331852406263351, "lambda_max": 0.6582831740379333, "S_mean": 0.3263142903645833, "G_mean": 0.35683103433499735, "G_std": 0.15906080114655197, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.4054765701293945, "grad_norm": 1.0941119194030762, "tokens_seen": 84058112, "tokens_per_sec": 15027.029311048704, "vram_gb": 6.442267648, "lambda": 0.3001634751756986, "lambda_min": 0.08331471681594849, "lambda_max": 0.6588149070739746, "S_mean": 0.3368021647135417, "G_mean": 0.35455567156895995, "G_std": 0.15959369658958167, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.38067626953125, "grad_norm": 0.9938342571258545, "tokens_seen": 84221952, "tokens_per_sec": 15027.464037873413, "vram_gb": 6.442267648, "lambda": 0.3002860962102811, "lambda_min": 0.08329811692237854, "lambda_max": 0.6593937277793884, "S_mean": 0.3330485026041667, "G_mean": 0.3569905702024698, "G_std": 0.15820754253460714, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.281987190246582, "grad_norm": 0.9223577976226807, "tokens_seen": 84385792, "tokens_per_sec": 15027.606236134188, "vram_gb": 6.442267648, "lambda": 0.3003728125865261, "lambda_min": 0.08329280465841293, "lambda_max": 0.6597341895103455, "S_mean": 0.3329569498697917, "G_mean": 0.3525042791540424, "G_std": 0.15719347835207978, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.229186534881592, "grad_norm": 0.8939169645309448, "tokens_seen": 84549632, "tokens_per_sec": 15027.893254101446, "vram_gb": 6.442267648, "lambda": 0.3004772945617636, "lambda_min": 0.08330289274454117, "lambda_max": 0.6602011322975159, "S_mean": 0.3322245279947917, "G_mean": 0.3497640168449531, "G_std": 0.15926377514066795, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.317463397979736, "grad_norm": 0.9858341813087463, "tokens_seen": 84713472, "tokens_per_sec": 15028.22770090392, "vram_gb": 6.442267648, "lambda": 0.3005873554696639, "lambda_min": 0.08329379558563232, "lambda_max": 0.6606296896934509, "S_mean": 0.3261973063151042, "G_mean": 0.3537204213595639, "G_std": 0.15861072627982745, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.40394139289856, "grad_norm": 0.9965940117835999, "tokens_seen": 84877312, "tokens_per_sec": 15028.584403621511, "vram_gb": 6.442267648, "lambda": 0.3006585069621603, "lambda_min": 0.0832904651761055, "lambda_max": 0.6609020233154297, "S_mean": 0.3378499348958333, "G_mean": 0.3563681039959192, "G_std": 0.15809736245622238, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.2411277294158936, "grad_norm": 0.9091622233390808, "tokens_seen": 85041152, "tokens_per_sec": 15028.999837018257, "vram_gb": 6.442267648, "lambda": 0.30072542900840443, "lambda_min": 0.08329242467880249, "lambda_max": 0.6611793041229248, "S_mean": 0.334381103515625, "G_mean": 0.35499131423421204, "G_std": 0.15907363403433314, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.363184452056885, "grad_norm": 0.977278470993042, "tokens_seen": 85204992, "tokens_per_sec": 15029.352772715483, "vram_gb": 6.442267648, "lambda": 0.3008216901992758, "lambda_min": 0.08329374343156815, "lambda_max": 0.6614495515823364, "S_mean": 0.3436991373697917, "G_mean": 0.35223177867010236, "G_std": 0.1589898179518059, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.388834238052368, "grad_norm": 1.0137953758239746, "tokens_seen": 85368832, "tokens_per_sec": 15029.571142438463, "vram_gb": 6.442267648, "lambda": 0.30091601020346087, "lambda_min": 0.08328991383314133, "lambda_max": 0.6616325378417969, "S_mean": 0.325164794921875, "G_mean": 0.3604333245505889, "G_std": 0.15814300863227496, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.424185752868652, "grad_norm": 1.0197232961654663, "tokens_seen": 85532672, "tokens_per_sec": 15029.822296325854, "vram_gb": 6.442267648, "lambda": 0.30101171880960464, "lambda_min": 0.0833057388663292, "lambda_max": 0.6618635058403015, "S_mean": 0.336517333984375, "G_mean": 0.35854766246241826, "G_std": 0.1597987411854168, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.467641353607178, "grad_norm": 1.0088372230529785, "tokens_seen": 85696512, "tokens_per_sec": 15030.074525727901, "vram_gb": 6.442267648, "lambda": 0.30109395862867433, "lambda_min": 0.08329886943101883, "lambda_max": 0.6621905565261841, "S_mean": 0.3297831217447917, "G_mean": 0.36066112147333723, "G_std": 0.15894729531525323, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.337714910507202, "grad_norm": 1.013427972793579, "tokens_seen": 85860352, "tokens_per_sec": 15030.391566194854, "vram_gb": 6.442267648, "lambda": 0.3011544942855835, "lambda_min": 0.0832846462726593, "lambda_max": 0.6624058485031128, "S_mean": 0.3441569010416667, "G_mean": 0.36058713821694255, "G_std": 0.15915062165974328, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.426962614059448, "grad_norm": 0.9598677754402161, "tokens_seen": 86024192, "tokens_per_sec": 15030.863624557964, "vram_gb": 6.442267648, "lambda": 0.3012528655429681, "lambda_min": 0.08329501748085022, "lambda_max": 0.6627078652381897, "S_mean": 0.335235595703125, "G_mean": 0.3568641215873261, "G_std": 0.1600849014163638, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.4351512551307675, "val_ppl": 84.36488454492857, "lambda": 0.3012567873423298, "lambda_min": 0.08329518884420395, "lambda_max": 0.6627259254455566, "lambda_per_layer": [0.18151463568210602, 0.14902061223983765, 0.15438425540924072, 0.13658404350280762, 0.12068073451519012, 0.08329518884420395, 0.12151429802179337, 0.09203378111124039, 0.15930677950382233, 0.31697171926498413, 0.3286533057689667, 0.45072245597839355, 0.31988102197647095, 0.5409426689147949, 0.5357829332351685, 0.4831858277320862, 0.6627259254455566, 0.45871835947036743, 0.3797757029533386, 0.5683215260505676, 0.19247028231620789, 0.34302425384521484, 0.27042627334594727, 0.18022631108760834], "S_mean": 0.3238932291666667, "G_mean": 0.35626507791069645, "G_std": 0.15939926639354476, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.230505704879761, "grad_norm": 0.9686294794082642, "tokens_seen": 86188032, "tokens_per_sec": 15020.557538378862, "vram_gb": 6.442267648, "lambda": 0.3013236640642087, "lambda_min": 0.08330201357603073, "lambda_max": 0.6629228591918945, "S_mean": 0.3249918619791667, "G_mean": 0.3546206353542705, "G_std": 0.15702441268755743, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.41196084022522, "grad_norm": 0.9329017996788025, "tokens_seen": 86351872, "tokens_per_sec": 15020.986475625772, "vram_gb": 6.442267648, "lambda": 0.3014307947208484, "lambda_min": 0.08331228047609329, "lambda_max": 0.6632576584815979, "S_mean": 0.3306070963541667, "G_mean": 0.35363122339670855, "G_std": 0.15715738257858902, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.354789733886719, "grad_norm": 0.9946582913398743, "tokens_seen": 86515712, "tokens_per_sec": 15020.968878543395, "vram_gb": 6.442267648, "lambda": 0.30151529765377444, "lambda_min": 0.08332381397485733, "lambda_max": 0.6636998653411865, "S_mean": 0.3266092936197917, "G_mean": 0.3494523138118287, "G_std": 0.15979556343518198, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.366533279418945, "grad_norm": 0.9454677104949951, "tokens_seen": 86679552, "tokens_per_sec": 15021.172553833294, "vram_gb": 6.442267648, "lambda": 0.30161114037036896, "lambda_min": 0.08331207185983658, "lambda_max": 0.6640674471855164, "S_mean": 0.335174560546875, "G_mean": 0.3549492438323796, "G_std": 0.15800849096073458, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.377943277359009, "grad_norm": 0.8980675339698792, "tokens_seen": 86843392, "tokens_per_sec": 15021.450165683344, "vram_gb": 6.442267648, "lambda": 0.3017227003971736, "lambda_min": 0.08331948518753052, "lambda_max": 0.6644792556762695, "S_mean": 0.3395589192708333, "G_mean": 0.3549186848104, "G_std": 0.15821394658026597, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.1942458152771, "grad_norm": 0.9190878868103027, "tokens_seen": 87007232, "tokens_per_sec": 15021.704140836804, "vram_gb": 6.442267648, "lambda": 0.30181511274228495, "lambda_min": 0.083319291472435, "lambda_max": 0.66476970911026, "S_mean": 0.3382568359375, "G_mean": 0.3555798502638936, "G_std": 0.1596464414615184, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.202578067779541, "grad_norm": 0.9488473534584045, "tokens_seen": 87171072, "tokens_per_sec": 15021.887409481757, "vram_gb": 6.442267648, "lambda": 0.30191439017653465, "lambda_min": 0.08333927392959595, "lambda_max": 0.6650046706199646, "S_mean": 0.3224589029947917, "G_mean": 0.35305707110092044, "G_std": 0.159239740188544, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.348737478256226, "grad_norm": 0.9680432081222534, "tokens_seen": 87334912, "tokens_per_sec": 15022.340881182916, "vram_gb": 6.442267648, "lambda": 0.3020153675849239, "lambda_min": 0.08333828300237656, "lambda_max": 0.6651234030723572, "S_mean": 0.3212890625, "G_mean": 0.3549336750681202, "G_std": 0.15847718374182782, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.429649591445923, "grad_norm": 0.909243106842041, "tokens_seen": 87498752, "tokens_per_sec": 15022.7746540293, "vram_gb": 6.442267648, "lambda": 0.3020895943045616, "lambda_min": 0.08330365270376205, "lambda_max": 0.6652798056602478, "S_mean": 0.3198598225911458, "G_mean": 0.35832432843744755, "G_std": 0.15852237275491157, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.241058588027954, "grad_norm": 0.9776111841201782, "tokens_seen": 87662592, "tokens_per_sec": 15023.204202533714, "vram_gb": 6.442267648, "lambda": 0.3021532253672679, "lambda_min": 0.08328044414520264, "lambda_max": 0.6654372811317444, "S_mean": 0.3366597493489583, "G_mean": 0.35331386234611273, "G_std": 0.1580997396570941, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.4644691944122314, "grad_norm": 1.018611192703247, "tokens_seen": 87826432, "tokens_per_sec": 15023.697456987298, "vram_gb": 6.442267648, "lambda": 0.3022256375600894, "lambda_min": 0.08324728161096573, "lambda_max": 0.6656020879745483, "S_mean": 0.331085205078125, "G_mean": 0.36003056991224486, "G_std": 0.15851931821089238, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.405058860778809, "grad_norm": 0.956038773059845, "tokens_seen": 87990272, "tokens_per_sec": 15024.288906790465, "vram_gb": 6.442267648, "lambda": 0.30232095004369813, "lambda_min": 0.08326306939125061, "lambda_max": 0.6659516096115112, "S_mean": 0.332061767578125, "G_mean": 0.35763002016271156, "G_std": 0.15682949028753987, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.296271562576294, "grad_norm": 0.922430157661438, "tokens_seen": 88154112, "tokens_per_sec": 15024.831031218302, "vram_gb": 6.442267648, "lambda": 0.3024042571584384, "lambda_min": 0.083298459649086, "lambda_max": 0.6661201119422913, "S_mean": 0.3318684895833333, "G_mean": 0.35448452721660334, "G_std": 0.15866177754166225, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.386836528778076, "grad_norm": 0.9817250370979309, "tokens_seen": 88317952, "tokens_per_sec": 15025.202429436544, "vram_gb": 6.442267648, "lambda": 0.30249367157618207, "lambda_min": 0.08329025655984879, "lambda_max": 0.6663685441017151, "S_mean": 0.3282674153645833, "G_mean": 0.35735314649840194, "G_std": 0.15937203238718212, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.472002983093262, "grad_norm": 1.0502763986587524, "tokens_seen": 88481792, "tokens_per_sec": 15025.628117623077, "vram_gb": 6.442267648, "lambda": 0.30258614011108875, "lambda_min": 0.08328400552272797, "lambda_max": 0.6667163968086243, "S_mean": 0.3314921061197917, "G_mean": 0.3596738283522427, "G_std": 0.15738639437283078, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.256236791610718, "grad_norm": 1.0613795518875122, "tokens_seen": 88645632, "tokens_per_sec": 15026.05640622999, "vram_gb": 6.442267648, "lambda": 0.302665904785196, "lambda_min": 0.0832960456609726, "lambda_max": 0.66702800989151, "S_mean": 0.3327128092447917, "G_mean": 0.3585772806157668, "G_std": 0.15882875983758518, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.368169546127319, "grad_norm": 0.9950992465019226, "tokens_seen": 88809472, "tokens_per_sec": 15026.535041870078, "vram_gb": 6.442267648, "lambda": 0.3027230082079768, "lambda_min": 0.08329510688781738, "lambda_max": 0.6673076152801514, "S_mean": 0.3440958658854167, "G_mean": 0.3486389470441888, "G_std": 0.1560692402999848, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.4627320766448975, "grad_norm": 1.0546432733535767, "tokens_seen": 88973312, "tokens_per_sec": 15026.973385306517, "vram_gb": 6.442267648, "lambda": 0.30280512167761725, "lambda_min": 0.08330618590116501, "lambda_max": 0.6676174402236938, "S_mean": 0.32794189453125, "G_mean": 0.3532739697645108, "G_std": 0.1577587944533055, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.278488397598267, "grad_norm": 0.9461482763290405, "tokens_seen": 89137152, "tokens_per_sec": 15027.532645188174, "vram_gb": 6.442267648, "lambda": 0.30287775211036205, "lambda_min": 0.08327529579401016, "lambda_max": 0.6679832935333252, "S_mean": 0.3262481689453125, "G_mean": 0.3460147036239505, "G_std": 0.15866154606919736, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.3999950885772705, "grad_norm": 0.9764474034309387, "tokens_seen": 89300992, "tokens_per_sec": 15027.856417031815, "vram_gb": 6.442267648, "lambda": 0.3029637100795905, "lambda_min": 0.08329673111438751, "lambda_max": 0.6684461832046509, "S_mean": 0.3397725423177083, "G_mean": 0.3560061762109399, "G_std": 0.15899100011059394, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.310083866119385, "grad_norm": 0.9485397338867188, "tokens_seen": 89464832, "tokens_per_sec": 15028.262298870899, "vram_gb": 6.442267648, "lambda": 0.3030237536877394, "lambda_min": 0.08328765630722046, "lambda_max": 0.6687440276145935, "S_mean": 0.3407084147135417, "G_mean": 0.35558117328522104, "G_std": 0.15846178296487778, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.39045262336731, "grad_norm": 0.9761127233505249, "tokens_seen": 89628672, "tokens_per_sec": 15028.810089652008, "vram_gb": 6.442267648, "lambda": 0.3030867511406541, "lambda_min": 0.08325531333684921, "lambda_max": 0.6689671277999878, "S_mean": 0.3321634928385417, "G_mean": 0.3543637775971244, "G_std": 0.15880605803492168, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.432351112365723, "grad_norm": 1.0286872386932373, "tokens_seen": 89792512, "tokens_per_sec": 15029.251251535583, "vram_gb": 6.442267648, "lambda": 0.30318172307064134, "lambda_min": 0.0832478329539299, "lambda_max": 0.669268786907196, "S_mean": 0.330657958984375, "G_mean": 0.3544179388942818, "G_std": 0.15759586434190473, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.341154098510742, "grad_norm": 1.0043226480484009, "tokens_seen": 89956352, "tokens_per_sec": 15029.696989797108, "vram_gb": 6.442267648, "lambda": 0.30324514272312325, "lambda_min": 0.08324828743934631, "lambda_max": 0.6693722605705261, "S_mean": 0.3396809895833333, "G_mean": 0.3545960403668384, "G_std": 0.15912636443196485, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.356797456741333, "grad_norm": 0.9607775211334229, "tokens_seen": 90120192, "tokens_per_sec": 15030.069667452548, "vram_gb": 6.442267648, "lambda": 0.303302401676774, "lambda_min": 0.0832565501332283, "lambda_max": 0.669506847858429, "S_mean": 0.3287150065104167, "G_mean": 0.34809166674191755, "G_std": 0.15746696467977017, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.3940312504768375, "val_ppl": 80.96615683224974, "lambda": 0.3033060310408473, "lambda_min": 0.08325648307800293, "lambda_max": 0.669518232345581, "lambda_per_layer": [0.1823100745677948, 0.1489475518465042, 0.15485042333602905, 0.13654106855392456, 0.12076804041862488, 0.08325648307800293, 0.1213025152683258, 0.09217757731676102, 0.1595553457736969, 0.31749042868614197, 0.32970473170280457, 0.45313674211502075, 0.3218613564968109, 0.5447759628295898, 0.5407372713088989, 0.48722636699676514, 0.669518232345581, 0.4627664387226105, 0.3840318024158478, 0.5734887719154358, 0.19378115236759186, 0.34677502512931824, 0.2731393873691559, 0.1812019944190979], "S_mean": 0.3317667643229167, "G_mean": 0.35351272858679295, "G_std": 0.15972542273811996, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.2265379428863525, "grad_norm": 0.9480246305465698, "tokens_seen": 90284032, "tokens_per_sec": 15020.399575458545, "vram_gb": 6.442267648, "lambda": 0.30338037976374227, "lambda_min": 0.08324802666902542, "lambda_max": 0.6697648763656616, "S_mean": 0.3287556966145833, "G_mean": 0.35431842335189384, "G_std": 0.15760480864749601, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.215819358825684, "grad_norm": 0.9681582450866699, "tokens_seen": 90447872, "tokens_per_sec": 15020.785524188621, "vram_gb": 6.442267648, "lambda": 0.30344449697683257, "lambda_min": 0.08326207101345062, "lambda_max": 0.6699577569961548, "S_mean": 0.3266499837239583, "G_mean": 0.3516675451149543, "G_std": 0.15787082153838128, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.424867391586304, "grad_norm": 1.069811224937439, "tokens_seen": 90611712, "tokens_per_sec": 15021.072439515881, "vram_gb": 6.442267648, "lambda": 0.30352770909667015, "lambda_min": 0.08328089863061905, "lambda_max": 0.6702755093574524, "S_mean": 0.3320210774739583, "G_mean": 0.35496364037195843, "G_std": 0.15765052222801992, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.355289697647095, "grad_norm": 1.0071158409118652, "tokens_seen": 90775552, "tokens_per_sec": 15021.579526602307, "vram_gb": 6.442267648, "lambda": 0.3036169521510601, "lambda_min": 0.08328934013843536, "lambda_max": 0.6705453991889954, "S_mean": 0.3225809733072917, "G_mean": 0.35138007610415417, "G_std": 0.15853381090952703, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.4281628131866455, "grad_norm": 0.9857382774353027, "tokens_seen": 90939392, "tokens_per_sec": 15022.091462180719, "vram_gb": 6.442267648, "lambda": 0.30369426403194666, "lambda_min": 0.08327288180589676, "lambda_max": 0.6707344055175781, "S_mean": 0.3282572428385417, "G_mean": 0.35393421406236786, "G_std": 0.15853367183202258, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.3023529052734375, "grad_norm": 0.9685032367706299, "tokens_seen": 91103232, "tokens_per_sec": 15022.467950973232, "vram_gb": 6.442267648, "lambda": 0.3037645888204376, "lambda_min": 0.08327983319759369, "lambda_max": 0.6710773706436157, "S_mean": 0.338043212890625, "G_mean": 0.34984520620976883, "G_std": 0.15813233330845833, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.183225154876709, "grad_norm": 0.9279104471206665, "tokens_seen": 91267072, "tokens_per_sec": 15022.91788323232, "vram_gb": 6.442267648, "lambda": 0.303836852312088, "lambda_min": 0.0832684189081192, "lambda_max": 0.6713669300079346, "S_mean": 0.3507588704427083, "G_mean": 0.3570408138912171, "G_std": 0.15868819830939174, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.279399156570435, "grad_norm": 1.027164340019226, "tokens_seen": 91430912, "tokens_per_sec": 15023.355171571378, "vram_gb": 6.442267648, "lambda": 0.30390045667688054, "lambda_min": 0.0832936018705368, "lambda_max": 0.671642005443573, "S_mean": 0.3469746907552083, "G_mean": 0.35645185426498455, "G_std": 0.1564291724547123, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.3600945472717285, "grad_norm": 0.9568514823913574, "tokens_seen": 91594752, "tokens_per_sec": 15023.718732203428, "vram_gb": 6.442267648, "lambda": 0.3040075519432624, "lambda_min": 0.08327048271894455, "lambda_max": 0.6719731092453003, "S_mean": 0.3281758626302083, "G_mean": 0.35261975058044, "G_std": 0.15877550355313966, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.358745098114014, "grad_norm": 0.9438033103942871, "tokens_seen": 91758592, "tokens_per_sec": 15024.139213732926, "vram_gb": 6.442267648, "lambda": 0.304081619096299, "lambda_min": 0.08326032757759094, "lambda_max": 0.6721546649932861, "S_mean": 0.3356119791666667, "G_mean": 0.35336496580081683, "G_std": 0.15714813865876445, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.353177309036255, "grad_norm": 0.9289277791976929, "tokens_seen": 91922432, "tokens_per_sec": 15024.44450785184, "vram_gb": 6.442267648, "lambda": 0.30416804800430935, "lambda_min": 0.08330100029706955, "lambda_max": 0.6724622249603271, "S_mean": 0.32647705078125, "G_mean": 0.3550302661024034, "G_std": 0.16095657999782512, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.1762471199035645, "grad_norm": 0.9185685515403748, "tokens_seen": 92086272, "tokens_per_sec": 15024.932534401903, "vram_gb": 6.442267648, "lambda": 0.3042526834954818, "lambda_min": 0.08330933004617691, "lambda_max": 0.6728055477142334, "S_mean": 0.333526611328125, "G_mean": 0.35358641560499865, "G_std": 0.15836513950489461, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.378087759017944, "grad_norm": 0.9922649264335632, "tokens_seen": 92250112, "tokens_per_sec": 15025.469181424862, "vram_gb": 6.442267648, "lambda": 0.30432180563608807, "lambda_min": 0.08328215032815933, "lambda_max": 0.6730561256408691, "S_mean": 0.3450419108072917, "G_mean": 0.3543307015982767, "G_std": 0.15739231339345375, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.38390851020813, "grad_norm": 1.163657546043396, "tokens_seen": 92413952, "tokens_per_sec": 15025.59743940308, "vram_gb": 6.442267648, "lambda": 0.3044009543955326, "lambda_min": 0.08324828743934631, "lambda_max": 0.6732911467552185, "S_mean": 0.3322041829427083, "G_mean": 0.3566186165747543, "G_std": 0.15915865590795875, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.36643123626709, "grad_norm": 0.9968581795692444, "tokens_seen": 92577792, "tokens_per_sec": 15025.925294668174, "vram_gb": 6.442267648, "lambda": 0.3044759665305416, "lambda_min": 0.08323871344327927, "lambda_max": 0.6734951138496399, "S_mean": 0.3275655110677083, "G_mean": 0.3548339807117979, "G_std": 0.15870600942677507, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.227801322937012, "grad_norm": 0.9746532440185547, "tokens_seen": 92741632, "tokens_per_sec": 15026.352732949139, "vram_gb": 6.442267648, "lambda": 0.30453793052583933, "lambda_min": 0.08325760811567307, "lambda_max": 0.673781156539917, "S_mean": 0.32489013671875, "G_mean": 0.35302414395846426, "G_std": 0.15836241014767438, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.306710243225098, "grad_norm": 1.0279502868652344, "tokens_seen": 92905472, "tokens_per_sec": 15026.752639567065, "vram_gb": 6.442267648, "lambda": 0.3046170463785529, "lambda_min": 0.08325543254613876, "lambda_max": 0.6741055846214294, "S_mean": 0.3335164388020833, "G_mean": 0.35333169748385745, "G_std": 0.15883475503263375, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.183293581008911, "grad_norm": 0.9790984988212585, "tokens_seen": 93069312, "tokens_per_sec": 15027.252257183432, "vram_gb": 6.442267648, "lambda": 0.3047155017654101, "lambda_min": 0.08325695246458054, "lambda_max": 0.6743925213813782, "S_mean": 0.3358052571614583, "G_mean": 0.3526973278882603, "G_std": 0.15956276880266765, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.379253387451172, "grad_norm": 0.9651256203651428, "tokens_seen": 93233152, "tokens_per_sec": 15027.667025902389, "vram_gb": 6.442267648, "lambda": 0.3047927661488454, "lambda_min": 0.08329671621322632, "lambda_max": 0.674605131149292, "S_mean": 0.3324178059895833, "G_mean": 0.3552943426184356, "G_std": 0.15942229970823973, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.288292646408081, "grad_norm": 0.9937219619750977, "tokens_seen": 93396992, "tokens_per_sec": 15028.058078009975, "vram_gb": 6.442267648, "lambda": 0.30486341565847397, "lambda_min": 0.08332093060016632, "lambda_max": 0.6748045086860657, "S_mean": 0.3309427897135417, "G_mean": 0.35069042979739606, "G_std": 0.15942482254467905, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.30789852142334, "grad_norm": 1.0284652709960938, "tokens_seen": 93560832, "tokens_per_sec": 15028.588363663535, "vram_gb": 6.442267648, "lambda": 0.3049508572245638, "lambda_min": 0.08334091305732727, "lambda_max": 0.6749470233917236, "S_mean": 0.32928466796875, "G_mean": 0.35548072036666173, "G_std": 0.1567641553701833, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.164791822433472, "grad_norm": 0.9831598997116089, "tokens_seen": 93724672, "tokens_per_sec": 15028.984127948417, "vram_gb": 6.442267648, "lambda": 0.30502159986644983, "lambda_min": 0.08334346860647202, "lambda_max": 0.6750888228416443, "S_mean": 0.3355305989583333, "G_mean": 0.35188748586612445, "G_std": 0.15925108552134284, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.3985655307769775, "grad_norm": 0.9681062698364258, "tokens_seen": 93888512, "tokens_per_sec": 15029.432997593121, "vram_gb": 6.442267648, "lambda": 0.3051067106425762, "lambda_min": 0.08332520723342896, "lambda_max": 0.6755267381668091, "S_mean": 0.3302510579427083, "G_mean": 0.3561136273977657, "G_std": 0.15831331574978927, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.2707130908966064, "grad_norm": 1.0176981687545776, "tokens_seen": 94052352, "tokens_per_sec": 15029.742377242135, "vram_gb": 6.442267648, "lambda": 0.30518741967777413, "lambda_min": 0.08331988751888275, "lambda_max": 0.6757336854934692, "S_mean": 0.3329264322916667, "G_mean": 0.3505121430692573, "G_std": 0.15776871637596437, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.287501096725464, "grad_norm": 0.9774022698402405, "tokens_seen": 94216192, "tokens_per_sec": 15030.156238732914, "vram_gb": 6.442267648, "lambda": 0.3052515930806597, "lambda_min": 0.08332601934671402, "lambda_max": 0.6759355664253235, "S_mean": 0.3222249348958333, "G_mean": 0.34763204368452233, "G_std": 0.15899080701638013, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.403920269012451, "val_ppl": 81.77080468932886, "lambda": 0.30525484929482144, "lambda_min": 0.08332601934671402, "lambda_max": 0.6759423613548279, "lambda_per_layer": [0.1828436553478241, 0.1491062343120575, 0.1552843302488327, 0.1365879476070404, 0.12093082070350647, 0.08332601934671402, 0.12131831794977188, 0.09200707077980042, 0.1601123958826065, 0.31840839982032776, 0.33115553855895996, 0.45531874895095825, 0.3232642412185669, 0.5481647849082947, 0.5445885062217712, 0.4910910725593567, 0.6759423613548279, 0.4661742150783539, 0.38808783888816833, 0.5786693096160889, 0.1950770914554596, 0.3508307635784149, 0.27562394738197327, 0.1822027713060379], "S_mean": 0.3383890787760417, "G_mean": 0.35009834294517833, "G_std": 0.15806764364242554, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.3040502071380615, "grad_norm": 0.9923292398452759, "tokens_seen": 94380032, "tokens_per_sec": 15020.774424811252, "vram_gb": 6.442267648, "lambda": 0.305330343854924, "lambda_min": 0.08334142714738846, "lambda_max": 0.6761566996574402, "S_mean": 0.3468119303385417, "G_mean": 0.3500541648051391, "G_std": 0.1585289346597468, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.426106214523315, "grad_norm": 1.0208379030227661, "tokens_seen": 94543872, "tokens_per_sec": 15021.131535313194, "vram_gb": 6.442267648, "lambda": 0.3053871961310506, "lambda_min": 0.08335080742835999, "lambda_max": 0.6764262914657593, "S_mean": 0.3286488850911458, "G_mean": 0.3533576432770739, "G_std": 0.15751920850016177, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.373720169067383, "grad_norm": 1.035332202911377, "tokens_seen": 94707712, "tokens_per_sec": 15021.55491029081, "vram_gb": 6.442267648, "lambda": 0.3054633044327299, "lambda_min": 0.08333555608987808, "lambda_max": 0.6767910122871399, "S_mean": 0.3341573079427083, "G_mean": 0.35542486468330026, "G_std": 0.15824882115703076, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.1795642375946045, "grad_norm": 0.9666481614112854, "tokens_seen": 94871552, "tokens_per_sec": 15021.98890662255, "vram_gb": 6.442267648, "lambda": 0.3055233849833409, "lambda_min": 0.08333349227905273, "lambda_max": 0.6769942045211792, "S_mean": 0.3282470703125, "G_mean": 0.3486354884225875, "G_std": 0.15745838859584183, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.310275316238403, "grad_norm": 0.98149174451828, "tokens_seen": 95035392, "tokens_per_sec": 15022.514983934689, "vram_gb": 6.442267648, "lambda": 0.3055758196860552, "lambda_min": 0.08331960439682007, "lambda_max": 0.6771689653396606, "S_mean": 0.3522847493489583, "G_mean": 0.3596073631973316, "G_std": 0.15710311282115677, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.37546181678772, "grad_norm": 0.9882040619850159, "tokens_seen": 95199232, "tokens_per_sec": 15023.094025520291, "vram_gb": 6.442267648, "lambda": 0.3056767834350467, "lambda_min": 0.08332156389951706, "lambda_max": 0.6774250268936157, "S_mean": 0.3309224446614583, "G_mean": 0.3549321588749687, "G_std": 0.15612016843321422, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.141449928283691, "grad_norm": 0.9951784610748291, "tokens_seen": 95363072, "tokens_per_sec": 15023.520566318057, "vram_gb": 6.442267648, "lambda": 0.30575469974428415, "lambda_min": 0.08333221822977066, "lambda_max": 0.6777811050415039, "S_mean": 0.3348897298177083, "G_mean": 0.35379307996481657, "G_std": 0.15840368814921627, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.28071928024292, "grad_norm": 0.9761941432952881, "tokens_seen": 95526912, "tokens_per_sec": 15023.86188409408, "vram_gb": 6.442267648, "lambda": 0.30581848820050556, "lambda_min": 0.083331398665905, "lambda_max": 0.6780989170074463, "S_mean": 0.324859619140625, "G_mean": 0.3464067072297136, "G_std": 0.15600911562796682, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.2597761154174805, "grad_norm": 0.9265913963317871, "tokens_seen": 95690752, "tokens_per_sec": 15024.222238483399, "vram_gb": 6.442267648, "lambda": 0.30589383747428656, "lambda_min": 0.08332061022520065, "lambda_max": 0.678380012512207, "S_mean": 0.3304036458333333, "G_mean": 0.3537711181367437, "G_std": 0.15845760854426771, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.2681732177734375, "grad_norm": 1.0330095291137695, "tokens_seen": 95854592, "tokens_per_sec": 15024.655326196984, "vram_gb": 6.442267648, "lambda": 0.30599048392226297, "lambda_min": 0.08331850171089172, "lambda_max": 0.6787547469139099, "S_mean": 0.3464457194010417, "G_mean": 0.35530768893659115, "G_std": 0.157514330620567, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.245497703552246, "grad_norm": 1.0315396785736084, "tokens_seen": 96018432, "tokens_per_sec": 15025.173728699874, "vram_gb": 6.442267648, "lambda": 0.30607374105602503, "lambda_min": 0.08334863185882568, "lambda_max": 0.6790668964385986, "S_mean": 0.3274943033854167, "G_mean": 0.35104059559913975, "G_std": 0.15778590979365012, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.424045085906982, "grad_norm": 1.013454556465149, "tokens_seen": 96182272, "tokens_per_sec": 15025.542733744303, "vram_gb": 6.442267648, "lambda": 0.30613534307728213, "lambda_min": 0.08335690945386887, "lambda_max": 0.6792824864387512, "S_mean": 0.3289693196614583, "G_mean": 0.3536324218536417, "G_std": 0.1583314842234055, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.355119228363037, "grad_norm": 1.0443696975708008, "tokens_seen": 96346112, "tokens_per_sec": 15025.900674082919, "vram_gb": 6.442267648, "lambda": 0.306194850243628, "lambda_min": 0.08338215947151184, "lambda_max": 0.6794981956481934, "S_mean": 0.3342386881510417, "G_mean": 0.3544285321452965, "G_std": 0.15962397198503217, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.358512878417969, "grad_norm": 1.0114247798919678, "tokens_seen": 96509952, "tokens_per_sec": 15026.333391172906, "vram_gb": 6.442267648, "lambda": 0.3062613724420468, "lambda_min": 0.0833992287516594, "lambda_max": 0.6796110272407532, "S_mean": 0.322418212890625, "G_mean": 0.34824644681066275, "G_std": 0.1570336721294249, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.3860650062561035, "grad_norm": 1.0018799304962158, "tokens_seen": 96673792, "tokens_per_sec": 15026.718475446634, "vram_gb": 6.442267648, "lambda": 0.3063259053354462, "lambda_min": 0.08338581770658493, "lambda_max": 0.679856538772583, "S_mean": 0.338165283203125, "G_mean": 0.35595441741558415, "G_std": 0.1579082493747895, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.286069393157959, "grad_norm": 0.9996907711029053, "tokens_seen": 96837632, "tokens_per_sec": 15027.193605571036, "vram_gb": 6.442267648, "lambda": 0.3063966992000739, "lambda_min": 0.08339504897594452, "lambda_max": 0.680008590221405, "S_mean": 0.3416748046875, "G_mean": 0.3542495178990066, "G_std": 0.15862268512137234, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.329220533370972, "grad_norm": 0.959978461265564, "tokens_seen": 97001472, "tokens_per_sec": 15027.628181122962, "vram_gb": 6.442267648, "lambda": 0.30647587186346453, "lambda_min": 0.08340640366077423, "lambda_max": 0.6802821755409241, "S_mean": 0.3192952473958333, "G_mean": 0.3518916078998397, "G_std": 0.157415184386385, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.129626035690308, "grad_norm": 0.9589694142341614, "tokens_seen": 97165312, "tokens_per_sec": 15028.062652201175, "vram_gb": 6.442267648, "lambda": 0.30654072389006615, "lambda_min": 0.08340234309434891, "lambda_max": 0.6805582046508789, "S_mean": 0.325286865234375, "G_mean": 0.34820271912030876, "G_std": 0.15732862489918867, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.4761645793914795, "grad_norm": 1.0960824489593506, "tokens_seen": 97329152, "tokens_per_sec": 15028.456818110293, "vram_gb": 6.442267648, "lambda": 0.3066139652704199, "lambda_min": 0.08341453224420547, "lambda_max": 0.680912971496582, "S_mean": 0.3395487467447917, "G_mean": 0.35586218621271354, "G_std": 0.1571919491592174, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.278066158294678, "grad_norm": 1.0623502731323242, "tokens_seen": 97492992, "tokens_per_sec": 15028.850158115934, "vram_gb": 6.442267648, "lambda": 0.3066550896813472, "lambda_min": 0.08338942378759384, "lambda_max": 0.6811544299125671, "S_mean": 0.32745361328125, "G_mean": 0.34946974491079646, "G_std": 0.1583287553706517, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.4473326206207275, "grad_norm": 1.066902756690979, "tokens_seen": 97656832, "tokens_per_sec": 15029.147004681163, "vram_gb": 6.442267648, "lambda": 0.3067334496105711, "lambda_min": 0.08338116854429245, "lambda_max": 0.6813886165618896, "S_mean": 0.3386433919270833, "G_mean": 0.3539520923513919, "G_std": 0.15860298675640175, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.141179800033569, "grad_norm": 1.0289204120635986, "tokens_seen": 97820672, "tokens_per_sec": 15029.594934745912, "vram_gb": 6.442267648, "lambda": 0.3068108210961024, "lambda_min": 0.08338465541601181, "lambda_max": 0.6816624999046326, "S_mean": 0.325439453125, "G_mean": 0.34945884401289123, "G_std": 0.15791437918475518, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.329986333847046, "grad_norm": 1.029549479484558, "tokens_seen": 97984512, "tokens_per_sec": 15029.933723478734, "vram_gb": 6.442267648, "lambda": 0.30687362390259904, "lambda_min": 0.08336734026670456, "lambda_max": 0.6818753480911255, "S_mean": 0.3274027506510417, "G_mean": 0.35756496984201175, "G_std": 0.15795296678940454, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.22503137588501, "grad_norm": 1.0157960653305054, "tokens_seen": 98148352, "tokens_per_sec": 15030.36823407994, "vram_gb": 6.442267648, "lambda": 0.3069125845407446, "lambda_min": 0.08335445076227188, "lambda_max": 0.6819084286689758, "S_mean": 0.3207906087239583, "G_mean": 0.3505655957075457, "G_std": 0.15715195195904622, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.354956531524659, "val_ppl": 77.86344065821494, "lambda": 0.30699489296724397, "lambda_min": 0.08336106687784195, "lambda_max": 0.6821404099464417, "lambda_per_layer": [0.18368050456047058, 0.14907410740852356, 0.1557440608739853, 0.13654100894927979, 0.12089323997497559, 0.08336106687784195, 0.12130295485258102, 0.0919935330748558, 0.16054677963256836, 0.3192470967769623, 0.33227404952049255, 0.456614226102829, 0.3248228132724762, 0.550731360912323, 0.5483548045158386, 0.4943457841873169, 0.6821404099464417, 0.46933409571647644, 0.3913608193397522, 0.584300696849823, 0.19613611698150635, 0.35359662771224976, 0.27814197540283203, 0.18333929777145386], "S_mean": 0.3415120442708333, "G_mean": 0.34948439166570705, "G_std": 0.15708909629999349, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 6544.294906139374, "nan_failures": 0} diff --git a/full_s1/config.json b/full_s1/config.json new file mode 100644 index 0000000000000000000000000000000000000000..1cac81f4278399b6d5bfc1c44d59f01bdac8732b --- /dev/null +++ b/full_s1/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 1, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s1", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1001340, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/full_s1/final.pt b/full_s1/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..c7dad22a63e11b890007f40e431c07d34326d954 --- /dev/null +++ b/full_s1/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e653be057f08c2c10bb1be3ee3633ef3c490f33dab83e9a3891339327c80f10 +size 256003111 diff --git a/full_s1/metrics.jsonl b/full_s1/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..1dc40feb4d05d52bc42cde7e7903fac5c1ed46c3 --- /dev/null +++ b/full_s1/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 1, "config": {"variant": "full", "seed": 1, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s1", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1001340, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.902008056640625, "grad_norm": 10.458535194396973, "tokens_seen": 8192, "tokens_per_sec": 7237.359036404446, "vram_gb": 5.920787456, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.1737213134765625, "G_mean": 0.4995858383675416, "G_std": 0.0870530813311537, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.759885787963867, "grad_norm": 2.1085288524627686, "tokens_seen": 172032, "tokens_per_sec": 14555.153879684265, "vram_gb": 6.442267648, "lambda": 0.1269188163181146, "lambda_min": 0.12686657905578613, "lambda_max": 0.12696336209774017, "S_mean": 0.45173899332682294, "G_mean": 0.4347797520458698, "G_std": 0.06715619533012311, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.124763488769531, "grad_norm": 1.626686692237854, "tokens_seen": 335872, "tokens_per_sec": 14995.19443647184, "vram_gb": 6.442267648, "lambda": 0.12684423414369425, "lambda_min": 0.12674367427825928, "lambda_max": 0.12693795561790466, "S_mean": 0.4795735677083333, "G_mean": 0.41958451891938847, "G_std": 0.07167885677578549, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.239702224731445, "grad_norm": 1.4519938230514526, "tokens_seen": 499712, "tokens_per_sec": 15142.869275575269, "vram_gb": 6.442267648, "lambda": 0.12674915107587972, "lambda_min": 0.12664858996868134, "lambda_max": 0.12695536017417908, "S_mean": 0.5074462890625, "G_mean": 0.4125487109025319, "G_std": 0.07927682460285723, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.390436172485352, "grad_norm": 0.8131384253501892, "tokens_seen": 663552, "tokens_per_sec": 15216.108802075, "vram_gb": 6.442267648, "lambda": 0.12662707827985287, "lambda_min": 0.1264747530221939, "lambda_max": 0.12705518305301666, "S_mean": 0.3780314127604167, "G_mean": 0.4209214635193348, "G_std": 0.09133562880257766, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.279054641723633, "grad_norm": 0.6336538791656494, "tokens_seen": 827392, "tokens_per_sec": 15239.340657587414, "vram_gb": 6.442267648, "lambda": 0.12651526803771654, "lambda_min": 0.12619748711585999, "lambda_max": 0.12712688744068146, "S_mean": 0.24797566731770834, "G_mean": 0.42261641720930737, "G_std": 0.10049366671591997, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.060963869094849, "grad_norm": 0.6534080505371094, "tokens_seen": 991232, "tokens_per_sec": 15256.087461336723, "vram_gb": 6.442267648, "lambda": 0.1264491273711125, "lambda_min": 0.12601613998413086, "lambda_max": 0.1272624433040619, "S_mean": 0.23869832356770834, "G_mean": 0.4161597763498624, "G_std": 0.11152348791559537, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.800248622894287, "grad_norm": 0.5962034463882446, "tokens_seen": 1155072, "tokens_per_sec": 15273.740396391884, "vram_gb": 6.442267648, "lambda": 0.12636985008915266, "lambda_min": 0.12582242488861084, "lambda_max": 0.12731435894966125, "S_mean": 0.24042765299479166, "G_mean": 0.4149595449368159, "G_std": 0.11406852258369327, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.781864881515503, "grad_norm": 0.5337608456611633, "tokens_seen": 1318912, "tokens_per_sec": 15283.732286381024, "vram_gb": 6.442267648, "lambda": 0.1263305749744177, "lambda_min": 0.12561406195163727, "lambda_max": 0.12733803689479828, "S_mean": 0.3123982747395833, "G_mean": 0.4113170268634955, "G_std": 0.13185743615031242, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.7422263622283936, "grad_norm": 0.536812961101532, "tokens_seen": 1482752, "tokens_per_sec": 15288.6306833607, "vram_gb": 6.442267648, "lambda": 0.1262933313846588, "lambda_min": 0.12552528083324432, "lambda_max": 0.12737815082073212, "S_mean": 0.2692972819010417, "G_mean": 0.39049290244777995, "G_std": 0.13048217802618942, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.59514045715332, "grad_norm": 0.5560492873191833, "tokens_seen": 1646592, "tokens_per_sec": 15295.002183555602, "vram_gb": 6.442267648, "lambda": 0.12629335994521776, "lambda_min": 0.12547916173934937, "lambda_max": 0.12735527753829956, "S_mean": 0.2747904459635417, "G_mean": 0.39893319581945735, "G_std": 0.14778481361766657, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.611188888549805, "grad_norm": 0.5373011827468872, "tokens_seen": 1810432, "tokens_per_sec": 15303.167489402884, "vram_gb": 6.442267648, "lambda": 0.1262515423198541, "lambda_min": 0.12529903650283813, "lambda_max": 0.12749622762203217, "S_mean": 0.2760416666666667, "G_mean": 0.39553646246592206, "G_std": 0.1492458862873415, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.6158154010772705, "grad_norm": 0.5015355348587036, "tokens_seen": 1974272, "tokens_per_sec": 15306.846470981187, "vram_gb": 6.442267648, "lambda": 0.126265499740839, "lambda_min": 0.12527307868003845, "lambda_max": 0.1275714784860611, "S_mean": 0.3050130208333333, "G_mean": 0.38961820552746457, "G_std": 0.15951658971607685, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.62297248840332, "grad_norm": 0.5313244462013245, "tokens_seen": 2138112, "tokens_per_sec": 15312.035622590207, "vram_gb": 6.442267648, "lambda": 0.1262509444107612, "lambda_min": 0.1252162605524063, "lambda_max": 0.12768524885177612, "S_mean": 0.2765401204427083, "G_mean": 0.39975054810444516, "G_std": 0.16564717112729946, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.698387145996094, "grad_norm": 0.5219796299934387, "tokens_seen": 2301952, "tokens_per_sec": 15315.019534347377, "vram_gb": 6.442267648, "lambda": 0.12626273992160955, "lambda_min": 0.12512144446372986, "lambda_max": 0.12782523036003113, "S_mean": 0.2839457194010417, "G_mean": 0.4032938467959563, "G_std": 0.16468698003639778, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.544940948486328, "grad_norm": 0.5026320219039917, "tokens_seen": 2465792, "tokens_per_sec": 15317.873131206426, "vram_gb": 6.442267648, "lambda": 0.12629778248568377, "lambda_min": 0.12513789534568787, "lambda_max": 0.12799771130084991, "S_mean": 0.277923583984375, "G_mean": 0.39266059547662735, "G_std": 0.16987673565745354, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.467379331588745, "grad_norm": 0.46692702174186707, "tokens_seen": 2629632, "tokens_per_sec": 15320.63278954088, "vram_gb": 6.442267648, "lambda": 0.1263255880524715, "lambda_min": 0.12503813207149506, "lambda_max": 0.12835541367530823, "S_mean": 0.2949625651041667, "G_mean": 0.4033467322587967, "G_std": 0.1795940383647879, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.485783576965332, "grad_norm": 0.465829998254776, "tokens_seen": 2793472, "tokens_per_sec": 15323.30336530619, "vram_gb": 6.442267648, "lambda": 0.12631176691502333, "lambda_min": 0.12494877725839615, "lambda_max": 0.1284661740064621, "S_mean": 0.2724711100260417, "G_mean": 0.41551605239510536, "G_std": 0.18526445422321558, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.23491096496582, "grad_norm": 0.477444589138031, "tokens_seen": 2957312, "tokens_per_sec": 15323.310016079224, "vram_gb": 6.442267648, "lambda": 0.12636415101587772, "lambda_min": 0.1250329464673996, "lambda_max": 0.1287025362253189, "S_mean": 0.33416748046875, "G_mean": 0.3895490989089012, "G_std": 0.18203202076256275, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.3028037548065186, "grad_norm": 0.46125364303588867, "tokens_seen": 3121152, "tokens_per_sec": 15322.623823582768, "vram_gb": 6.442267648, "lambda": 0.12646240182220936, "lambda_min": 0.12504197657108307, "lambda_max": 0.12908606231212616, "S_mean": 0.301239013671875, "G_mean": 0.4022495821118355, "G_std": 0.19193154449264208, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.262359142303467, "grad_norm": 0.471969872713089, "tokens_seen": 3284992, "tokens_per_sec": 15324.203726571814, "vram_gb": 6.442267648, "lambda": 0.12657801620662212, "lambda_min": 0.12490208446979523, "lambda_max": 0.1294812709093094, "S_mean": 0.2967427571614583, "G_mean": 0.4217920737961928, "G_std": 0.20208207331597805, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.360418319702148, "grad_norm": 0.45663315057754517, "tokens_seen": 3448832, "tokens_per_sec": 15327.138677204064, "vram_gb": 6.442267648, "lambda": 0.1266706862176458, "lambda_min": 0.12497340887784958, "lambda_max": 0.12986528873443604, "S_mean": 0.34686279296875, "G_mean": 0.408292718231678, "G_std": 0.19646399995932975, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.265358924865723, "grad_norm": 0.4997898042201996, "tokens_seen": 3612672, "tokens_per_sec": 15327.585279147455, "vram_gb": 6.442267648, "lambda": 0.126765969209373, "lambda_min": 0.12485241144895554, "lambda_max": 0.13018757104873657, "S_mean": 0.3246866861979167, "G_mean": 0.4287540478010972, "G_std": 0.20371740746001402, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.327577590942383, "grad_norm": 0.5015866160392761, "tokens_seen": 3776512, "tokens_per_sec": 15321.183185888358, "vram_gb": 6.442267648, "lambda": 0.12691457755863667, "lambda_min": 0.12464453279972076, "lambda_max": 0.1305653601884842, "S_mean": 0.3348185221354167, "G_mean": 0.4263387819131215, "G_std": 0.20772285231699547, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.197047710418701, "grad_norm": 0.4670758843421936, "tokens_seen": 3940352, "tokens_per_sec": 15325.676006718806, "vram_gb": 6.442267648, "lambda": 0.127042882454892, "lambda_min": 0.12453304976224899, "lambda_max": 0.13096025586128235, "S_mean": 0.3962809244791667, "G_mean": 0.4007243352631728, "G_std": 0.2000454245135188, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.322761297225952, "grad_norm": 0.4810853898525238, "tokens_seen": 4104192, "tokens_per_sec": 15327.272959715387, "vram_gb": 6.442267648, "lambda": 0.1271931075801452, "lambda_min": 0.12435391545295715, "lambda_max": 0.13112100958824158, "S_mean": 0.3610636393229167, "G_mean": 0.45453549176454544, "G_std": 0.2095284747580687, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.249036610126495, "val_ppl": 517.5140166705775, "lambda": 0.12719825437913337, "lambda_min": 0.12433648854494095, "lambda_max": 0.13112607598304749, "lambda_per_layer": [0.13112607598304749, 0.12811502814292908, 0.1275411695241928, 0.12842203676700592, 0.12666940689086914, 0.12681259214878082, 0.12433648854494095, 0.12645360827445984, 0.12912020087242126, 0.12793301045894623, 0.12770412862300873, 0.12782417237758636, 0.1274869441986084, 0.12888866662979126, 0.1274515986442566, 0.12701910734176636, 0.12617649137973785, 0.12586627900600433, 0.12616631388664246, 0.1260969489812851, 0.12670069932937622, 0.1251976042985916, 0.126573845744133, 0.1270756870508194], "S_mean": 0.3885701497395833, "G_mean": 0.45895810176928836, "G_std": 0.20984219387173653, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.167030572891235, "grad_norm": 0.517214834690094, "tokens_seen": 4268032, "tokens_per_sec": 15104.13255555622, "vram_gb": 6.442267648, "lambda": 0.1273673657948772, "lambda_min": 0.12425286322832108, "lambda_max": 0.13149169087409973, "S_mean": 0.3790995279947917, "G_mean": 0.46657221764326096, "G_std": 0.2162543429682652, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.253943204879761, "grad_norm": 0.48115912079811096, "tokens_seen": 4431872, "tokens_per_sec": 15110.761691186484, "vram_gb": 6.442267648, "lambda": 0.12753031154473624, "lambda_min": 0.12411439418792725, "lambda_max": 0.13186077773571014, "S_mean": 0.3548583984375, "G_mean": 0.4478190665443738, "G_std": 0.2169957390675942, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.13905143737793, "grad_norm": 0.4561431407928467, "tokens_seen": 4595712, "tokens_per_sec": 15115.975682485074, "vram_gb": 6.442267648, "lambda": 0.12767995055764914, "lambda_min": 0.12401295453310013, "lambda_max": 0.13214556872844696, "S_mean": 0.3654073079427083, "G_mean": 0.46714531257748604, "G_std": 0.2142923939973116, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.093328475952148, "grad_norm": 0.5221611261367798, "tokens_seen": 4759552, "tokens_per_sec": 15124.346516957692, "vram_gb": 6.442267648, "lambda": 0.12791364857306084, "lambda_min": 0.12395400553941727, "lambda_max": 0.13259942829608917, "S_mean": 0.41650390625, "G_mean": 0.44959703211983043, "G_std": 0.2153396662324667, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.137373208999634, "grad_norm": 0.5025668740272522, "tokens_seen": 4923392, "tokens_per_sec": 15130.572288258343, "vram_gb": 6.442267648, "lambda": 0.12814220475653806, "lambda_min": 0.12385274469852448, "lambda_max": 0.1330040544271469, "S_mean": 0.3750508626302083, "G_mean": 0.49712130924065906, "G_std": 0.2167811126758655, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.241998672485352, "grad_norm": 0.4645451009273529, "tokens_seen": 5087232, "tokens_per_sec": 15132.348607949612, "vram_gb": 6.442267648, "lambda": 0.12846932963778576, "lambda_min": 0.12373156100511551, "lambda_max": 0.1335141807794571, "S_mean": 0.3756001790364583, "G_mean": 0.5221152193844318, "G_std": 0.21520147348443666, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.234457969665527, "grad_norm": 0.5295551419258118, "tokens_seen": 5251072, "tokens_per_sec": 15129.97450888231, "vram_gb": 6.442267648, "lambda": 0.12868615301946798, "lambda_min": 0.12349596619606018, "lambda_max": 0.1338622272014618, "S_mean": 0.395721435546875, "G_mean": 0.48895440623164177, "G_std": 0.21096296577403942, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.966493129730225, "grad_norm": 0.4947061836719513, "tokens_seen": 5414912, "tokens_per_sec": 15117.120262634178, "vram_gb": 6.442267648, "lambda": 0.1289492187400659, "lambda_min": 0.12337572872638702, "lambda_max": 0.13425667583942413, "S_mean": 0.3953043619791667, "G_mean": 0.5151641977330049, "G_std": 0.2084288944800695, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.0000669956207275, "grad_norm": 0.5240530967712402, "tokens_seen": 5578752, "tokens_per_sec": 15109.651491532108, "vram_gb": 6.442267648, "lambda": 0.12923613532135883, "lambda_min": 0.12306613475084305, "lambda_max": 0.1345335841178894, "S_mean": 0.3792826334635417, "G_mean": 0.4942400325089693, "G_std": 0.21215920336544514, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.986919403076172, "grad_norm": 0.4835030138492584, "tokens_seen": 5742592, "tokens_per_sec": 15100.56498542625, "vram_gb": 6.442267648, "lambda": 0.12954013825704655, "lambda_min": 0.122867651283741, "lambda_max": 0.13536272943019867, "S_mean": 0.3921305338541667, "G_mean": 0.5145861580967903, "G_std": 0.20840180882563195, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.069324016571045, "grad_norm": 0.5246618390083313, "tokens_seen": 5906432, "tokens_per_sec": 15090.854833281819, "vram_gb": 6.442267648, "lambda": 0.12988313163320223, "lambda_min": 0.12279212474822998, "lambda_max": 0.1361108124256134, "S_mean": 0.3667805989583333, "G_mean": 0.5187281928956509, "G_std": 0.20563854649662971, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.984170913696289, "grad_norm": 0.480779767036438, "tokens_seen": 6070272, "tokens_per_sec": 15078.6611955654, "vram_gb": 6.442267648, "lambda": 0.130269353898863, "lambda_min": 0.12249822169542313, "lambda_max": 0.13718192279338837, "S_mean": 0.4214680989583333, "G_mean": 0.5354160778224468, "G_std": 0.20948268317927918, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.0367066860198975, "grad_norm": 0.46984338760375977, "tokens_seen": 6234112, "tokens_per_sec": 15068.805822379789, "vram_gb": 6.442267648, "lambda": 0.13060892187058926, "lambda_min": 0.12237872183322906, "lambda_max": 0.13822193443775177, "S_mean": 0.390869140625, "G_mean": 0.5487556271255016, "G_std": 0.20172605011612177, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.018792629241943, "grad_norm": 0.48747628927230835, "tokens_seen": 6397952, "tokens_per_sec": 15065.856236947222, "vram_gb": 6.442267648, "lambda": 0.13101794136067232, "lambda_min": 0.1220168024301529, "lambda_max": 0.13939671218395233, "S_mean": 0.368804931640625, "G_mean": 0.5220799595117569, "G_std": 0.19979185548921427, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.938720464706421, "grad_norm": 0.53873610496521, "tokens_seen": 6561792, "tokens_per_sec": 15063.710863601984, "vram_gb": 6.442267648, "lambda": 0.1314238477498293, "lambda_min": 0.12206117808818817, "lambda_max": 0.14055447280406952, "S_mean": 0.3710530598958333, "G_mean": 0.5579956161479155, "G_std": 0.19605815348525843, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8457112312316895, "grad_norm": 0.5460004806518555, "tokens_seen": 6725632, "tokens_per_sec": 15060.177842250336, "vram_gb": 6.442267648, "lambda": 0.1318300971761346, "lambda_min": 0.12187009304761887, "lambda_max": 0.14181506633758545, "S_mean": 0.3485514322916667, "G_mean": 0.5208019266525904, "G_std": 0.20084991306066513, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.7735655307769775, "grad_norm": 0.53466796875, "tokens_seen": 6889472, "tokens_per_sec": 15065.273848354691, "vram_gb": 6.442267648, "lambda": 0.13224542948106924, "lambda_min": 0.12166085839271545, "lambda_max": 0.1430007964372635, "S_mean": 0.3752848307291667, "G_mean": 0.5339502487331629, "G_std": 0.19450776682545742, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.6955132484436035, "grad_norm": 0.49797323346138, "tokens_seen": 7053312, "tokens_per_sec": 15070.34307432588, "vram_gb": 6.442267648, "lambda": 0.13265944675852856, "lambda_min": 0.1216290220618248, "lambda_max": 0.1442461460828781, "S_mean": 0.366180419921875, "G_mean": 0.550466320787867, "G_std": 0.18984886383016905, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.763384580612183, "grad_norm": 0.5309057235717773, "tokens_seen": 7217152, "tokens_per_sec": 15075.501206356159, "vram_gb": 6.442267648, "lambda": 0.13301804071913162, "lambda_min": 0.1212332472205162, "lambda_max": 0.14541949331760406, "S_mean": 0.357635498046875, "G_mean": 0.5376573378841082, "G_std": 0.19855209408948818, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.689276218414307, "grad_norm": 0.5035059452056885, "tokens_seen": 7380992, "tokens_per_sec": 15080.910724216652, "vram_gb": 6.442267648, "lambda": 0.13344700230906406, "lambda_min": 0.12071152776479721, "lambda_max": 0.1466432809829712, "S_mean": 0.3642781575520833, "G_mean": 0.5317534307638804, "G_std": 0.19414084063222012, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.874760389328003, "grad_norm": 0.5102369785308838, "tokens_seen": 7544832, "tokens_per_sec": 15086.356788010627, "vram_gb": 6.442267648, "lambda": 0.13387259965141615, "lambda_min": 0.12036719918251038, "lambda_max": 0.14804482460021973, "S_mean": 0.362396240234375, "G_mean": 0.5500236116349697, "G_std": 0.1894686051333944, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.773984909057617, "grad_norm": 0.5008283853530884, "tokens_seen": 7708672, "tokens_per_sec": 15092.385111525122, "vram_gb": 6.442267648, "lambda": 0.13431155091772476, "lambda_min": 0.12015143781900406, "lambda_max": 0.14938032627105713, "S_mean": 0.3674112955729167, "G_mean": 0.539446696639061, "G_std": 0.1878094496205449, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.913170099258423, "grad_norm": 0.48691290616989136, "tokens_seen": 7872512, "tokens_per_sec": 15096.442166184781, "vram_gb": 6.442267648, "lambda": 0.134713439270854, "lambda_min": 0.1197281926870346, "lambda_max": 0.15068939328193665, "S_mean": 0.3862406412760417, "G_mean": 0.5332141350954771, "G_std": 0.18531103090693554, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.628031969070435, "grad_norm": 0.5202297568321228, "tokens_seen": 8036352, "tokens_per_sec": 15100.401092392498, "vram_gb": 6.442267648, "lambda": 0.1351950994382302, "lambda_min": 0.11936236917972565, "lambda_max": 0.15197113156318665, "S_mean": 0.3470255533854167, "G_mean": 0.5231226769586405, "G_std": 0.18805518994728723, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.749140739440918, "grad_norm": 0.4804316461086273, "tokens_seen": 8200192, "tokens_per_sec": 15104.85534425908, "vram_gb": 6.442267648, "lambda": 0.13570013331870237, "lambda_min": 0.11896251142024994, "lambda_max": 0.15336281061172485, "S_mean": 0.3685506184895833, "G_mean": 0.5336024121691784, "G_std": 0.19223345909267664, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.783773362636566, "val_ppl": 324.9831590360699, "lambda": 0.13572661144038042, "lambda_min": 0.11895628273487091, "lambda_max": 0.15344001352787018, "lambda_per_layer": [0.1390264928340912, 0.13316211104393005, 0.13182660937309265, 0.13185396790504456, 0.12662744522094727, 0.12648910284042358, 0.11895628273487091, 0.12916457653045654, 0.13357500731945038, 0.1439833790063858, 0.1443495899438858, 0.15344001352787018, 0.1496298611164093, 0.15281492471694946, 0.14497995376586914, 0.13644880056381226, 0.1335911899805069, 0.13410180807113647, 0.12885664403438568, 0.1335878223180771, 0.13497568666934967, 0.13420456647872925, 0.1320439875125885, 0.1297488510608673], "S_mean": 0.3712361653645833, "G_mean": 0.5440575331449509, "G_std": 0.18847663638492426, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.667198181152344, "grad_norm": 0.537924587726593, "tokens_seen": 8364032, "tokens_per_sec": 14995.688987523696, "vram_gb": 6.442267648, "lambda": 0.1361740588520964, "lambda_min": 0.11860344558954239, "lambda_max": 0.1547372192144394, "S_mean": 0.3594970703125, "G_mean": 0.5432699968417486, "G_std": 0.1886260515699784, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.724745273590088, "grad_norm": 0.5511099100112915, "tokens_seen": 8527872, "tokens_per_sec": 15001.617341324536, "vram_gb": 6.442267648, "lambda": 0.13660040590912104, "lambda_min": 0.1183343306183815, "lambda_max": 0.15596166253089905, "S_mean": 0.3703206380208333, "G_mean": 0.5442267836381992, "G_std": 0.1880516977980733, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.776203632354736, "grad_norm": 0.5230923891067505, "tokens_seen": 8691712, "tokens_per_sec": 15008.528827844391, "vram_gb": 6.442267648, "lambda": 0.13701843253026405, "lambda_min": 0.1179705485701561, "lambda_max": 0.1572633981704712, "S_mean": 0.3420003255208333, "G_mean": 0.5248871830602487, "G_std": 0.18870886880904436, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.581274032592773, "grad_norm": 0.5307595133781433, "tokens_seen": 8855552, "tokens_per_sec": 15014.94618905837, "vram_gb": 6.442267648, "lambda": 0.13744254782795906, "lambda_min": 0.11773303151130676, "lambda_max": 0.15852424502372742, "S_mean": 0.3791097005208333, "G_mean": 0.5443419044216474, "G_std": 0.18720095915098986, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.791805982589722, "grad_norm": 0.5268341302871704, "tokens_seen": 9019392, "tokens_per_sec": 15020.35356005436, "vram_gb": 6.442267648, "lambda": 0.13791457129021487, "lambda_min": 0.11724188923835754, "lambda_max": 0.15970636904239655, "S_mean": 0.3686319986979167, "G_mean": 0.5344842790315548, "G_std": 0.18268961738795042, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.650755405426025, "grad_norm": 0.6041169166564941, "tokens_seen": 9183232, "tokens_per_sec": 15026.56416160012, "vram_gb": 6.442267648, "lambda": 0.13837703472624222, "lambda_min": 0.1168922558426857, "lambda_max": 0.1609663963317871, "S_mean": 0.3571573893229167, "G_mean": 0.5129796254138151, "G_std": 0.1825084239244461, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.684318780899048, "grad_norm": 0.4863232672214508, "tokens_seen": 9347072, "tokens_per_sec": 15031.381153866028, "vram_gb": 6.442267648, "lambda": 0.13889473676681519, "lambda_min": 0.11639101803302765, "lambda_max": 0.16237789392471313, "S_mean": 0.3493448893229167, "G_mean": 0.5187325533479452, "G_std": 0.18669264391064644, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.659592866897583, "grad_norm": 0.538981556892395, "tokens_seen": 9510912, "tokens_per_sec": 15036.552303384928, "vram_gb": 6.442267648, "lambda": 0.13932060108830532, "lambda_min": 0.11573313921689987, "lambda_max": 0.1638471484184265, "S_mean": 0.3436279296875, "G_mean": 0.5307530878732601, "G_std": 0.18330404441803694, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.702739477157593, "grad_norm": 0.5296981930732727, "tokens_seen": 9674752, "tokens_per_sec": 15042.099637335903, "vram_gb": 6.442267648, "lambda": 0.1397992434600989, "lambda_min": 0.11562216281890869, "lambda_max": 0.1651519536972046, "S_mean": 0.3734130859375, "G_mean": 0.5250659603625536, "G_std": 0.18204332298288742, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.65081262588501, "grad_norm": 0.5416917204856873, "tokens_seen": 9838592, "tokens_per_sec": 15045.532461006536, "vram_gb": 6.442267648, "lambda": 0.14024506974965334, "lambda_min": 0.11549615114927292, "lambda_max": 0.16647987067699432, "S_mean": 0.3436686197916667, "G_mean": 0.5201472174376249, "G_std": 0.17960221134126186, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.82702374458313, "grad_norm": 0.522251546382904, "tokens_seen": 10002432, "tokens_per_sec": 15049.989218574985, "vram_gb": 6.442267648, "lambda": 0.14067930790285268, "lambda_min": 0.11527208983898163, "lambda_max": 0.16780151426792145, "S_mean": 0.4000651041666667, "G_mean": 0.5142818142970403, "G_std": 0.18230339077611765, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.605147361755371, "grad_norm": 0.5459177494049072, "tokens_seen": 10166272, "tokens_per_sec": 15053.608257118927, "vram_gb": 6.442267648, "lambda": 0.1411820643891891, "lambda_min": 0.1149461567401886, "lambda_max": 0.16913814842700958, "S_mean": 0.355224609375, "G_mean": 0.5086755454540253, "G_std": 0.18506068736314774, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.588300466537476, "grad_norm": 0.568435788154602, "tokens_seen": 10330112, "tokens_per_sec": 15057.959955485276, "vram_gb": 6.442267648, "lambda": 0.14161355855564275, "lambda_min": 0.1148676723241806, "lambda_max": 0.17036540806293488, "S_mean": 0.37109375, "G_mean": 0.5008563883602619, "G_std": 0.18332367856055498, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.493361949920654, "grad_norm": 0.6325851678848267, "tokens_seen": 10493952, "tokens_per_sec": 15062.479675913391, "vram_gb": 6.442267648, "lambda": 0.14210093518098196, "lambda_min": 0.11475925147533417, "lambda_max": 0.1717897206544876, "S_mean": 0.3339335123697917, "G_mean": 0.5111561908076206, "G_std": 0.18292257903764644, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.596926212310791, "grad_norm": 0.5158584117889404, "tokens_seen": 10657792, "tokens_per_sec": 15066.83639448327, "vram_gb": 6.442267648, "lambda": 0.14261136731753746, "lambda_min": 0.11429593712091446, "lambda_max": 0.1732843667268753, "S_mean": 0.34918212890625, "G_mean": 0.49679382890462875, "G_std": 0.1849870284398397, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.472986936569214, "grad_norm": 0.5261136293411255, "tokens_seen": 10821632, "tokens_per_sec": 15069.60410355178, "vram_gb": 6.442267648, "lambda": 0.14309041605641445, "lambda_min": 0.11395924538373947, "lambda_max": 0.17447471618652344, "S_mean": 0.342071533203125, "G_mean": 0.49579310851792496, "G_std": 0.18544119037687778, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.719213485717773, "grad_norm": 0.6206471920013428, "tokens_seen": 10985472, "tokens_per_sec": 15073.867890965055, "vram_gb": 6.442267648, "lambda": 0.14360974449664354, "lambda_min": 0.11367636173963547, "lambda_max": 0.17578409612178802, "S_mean": 0.3835652669270833, "G_mean": 0.48786865050594014, "G_std": 0.17986944721390805, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.550861835479736, "grad_norm": 0.5561574101448059, "tokens_seen": 11149312, "tokens_per_sec": 15078.340587300987, "vram_gb": 6.442267648, "lambda": 0.144102250225842, "lambda_min": 0.1131429597735405, "lambda_max": 0.17735576629638672, "S_mean": 0.3496297200520833, "G_mean": 0.49437601305544376, "G_std": 0.18139217731853327, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.466156721115112, "grad_norm": 0.5311726927757263, "tokens_seen": 11313152, "tokens_per_sec": 15082.12011372401, "vram_gb": 6.442267648, "lambda": 0.1445565310617288, "lambda_min": 0.11291098594665527, "lambda_max": 0.17865322530269623, "S_mean": 0.3573404947916667, "G_mean": 0.4992907239745061, "G_std": 0.18202816663930813, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.673633575439453, "grad_norm": 0.5441731810569763, "tokens_seen": 11476992, "tokens_per_sec": 15085.876351391526, "vram_gb": 6.442267648, "lambda": 0.14502672757953405, "lambda_min": 0.1127457395195961, "lambda_max": 0.1799522340297699, "S_mean": 0.3671875, "G_mean": 0.48423727105061215, "G_std": 0.18476627053072056, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.4427502155303955, "grad_norm": 0.5441908836364746, "tokens_seen": 11640832, "tokens_per_sec": 15088.615566079818, "vram_gb": 6.442267648, "lambda": 0.14550234439472357, "lambda_min": 0.11268433928489685, "lambda_max": 0.18138758838176727, "S_mean": 0.3564249674479167, "G_mean": 0.47503861722846824, "G_std": 0.1865861687498788, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.568061351776123, "grad_norm": 0.5296436548233032, "tokens_seen": 11804672, "tokens_per_sec": 15091.895851807913, "vram_gb": 6.442267648, "lambda": 0.14598066949596009, "lambda_min": 0.11238870769739151, "lambda_max": 0.1829061508178711, "S_mean": 0.34185791015625, "G_mean": 0.49535263205568, "G_std": 0.1846285288532575, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.5947020053863525, "grad_norm": 0.6018644571304321, "tokens_seen": 11968512, "tokens_per_sec": 15094.91277487437, "vram_gb": 6.442267648, "lambda": 0.1464797913407286, "lambda_min": 0.1119563952088356, "lambda_max": 0.1842634528875351, "S_mean": 0.37396240234375, "G_mean": 0.48866956618924934, "G_std": 0.1814192154755195, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.48592472076416, "grad_norm": 0.537384569644928, "tokens_seen": 12132352, "tokens_per_sec": 15098.250388548477, "vram_gb": 6.442267648, "lambda": 0.1469409835214416, "lambda_min": 0.1114950105547905, "lambda_max": 0.18563689291477203, "S_mean": 0.36480712890625, "G_mean": 0.4797961926087737, "G_std": 0.1844734555731217, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.292418718338013, "grad_norm": 0.6419336199760437, "tokens_seen": 12296192, "tokens_per_sec": 15100.509689069993, "vram_gb": 6.442267648, "lambda": 0.14740437424431244, "lambda_min": 0.11098974198102951, "lambda_max": 0.18708913028240204, "S_mean": 0.3444010416666667, "G_mean": 0.49094694790740806, "G_std": 0.18544593391319117, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.499758696556091, "val_ppl": 244.63289438157693, "lambda": 0.14742931226889291, "lambda_min": 0.11096426844596863, "lambda_max": 0.18713749945163727, "lambda_per_layer": [0.14345543086528778, 0.13494305312633514, 0.1330583393573761, 0.13260795176029205, 0.12661640346050262, 0.1259779930114746, 0.11096426844596863, 0.13005010783672333, 0.13626503944396973, 0.16991858184337616, 0.16727417707443237, 0.1847391277551651, 0.1768326759338379, 0.18713749945163727, 0.17057953774929047, 0.1586134135723114, 0.14843730628490448, 0.1461152881383896, 0.13582149147987366, 0.14413128793239594, 0.15131811797618866, 0.14957256615161896, 0.13876986503601074, 0.1351039707660675], "S_mean": 0.3859456380208333, "G_mean": 0.48634490619103116, "G_std": 0.18502930086106062, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.485316514968872, "grad_norm": 0.5714280009269714, "tokens_seen": 12460032, "tokens_per_sec": 15029.224295903105, "vram_gb": 6.442267648, "lambda": 0.14792320939401785, "lambda_min": 0.1105896532535553, "lambda_max": 0.18837612867355347, "S_mean": 0.34539794921875, "G_mean": 0.49411463861664134, "G_std": 0.18801625507573286, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.468908071517944, "grad_norm": 0.5755279660224915, "tokens_seen": 12623872, "tokens_per_sec": 15033.917088255013, "vram_gb": 6.442267648, "lambda": 0.14836191199719906, "lambda_min": 0.11006532609462738, "lambda_max": 0.1897186040878296, "S_mean": 0.3673502604166667, "G_mean": 0.4858884538213412, "G_std": 0.18426146125420928, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.470731258392334, "grad_norm": 0.5601330995559692, "tokens_seen": 12787712, "tokens_per_sec": 15037.07803573567, "vram_gb": 6.442267648, "lambda": 0.14881866735716662, "lambda_min": 0.10946895182132721, "lambda_max": 0.191091388463974, "S_mean": 0.3436686197916667, "G_mean": 0.46717609899739426, "G_std": 0.1821450898423791, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.154462814331055, "grad_norm": 0.5286864638328552, "tokens_seen": 12951552, "tokens_per_sec": 15036.418767635703, "vram_gb": 6.442267648, "lambda": 0.14933893146614233, "lambda_min": 0.10871551930904388, "lambda_max": 0.19238059222698212, "S_mean": 0.3560384114583333, "G_mean": 0.4865201183905204, "G_std": 0.18206843826919794, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.238428354263306, "grad_norm": 0.5678160786628723, "tokens_seen": 13115392, "tokens_per_sec": 15035.21179778085, "vram_gb": 6.442267648, "lambda": 0.14979441122462353, "lambda_min": 0.10860412567853928, "lambda_max": 0.19383341073989868, "S_mean": 0.37432861328125, "G_mean": 0.45702402604122955, "G_std": 0.1832351287206014, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.242980480194092, "grad_norm": 0.5800874829292297, "tokens_seen": 13279232, "tokens_per_sec": 15034.863941381755, "vram_gb": 6.442267648, "lambda": 0.15030948352068663, "lambda_min": 0.10865028947591782, "lambda_max": 0.19513770937919617, "S_mean": 0.36181640625, "G_mean": 0.4778065225109458, "G_std": 0.1849853388654689, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.245288133621216, "grad_norm": 0.5519765615463257, "tokens_seen": 13443072, "tokens_per_sec": 15035.517792377394, "vram_gb": 6.442267648, "lambda": 0.15086202509701252, "lambda_min": 0.1084354817867279, "lambda_max": 0.1965719312429428, "S_mean": 0.3595377604166667, "G_mean": 0.46451487361143035, "G_std": 0.18310711113736033, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.379663944244385, "grad_norm": 0.6075241565704346, "tokens_seen": 13606912, "tokens_per_sec": 15036.710853005952, "vram_gb": 6.442267648, "lambda": 0.1513464174543818, "lambda_min": 0.10792446881532669, "lambda_max": 0.19831936061382294, "S_mean": 0.4187418619791667, "G_mean": 0.46604346701254445, "G_std": 0.1877673463895917, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.190176486968994, "grad_norm": 0.5484301447868347, "tokens_seen": 13770752, "tokens_per_sec": 15037.584405092091, "vram_gb": 6.442267648, "lambda": 0.15187862142920494, "lambda_min": 0.10748325288295746, "lambda_max": 0.20003266632556915, "S_mean": 0.3624267578125, "G_mean": 0.46669851150363684, "G_std": 0.17983418966953954, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.277570009231567, "grad_norm": 0.5344012975692749, "tokens_seen": 13934592, "tokens_per_sec": 15037.678357862385, "vram_gb": 6.442267648, "lambda": 0.1523896176367998, "lambda_min": 0.106999471783638, "lambda_max": 0.20156343281269073, "S_mean": 0.3642374674479167, "G_mean": 0.46310600141684216, "G_std": 0.18431306288888058, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.341869831085205, "grad_norm": 0.5438724756240845, "tokens_seen": 14098432, "tokens_per_sec": 15039.207786749539, "vram_gb": 6.442267648, "lambda": 0.15285915229469538, "lambda_min": 0.10653731226921082, "lambda_max": 0.20297901332378387, "S_mean": 0.3601888020833333, "G_mean": 0.4667445082838337, "G_std": 0.17911538605888686, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.274724006652832, "grad_norm": 0.5560708045959473, "tokens_seen": 14262272, "tokens_per_sec": 15040.688706192586, "vram_gb": 6.442267648, "lambda": 0.1533452638735374, "lambda_min": 0.10619840025901794, "lambda_max": 0.2043808251619339, "S_mean": 0.38555908203125, "G_mean": 0.4592539019261797, "G_std": 0.17878653279816112, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.321974277496338, "grad_norm": 0.5663159489631653, "tokens_seen": 14426112, "tokens_per_sec": 15041.017096124237, "vram_gb": 6.442267648, "lambda": 0.15379787981510162, "lambda_min": 0.10591667890548706, "lambda_max": 0.2054879069328308, "S_mean": 0.3637898763020833, "G_mean": 0.4685449091096719, "G_std": 0.1850039528993269, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.325016021728516, "grad_norm": 0.5494656562805176, "tokens_seen": 14589952, "tokens_per_sec": 15040.853929462854, "vram_gb": 6.442267648, "lambda": 0.15428432046125332, "lambda_min": 0.10553719848394394, "lambda_max": 0.20680949091911316, "S_mean": 0.3665873209635417, "G_mean": 0.4611059765641888, "G_std": 0.18095755903050303, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.421743631362915, "grad_norm": 0.6243599057197571, "tokens_seen": 14753792, "tokens_per_sec": 15041.382076005493, "vram_gb": 6.442267648, "lambda": 0.1548013056938847, "lambda_min": 0.10514430701732635, "lambda_max": 0.20825324952602386, "S_mean": 0.3438313802083333, "G_mean": 0.4686073859532674, "G_std": 0.18202187803884348, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.422279596328735, "grad_norm": 0.5635290145874023, "tokens_seen": 14917632, "tokens_per_sec": 15042.46542207115, "vram_gb": 6.442267648, "lambda": 0.15531393947700659, "lambda_min": 0.10493044555187225, "lambda_max": 0.20998647809028625, "S_mean": 0.36785888671875, "G_mean": 0.45914786743621033, "G_std": 0.18110557878389955, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.1764702796936035, "grad_norm": 0.5985127687454224, "tokens_seen": 15081472, "tokens_per_sec": 15043.393049095414, "vram_gb": 6.442267648, "lambda": 0.1557793046037356, "lambda_min": 0.10448956489562988, "lambda_max": 0.21139582991600037, "S_mean": 0.36871337890625, "G_mean": 0.4511067798982064, "G_std": 0.18200095432500044, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.271736145019531, "grad_norm": 0.6226722002029419, "tokens_seen": 15245312, "tokens_per_sec": 15043.08435835922, "vram_gb": 6.442267648, "lambda": 0.15625694238891205, "lambda_min": 0.10388105362653732, "lambda_max": 0.2130461037158966, "S_mean": 0.36285400390625, "G_mean": 0.45839044141272706, "G_std": 0.1770147249723474, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.124060392379761, "grad_norm": 0.583362340927124, "tokens_seen": 15409152, "tokens_per_sec": 15043.733281040992, "vram_gb": 6.442267648, "lambda": 0.15679655193040767, "lambda_min": 0.10336490720510483, "lambda_max": 0.21477340161800385, "S_mean": 0.33831787109375, "G_mean": 0.45472002153595287, "G_std": 0.17712325463071465, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.349467515945435, "grad_norm": 0.5867902636528015, "tokens_seen": 15572992, "tokens_per_sec": 15044.140662291478, "vram_gb": 6.442267648, "lambda": 0.15727647269765535, "lambda_min": 0.10290879756212234, "lambda_max": 0.21625356376171112, "S_mean": 0.345947265625, "G_mean": 0.4490290687729915, "G_std": 0.1809350550174713, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.37505578994751, "grad_norm": 0.6170260310173035, "tokens_seen": 15736832, "tokens_per_sec": 15044.999085841988, "vram_gb": 6.442267648, "lambda": 0.15773574045548835, "lambda_min": 0.10241494327783585, "lambda_max": 0.2177678495645523, "S_mean": 0.3500569661458333, "G_mean": 0.4572247117757797, "G_std": 0.1826470885425806, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.4825356006622314, "grad_norm": 0.5601028203964233, "tokens_seen": 15900672, "tokens_per_sec": 15044.954507425271, "vram_gb": 6.442267648, "lambda": 0.15828591274718443, "lambda_min": 0.10219380259513855, "lambda_max": 0.21916772425174713, "S_mean": 0.3604939778645833, "G_mean": 0.4475600505247712, "G_std": 0.18157807954897484, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.125859498977661, "grad_norm": 0.5839141607284546, "tokens_seen": 16064512, "tokens_per_sec": 15046.557732200792, "vram_gb": 6.442267648, "lambda": 0.1587673577790459, "lambda_min": 0.10215974599123001, "lambda_max": 0.2207261025905609, "S_mean": 0.3520100911458333, "G_mean": 0.43393760329733294, "G_std": 0.17996295650179187, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.251519203186035, "grad_norm": 0.5999373197555542, "tokens_seen": 16228352, "tokens_per_sec": 15047.727814445934, "vram_gb": 6.442267648, "lambda": 0.15932540564487377, "lambda_min": 0.10196378082036972, "lambda_max": 0.22247889637947083, "S_mean": 0.3408610026041667, "G_mean": 0.4404917157565554, "G_std": 0.1793982827415069, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.3138017654418945, "grad_norm": 0.6243618726730347, "tokens_seen": 16392192, "tokens_per_sec": 15047.95269335272, "vram_gb": 6.442267648, "lambda": 0.15990498196333647, "lambda_min": 0.10163281857967377, "lambda_max": 0.22413387894630432, "S_mean": 0.3740437825520833, "G_mean": 0.44254679962371785, "G_std": 0.1833601181084911, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.339651262760162, "val_ppl": 208.44000681997935, "lambda": 0.1599373227606217, "lambda_min": 0.10160969197750092, "lambda_max": 0.22422383725643158, "lambda_per_layer": [0.14676517248153687, 0.136132150888443, 0.1341589093208313, 0.13298802077770233, 0.12617677450180054, 0.12324081361293793, 0.10160969197750092, 0.13027580082416534, 0.13610416650772095, 0.19096635282039642, 0.18973326683044434, 0.21575623750686646, 0.2043571174144745, 0.22422383725643158, 0.19957196712493896, 0.19000354409217834, 0.16660310328006744, 0.16307052969932556, 0.1454073041677475, 0.1552092432975769, 0.17109614610671997, 0.1675378382205963, 0.14655627310276031, 0.1409514844417572], "S_mean": 0.3690388997395833, "G_mean": 0.4385765877862771, "G_std": 0.1836605629262825, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.22559380531311, "grad_norm": 0.6192225813865662, "tokens_seen": 16556032, "tokens_per_sec": 14993.040273548206, "vram_gb": 6.442267648, "lambda": 0.16048197324077287, "lambda_min": 0.10118237882852554, "lambda_max": 0.22591739892959595, "S_mean": 0.3544921875, "G_mean": 0.4400143741319577, "G_std": 0.17987420549616218, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.277403831481934, "grad_norm": 0.5889644026756287, "tokens_seen": 16719872, "tokens_per_sec": 14994.287895405343, "vram_gb": 6.442267648, "lambda": 0.1609654643883308, "lambda_min": 0.10077713429927826, "lambda_max": 0.22763752937316895, "S_mean": 0.3586629231770833, "G_mean": 0.44182249748458463, "G_std": 0.17802936040485898, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.235840797424316, "grad_norm": 0.5617243051528931, "tokens_seen": 16883712, "tokens_per_sec": 14994.255277148512, "vram_gb": 6.442267648, "lambda": 0.16149225706855455, "lambda_min": 0.10037264227867126, "lambda_max": 0.22948706150054932, "S_mean": 0.3601175944010417, "G_mean": 0.44253334620346624, "G_std": 0.17770238779485226, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.248833656311035, "grad_norm": 0.6234917044639587, "tokens_seen": 17047552, "tokens_per_sec": 14995.358406683563, "vram_gb": 6.442267648, "lambda": 0.1619664803147316, "lambda_min": 0.10010722279548645, "lambda_max": 0.23096153140068054, "S_mean": 0.35589599609375, "G_mean": 0.4571499613424142, "G_std": 0.18050760248055062, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.286373853683472, "grad_norm": 0.5611773133277893, "tokens_seen": 17211392, "tokens_per_sec": 14995.938443818808, "vram_gb": 6.442267648, "lambda": 0.16239299376805624, "lambda_min": 0.09981238096952438, "lambda_max": 0.23228858411312103, "S_mean": 0.3749796549479167, "G_mean": 0.42950112217416364, "G_std": 0.1774836778640747, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.148777484893799, "grad_norm": 0.5640996098518372, "tokens_seen": 17375232, "tokens_per_sec": 14996.922707594336, "vram_gb": 6.442267648, "lambda": 0.16285897139459848, "lambda_min": 0.09947941452264786, "lambda_max": 0.23353834450244904, "S_mean": 0.3631184895833333, "G_mean": 0.4387456787129243, "G_std": 0.1788953331609567, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.110760450363159, "grad_norm": 0.6697447896003723, "tokens_seen": 17539072, "tokens_per_sec": 14998.05121398387, "vram_gb": 6.442267648, "lambda": 0.16338208566109338, "lambda_min": 0.09908170253038406, "lambda_max": 0.2351834625005722, "S_mean": 0.3518473307291667, "G_mean": 0.4414132108601431, "G_std": 0.17754534476747116, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.180637359619141, "grad_norm": 0.5765979886054993, "tokens_seen": 17702912, "tokens_per_sec": 14999.385055600225, "vram_gb": 6.442267648, "lambda": 0.16390493356933197, "lambda_min": 0.0989169105887413, "lambda_max": 0.23680512607097626, "S_mean": 0.36651611328125, "G_mean": 0.43529749289155006, "G_std": 0.18013201793655753, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.187190055847168, "grad_norm": 0.5970165729522705, "tokens_seen": 17866752, "tokens_per_sec": 15000.658686358993, "vram_gb": 6.442267648, "lambda": 0.16443953973551592, "lambda_min": 0.0987050011754036, "lambda_max": 0.23840640485286713, "S_mean": 0.3896484375, "G_mean": 0.4204845493659377, "G_std": 0.17658888921141624, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.14050030708313, "grad_norm": 0.5773305296897888, "tokens_seen": 18030592, "tokens_per_sec": 15002.551088268108, "vram_gb": 6.442267648, "lambda": 0.16494531774272522, "lambda_min": 0.09890631586313248, "lambda_max": 0.2399415671825409, "S_mean": 0.35040283203125, "G_mean": 0.42809480025122565, "G_std": 0.1751376575169464, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.278674125671387, "grad_norm": 0.6047620177268982, "tokens_seen": 18194432, "tokens_per_sec": 15003.65156857041, "vram_gb": 6.442267648, "lambda": 0.1654657982289791, "lambda_min": 0.09868185967206955, "lambda_max": 0.24156403541564941, "S_mean": 0.38092041015625, "G_mean": 0.42853371193632483, "G_std": 0.17640085821039975, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.063074111938477, "grad_norm": 0.5765852332115173, "tokens_seen": 18358272, "tokens_per_sec": 15004.931017984864, "vram_gb": 6.442267648, "lambda": 0.1660358545680841, "lambda_min": 0.09862463176250458, "lambda_max": 0.24337202310562134, "S_mean": 0.3459065755208333, "G_mean": 0.42935371507580083, "G_std": 0.17583881124543646, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.0516746044158936, "grad_norm": 0.5692353248596191, "tokens_seen": 18522112, "tokens_per_sec": 15006.321906844858, "vram_gb": 6.442267648, "lambda": 0.1665091086179018, "lambda_min": 0.09832760691642761, "lambda_max": 0.2449418157339096, "S_mean": 0.37933349609375, "G_mean": 0.42736871757855016, "G_std": 0.17616632968808213, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.17138671875, "grad_norm": 0.6260473728179932, "tokens_seen": 18685952, "tokens_per_sec": 15007.571961580876, "vram_gb": 6.442267648, "lambda": 0.16707039314011732, "lambda_min": 0.09789778292179108, "lambda_max": 0.24672949314117432, "S_mean": 0.3803304036458333, "G_mean": 0.4278221872324745, "G_std": 0.17327115451917052, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.094193458557129, "grad_norm": 0.5845456719398499, "tokens_seen": 18849792, "tokens_per_sec": 15008.522683667428, "vram_gb": 6.442267648, "lambda": 0.16755416740973791, "lambda_min": 0.0975065752863884, "lambda_max": 0.248414546251297, "S_mean": 0.3721516927083333, "G_mean": 0.42178585814932984, "G_std": 0.17483725134904185, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 4.972527265548706, "grad_norm": 0.6341760158538818, "tokens_seen": 19013632, "tokens_per_sec": 15009.609959263818, "vram_gb": 6.442267648, "lambda": 0.16808387668182453, "lambda_min": 0.09725268185138702, "lambda_max": 0.24973371624946594, "S_mean": 0.3604736328125, "G_mean": 0.4178652417225142, "G_std": 0.17626430567664406, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.153456211090088, "grad_norm": 0.5786718726158142, "tokens_seen": 19177472, "tokens_per_sec": 15011.000670715384, "vram_gb": 6.442267648, "lambda": 0.1685764662300547, "lambda_min": 0.0971314013004303, "lambda_max": 0.2510306239128113, "S_mean": 0.3539225260416667, "G_mean": 0.4267597170546651, "G_std": 0.17075310880318284, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.221965789794922, "grad_norm": 0.6241642832756042, "tokens_seen": 19341312, "tokens_per_sec": 15011.758261102463, "vram_gb": 6.442267648, "lambda": 0.16904318425804377, "lambda_min": 0.09699133783578873, "lambda_max": 0.25260651111602783, "S_mean": 0.3551432291666667, "G_mean": 0.42788540354619425, "G_std": 0.1785259268557032, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.17497706413269, "grad_norm": 0.6367751359939575, "tokens_seen": 19505152, "tokens_per_sec": 15012.99856634971, "vram_gb": 6.442267648, "lambda": 0.169654021350046, "lambda_min": 0.09694427251815796, "lambda_max": 0.25445854663848877, "S_mean": 0.360107421875, "G_mean": 0.42358839480827254, "G_std": 0.1768934199741731, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.081995010375977, "grad_norm": 0.6469619274139404, "tokens_seen": 19668992, "tokens_per_sec": 15014.612560184729, "vram_gb": 6.442267648, "lambda": 0.17022556470086178, "lambda_min": 0.09685266762971878, "lambda_max": 0.25605523586273193, "S_mean": 0.3647664388020833, "G_mean": 0.4310847794016202, "G_std": 0.17668996937572956, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.120599269866943, "grad_norm": 0.5958294868469238, "tokens_seen": 19832832, "tokens_per_sec": 15015.433524382363, "vram_gb": 6.442267648, "lambda": 0.17069197011490664, "lambda_min": 0.09650598466396332, "lambda_max": 0.2578231990337372, "S_mean": 0.3527018229166667, "G_mean": 0.41329746910681325, "G_std": 0.17502547544427216, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.121206283569336, "grad_norm": 0.6358193755149841, "tokens_seen": 19996672, "tokens_per_sec": 15016.220446273526, "vram_gb": 6.442267648, "lambda": 0.1712501710280776, "lambda_min": 0.09654287993907928, "lambda_max": 0.2595966160297394, "S_mean": 0.37481689453125, "G_mean": 0.42022023210301995, "G_std": 0.17405037116259336, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.135196924209595, "grad_norm": 0.6214128732681274, "tokens_seen": 20160512, "tokens_per_sec": 15017.490974417238, "vram_gb": 6.442267648, "lambda": 0.17179489663491646, "lambda_min": 0.09665212035179138, "lambda_max": 0.2614705562591553, "S_mean": 0.3538004557291667, "G_mean": 0.42324206481377286, "G_std": 0.17363201330105463, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.113698720932007, "grad_norm": 0.6097403168678284, "tokens_seen": 20324352, "tokens_per_sec": 15018.843409699188, "vram_gb": 6.442267648, "lambda": 0.17226049428184828, "lambda_min": 0.09663353860378265, "lambda_max": 0.2631186246871948, "S_mean": 0.3497314453125, "G_mean": 0.41971947128574055, "G_std": 0.17533626430667937, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.131800651550293, "grad_norm": 0.6672024130821228, "tokens_seen": 20488192, "tokens_per_sec": 15020.133686221094, "vram_gb": 6.442267648, "lambda": 0.17278560002644858, "lambda_min": 0.09636682271957397, "lambda_max": 0.2649993300437927, "S_mean": 0.3668619791666667, "G_mean": 0.41679671173915267, "G_std": 0.17409298033453524, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.168568420410156, "val_ppl": 175.66318156888738, "lambda": 0.17281074884037176, "lambda_min": 0.09635722637176514, "lambda_max": 0.26508185267448425, "lambda_per_layer": [0.1488613784313202, 0.1371639370918274, 0.13486738502979279, 0.1333557367324829, 0.12572984397411346, 0.12088477611541748, 0.09635722637176514, 0.13002611696720123, 0.13548457622528076, 0.2067021131515503, 0.21059080958366394, 0.248128741979599, 0.23041589558124542, 0.26508185267448425, 0.22742576897144318, 0.22341430187225342, 0.1877863109111786, 0.18322697281837463, 0.1569189578294754, 0.16535434126853943, 0.1926669180393219, 0.18399547040462494, 0.1550457775592804, 0.14797276258468628], "S_mean": 0.3544108072916667, "G_mean": 0.41601955875133473, "G_std": 0.17415511763344207, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.215679883956909, "grad_norm": 0.6757616400718689, "tokens_seen": 20652032, "tokens_per_sec": 14976.230378760058, "vram_gb": 6.442267648, "lambda": 0.1733467181523641, "lambda_min": 0.09620606899261475, "lambda_max": 0.2669607996940613, "S_mean": 0.347412109375, "G_mean": 0.4191951248794794, "G_std": 0.17660143529064953, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.174542188644409, "grad_norm": 0.6511585712432861, "tokens_seen": 20815872, "tokens_per_sec": 14977.412994293965, "vram_gb": 6.442267648, "lambda": 0.17391117879499993, "lambda_min": 0.09587545692920685, "lambda_max": 0.2687865197658539, "S_mean": 0.35589599609375, "G_mean": 0.4185059356192748, "G_std": 0.1759508861384044, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.150973558425903, "grad_norm": 0.6188955903053284, "tokens_seen": 20979712, "tokens_per_sec": 14978.664718751374, "vram_gb": 6.442267648, "lambda": 0.17443909402936697, "lambda_min": 0.09570340812206268, "lambda_max": 0.27038711309432983, "S_mean": 0.36993408203125, "G_mean": 0.4222052649905284, "G_std": 0.1733484136251112, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 4.994162082672119, "grad_norm": 0.6684100031852722, "tokens_seen": 21143552, "tokens_per_sec": 14979.885553301292, "vram_gb": 6.442267648, "lambda": 0.17497245067109665, "lambda_min": 0.09556279331445694, "lambda_max": 0.2722320854663849, "S_mean": 0.3586832682291667, "G_mean": 0.4096009056083858, "G_std": 0.16928584268316627, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.068352937698364, "grad_norm": 0.6276954412460327, "tokens_seen": 21307392, "tokens_per_sec": 14980.949341348056, "vram_gb": 6.442267648, "lambda": 0.17552269777903953, "lambda_min": 0.09497325122356415, "lambda_max": 0.27393439412117004, "S_mean": 0.3560994466145833, "G_mean": 0.4104561192604403, "G_std": 0.17617346765473485, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.977700710296631, "grad_norm": 0.6326372027397156, "tokens_seen": 21471232, "tokens_per_sec": 14982.306413692088, "vram_gb": 6.442267648, "lambda": 0.17607388831675053, "lambda_min": 0.09482774883508682, "lambda_max": 0.2754055857658386, "S_mean": 0.339111328125, "G_mean": 0.4036020350952943, "G_std": 0.174259361733372, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.178728103637695, "grad_norm": 0.6613171696662903, "tokens_seen": 21635072, "tokens_per_sec": 14983.597758074913, "vram_gb": 6.442267648, "lambda": 0.17660792544484138, "lambda_min": 0.09497273713350296, "lambda_max": 0.27731233835220337, "S_mean": 0.3674112955729167, "G_mean": 0.4197468024988969, "G_std": 0.17452515188294151, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.006307125091553, "grad_norm": 0.6396279335021973, "tokens_seen": 21798912, "tokens_per_sec": 14985.151021529, "vram_gb": 6.442267648, "lambda": 0.1771103631084164, "lambda_min": 0.09479571878910065, "lambda_max": 0.27901801466941833, "S_mean": 0.35845947265625, "G_mean": 0.40465116982037824, "G_std": 0.17263885439994434, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.014627933502197, "grad_norm": 0.6692155599594116, "tokens_seen": 21962752, "tokens_per_sec": 14985.95519272264, "vram_gb": 6.442267648, "lambda": 0.17758611869066954, "lambda_min": 0.0945093110203743, "lambda_max": 0.28061655163764954, "S_mean": 0.339385986328125, "G_mean": 0.4134108276727299, "G_std": 0.17172042047604918, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.164994716644287, "grad_norm": 0.6850765943527222, "tokens_seen": 22126592, "tokens_per_sec": 14987.428330973993, "vram_gb": 6.442267648, "lambda": 0.17816964350640774, "lambda_min": 0.0945550948381424, "lambda_max": 0.28251537680625916, "S_mean": 0.3580525716145833, "G_mean": 0.40519791515544057, "G_std": 0.1726872584937761, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.013911962509155, "grad_norm": 0.6390072107315063, "tokens_seen": 22290432, "tokens_per_sec": 14988.709745111437, "vram_gb": 6.442267648, "lambda": 0.17870614056785902, "lambda_min": 0.0945853739976883, "lambda_max": 0.28433459997177124, "S_mean": 0.3408406575520833, "G_mean": 0.4123427518643439, "G_std": 0.1727110545616597, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.160232067108154, "grad_norm": 0.6647241711616516, "tokens_seen": 22454272, "tokens_per_sec": 14989.976364570777, "vram_gb": 6.442267648, "lambda": 0.17928303064157566, "lambda_min": 0.09452828764915466, "lambda_max": 0.2862655520439148, "S_mean": 0.33538818359375, "G_mean": 0.41071807344754535, "G_std": 0.1770435639191419, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 4.984401226043701, "grad_norm": 0.7483488917350769, "tokens_seen": 22618112, "tokens_per_sec": 14991.155155920944, "vram_gb": 6.442267648, "lambda": 0.17973885405808687, "lambda_min": 0.0946669653058052, "lambda_max": 0.28765082359313965, "S_mean": 0.3476969401041667, "G_mean": 0.4159242582196991, "G_std": 0.17082312245232364, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.111650466918945, "grad_norm": 0.6714873909950256, "tokens_seen": 22781952, "tokens_per_sec": 14992.455121411222, "vram_gb": 6.442267648, "lambda": 0.18017095141112804, "lambda_min": 0.09462468326091766, "lambda_max": 0.28899049758911133, "S_mean": 0.35760498046875, "G_mean": 0.40580977173522115, "G_std": 0.1717022906523198, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.880175590515137, "grad_norm": 0.6456176042556763, "tokens_seen": 22945792, "tokens_per_sec": 14993.213400051514, "vram_gb": 6.442267648, "lambda": 0.1806250453616182, "lambda_min": 0.09473896771669388, "lambda_max": 0.2903444468975067, "S_mean": 0.348388671875, "G_mean": 0.40025877781833213, "G_std": 0.1686066920713832, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.192298412322998, "grad_norm": 0.6566681265830994, "tokens_seen": 23109632, "tokens_per_sec": 14994.43292404763, "vram_gb": 6.442267648, "lambda": 0.1811371094857653, "lambda_min": 0.09500160068273544, "lambda_max": 0.29174625873565674, "S_mean": 0.3601481119791667, "G_mean": 0.40688813136269647, "G_std": 0.17320482657911876, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.033447742462158, "grad_norm": 0.6661161184310913, "tokens_seen": 23273472, "tokens_per_sec": 14994.181642028087, "vram_gb": 6.442267648, "lambda": 0.18173559724042812, "lambda_min": 0.09500477463006973, "lambda_max": 0.2935943007469177, "S_mean": 0.3556111653645833, "G_mean": 0.4058922144273917, "G_std": 0.16898076670865217, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.974839925765991, "grad_norm": 0.6377353668212891, "tokens_seen": 23437312, "tokens_per_sec": 14995.644140451355, "vram_gb": 6.442267648, "lambda": 0.18220047559589148, "lambda_min": 0.09484492242336273, "lambda_max": 0.29521051049232483, "S_mean": 0.3643798828125, "G_mean": 0.4108061242538194, "G_std": 0.17541603200758496, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.034541368484497, "grad_norm": 0.7089442014694214, "tokens_seen": 23601152, "tokens_per_sec": 14996.776359753849, "vram_gb": 6.442267648, "lambda": 0.18270055763423443, "lambda_min": 0.09485399723052979, "lambda_max": 0.29687559604644775, "S_mean": 0.36358642578125, "G_mean": 0.3939345497637987, "G_std": 0.1722666653804481, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.0398900508880615, "grad_norm": 0.6571464538574219, "tokens_seen": 23764992, "tokens_per_sec": 14998.213447632204, "vram_gb": 6.442267648, "lambda": 0.18319883042325577, "lambda_min": 0.09472034126520157, "lambda_max": 0.29872527718544006, "S_mean": 0.35272216796875, "G_mean": 0.3984602933439116, "G_std": 0.1687144705404838, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.9871697425842285, "grad_norm": 0.8074601292610168, "tokens_seen": 23928832, "tokens_per_sec": 14999.284195448601, "vram_gb": 6.442267648, "lambda": 0.18365457964440188, "lambda_min": 0.0945853739976883, "lambda_max": 0.30058416724205017, "S_mean": 0.3671671549479167, "G_mean": 0.3983884967553119, "G_std": 0.16814642096869648, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.060928583145142, "grad_norm": 0.651003897190094, "tokens_seen": 24092672, "tokens_per_sec": 15000.495588544169, "vram_gb": 6.442267648, "lambda": 0.18414263365169367, "lambda_min": 0.09429843723773956, "lambda_max": 0.30197077989578247, "S_mean": 0.3761393229166667, "G_mean": 0.3946270962866644, "G_std": 0.16893109764593342, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.855625152587891, "grad_norm": 0.6842800974845886, "tokens_seen": 24256512, "tokens_per_sec": 15001.611237799738, "vram_gb": 6.442267648, "lambda": 0.18457439665993056, "lambda_min": 0.09421535581350327, "lambda_max": 0.3033868372440338, "S_mean": 0.3673909505208333, "G_mean": 0.3923480113347371, "G_std": 0.17331117865008613, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.94897723197937, "grad_norm": 0.6132776141166687, "tokens_seen": 24420352, "tokens_per_sec": 15002.272696881948, "vram_gb": 6.442267648, "lambda": 0.18497683635602394, "lambda_min": 0.0941881313920021, "lambda_max": 0.3050730228424072, "S_mean": 0.3666585286458333, "G_mean": 0.39650011404107016, "G_std": 0.1701076382305473, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.959791660308838, "grad_norm": 0.7007431983947754, "tokens_seen": 24584192, "tokens_per_sec": 15002.809425997522, "vram_gb": 6.442267648, "lambda": 0.1854154352719585, "lambda_min": 0.09437752515077591, "lambda_max": 0.30636411905288696, "S_mean": 0.3731892903645833, "G_mean": 0.3968866447297235, "G_std": 0.17157603800296783, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.031508696079254, "val_ppl": 153.16391627011805, "lambda": 0.18544396571815014, "lambda_min": 0.09438782930374146, "lambda_max": 0.3064395487308502, "lambda_per_layer": [0.1508040875196457, 0.13791652023792267, 0.13597746193408966, 0.13376493752002716, 0.12467186897993088, 0.12105724960565567, 0.09438782930374146, 0.12959882616996765, 0.1347554475069046, 0.22004595398902893, 0.22996406257152557, 0.27779829502105713, 0.2529531717300415, 0.3064395487308502, 0.2541854679584503, 0.2568318247795105, 0.20989882946014404, 0.2043234258890152, 0.16849635541439056, 0.17475007474422455, 0.21290577948093414, 0.20241864025592804, 0.16319121420383453, 0.15351830422878265], "S_mean": 0.3873087565104167, "G_mean": 0.40403110704695183, "G_std": 0.1709929930511862, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.988094329833984, "grad_norm": 0.6592842936515808, "tokens_seen": 24748032, "tokens_per_sec": 14966.587853611944, "vram_gb": 6.442267648, "lambda": 0.18591970826188722, "lambda_min": 0.09425855427980423, "lambda_max": 0.3080317974090576, "S_mean": 0.3544514973958333, "G_mean": 0.39868138792614144, "G_std": 0.16880249969350794, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.823719024658203, "grad_norm": 0.6159408092498779, "tokens_seen": 24911872, "tokens_per_sec": 14967.158123193753, "vram_gb": 6.442267648, "lambda": 0.18643991245577732, "lambda_min": 0.09440299868583679, "lambda_max": 0.30984437465667725, "S_mean": 0.3514607747395833, "G_mean": 0.39237305149435997, "G_std": 0.17312515752079585, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.988168716430664, "grad_norm": 0.690544843673706, "tokens_seen": 25075712, "tokens_per_sec": 14968.051065255064, "vram_gb": 6.442267648, "lambda": 0.1870363550260663, "lambda_min": 0.09444385766983032, "lambda_max": 0.31161031126976013, "S_mean": 0.347747802734375, "G_mean": 0.3957376967494686, "G_std": 0.17221335864936313, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.810473442077637, "grad_norm": 0.6881431937217712, "tokens_seen": 25239552, "tokens_per_sec": 14969.180906804495, "vram_gb": 6.442267648, "lambda": 0.18751942863066992, "lambda_min": 0.09442112594842911, "lambda_max": 0.31334611773490906, "S_mean": 0.359710693359375, "G_mean": 0.3923614569939673, "G_std": 0.17166477527158955, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.982725620269775, "grad_norm": 0.6788945198059082, "tokens_seen": 25403392, "tokens_per_sec": 14970.569923787154, "vram_gb": 6.442267648, "lambda": 0.1879943838963906, "lambda_min": 0.09423433989286423, "lambda_max": 0.3150157630443573, "S_mean": 0.3589070638020833, "G_mean": 0.3923491245756547, "G_std": 0.1731744868836055, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.971843957901001, "grad_norm": 0.7080022692680359, "tokens_seen": 25567232, "tokens_per_sec": 14971.505430422763, "vram_gb": 6.442267648, "lambda": 0.18843579292297363, "lambda_min": 0.0942462906241417, "lambda_max": 0.3167557716369629, "S_mean": 0.3404337565104167, "G_mean": 0.4027436982529859, "G_std": 0.17164442393307885, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.930303573608398, "grad_norm": 0.683189332485199, "tokens_seen": 25731072, "tokens_per_sec": 14972.547960200074, "vram_gb": 6.442267648, "lambda": 0.18894681055098772, "lambda_min": 0.09432676434516907, "lambda_max": 0.31851306557655334, "S_mean": 0.3549397786458333, "G_mean": 0.39573678203547996, "G_std": 0.16900050973830125, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.954830646514893, "grad_norm": 0.6596022248268127, "tokens_seen": 25894912, "tokens_per_sec": 14973.743532847246, "vram_gb": 6.442267648, "lambda": 0.1894544381648302, "lambda_min": 0.09448061138391495, "lambda_max": 0.31968241930007935, "S_mean": 0.33868408203125, "G_mean": 0.39118253346532583, "G_std": 0.17030445369891822, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.988365888595581, "grad_norm": 0.8007938265800476, "tokens_seen": 26058752, "tokens_per_sec": 14975.024165517993, "vram_gb": 6.442267648, "lambda": 0.1899393325050672, "lambda_min": 0.0946364775300026, "lambda_max": 0.3206913471221924, "S_mean": 0.3404541015625, "G_mean": 0.3957154434174299, "G_std": 0.17214420841385922, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.772445440292358, "grad_norm": 0.646380603313446, "tokens_seen": 26222592, "tokens_per_sec": 14975.770407337182, "vram_gb": 6.442267648, "lambda": 0.1903568860143423, "lambda_min": 0.09460216760635376, "lambda_max": 0.3221520483493805, "S_mean": 0.3601175944010417, "G_mean": 0.38460491349299747, "G_std": 0.16784144813815752, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.9609832763671875, "grad_norm": 0.7654547095298767, "tokens_seen": 26386432, "tokens_per_sec": 14976.10388520559, "vram_gb": 6.442267648, "lambda": 0.1908245130131642, "lambda_min": 0.09442491084337234, "lambda_max": 0.3237379789352417, "S_mean": 0.3481241861979167, "G_mean": 0.39597682701423764, "G_std": 0.1695021060295403, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.021539688110352, "grad_norm": 0.7004066705703735, "tokens_seen": 26550272, "tokens_per_sec": 14976.754879360558, "vram_gb": 6.442267648, "lambda": 0.19130467313031355, "lambda_min": 0.09419548511505127, "lambda_max": 0.32531827688217163, "S_mean": 0.3466796875, "G_mean": 0.3976783783485492, "G_std": 0.16971452231518924, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.877844333648682, "grad_norm": 0.6660382747650146, "tokens_seen": 26714112, "tokens_per_sec": 14977.655065884688, "vram_gb": 6.442267648, "lambda": 0.1918546153853337, "lambda_min": 0.09411456435918808, "lambda_max": 0.32742106914520264, "S_mean": 0.3513387044270833, "G_mean": 0.389945635261635, "G_std": 0.16824214059549072, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.822800159454346, "grad_norm": 0.7546278834342957, "tokens_seen": 26877952, "tokens_per_sec": 14978.56420989693, "vram_gb": 6.442267648, "lambda": 0.1923148650676012, "lambda_min": 0.09401235729455948, "lambda_max": 0.32887187600135803, "S_mean": 0.3480021158854167, "G_mean": 0.3916345052421093, "G_std": 0.17049037230511507, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.993925094604492, "grad_norm": 0.6834532618522644, "tokens_seen": 27041792, "tokens_per_sec": 14979.57936942617, "vram_gb": 6.442267648, "lambda": 0.19285838368038336, "lambda_min": 0.09390459954738617, "lambda_max": 0.33050641417503357, "S_mean": 0.3577677408854167, "G_mean": 0.39586646544436616, "G_std": 0.1673274621522675, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 5.022613763809204, "grad_norm": 0.6886204481124878, "tokens_seen": 27205632, "tokens_per_sec": 14981.076752457808, "vram_gb": 6.442267648, "lambda": 0.19338323455303907, "lambda_min": 0.09387064725160599, "lambda_max": 0.3321389853954315, "S_mean": 0.34698486328125, "G_mean": 0.39684691388780874, "G_std": 0.16920610579351583, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.859388113021851, "grad_norm": 0.6515411138534546, "tokens_seen": 27369472, "tokens_per_sec": 14982.18754622138, "vram_gb": 6.442267648, "lambda": 0.19379915048678717, "lambda_min": 0.09401242434978485, "lambda_max": 0.33365145325660706, "S_mean": 0.3389180501302083, "G_mean": 0.3942444010948141, "G_std": 0.17042338328125575, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.790692090988159, "grad_norm": 0.6637119650840759, "tokens_seen": 27533312, "tokens_per_sec": 14983.233958683639, "vram_gb": 6.442267648, "lambda": 0.1942577799782157, "lambda_min": 0.09389673918485641, "lambda_max": 0.33593037724494934, "S_mean": 0.341033935546875, "G_mean": 0.3938445254849891, "G_std": 0.16824572215167186, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.047780513763428, "grad_norm": 0.674608051776886, "tokens_seen": 27697152, "tokens_per_sec": 14984.019328106542, "vram_gb": 6.442267648, "lambda": 0.1947017451748252, "lambda_min": 0.09383556991815567, "lambda_max": 0.33752748370170593, "S_mean": 0.359039306640625, "G_mean": 0.3959193304181099, "G_std": 0.16709967508601645, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 5.133354902267456, "grad_norm": 0.7037174105644226, "tokens_seen": 27860992, "tokens_per_sec": 14984.687389460156, "vram_gb": 6.442267648, "lambda": 0.19520254681507745, "lambda_min": 0.09350981563329697, "lambda_max": 0.3390406370162964, "S_mean": 0.3418375651041667, "G_mean": 0.3968056154747804, "G_std": 0.16801689107281467, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.041635274887085, "grad_norm": 0.7737427949905396, "tokens_seen": 28024832, "tokens_per_sec": 14985.762860885787, "vram_gb": 6.442267648, "lambda": 0.19567940601458153, "lambda_min": 0.09359528869390488, "lambda_max": 0.34051889181137085, "S_mean": 0.3672587076822917, "G_mean": 0.3865506239235401, "G_std": 0.16681378652962545, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.835792541503906, "grad_norm": 0.6925722360610962, "tokens_seen": 28188672, "tokens_per_sec": 14986.518066274984, "vram_gb": 6.442267648, "lambda": 0.19615983155866465, "lambda_min": 0.09369111061096191, "lambda_max": 0.3421555757522583, "S_mean": 0.352142333984375, "G_mean": 0.3851083545014262, "G_std": 0.16765729368974766, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.962696313858032, "grad_norm": 0.7007961273193359, "tokens_seen": 28352512, "tokens_per_sec": 14987.061352520535, "vram_gb": 6.442267648, "lambda": 0.19670069621255, "lambda_min": 0.0939682349562645, "lambda_max": 0.34383130073547363, "S_mean": 0.3502298990885417, "G_mean": 0.38711169579376775, "G_std": 0.16622707527130842, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.857001066207886, "grad_norm": 0.7143704295158386, "tokens_seen": 28516352, "tokens_per_sec": 14987.706455950838, "vram_gb": 6.442267648, "lambda": 0.1972318027789394, "lambda_min": 0.09394291788339615, "lambda_max": 0.345723420381546, "S_mean": 0.3535563151041667, "G_mean": 0.390395356186976, "G_std": 0.16556147951632738, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.890896797180176, "grad_norm": 0.7247089147567749, "tokens_seen": 28680192, "tokens_per_sec": 14988.707107733531, "vram_gb": 6.442267648, "lambda": 0.19771316864838204, "lambda_min": 0.09374721348285675, "lambda_max": 0.3475453853607178, "S_mean": 0.371337890625, "G_mean": 0.38419649897453684, "G_std": 0.16573417178976038, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.986731505393982, "val_ppl": 146.45694659875366, "lambda": 0.1977351102977991, "lambda_min": 0.0937575176358223, "lambda_max": 0.347623348236084, "lambda_per_layer": [0.15272611379623413, 0.13947507739067078, 0.13669802248477936, 0.1341836005449295, 0.12402597069740295, 0.1197851225733757, 0.0937575176358223, 0.1299191415309906, 0.13418319821357727, 0.23039610683918, 0.2499375194311142, 0.30584800243377686, 0.2726336419582367, 0.347623348236084, 0.2805521488189697, 0.28675970435142517, 0.23395605385303497, 0.22542399168014526, 0.179195374250412, 0.18476352095603943, 0.23596376180648804, 0.21940523386001587, 0.16978402435779572, 0.15864644944667816], "S_mean": 0.3599446614583333, "G_mean": 0.38389152971406776, "G_std": 0.16547637370725474, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.832041501998901, "grad_norm": 0.6908226013183594, "tokens_seen": 28844032, "tokens_per_sec": 14957.790616019125, "vram_gb": 6.442267648, "lambda": 0.19817129212121168, "lambda_min": 0.09387703984975815, "lambda_max": 0.3492303788661957, "S_mean": 0.3542989095052083, "G_mean": 0.3906831302059193, "G_std": 0.16828523312384883, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.926745176315308, "grad_norm": 0.7242051959037781, "tokens_seen": 29007872, "tokens_per_sec": 14958.691680314925, "vram_gb": 6.442267648, "lambda": 0.19855653091023365, "lambda_min": 0.09373842179775238, "lambda_max": 0.3507935106754303, "S_mean": 0.34375, "G_mean": 0.3963502161204815, "G_std": 0.17031945932346085, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.906310796737671, "grad_norm": 0.6947289109230042, "tokens_seen": 29171712, "tokens_per_sec": 14959.281500924035, "vram_gb": 6.442267648, "lambda": 0.19909179055442414, "lambda_min": 0.09349570423364639, "lambda_max": 0.35236480832099915, "S_mean": 0.3494364420572917, "G_mean": 0.38569319900125265, "G_std": 0.16453998225430647, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.984635829925537, "grad_norm": 0.7613537311553955, "tokens_seen": 29335552, "tokens_per_sec": 14959.776547993346, "vram_gb": 6.442267648, "lambda": 0.19963449022422233, "lambda_min": 0.09352180361747742, "lambda_max": 0.35408586263656616, "S_mean": 0.37353515625, "G_mean": 0.3832656890153885, "G_std": 0.1657625245861709, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 5.050743579864502, "grad_norm": 0.7328582406044006, "tokens_seen": 29499392, "tokens_per_sec": 14960.306409967465, "vram_gb": 6.442267648, "lambda": 0.20007767590383688, "lambda_min": 0.09348036348819733, "lambda_max": 0.35580864548683167, "S_mean": 0.34112548828125, "G_mean": 0.389210515966018, "G_std": 0.1669699929188937, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.904698848724365, "grad_norm": 0.7191260457038879, "tokens_seen": 29663232, "tokens_per_sec": 14961.124791694516, "vram_gb": 6.442267648, "lambda": 0.2005883470798532, "lambda_min": 0.09333755075931549, "lambda_max": 0.357571005821228, "S_mean": 0.3511454264322917, "G_mean": 0.3864594671564798, "G_std": 0.16410423729879162, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.774038553237915, "grad_norm": 0.7735600471496582, "tokens_seen": 29827072, "tokens_per_sec": 14961.69666145894, "vram_gb": 6.442267648, "lambda": 0.2011195303251346, "lambda_min": 0.09337671846151352, "lambda_max": 0.3597220778465271, "S_mean": 0.339111328125, "G_mean": 0.38664374127984047, "G_std": 0.16500795244549712, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.730050086975098, "grad_norm": 0.709846556186676, "tokens_seen": 29990912, "tokens_per_sec": 14962.604846320051, "vram_gb": 6.442267648, "lambda": 0.20154940647383532, "lambda_min": 0.0935128852725029, "lambda_max": 0.3611850440502167, "S_mean": 0.3498942057291667, "G_mean": 0.37231302199264366, "G_std": 0.16593450587242842, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.712651014328003, "grad_norm": 0.6588510870933533, "tokens_seen": 30154752, "tokens_per_sec": 14963.563203291551, "vram_gb": 6.442267648, "lambda": 0.20195918809622526, "lambda_min": 0.09347419440746307, "lambda_max": 0.362485408782959, "S_mean": 0.359130859375, "G_mean": 0.38360304951978225, "G_std": 0.16823256838445863, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.941730976104736, "grad_norm": 0.7251073718070984, "tokens_seen": 30318592, "tokens_per_sec": 14964.931076823354, "vram_gb": 6.442267648, "lambda": 0.20242032315582037, "lambda_min": 0.09359360486268997, "lambda_max": 0.36424520611763, "S_mean": 0.3652140299479167, "G_mean": 0.38928530846411985, "G_std": 0.16334691744608185, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.884069442749023, "grad_norm": 0.6990702748298645, "tokens_seen": 30482432, "tokens_per_sec": 14965.649716909978, "vram_gb": 6.442267648, "lambda": 0.20285006643583378, "lambda_min": 0.09356434643268585, "lambda_max": 0.3655931353569031, "S_mean": 0.3638509114583333, "G_mean": 0.3776211114600301, "G_std": 0.1661692140623927, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.898437023162842, "grad_norm": 0.6977521777153015, "tokens_seen": 30646272, "tokens_per_sec": 14966.301573363133, "vram_gb": 6.442267648, "lambda": 0.20331771858036518, "lambda_min": 0.09360304474830627, "lambda_max": 0.3673585057258606, "S_mean": 0.3440348307291667, "G_mean": 0.3829444528867801, "G_std": 0.16490361676551402, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.8615312576293945, "grad_norm": 0.7257525324821472, "tokens_seen": 30810112, "tokens_per_sec": 14966.960347770955, "vram_gb": 6.442267648, "lambda": 0.20384814900656542, "lambda_min": 0.09389550238847733, "lambda_max": 0.3692944645881653, "S_mean": 0.36102294921875, "G_mean": 0.3852472440339625, "G_std": 0.1681192802886168, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.774036169052124, "grad_norm": 0.7305677533149719, "tokens_seen": 30973952, "tokens_per_sec": 14967.64566763788, "vram_gb": 6.442267648, "lambda": 0.20426123340924582, "lambda_min": 0.09366799890995026, "lambda_max": 0.37084320187568665, "S_mean": 0.3514200846354167, "G_mean": 0.3814217286805312, "G_std": 0.1661629043519497, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.790410995483398, "grad_norm": 0.7058204412460327, "tokens_seen": 31137792, "tokens_per_sec": 14968.54352628929, "vram_gb": 6.442267648, "lambda": 0.20474778302013874, "lambda_min": 0.09341002255678177, "lambda_max": 0.3725699186325073, "S_mean": 0.3564453125, "G_mean": 0.38044378510676324, "G_std": 0.166069544153288, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.835687160491943, "grad_norm": 0.6936566829681396, "tokens_seen": 31301632, "tokens_per_sec": 14969.363893305548, "vram_gb": 6.442267648, "lambda": 0.20519877411425114, "lambda_min": 0.09330882877111435, "lambda_max": 0.3740805387496948, "S_mean": 0.35821533203125, "G_mean": 0.3849569895149519, "G_std": 0.16933902174544832, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.922081232070923, "grad_norm": 0.6930922865867615, "tokens_seen": 31465472, "tokens_per_sec": 14970.352566522644, "vram_gb": 6.442267648, "lambda": 0.20564522873610258, "lambda_min": 0.09320677816867828, "lambda_max": 0.3755750060081482, "S_mean": 0.3634236653645833, "G_mean": 0.3805615307452778, "G_std": 0.1647332803501437, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.796996593475342, "grad_norm": 0.7427541613578796, "tokens_seen": 31629312, "tokens_per_sec": 14971.216708481374, "vram_gb": 6.442267648, "lambda": 0.2061215927824378, "lambda_min": 0.09321842342615128, "lambda_max": 0.37723222374916077, "S_mean": 0.3596394856770833, "G_mean": 0.37187275802716613, "G_std": 0.16709579705881575, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.666410684585571, "grad_norm": 0.6702901124954224, "tokens_seen": 31793152, "tokens_per_sec": 14971.785333011976, "vram_gb": 6.442267648, "lambda": 0.2066107140854001, "lambda_min": 0.09345371276140213, "lambda_max": 0.3788806200027466, "S_mean": 0.3573506673177083, "G_mean": 0.3797315318758289, "G_std": 0.16508856896931925, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.7961061000823975, "grad_norm": 0.7451735138893127, "tokens_seen": 31956992, "tokens_per_sec": 14972.64004509954, "vram_gb": 6.442267648, "lambda": 0.20712714393933615, "lambda_min": 0.09354696422815323, "lambda_max": 0.38084086775779724, "S_mean": 0.357757568359375, "G_mean": 0.37564280272151035, "G_std": 0.16540856667173406, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.90155029296875, "grad_norm": 0.7255696058273315, "tokens_seen": 32120832, "tokens_per_sec": 14973.432442858684, "vram_gb": 6.442267648, "lambda": 0.20762438358118138, "lambda_min": 0.09348617494106293, "lambda_max": 0.3823005259037018, "S_mean": 0.345489501953125, "G_mean": 0.3796211796967934, "G_std": 0.1642184886150062, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.7344207763671875, "grad_norm": 0.7037197351455688, "tokens_seen": 32284672, "tokens_per_sec": 14973.567491198635, "vram_gb": 6.442267648, "lambda": 0.20810106055190167, "lambda_min": 0.09348783642053604, "lambda_max": 0.3841119110584259, "S_mean": 0.365325927734375, "G_mean": 0.37997589260339737, "G_std": 0.16462088438371816, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.970690488815308, "grad_norm": 0.7152222990989685, "tokens_seen": 32448512, "tokens_per_sec": 14974.24019462362, "vram_gb": 6.442267648, "lambda": 0.2086110496893525, "lambda_min": 0.09349612891674042, "lambda_max": 0.3859299123287201, "S_mean": 0.3510030110677083, "G_mean": 0.37405933486297727, "G_std": 0.1636260423498849, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 5.019201278686523, "grad_norm": 0.8009392023086548, "tokens_seen": 32612352, "tokens_per_sec": 14975.091801040684, "vram_gb": 6.442267648, "lambda": 0.20913751485447088, "lambda_min": 0.09358751028776169, "lambda_max": 0.3877987265586853, "S_mean": 0.3561503092447917, "G_mean": 0.3787727613622944, "G_std": 0.16502531974886855, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.926419973373413, "grad_norm": 0.7341143488883972, "tokens_seen": 32776192, "tokens_per_sec": 14976.260839071241, "vram_gb": 6.442267648, "lambda": 0.20954076293855906, "lambda_min": 0.09374503046274185, "lambda_max": 0.38950997591018677, "S_mean": 0.3392842610677083, "G_mean": 0.37509435741230845, "G_std": 0.1633923964885374, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.8458712816238405, "val_ppl": 127.21407300574555, "lambda": 0.20955694684137902, "lambda_min": 0.09374671429395676, "lambda_max": 0.38956212997436523, "lambda_per_layer": [0.15550537407398224, 0.13994412124156952, 0.13773410022258759, 0.13456495106220245, 0.12344233691692352, 0.11930805444717407, 0.09374671429395676, 0.12976022064685822, 0.13368621468544006, 0.23873837292194366, 0.265573650598526, 0.33218252658843994, 0.2916460335254669, 0.38956212997436523, 0.3059329092502594, 0.31537631154060364, 0.2533113360404968, 0.24678416550159454, 0.1900395154953003, 0.1947326511144638, 0.2593371272087097, 0.2378273457288742, 0.17565515637397766, 0.16497540473937988], "S_mean": 0.3680013020833333, "G_mean": 0.38343982320899767, "G_std": 0.16721812325219312, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.572962045669556, "grad_norm": 0.659587562084198, "tokens_seen": 32940032, "tokens_per_sec": 14948.906659719274, "vram_gb": 6.442267648, "lambda": 0.20995215947429338, "lambda_min": 0.09363483637571335, "lambda_max": 0.39067256450653076, "S_mean": 0.3568318684895833, "G_mean": 0.3806492208192746, "G_std": 0.16679074297038218, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.836391925811768, "grad_norm": 0.7189354300498962, "tokens_seen": 33103872, "tokens_per_sec": 14949.751274858225, "vram_gb": 6.442267648, "lambda": 0.21045360248535872, "lambda_min": 0.09344511479139328, "lambda_max": 0.3922872245311737, "S_mean": 0.3468017578125, "G_mean": 0.3771408104027311, "G_std": 0.1643551018399497, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.912526845932007, "grad_norm": 0.7767853140830994, "tokens_seen": 33267712, "tokens_per_sec": 14950.590687530339, "vram_gb": 6.442267648, "lambda": 0.2110036676749587, "lambda_min": 0.093536376953125, "lambda_max": 0.39387083053588867, "S_mean": 0.3593343098958333, "G_mean": 0.3686733143404126, "G_std": 0.1648035477846861, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.7035112380981445, "grad_norm": 0.7171063423156738, "tokens_seen": 33431552, "tokens_per_sec": 14951.183644474346, "vram_gb": 6.442267648, "lambda": 0.21147102086494365, "lambda_min": 0.09350430965423584, "lambda_max": 0.3954415023326874, "S_mean": 0.3608194986979167, "G_mean": 0.3686879393644631, "G_std": 0.16319685025761524, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.837684392929077, "grad_norm": 0.7506311535835266, "tokens_seen": 33595392, "tokens_per_sec": 14951.669016666858, "vram_gb": 6.442267648, "lambda": 0.21188004625340304, "lambda_min": 0.0934651717543602, "lambda_max": 0.39672529697418213, "S_mean": 0.3585611979166667, "G_mean": 0.37463575570533675, "G_std": 0.16556266749588153, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.826378345489502, "grad_norm": 0.7829037308692932, "tokens_seen": 33759232, "tokens_per_sec": 14952.090803314813, "vram_gb": 6.442267648, "lambda": 0.21229513517270485, "lambda_min": 0.0931810811161995, "lambda_max": 0.3980858325958252, "S_mean": 0.3620707194010417, "G_mean": 0.37910475620689493, "G_std": 0.16524465048375228, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.89382266998291, "grad_norm": 0.7546592354774475, "tokens_seen": 33923072, "tokens_per_sec": 14953.048077080013, "vram_gb": 6.442267648, "lambda": 0.21282581984996796, "lambda_min": 0.09303661435842514, "lambda_max": 0.3995512127876282, "S_mean": 0.3568522135416667, "G_mean": 0.3812202579962711, "G_std": 0.1649323704186827, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.721264839172363, "grad_norm": 0.7558673620223999, "tokens_seen": 34086912, "tokens_per_sec": 14953.653773304268, "vram_gb": 6.442267648, "lambda": 0.21322089744110903, "lambda_min": 0.09287471324205399, "lambda_max": 0.40089040994644165, "S_mean": 0.3431396484375, "G_mean": 0.3751378464512527, "G_std": 0.16384696083453795, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.96013069152832, "grad_norm": 0.7553685903549194, "tokens_seen": 34250752, "tokens_per_sec": 14954.92662232726, "vram_gb": 6.442267648, "lambda": 0.21367353418221077, "lambda_min": 0.09289546310901642, "lambda_max": 0.40259549021720886, "S_mean": 0.3415425618489583, "G_mean": 0.37702339196888107, "G_std": 0.16518369348098835, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.666858196258545, "grad_norm": 0.7426431775093079, "tokens_seen": 34414592, "tokens_per_sec": 14955.96616738475, "vram_gb": 6.442267648, "lambda": 0.2141635020573934, "lambda_min": 0.09275533258914948, "lambda_max": 0.404307097196579, "S_mean": 0.3631591796875, "G_mean": 0.3800294892862439, "G_std": 0.16681161693607768, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.736061334609985, "grad_norm": 0.7267663478851318, "tokens_seen": 34578432, "tokens_per_sec": 14956.779746099886, "vram_gb": 6.442267648, "lambda": 0.2146925569201509, "lambda_min": 0.09279379993677139, "lambda_max": 0.40582212805747986, "S_mean": 0.3442789713541667, "G_mean": 0.37625679032256204, "G_std": 0.16765925637446344, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.689302682876587, "grad_norm": 0.6818296313285828, "tokens_seen": 34742272, "tokens_per_sec": 14957.64546056213, "vram_gb": 6.442267648, "lambda": 0.21521606078992286, "lambda_min": 0.09293108433485031, "lambda_max": 0.40790775418281555, "S_mean": 0.3426106770833333, "G_mean": 0.37928203664099175, "G_std": 0.16292175084042052, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.733527898788452, "grad_norm": 0.7153716087341309, "tokens_seen": 34906112, "tokens_per_sec": 14958.51181562161, "vram_gb": 6.442267648, "lambda": 0.2157188830897212, "lambda_min": 0.09294284880161285, "lambda_max": 0.4095943570137024, "S_mean": 0.34124755859375, "G_mean": 0.3730606504250318, "G_std": 0.16612234098526338, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.863258361816406, "grad_norm": 0.8162981867790222, "tokens_seen": 35069952, "tokens_per_sec": 14959.513682832774, "vram_gb": 6.442267648, "lambda": 0.21619992734243473, "lambda_min": 0.09279347956180573, "lambda_max": 0.4111954867839813, "S_mean": 0.3457845052083333, "G_mean": 0.38272624167924124, "G_std": 0.16532340187889835, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.912982702255249, "grad_norm": 0.740095317363739, "tokens_seen": 35233792, "tokens_per_sec": 14960.334031811402, "vram_gb": 6.442267648, "lambda": 0.21671670023351908, "lambda_min": 0.0924575999379158, "lambda_max": 0.4128933548927307, "S_mean": 0.3410237630208333, "G_mean": 0.37422184894482297, "G_std": 0.16553127222384015, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.702563524246216, "grad_norm": 0.7224174737930298, "tokens_seen": 35397632, "tokens_per_sec": 14960.582533899158, "vram_gb": 6.442267648, "lambda": 0.2172152598698934, "lambda_min": 0.09210172295570374, "lambda_max": 0.4143318235874176, "S_mean": 0.3507893880208333, "G_mean": 0.37501811461212736, "G_std": 0.16448035882785916, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.74778938293457, "grad_norm": 0.6991252303123474, "tokens_seen": 35561472, "tokens_per_sec": 14960.856924067291, "vram_gb": 6.442267648, "lambda": 0.21766561611245075, "lambda_min": 0.0921151340007782, "lambda_max": 0.41584381461143494, "S_mean": 0.3537801106770833, "G_mean": 0.37222437079374987, "G_std": 0.16220587437661985, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.824889659881592, "grad_norm": 0.7466633915901184, "tokens_seen": 35725312, "tokens_per_sec": 14961.830735662967, "vram_gb": 6.442267648, "lambda": 0.2182499667008718, "lambda_min": 0.0921730101108551, "lambda_max": 0.41774895787239075, "S_mean": 0.3672281901041667, "G_mean": 0.37534508605798084, "G_std": 0.1641764136341711, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.7022833824157715, "grad_norm": 0.7403384447097778, "tokens_seen": 35889152, "tokens_per_sec": 14962.608066678433, "vram_gb": 6.442267648, "lambda": 0.21872822598864636, "lambda_min": 0.09201526641845703, "lambda_max": 0.41918429732322693, "S_mean": 0.3538411458333333, "G_mean": 0.37400468112900853, "G_std": 0.16523342692138007, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.795382738113403, "grad_norm": 0.7582743167877197, "tokens_seen": 36052992, "tokens_per_sec": 14963.277807444927, "vram_gb": 6.442267648, "lambda": 0.2191599498813351, "lambda_min": 0.09190624207258224, "lambda_max": 0.42046108841896057, "S_mean": 0.3460489908854167, "G_mean": 0.37007787901287276, "G_std": 0.16623659098210433, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.793369293212891, "grad_norm": 0.770072877407074, "tokens_seen": 36216832, "tokens_per_sec": 14964.0618558975, "vram_gb": 6.442267648, "lambda": 0.2196408354987701, "lambda_min": 0.09191835671663284, "lambda_max": 0.42186853289604187, "S_mean": 0.3439839680989583, "G_mean": 0.3758167006696264, "G_std": 0.16484898057145378, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.705977439880371, "grad_norm": 0.755837082862854, "tokens_seen": 36380672, "tokens_per_sec": 14965.120202469618, "vram_gb": 6.442267648, "lambda": 0.2201397263755401, "lambda_min": 0.09192657470703125, "lambda_max": 0.4235735535621643, "S_mean": 0.3535359700520833, "G_mean": 0.37172647185313207, "G_std": 0.16282860181915262, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.524242401123047, "grad_norm": 0.7427080273628235, "tokens_seen": 36544512, "tokens_per_sec": 14965.169233497061, "vram_gb": 6.442267648, "lambda": 0.22058281457672516, "lambda_min": 0.09204316884279251, "lambda_max": 0.4252946674823761, "S_mean": 0.3392842610677083, "G_mean": 0.3712911718369772, "G_std": 0.16362817507858077, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.65875244140625, "grad_norm": 0.701123058795929, "tokens_seen": 36708352, "tokens_per_sec": 14965.290306939798, "vram_gb": 6.442267648, "lambda": 0.22119973910351595, "lambda_min": 0.09204558283090591, "lambda_max": 0.4274047017097473, "S_mean": 0.3640238444010417, "G_mean": 0.37429977968956035, "G_std": 0.16544497401143113, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.938594341278076, "grad_norm": 0.7175213098526001, "tokens_seen": 36872192, "tokens_per_sec": 14966.177397282869, "vram_gb": 6.442267648, "lambda": 0.221638402591149, "lambda_min": 0.09212798625230789, "lambda_max": 0.42902258038520813, "S_mean": 0.3616739908854167, "G_mean": 0.36432913481257856, "G_std": 0.16287093167193234, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.762783992290497, "val_ppl": 117.07139850303214, "lambda": 0.22165990869204202, "lambda_min": 0.09214075654745102, "lambda_max": 0.429113507270813, "lambda_per_layer": [0.15865907073020935, 0.1406642645597458, 0.13875992596149445, 0.13489855825901031, 0.12284549325704575, 0.11863528192043304, 0.09214075654745102, 0.12923508882522583, 0.13316422700881958, 0.24859386682510376, 0.2809579074382782, 0.35760611295700073, 0.3120424151420593, 0.429113507270813, 0.3335786461830139, 0.3430546522140503, 0.27412149310112, 0.2711404263973236, 0.20153696835041046, 0.20522598922252655, 0.28347623348236084, 0.2572091519832611, 0.18319997191429138, 0.1699777990579605], "S_mean": 0.3548583984375, "G_mean": 0.36873408531149227, "G_std": 0.16338593101439378, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.8204076290130615, "grad_norm": 0.7808176875114441, "tokens_seen": 37036032, "tokens_per_sec": 14942.05861633222, "vram_gb": 6.442267648, "lambda": 0.22209187783300877, "lambda_min": 0.09209883958101273, "lambda_max": 0.4310058355331421, "S_mean": 0.3528238932291667, "G_mean": 0.37729213216031593, "G_std": 0.16410378032984832, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.7875072956085205, "grad_norm": 0.7221584916114807, "tokens_seen": 37199872, "tokens_per_sec": 14942.87174749974, "vram_gb": 6.442267648, "lambda": 0.22255874735613665, "lambda_min": 0.09221860021352768, "lambda_max": 0.4325249195098877, "S_mean": 0.3664652506510417, "G_mean": 0.37067429806726676, "G_std": 0.1609570039436221, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.659725904464722, "grad_norm": 0.7283861637115479, "tokens_seen": 37363712, "tokens_per_sec": 14943.716239252717, "vram_gb": 6.442267648, "lambda": 0.22309325262904167, "lambda_min": 0.09226518124341965, "lambda_max": 0.43439361453056335, "S_mean": 0.353668212890625, "G_mean": 0.36570760514587164, "G_std": 0.1642876568560799, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.838052988052368, "grad_norm": 0.7776515483856201, "tokens_seen": 37527552, "tokens_per_sec": 14944.5350055077, "vram_gb": 6.442267648, "lambda": 0.22356062040974697, "lambda_min": 0.0924186259508133, "lambda_max": 0.4359493851661682, "S_mean": 0.3380940755208333, "G_mean": 0.38268600264564157, "G_std": 0.16372426428521672, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.66676139831543, "grad_norm": 0.8216766119003296, "tokens_seen": 37691392, "tokens_per_sec": 14944.9409372208, "vram_gb": 6.442267648, "lambda": 0.2240753968556722, "lambda_min": 0.09244946390390396, "lambda_max": 0.4373224377632141, "S_mean": 0.3312276204427083, "G_mean": 0.3727439334616065, "G_std": 0.16384827819031975, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.680222988128662, "grad_norm": 0.8044083714485168, "tokens_seen": 37855232, "tokens_per_sec": 14945.815160869393, "vram_gb": 6.442267648, "lambda": 0.2244894653558731, "lambda_min": 0.09229863435029984, "lambda_max": 0.4390113353729248, "S_mean": 0.3422953287760417, "G_mean": 0.3703835306999584, "G_std": 0.16056174497740963, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.824900388717651, "grad_norm": 0.7760903835296631, "tokens_seen": 38019072, "tokens_per_sec": 14946.845288521148, "vram_gb": 6.442267648, "lambda": 0.224920516833663, "lambda_min": 0.09216754883527756, "lambda_max": 0.44058874249458313, "S_mean": 0.3583780924479167, "G_mean": 0.36716553568840027, "G_std": 0.16246647123868266, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.6951470375061035, "grad_norm": 0.7583127021789551, "tokens_seen": 38182912, "tokens_per_sec": 14947.792067358216, "vram_gb": 6.442267648, "lambda": 0.2254283813138803, "lambda_min": 0.09177759289741516, "lambda_max": 0.4419364929199219, "S_mean": 0.3432820638020833, "G_mean": 0.3668460439269741, "G_std": 0.1632818439975381, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.819509029388428, "grad_norm": 0.7386686205863953, "tokens_seen": 38346752, "tokens_per_sec": 14948.542104702288, "vram_gb": 6.442267648, "lambda": 0.22585028813531002, "lambda_min": 0.09152461588382721, "lambda_max": 0.4432733654975891, "S_mean": 0.3525594075520833, "G_mean": 0.3677620122519632, "G_std": 0.1628010484855622, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.760105609893799, "grad_norm": 0.7933645248413086, "tokens_seen": 38510592, "tokens_per_sec": 14949.479697077208, "vram_gb": 6.442267648, "lambda": 0.2261975103368362, "lambda_min": 0.09143509715795517, "lambda_max": 0.44456028938293457, "S_mean": 0.3503926595052083, "G_mean": 0.3657983403342466, "G_std": 0.1618801129516214, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.783237934112549, "grad_norm": 0.7653921246528625, "tokens_seen": 38674432, "tokens_per_sec": 14950.306478312223, "vram_gb": 6.442267648, "lambda": 0.22659939713776112, "lambda_min": 0.09126467257738113, "lambda_max": 0.4458428919315338, "S_mean": 0.3355712890625, "G_mean": 0.3694853999962409, "G_std": 0.1647643456235528, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.749821186065674, "grad_norm": 0.7599350810050964, "tokens_seen": 38838272, "tokens_per_sec": 14951.142751888101, "vram_gb": 6.442267648, "lambda": 0.226980771869421, "lambda_min": 0.09093307703733444, "lambda_max": 0.4467817544937134, "S_mean": 0.3397623697916667, "G_mean": 0.3716899997089058, "G_std": 0.16273930350628993, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.806659936904907, "grad_norm": 0.7451069355010986, "tokens_seen": 39002112, "tokens_per_sec": 14951.786316676047, "vram_gb": 6.442267648, "lambda": 0.22744736510018507, "lambda_min": 0.09089573472738266, "lambda_max": 0.4484533667564392, "S_mean": 0.342529296875, "G_mean": 0.3633918932173401, "G_std": 0.1603720939407746, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.616800546646118, "grad_norm": 0.82294762134552, "tokens_seen": 39165952, "tokens_per_sec": 14952.629314077221, "vram_gb": 6.442267648, "lambda": 0.22802451035628715, "lambda_min": 0.09085245430469513, "lambda_max": 0.44968369603157043, "S_mean": 0.3379720052083333, "G_mean": 0.36724060455647606, "G_std": 0.16355448906930783, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.578568935394287, "grad_norm": 0.7516044974327087, "tokens_seen": 39329792, "tokens_per_sec": 14953.407291764308, "vram_gb": 6.442267648, "lambda": 0.22842577969034514, "lambda_min": 0.09094275534152985, "lambda_max": 0.45114195346832275, "S_mean": 0.3510640462239583, "G_mean": 0.36432265241940814, "G_std": 0.1611548124346882, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.748439311981201, "grad_norm": 0.7003540396690369, "tokens_seen": 39493632, "tokens_per_sec": 14954.29721063266, "vram_gb": 6.442267648, "lambda": 0.2288939127077659, "lambda_min": 0.09108225256204605, "lambda_max": 0.45220059156417847, "S_mean": 0.3504740397135417, "G_mean": 0.3663145396082352, "G_std": 0.16175452247262, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.742399215698242, "grad_norm": 0.7565218806266785, "tokens_seen": 39657472, "tokens_per_sec": 14955.012226653691, "vram_gb": 6.442267648, "lambda": 0.22933503209302822, "lambda_min": 0.09110618382692337, "lambda_max": 0.4537615180015564, "S_mean": 0.3504231770833333, "G_mean": 0.3626585485568891, "G_std": 0.16066978561381498, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.827755451202393, "grad_norm": 0.6998435258865356, "tokens_seen": 39821312, "tokens_per_sec": 14956.026921359333, "vram_gb": 6.442267648, "lambda": 0.22985996150722107, "lambda_min": 0.09113314002752304, "lambda_max": 0.455009400844574, "S_mean": 0.3498738606770833, "G_mean": 0.36458040758346516, "G_std": 0.16355925030075014, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.61467981338501, "grad_norm": 0.7409037947654724, "tokens_seen": 39985152, "tokens_per_sec": 14956.757209311007, "vram_gb": 6.442267648, "lambda": 0.2302693168943127, "lambda_min": 0.09106186032295227, "lambda_max": 0.456349641084671, "S_mean": 0.3359578450520833, "G_mean": 0.37304140836931765, "G_std": 0.16054876777343452, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.726568937301636, "grad_norm": 0.7328436970710754, "tokens_seen": 40148992, "tokens_per_sec": 14957.285462142047, "vram_gb": 6.442267648, "lambda": 0.23078510196258625, "lambda_min": 0.09097882360219955, "lambda_max": 0.4578092098236084, "S_mean": 0.3535257975260417, "G_mean": 0.36698070998924476, "G_std": 0.16389430523850024, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.756451606750488, "grad_norm": 0.7347059845924377, "tokens_seen": 40312832, "tokens_per_sec": 14957.84307582818, "vram_gb": 6.442267648, "lambda": 0.23123980034142733, "lambda_min": 0.09085139632225037, "lambda_max": 0.4590793251991272, "S_mean": 0.3559163411458333, "G_mean": 0.36269292514771223, "G_std": 0.1615046712104231, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.515530347824097, "grad_norm": 0.766558051109314, "tokens_seen": 40476672, "tokens_per_sec": 14958.38947368271, "vram_gb": 6.442267648, "lambda": 0.2316896210735043, "lambda_min": 0.09079693257808685, "lambda_max": 0.4606563448905945, "S_mean": 0.3437906901041667, "G_mean": 0.3555888255747656, "G_std": 0.16051079767445722, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.728588104248047, "grad_norm": 0.8283942937850952, "tokens_seen": 40640512, "tokens_per_sec": 14959.026843203506, "vram_gb": 6.442267648, "lambda": 0.23206498908499876, "lambda_min": 0.09066098183393478, "lambda_max": 0.4619552493095398, "S_mean": 0.3583984375, "G_mean": 0.3665480197717746, "G_std": 0.16224760422483087, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.829557657241821, "grad_norm": 0.7454447746276855, "tokens_seen": 40804352, "tokens_per_sec": 14959.60445101209, "vram_gb": 6.442267648, "lambda": 0.23243206708381572, "lambda_min": 0.09071801602840424, "lambda_max": 0.4636691212654114, "S_mean": 0.35009765625, "G_mean": 0.36154867607789737, "G_std": 0.161949361829708, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.707052707672119, "grad_norm": 0.8290592432022095, "tokens_seen": 40968192, "tokens_per_sec": 14960.299458699643, "vram_gb": 6.442267648, "lambda": 0.23282447333137193, "lambda_min": 0.0908288061618805, "lambda_max": 0.46494418382644653, "S_mean": 0.3499552408854167, "G_mean": 0.36828932468779385, "G_std": 0.16070663936746618, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.731944406032563, "val_ppl": 113.51606923153702, "lambda": 0.23285239127775034, "lambda_min": 0.09082267433404922, "lambda_max": 0.46503040194511414, "lambda_per_layer": [0.1607099026441574, 0.14045226573944092, 0.13957110047340393, 0.13514205813407898, 0.12259044498205185, 0.11893695592880249, 0.09082267433404922, 0.1291569471359253, 0.13287116587162018, 0.2567298710346222, 0.2962147295475006, 0.38106200098991394, 0.32808753848075867, 0.46503040194511414, 0.3577825129032135, 0.3690206706523895, 0.2920509874820709, 0.2938118577003479, 0.21229392290115356, 0.21380336582660675, 0.309128999710083, 0.2763102054595947, 0.19052253663539886, 0.1763542741537094], "S_mean": 0.3499857584635417, "G_mean": 0.3563940979850789, "G_std": 0.16106931391792992, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.658163547515869, "grad_norm": 0.7642408609390259, "tokens_seen": 41132032, "tokens_per_sec": 14938.441049274608, "vram_gb": 6.442267648, "lambda": 0.2333515811090668, "lambda_min": 0.09075790643692017, "lambda_max": 0.46672698855400085, "S_mean": 0.3548787434895833, "G_mean": 0.3608268881992747, "G_std": 0.16149762940282622, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.609882593154907, "grad_norm": 0.7650431394577026, "tokens_seen": 41295872, "tokens_per_sec": 14939.128409200264, "vram_gb": 6.442267648, "lambda": 0.23382905467102924, "lambda_min": 0.09090696275234222, "lambda_max": 0.4679049253463745, "S_mean": 0.3488260904947917, "G_mean": 0.36591486919981736, "G_std": 0.16006658280578753, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.739856481552124, "grad_norm": 0.7541913986206055, "tokens_seen": 41459712, "tokens_per_sec": 14939.575393340903, "vram_gb": 6.442267648, "lambda": 0.23427053758253655, "lambda_min": 0.0910433903336525, "lambda_max": 0.4691133201122284, "S_mean": 0.3495890299479167, "G_mean": 0.36017881869338453, "G_std": 0.1615553874677668, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.742357969284058, "grad_norm": 0.7898088693618774, "tokens_seen": 41623552, "tokens_per_sec": 14940.466026991135, "vram_gb": 6.442267648, "lambda": 0.23478025694688162, "lambda_min": 0.09101375937461853, "lambda_max": 0.4704517126083374, "S_mean": 0.3400980631510417, "G_mean": 0.3644076492637396, "G_std": 0.16325070972864827, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.6319661140441895, "grad_norm": 0.7908825278282166, "tokens_seen": 41787392, "tokens_per_sec": 14941.33189857172, "vram_gb": 6.442267648, "lambda": 0.2351724530259768, "lambda_min": 0.09078652411699295, "lambda_max": 0.47182631492614746, "S_mean": 0.36199951171875, "G_mean": 0.3611690717904518, "G_std": 0.1603031406799952, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.751363039016724, "grad_norm": 0.7347491383552551, "tokens_seen": 41951232, "tokens_per_sec": 14942.122666244806, "vram_gb": 6.442267648, "lambda": 0.23561857702831426, "lambda_min": 0.09076167643070221, "lambda_max": 0.47306400537490845, "S_mean": 0.3443603515625, "G_mean": 0.3644266583335896, "G_std": 0.1633606934143851, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.701833724975586, "grad_norm": 0.7181615829467773, "tokens_seen": 42115072, "tokens_per_sec": 14942.370643248634, "vram_gb": 6.442267648, "lambda": 0.23607436319192251, "lambda_min": 0.09082957357168198, "lambda_max": 0.4737728238105774, "S_mean": 0.341888427734375, "G_mean": 0.3563565543542306, "G_std": 0.1603432200693836, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.735419034957886, "grad_norm": 0.7689650058746338, "tokens_seen": 42278912, "tokens_per_sec": 14942.970051088083, "vram_gb": 6.442267648, "lambda": 0.23655954655259848, "lambda_min": 0.09101351350545883, "lambda_max": 0.4753109812736511, "S_mean": 0.3539632161458333, "G_mean": 0.36777909006923437, "G_std": 0.16207960566195348, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.764625310897827, "grad_norm": 0.8344626426696777, "tokens_seen": 42442752, "tokens_per_sec": 14943.730167463636, "vram_gb": 6.442267648, "lambda": 0.23710218320290247, "lambda_min": 0.09125538915395737, "lambda_max": 0.477342426776886, "S_mean": 0.37725830078125, "G_mean": 0.3640374782650421, "G_std": 0.16130817658267915, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.7404868602752686, "grad_norm": 0.8224921226501465, "tokens_seen": 42606592, "tokens_per_sec": 14944.20369449882, "vram_gb": 6.442267648, "lambda": 0.23754268481085697, "lambda_min": 0.09131323546171188, "lambda_max": 0.4789014756679535, "S_mean": 0.3503519694010417, "G_mean": 0.3649633709186067, "G_std": 0.1632276064095398, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.541479110717773, "grad_norm": 0.7600233554840088, "tokens_seen": 42770432, "tokens_per_sec": 14944.636802553576, "vram_gb": 6.442267648, "lambda": 0.23798670681814352, "lambda_min": 0.09131844341754913, "lambda_max": 0.4803173840045929, "S_mean": 0.3584391276041667, "G_mean": 0.3614440616996338, "G_std": 0.1604390429177632, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.877533912658691, "grad_norm": 0.8202681541442871, "tokens_seen": 42934272, "tokens_per_sec": 14945.21606071288, "vram_gb": 6.442267648, "lambda": 0.23840092246731123, "lambda_min": 0.09130637347698212, "lambda_max": 0.48185545206069946, "S_mean": 0.330535888671875, "G_mean": 0.3659230852499604, "G_std": 0.16018424501332143, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.6210572719573975, "grad_norm": 0.7690300345420837, "tokens_seen": 43098112, "tokens_per_sec": 14945.8867229782, "vram_gb": 6.442267648, "lambda": 0.2388043226674199, "lambda_min": 0.09143209457397461, "lambda_max": 0.48331815004348755, "S_mean": 0.356048583984375, "G_mean": 0.36463975238924223, "G_std": 0.16432413435541093, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.5235350131988525, "grad_norm": 0.7538413405418396, "tokens_seen": 43261952, "tokens_per_sec": 14946.541919751673, "vram_gb": 6.442267648, "lambda": 0.23911674538006386, "lambda_min": 0.0916772112250328, "lambda_max": 0.4842711091041565, "S_mean": 0.3455708821614583, "G_mean": 0.36188951080354553, "G_std": 0.15901599490704635, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.664088010787964, "grad_norm": 0.7836763262748718, "tokens_seen": 43425792, "tokens_per_sec": 14947.200905294683, "vram_gb": 6.442267648, "lambda": 0.23948627368857464, "lambda_min": 0.09191731363534927, "lambda_max": 0.48556840419769287, "S_mean": 0.3545939127604167, "G_mean": 0.3604513762208323, "G_std": 0.157894025867184, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.637540340423584, "grad_norm": 0.7732751965522766, "tokens_seen": 43589632, "tokens_per_sec": 14947.937218062016, "vram_gb": 6.442267648, "lambda": 0.23991193156689405, "lambda_min": 0.09196031093597412, "lambda_max": 0.48666274547576904, "S_mean": 0.3481547037760417, "G_mean": 0.3604962717120846, "G_std": 0.16195249961068234, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.724172115325928, "grad_norm": 0.7678517699241638, "tokens_seen": 43753472, "tokens_per_sec": 14948.826864123686, "vram_gb": 6.442267648, "lambda": 0.2403493756428361, "lambda_min": 0.09198565781116486, "lambda_max": 0.4883117079734802, "S_mean": 0.3438924153645833, "G_mean": 0.3652848042547703, "G_std": 0.15971462009474635, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.498904466629028, "grad_norm": 0.7591535449028015, "tokens_seen": 43917312, "tokens_per_sec": 14949.32989954839, "vram_gb": 6.442267648, "lambda": 0.2407885016873479, "lambda_min": 0.09191948920488358, "lambda_max": 0.4895465672016144, "S_mean": 0.3460489908854167, "G_mean": 0.3511878954401861, "G_std": 0.1587901059538126, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.522265911102295, "grad_norm": 0.8191450238227844, "tokens_seen": 44081152, "tokens_per_sec": 14949.956190752502, "vram_gb": 6.442267648, "lambda": 0.24116604837278524, "lambda_min": 0.0921413004398346, "lambda_max": 0.4908551573753357, "S_mean": 0.351226806640625, "G_mean": 0.3664879105829944, "G_std": 0.16050770265671113, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.55047607421875, "grad_norm": 0.7634784579277039, "tokens_seen": 44244992, "tokens_per_sec": 14950.778310506705, "vram_gb": 6.442267648, "lambda": 0.24158635331938663, "lambda_min": 0.09217151999473572, "lambda_max": 0.4924095869064331, "S_mean": 0.3542277018229167, "G_mean": 0.36109936485687893, "G_std": 0.158915970629702, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.632843971252441, "grad_norm": 0.7410336136817932, "tokens_seen": 44408832, "tokens_per_sec": 14951.374918914174, "vram_gb": 6.442267648, "lambda": 0.24204647106428942, "lambda_min": 0.09207499772310257, "lambda_max": 0.4939776062965393, "S_mean": 0.3501180013020833, "G_mean": 0.35352831791775924, "G_std": 0.16095604235306382, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.769007921218872, "grad_norm": 0.794490396976471, "tokens_seen": 44572672, "tokens_per_sec": 14951.985858853735, "vram_gb": 6.442267648, "lambda": 0.24247047708680233, "lambda_min": 0.09212545305490494, "lambda_max": 0.49522316455841064, "S_mean": 0.34759521484375, "G_mean": 0.36126538272947073, "G_std": 0.16228265633496144, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.657814025878906, "grad_norm": 0.7944726347923279, "tokens_seen": 44736512, "tokens_per_sec": 14952.626783993417, "vram_gb": 6.442267648, "lambda": 0.24286914492646852, "lambda_min": 0.09217700362205505, "lambda_max": 0.4966190457344055, "S_mean": 0.3357340494791667, "G_mean": 0.3565722156781703, "G_std": 0.16031142277643085, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.652551174163818, "grad_norm": 0.7739026546478271, "tokens_seen": 44900352, "tokens_per_sec": 14953.590318032433, "vram_gb": 6.442267648, "lambda": 0.24332997233917317, "lambda_min": 0.09217878431081772, "lambda_max": 0.498185396194458, "S_mean": 0.3453776041666667, "G_mean": 0.35821183398365974, "G_std": 0.16108927767102918, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.642356872558594, "grad_norm": 0.8133164048194885, "tokens_seen": 45064192, "tokens_per_sec": 14954.384261489598, "vram_gb": 6.442267648, "lambda": 0.24383877466122308, "lambda_min": 0.09214165806770325, "lambda_max": 0.49991342425346375, "S_mean": 0.3380330403645833, "G_mean": 0.3584972876124084, "G_std": 0.16051056561991572, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.67000390291214, "val_ppl": 106.69815886517773, "lambda": 0.24385813685754934, "lambda_min": 0.09214049577713013, "lambda_max": 0.4999983012676239, "lambda_per_layer": [0.16345354914665222, 0.14134861528873444, 0.1403546929359436, 0.13561849296092987, 0.12241953611373901, 0.11882179975509644, 0.09214049577713013, 0.12889379262924194, 0.13308623433113098, 0.26494327187538147, 0.3077579736709595, 0.4033767282962799, 0.3415338397026062, 0.4999983012676239, 0.38209325075149536, 0.39689525961875916, 0.3117201626300812, 0.3141467273235321, 0.2214781939983368, 0.2222103625535965, 0.33263450860977173, 0.2979433834552765, 0.19786985218524933, 0.18185625970363617], "S_mean": 0.3310648600260417, "G_mean": 0.354944600413243, "G_std": 0.16098405645849803, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.496841669082642, "grad_norm": 0.7582670450210571, "tokens_seen": 45228032, "tokens_per_sec": 14935.065367045563, "vram_gb": 6.442267648, "lambda": 0.24429708377768597, "lambda_min": 0.0921604111790657, "lambda_max": 0.5014298558235168, "S_mean": 0.3254801432291667, "G_mean": 0.3590108932306369, "G_std": 0.159815886678795, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.625713109970093, "grad_norm": 0.8340216875076294, "tokens_seen": 45391872, "tokens_per_sec": 14935.830823540638, "vram_gb": 6.442267648, "lambda": 0.24473298185815415, "lambda_min": 0.09235140681266785, "lambda_max": 0.5027926564216614, "S_mean": 0.3480631510416667, "G_mean": 0.3618115121498704, "G_std": 0.1613574407529086, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.650115013122559, "grad_norm": 0.8026201725006104, "tokens_seen": 45555712, "tokens_per_sec": 14936.28226060252, "vram_gb": 6.442267648, "lambda": 0.24521021420756975, "lambda_min": 0.09254375100135803, "lambda_max": 0.5043038725852966, "S_mean": 0.3207906087239583, "G_mean": 0.3657219629579534, "G_std": 0.1627287797164172, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.484262466430664, "grad_norm": 0.7676271200180054, "tokens_seen": 45719552, "tokens_per_sec": 14936.862714525143, "vram_gb": 6.442267648, "lambda": 0.24560947405795255, "lambda_min": 0.09272035211324692, "lambda_max": 0.5053584575653076, "S_mean": 0.3326212565104167, "G_mean": 0.3584932784239451, "G_std": 0.16091660200618207, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.693345546722412, "grad_norm": 0.8382835388183594, "tokens_seen": 45883392, "tokens_per_sec": 14937.622856107959, "vram_gb": 6.442267648, "lambda": 0.24600102876623473, "lambda_min": 0.09262886643409729, "lambda_max": 0.505996823310852, "S_mean": 0.3513997395833333, "G_mean": 0.35319007793441415, "G_std": 0.16004470999663076, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.697345018386841, "grad_norm": 0.7863849997520447, "tokens_seen": 46047232, "tokens_per_sec": 14938.529217227648, "vram_gb": 6.442267648, "lambda": 0.24634672546138367, "lambda_min": 0.0925438404083252, "lambda_max": 0.5067480802536011, "S_mean": 0.3313802083333333, "G_mean": 0.35618873957234126, "G_std": 0.1617474187320719, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.628992557525635, "grad_norm": 0.763202428817749, "tokens_seen": 46211072, "tokens_per_sec": 14939.264981689024, "vram_gb": 6.442267648, "lambda": 0.2467571816717585, "lambda_min": 0.09274151921272278, "lambda_max": 0.5079578757286072, "S_mean": 0.342529296875, "G_mean": 0.35739188532655436, "G_std": 0.16155644382039705, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.74648642539978, "grad_norm": 0.8191177248954773, "tokens_seen": 46374912, "tokens_per_sec": 14940.06889618567, "vram_gb": 6.442267648, "lambda": 0.24718666914850473, "lambda_min": 0.09269598126411438, "lambda_max": 0.5093175172805786, "S_mean": 0.3394571940104167, "G_mean": 0.3566814415777723, "G_std": 0.1593598018710812, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.780634880065918, "grad_norm": 0.8267871737480164, "tokens_seen": 46538752, "tokens_per_sec": 14940.876671833912, "vram_gb": 6.442267648, "lambda": 0.24764590648313364, "lambda_min": 0.09267624467611313, "lambda_max": 0.5105971693992615, "S_mean": 0.3415018717447917, "G_mean": 0.3571089179410289, "G_std": 0.15968936759357652, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.590611934661865, "grad_norm": 0.7870538830757141, "tokens_seen": 46702592, "tokens_per_sec": 14941.821746310412, "vram_gb": 6.442267648, "lambda": 0.2480700397863984, "lambda_min": 0.09270121902227402, "lambda_max": 0.5116732716560364, "S_mean": 0.332122802734375, "G_mean": 0.352999451259772, "G_std": 0.1617538418310384, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.487732648849487, "grad_norm": 0.8215048313140869, "tokens_seen": 46866432, "tokens_per_sec": 14942.6674210334, "vram_gb": 6.442267648, "lambda": 0.248445271824797, "lambda_min": 0.09276922792196274, "lambda_max": 0.5130230188369751, "S_mean": 0.3422444661458333, "G_mean": 0.35537890042178333, "G_std": 0.15863800584338605, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.614208698272705, "grad_norm": 0.7790441513061523, "tokens_seen": 47030272, "tokens_per_sec": 14943.347320219247, "vram_gb": 6.442267648, "lambda": 0.24892320235570273, "lambda_min": 0.09283076226711273, "lambda_max": 0.5143323540687561, "S_mean": 0.3436279296875, "G_mean": 0.35398017001959187, "G_std": 0.16019859739268819, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.678285121917725, "grad_norm": 0.7640524506568909, "tokens_seen": 47194112, "tokens_per_sec": 14943.968881349385, "vram_gb": 6.442267648, "lambda": 0.2492934986948967, "lambda_min": 0.09284332394599915, "lambda_max": 0.5155850648880005, "S_mean": 0.347625732421875, "G_mean": 0.3527222261764109, "G_std": 0.15966467567098638, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.608823537826538, "grad_norm": 0.8011114597320557, "tokens_seen": 47357952, "tokens_per_sec": 14944.576585074185, "vram_gb": 6.442267648, "lambda": 0.24963949931164583, "lambda_min": 0.09289102256298065, "lambda_max": 0.5168377757072449, "S_mean": 0.3371073404947917, "G_mean": 0.35647912230342627, "G_std": 0.16105741262435913, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.58683180809021, "grad_norm": 0.8067581653594971, "tokens_seen": 47521792, "tokens_per_sec": 14945.022191885146, "vram_gb": 6.442267648, "lambda": 0.2500800058866541, "lambda_min": 0.09291253238916397, "lambda_max": 0.5180702209472656, "S_mean": 0.33544921875, "G_mean": 0.3555862130597234, "G_std": 0.1607003315196683, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.56153678894043, "grad_norm": 0.7625550031661987, "tokens_seen": 47685632, "tokens_per_sec": 14945.547882703298, "vram_gb": 6.442267648, "lambda": 0.25037563281754655, "lambda_min": 0.09308464080095291, "lambda_max": 0.5191401243209839, "S_mean": 0.3454996744791667, "G_mean": 0.35599654835338396, "G_std": 0.15880124433897436, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.555909872055054, "grad_norm": 0.8085063695907593, "tokens_seen": 47849472, "tokens_per_sec": 14946.017866973325, "vram_gb": 6.442267648, "lambda": 0.250719763028125, "lambda_min": 0.09306637942790985, "lambda_max": 0.5203598737716675, "S_mean": 0.339324951171875, "G_mean": 0.354915084472547, "G_std": 0.16025755849356452, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.5478997230529785, "grad_norm": 0.7609067559242249, "tokens_seen": 48013312, "tokens_per_sec": 14946.527020237974, "vram_gb": 6.442267648, "lambda": 0.2510459451004863, "lambda_min": 0.09296217560768127, "lambda_max": 0.5212231874465942, "S_mean": 0.3459065755208333, "G_mean": 0.35425161422851187, "G_std": 0.1592990313656628, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.532185077667236, "grad_norm": 0.7745428085327148, "tokens_seen": 48177152, "tokens_per_sec": 14947.110307204612, "vram_gb": 6.442267648, "lambda": 0.25139578897506, "lambda_min": 0.09290779381990433, "lambda_max": 0.5227399468421936, "S_mean": 0.3580322265625, "G_mean": 0.35190622399871546, "G_std": 0.1597727567423135, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.4983811378479, "grad_norm": 0.7513786554336548, "tokens_seen": 48340992, "tokens_per_sec": 14947.65339849346, "vram_gb": 6.442267648, "lambda": 0.25180616540213424, "lambda_min": 0.09264710545539856, "lambda_max": 0.5239930152893066, "S_mean": 0.33599853515625, "G_mean": 0.35850760193231207, "G_std": 0.15980547736398876, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.617971181869507, "grad_norm": 0.7928863167762756, "tokens_seen": 48504832, "tokens_per_sec": 14948.281102408751, "vram_gb": 6.442267648, "lambda": 0.2521887455756466, "lambda_min": 0.09258409589529037, "lambda_max": 0.5251655578613281, "S_mean": 0.33966064453125, "G_mean": 0.3557229045157631, "G_std": 0.15984629350714386, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.591660737991333, "grad_norm": 0.833747923374176, "tokens_seen": 48668672, "tokens_per_sec": 14948.980305167246, "vram_gb": 6.442267648, "lambda": 0.25253822685529786, "lambda_min": 0.09261707216501236, "lambda_max": 0.5261405110359192, "S_mean": 0.3364664713541667, "G_mean": 0.35688715195283294, "G_std": 0.16156872594729066, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.7371838092803955, "grad_norm": 0.831041693687439, "tokens_seen": 48832512, "tokens_per_sec": 14949.778728707688, "vram_gb": 6.442267648, "lambda": 0.2529512144004305, "lambda_min": 0.09260225296020508, "lambda_max": 0.5273484587669373, "S_mean": 0.34625244140625, "G_mean": 0.3608042790244023, "G_std": 0.16122953842083612, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.529332637786865, "grad_norm": 0.767030656337738, "tokens_seen": 48996352, "tokens_per_sec": 14950.459988762095, "vram_gb": 6.442267648, "lambda": 0.25331244245171547, "lambda_min": 0.09255249798297882, "lambda_max": 0.528659999370575, "S_mean": 0.349365234375, "G_mean": 0.34914714004844427, "G_std": 0.1594164336565882, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.556685924530029, "grad_norm": 0.8392241597175598, "tokens_seen": 49160192, "tokens_per_sec": 14951.124439694542, "vram_gb": 6.442267648, "lambda": 0.2536272695288062, "lambda_min": 0.09251778572797775, "lambda_max": 0.5293651223182678, "S_mean": 0.3493143717447917, "G_mean": 0.349960605148226, "G_std": 0.16014344641007483, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.683038997650146, "val_ppl": 108.09808372882365, "lambda": 0.25363995383183163, "lambda_min": 0.09251642227172852, "lambda_max": 0.5294021368026733, "lambda_per_layer": [0.16552026569843292, 0.14097443222999573, 0.14122851192951202, 0.1359502077102661, 0.12188171595335007, 0.11966634541749954, 0.09251642227172852, 0.12794318795204163, 0.1323608011007309, 0.2708246111869812, 0.31926366686820984, 0.42136457562446594, 0.353329062461853, 0.5294021368026733, 0.4040609896183014, 0.4216923117637634, 0.3304087817668915, 0.33286890387535095, 0.2310563176870346, 0.23010878264904022, 0.3544543385505676, 0.3177002966403961, 0.2050570398569107, 0.18772518634796143], "S_mean": 0.34881591796875, "G_mean": 0.3593328814022243, "G_std": 0.16091886915576956, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.6428022384643555, "grad_norm": 0.8299883008003235, "tokens_seen": 49324032, "tokens_per_sec": 14933.268764871087, "vram_gb": 6.442267648, "lambda": 0.2539903636400898, "lambda_min": 0.09253253787755966, "lambda_max": 0.5308772325515747, "S_mean": 0.3605143229166667, "G_mean": 0.35764261754229665, "G_std": 0.16011639048034945, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.508270740509033, "grad_norm": 0.8103289008140564, "tokens_seen": 49487872, "tokens_per_sec": 14933.839831869354, "vram_gb": 6.442267648, "lambda": 0.2544513779381911, "lambda_min": 0.09263687580823898, "lambda_max": 0.5323760509490967, "S_mean": 0.3407999674479167, "G_mean": 0.3578381221741438, "G_std": 0.16170298346939185, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.433924674987793, "grad_norm": 0.7480409741401672, "tokens_seen": 49651712, "tokens_per_sec": 14934.526154920519, "vram_gb": 6.442267648, "lambda": 0.2547763880963127, "lambda_min": 0.09271036088466644, "lambda_max": 0.533511221408844, "S_mean": 0.3563028971354167, "G_mean": 0.34918899554759264, "G_std": 0.1572897486233463, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.435328960418701, "grad_norm": 0.8119040727615356, "tokens_seen": 49815552, "tokens_per_sec": 14935.034620912109, "vram_gb": 6.442267648, "lambda": 0.25517619370172423, "lambda_min": 0.09267614781856537, "lambda_max": 0.5349963307380676, "S_mean": 0.3535563151041667, "G_mean": 0.35176146410716075, "G_std": 0.1600465717880676, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.45570182800293, "grad_norm": 0.7683441638946533, "tokens_seen": 49979392, "tokens_per_sec": 14935.613787929336, "vram_gb": 6.442267648, "lambda": 0.2555143569285671, "lambda_min": 0.09265631437301636, "lambda_max": 0.5356365442276001, "S_mean": 0.3550211588541667, "G_mean": 0.352242373318101, "G_std": 0.16189163628344735, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.553733825683594, "grad_norm": 0.7775483131408691, "tokens_seen": 50143232, "tokens_per_sec": 14935.956037705873, "vram_gb": 6.442267648, "lambda": 0.25584579569598037, "lambda_min": 0.09251397848129272, "lambda_max": 0.5371044874191284, "S_mean": 0.3602294921875, "G_mean": 0.3511818471985559, "G_std": 0.16017157522340617, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.479326248168945, "grad_norm": 0.7818173170089722, "tokens_seen": 50307072, "tokens_per_sec": 14936.423796273075, "vram_gb": 6.442267648, "lambda": 0.256162135861814, "lambda_min": 0.09248095005750656, "lambda_max": 0.53841233253479, "S_mean": 0.3532307942708333, "G_mean": 0.3465487485130628, "G_std": 0.15983384498395026, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.488524913787842, "grad_norm": 0.834567666053772, "tokens_seen": 50470912, "tokens_per_sec": 14936.954261961242, "vram_gb": 6.442267648, "lambda": 0.2565131727606058, "lambda_min": 0.09263411164283752, "lambda_max": 0.5397502183914185, "S_mean": 0.3508504231770833, "G_mean": 0.35290113432953757, "G_std": 0.16018534071433047, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.624014377593994, "grad_norm": 0.8928918242454529, "tokens_seen": 50634752, "tokens_per_sec": 14937.601662801651, "vram_gb": 6.442267648, "lambda": 0.2568654340381424, "lambda_min": 0.09260633587837219, "lambda_max": 0.5412040948867798, "S_mean": 0.3453776041666667, "G_mean": 0.3530101820360869, "G_std": 0.16204463395600519, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.548815011978149, "grad_norm": 0.8035106658935547, "tokens_seen": 50798592, "tokens_per_sec": 14938.22796785331, "vram_gb": 6.442267648, "lambda": 0.2571197769915064, "lambda_min": 0.09259776026010513, "lambda_max": 0.541719377040863, "S_mean": 0.3516031901041667, "G_mean": 0.3543983727383117, "G_std": 0.16302607161924243, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.581708908081055, "grad_norm": 0.8158683776855469, "tokens_seen": 50962432, "tokens_per_sec": 14938.684963856034, "vram_gb": 6.442267648, "lambda": 0.25745477589468163, "lambda_min": 0.09255437552928925, "lambda_max": 0.5427065491676331, "S_mean": 0.3474527994791667, "G_mean": 0.3473441960910956, "G_std": 0.16031282022595406, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.517530679702759, "grad_norm": 0.8276153206825256, "tokens_seen": 51126272, "tokens_per_sec": 14939.05683442869, "vram_gb": 6.442267648, "lambda": 0.2578473376731078, "lambda_min": 0.09255167841911316, "lambda_max": 0.5439901351928711, "S_mean": 0.3558146158854167, "G_mean": 0.35217455339928466, "G_std": 0.16121924008863667, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.541991472244263, "grad_norm": 0.8091548681259155, "tokens_seen": 51290112, "tokens_per_sec": 14939.210447561409, "vram_gb": 6.442267648, "lambda": 0.25821321550756693, "lambda_min": 0.0924597829580307, "lambda_max": 0.5452892780303955, "S_mean": 0.344085693359375, "G_mean": 0.35635251831263304, "G_std": 0.15848677807177106, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.613469123840332, "grad_norm": 0.8237268328666687, "tokens_seen": 51453952, "tokens_per_sec": 14939.85228364589, "vram_gb": 6.442267648, "lambda": 0.25856153449664515, "lambda_min": 0.09246862679719925, "lambda_max": 0.5461412668228149, "S_mean": 0.347747802734375, "G_mean": 0.3508106314887603, "G_std": 0.1598949239899715, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.476658821105957, "grad_norm": 0.8207014799118042, "tokens_seen": 51617792, "tokens_per_sec": 14940.489860657868, "vram_gb": 6.442267648, "lambda": 0.25889758486300707, "lambda_min": 0.0925838053226471, "lambda_max": 0.5470849871635437, "S_mean": 0.3446756998697917, "G_mean": 0.3486305832241972, "G_std": 0.16043244185857475, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.6250951290130615, "grad_norm": 0.8260527849197388, "tokens_seen": 51781632, "tokens_per_sec": 14941.024028249585, "vram_gb": 6.442267648, "lambda": 0.25918776510904235, "lambda_min": 0.09260241687297821, "lambda_max": 0.5479332208633423, "S_mean": 0.338043212890625, "G_mean": 0.34927801275625825, "G_std": 0.1608201953737686, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.456175804138184, "grad_norm": 0.8207668662071228, "tokens_seen": 51945472, "tokens_per_sec": 14941.786874457097, "vram_gb": 6.442267648, "lambda": 0.2595130726695061, "lambda_min": 0.09272711724042892, "lambda_max": 0.5491175055503845, "S_mean": 0.3421834309895833, "G_mean": 0.3484239195628713, "G_std": 0.15973071629802385, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.547425746917725, "grad_norm": 0.8529886603355408, "tokens_seen": 52109312, "tokens_per_sec": 14942.142737570062, "vram_gb": 6.442267648, "lambda": 0.25988756337513524, "lambda_min": 0.09274066984653473, "lambda_max": 0.5505583882331848, "S_mean": 0.3572489420572917, "G_mean": 0.35815933633906144, "G_std": 0.16075456996137896, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.584128141403198, "grad_norm": 0.8580735325813293, "tokens_seen": 52273152, "tokens_per_sec": 14942.781251991746, "vram_gb": 6.442267648, "lambda": 0.26029288582503796, "lambda_min": 0.09271508455276489, "lambda_max": 0.5517944693565369, "S_mean": 0.34552001953125, "G_mean": 0.35128619133805233, "G_std": 0.15844239317812026, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.521570682525635, "grad_norm": 0.8358850479125977, "tokens_seen": 52436992, "tokens_per_sec": 14943.408164737151, "vram_gb": 6.442267648, "lambda": 0.260696267709136, "lambda_min": 0.09258171170949936, "lambda_max": 0.5530484914779663, "S_mean": 0.36474609375, "G_mean": 0.35674855252727866, "G_std": 0.1602527516273161, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.573185443878174, "grad_norm": 0.8441569209098816, "tokens_seen": 52600832, "tokens_per_sec": 14944.074364940745, "vram_gb": 6.442267648, "lambda": 0.26106796599924564, "lambda_min": 0.09270195662975311, "lambda_max": 0.5542044043540955, "S_mean": 0.353759765625, "G_mean": 0.3527419176728775, "G_std": 0.15912978110524514, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.542093515396118, "grad_norm": 0.7950953245162964, "tokens_seen": 52764672, "tokens_per_sec": 14944.552893946246, "vram_gb": 6.442267648, "lambda": 0.2614184453462561, "lambda_min": 0.09270571172237396, "lambda_max": 0.5553386807441711, "S_mean": 0.33856201171875, "G_mean": 0.35877512434187037, "G_std": 0.1612077991788586, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.619701147079468, "grad_norm": 0.8325158953666687, "tokens_seen": 52928512, "tokens_per_sec": 14945.088094737866, "vram_gb": 6.442267648, "lambda": 0.26175570612152416, "lambda_min": 0.09272019565105438, "lambda_max": 0.5559559464454651, "S_mean": 0.343170166015625, "G_mean": 0.35162812906006974, "G_std": 0.1588913882151246, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.414281845092773, "grad_norm": 0.809623658657074, "tokens_seen": 53092352, "tokens_per_sec": 14945.615641833447, "vram_gb": 6.442267648, "lambda": 0.2621135550240676, "lambda_min": 0.09285847097635269, "lambda_max": 0.5571504831314087, "S_mean": 0.3415120442708333, "G_mean": 0.347916381744047, "G_std": 0.1593043312896043, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.503525257110596, "grad_norm": 0.8062947988510132, "tokens_seen": 53256192, "tokens_per_sec": 14946.166184626785, "vram_gb": 6.442267648, "lambda": 0.26240216195583344, "lambda_min": 0.09302084892988205, "lambda_max": 0.5582990050315857, "S_mean": 0.3382466634114583, "G_mean": 0.3524323896660159, "G_std": 0.1619461098841081, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.631424367427826, "val_ppl": 102.66018584512157, "lambda": 0.2624150461827715, "lambda_min": 0.09302400797605515, "lambda_max": 0.5583599805831909, "lambda_per_layer": [0.1669444441795349, 0.14154881238937378, 0.14213255047798157, 0.13660451769828796, 0.12160956859588623, 0.11856536567211151, 0.09302400797605515, 0.12771154940128326, 0.13320957124233246, 0.27595874667167664, 0.32819315791130066, 0.43706363439559937, 0.3636455237865448, 0.5583599805831909, 0.42347094416618347, 0.4429659843444824, 0.3467627167701721, 0.3492734134197235, 0.2401648312807083, 0.23856358230113983, 0.37451013922691345, 0.3341681957244873, 0.21121542155742645, 0.19229444861412048], "S_mean": 0.3365987141927083, "G_mean": 0.349411576675872, "G_std": 0.16046778244587281, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.480672836303711, "grad_norm": 0.7640199065208435, "tokens_seen": 53420032, "tokens_per_sec": 14929.319083490334, "vram_gb": 6.442267648, "lambda": 0.2627305034548044, "lambda_min": 0.09308820217847824, "lambda_max": 0.5599650144577026, "S_mean": 0.3446858723958333, "G_mean": 0.3442771936922024, "G_std": 0.1598358512079964, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.624432802200317, "grad_norm": 0.8165052533149719, "tokens_seen": 53583872, "tokens_per_sec": 14929.69937950588, "vram_gb": 6.442267648, "lambda": 0.2630973511065046, "lambda_min": 0.09301704168319702, "lambda_max": 0.5610051155090332, "S_mean": 0.3299967447916667, "G_mean": 0.3450061803838859, "G_std": 0.15879127856654426, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.57987380027771, "grad_norm": 0.829293429851532, "tokens_seen": 53747712, "tokens_per_sec": 14930.040565597636, "vram_gb": 6.442267648, "lambda": 0.26348271903892356, "lambda_min": 0.09309165924787521, "lambda_max": 0.5620871782302856, "S_mean": 0.348419189453125, "G_mean": 0.35383700113743544, "G_std": 0.15868659814198813, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.521803140640259, "grad_norm": 0.7783628702163696, "tokens_seen": 53911552, "tokens_per_sec": 14930.751704814553, "vram_gb": 6.442267648, "lambda": 0.26382218146075803, "lambda_min": 0.09300264716148376, "lambda_max": 0.5632978677749634, "S_mean": 0.34161376953125, "G_mean": 0.34668865248871344, "G_std": 0.157362707114468, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.327305555343628, "grad_norm": 0.7726619839668274, "tokens_seen": 54075392, "tokens_per_sec": 14931.649005662042, "vram_gb": 6.442267648, "lambda": 0.264100993052125, "lambda_min": 0.09299203753471375, "lambda_max": 0.5642259120941162, "S_mean": 0.34478759765625, "G_mean": 0.3442901980597526, "G_std": 0.15864643330375353, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.746875047683716, "grad_norm": 0.888484001159668, "tokens_seen": 54239232, "tokens_per_sec": 14932.244044261197, "vram_gb": 6.442267648, "lambda": 0.26442968907455605, "lambda_min": 0.09293652325868607, "lambda_max": 0.5651529431343079, "S_mean": 0.3288370768229167, "G_mean": 0.345340166396151, "G_std": 0.15814332648490867, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.537554502487183, "grad_norm": 0.8388636112213135, "tokens_seen": 54403072, "tokens_per_sec": 14932.754037618073, "vram_gb": 6.442267648, "lambda": 0.26477554192145664, "lambda_min": 0.09277110546827316, "lambda_max": 0.5661548376083374, "S_mean": 0.3418680826822917, "G_mean": 0.3517673848352085, "G_std": 0.160913618048653, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.497804880142212, "grad_norm": 0.8335444927215576, "tokens_seen": 54566912, "tokens_per_sec": 14932.739174691074, "vram_gb": 6.442267648, "lambda": 0.26517605905731517, "lambda_min": 0.09266526252031326, "lambda_max": 0.5675209760665894, "S_mean": 0.3255208333333333, "G_mean": 0.35189354776715237, "G_std": 0.15875417017377913, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.492546558380127, "grad_norm": 0.7705349922180176, "tokens_seen": 54730752, "tokens_per_sec": 14933.24804546335, "vram_gb": 6.442267648, "lambda": 0.2654828081528346, "lambda_min": 0.09280972927808762, "lambda_max": 0.5683460235595703, "S_mean": 0.3345743815104167, "G_mean": 0.3473820990572373, "G_std": 0.15864151374747357, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.4991302490234375, "grad_norm": 0.8689711093902588, "tokens_seen": 54894592, "tokens_per_sec": 14934.04108788178, "vram_gb": 6.442267648, "lambda": 0.2658550764123599, "lambda_min": 0.09274517744779587, "lambda_max": 0.5691332817077637, "S_mean": 0.3284708658854167, "G_mean": 0.345112725161016, "G_std": 0.15849836139629284, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.518761396408081, "grad_norm": 0.8518868684768677, "tokens_seen": 55058432, "tokens_per_sec": 14934.479973690422, "vram_gb": 6.442267648, "lambda": 0.2661411914353569, "lambda_min": 0.09284836798906326, "lambda_max": 0.5705409049987793, "S_mean": 0.3509928385416667, "G_mean": 0.3507055505178869, "G_std": 0.15856373651574054, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.488066911697388, "grad_norm": 0.8337095379829407, "tokens_seen": 55222272, "tokens_per_sec": 14934.830711942695, "vram_gb": 6.442267648, "lambda": 0.2664645789191127, "lambda_min": 0.09298301488161087, "lambda_max": 0.5716745853424072, "S_mean": 0.3616129557291667, "G_mean": 0.3466409300453961, "G_std": 0.15593333371604481, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.566265821456909, "grad_norm": 0.8049474358558655, "tokens_seen": 55386112, "tokens_per_sec": 14935.560753782207, "vram_gb": 6.442267648, "lambda": 0.2667553663874666, "lambda_min": 0.09293802082538605, "lambda_max": 0.572733998298645, "S_mean": 0.3434855143229167, "G_mean": 0.35291034712766606, "G_std": 0.1607185365476956, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.547971487045288, "grad_norm": 0.8021165132522583, "tokens_seen": 55549952, "tokens_per_sec": 14936.26109225852, "vram_gb": 6.442267648, "lambda": 0.2670265457903345, "lambda_min": 0.09286218881607056, "lambda_max": 0.5737457275390625, "S_mean": 0.3434041341145833, "G_mean": 0.34344490244984627, "G_std": 0.1582081518135965, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.495293855667114, "grad_norm": 0.9015806317329407, "tokens_seen": 55713792, "tokens_per_sec": 14937.008368721377, "vram_gb": 6.442267648, "lambda": 0.26728153973817825, "lambda_min": 0.09285929799079895, "lambda_max": 0.5745196342468262, "S_mean": 0.3407185872395833, "G_mean": 0.3486279157611231, "G_std": 0.16001289260263243, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.670623540878296, "grad_norm": 0.8449932336807251, "tokens_seen": 55877632, "tokens_per_sec": 14937.859786236866, "vram_gb": 6.442267648, "lambda": 0.2676221566895644, "lambda_min": 0.09301449358463287, "lambda_max": 0.5760753750801086, "S_mean": 0.3421834309895833, "G_mean": 0.34811893198639154, "G_std": 0.15939008238880584, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.565001964569092, "grad_norm": 0.8456317186355591, "tokens_seen": 56041472, "tokens_per_sec": 14938.548305371494, "vram_gb": 6.442267648, "lambda": 0.2679725084453821, "lambda_min": 0.09305355697870255, "lambda_max": 0.5774657130241394, "S_mean": 0.3379109700520833, "G_mean": 0.3540763700536142, "G_std": 0.15989994343059757, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.459850072860718, "grad_norm": 0.8856051564216614, "tokens_seen": 56205312, "tokens_per_sec": 14939.062706052517, "vram_gb": 6.442267648, "lambda": 0.2682387555638949, "lambda_min": 0.0930062010884285, "lambda_max": 0.5784350037574768, "S_mean": 0.3600056966145833, "G_mean": 0.34175121209894616, "G_std": 0.15682322229258716, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.498831272125244, "grad_norm": 0.8450961709022522, "tokens_seen": 56369152, "tokens_per_sec": 14939.805590660864, "vram_gb": 6.442267648, "lambda": 0.26855796047796804, "lambda_min": 0.0929865762591362, "lambda_max": 0.5789346694946289, "S_mean": 0.3389078776041667, "G_mean": 0.34704178432002664, "G_std": 0.15940072985055545, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.481740474700928, "grad_norm": 0.8943323493003845, "tokens_seen": 56532992, "tokens_per_sec": 14940.557871120887, "vram_gb": 6.442267648, "lambda": 0.268855062002937, "lambda_min": 0.09300531446933746, "lambda_max": 0.5797836780548096, "S_mean": 0.3266194661458333, "G_mean": 0.3510745828971267, "G_std": 0.15938485126631954, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.43608546257019, "grad_norm": 0.8478668332099915, "tokens_seen": 56696832, "tokens_per_sec": 14941.256591825595, "vram_gb": 6.442267648, "lambda": 0.2692021771023671, "lambda_min": 0.09302647411823273, "lambda_max": 0.5804880857467651, "S_mean": 0.3539225260416667, "G_mean": 0.3504328644679238, "G_std": 0.16002305589305857, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.431512355804443, "grad_norm": 0.841858983039856, "tokens_seen": 56860672, "tokens_per_sec": 14941.672710702307, "vram_gb": 6.442267648, "lambda": 0.2695480001469453, "lambda_min": 0.09293507784605026, "lambda_max": 0.5809868574142456, "S_mean": 0.3394978841145833, "G_mean": 0.3522706561877082, "G_std": 0.16058477441159388, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.488918304443359, "grad_norm": 0.8249927759170532, "tokens_seen": 57024512, "tokens_per_sec": 14942.288403874398, "vram_gb": 6.442267648, "lambda": 0.26979978444675606, "lambda_min": 0.09304340928792953, "lambda_max": 0.5816521048545837, "S_mean": 0.3425089518229167, "G_mean": 0.34353846684098244, "G_std": 0.1582834533570955, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.362576723098755, "grad_norm": 0.8721405267715454, "tokens_seen": 57188352, "tokens_per_sec": 14942.984734132153, "vram_gb": 6.442267648, "lambda": 0.27012288104742765, "lambda_min": 0.09318340569734573, "lambda_max": 0.5826768279075623, "S_mean": 0.3282470703125, "G_mean": 0.3484779993693034, "G_std": 0.16141091768319407, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.353314161300659, "grad_norm": 0.8368335366249084, "tokens_seen": 57352192, "tokens_per_sec": 14943.673195567055, "vram_gb": 6.442267648, "lambda": 0.2704320962851246, "lambda_min": 0.09320444613695145, "lambda_max": 0.5833789706230164, "S_mean": 0.3409423828125, "G_mean": 0.3413316130948563, "G_std": 0.15816058521158993, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.567519903182983, "val_ppl": 96.30496770381527, "lambda": 0.2704446030159791, "lambda_min": 0.0932115986943245, "lambda_max": 0.5834275484085083, "lambda_per_layer": [0.16872236132621765, 0.14160850644111633, 0.14271052181720734, 0.13685397803783417, 0.12173784524202347, 0.11984580755233765, 0.0932115986943245, 0.12809674441814423, 0.13304653763771057, 0.28025680780410767, 0.33531665802001953, 0.45169752836227417, 0.3712189495563507, 0.5834275484085083, 0.4416094720363617, 0.46403372287750244, 0.362228661775589, 0.36555343866348267, 0.2478044182062149, 0.24705517292022705, 0.39126867055892944, 0.349905401468277, 0.2166539430618286, 0.1968061774969101], "S_mean": 0.3487752278645833, "G_mean": 0.34891659937178093, "G_std": 0.15996149787679315, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.52497673034668, "grad_norm": 0.8109148144721985, "tokens_seen": 57516032, "tokens_per_sec": 14928.513203458397, "vram_gb": 6.442267648, "lambda": 0.27072847727686167, "lambda_min": 0.09321556240320206, "lambda_max": 0.5845556259155273, "S_mean": 0.3410135904947917, "G_mean": 0.35441024011621874, "G_std": 0.1605358930925528, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.595597267150879, "grad_norm": 0.8344290256500244, "tokens_seen": 57679872, "tokens_per_sec": 14929.222342848841, "vram_gb": 6.442267648, "lambda": 0.27101048609862727, "lambda_min": 0.0931568294763565, "lambda_max": 0.5855735540390015, "S_mean": 0.3469645182291667, "G_mean": 0.35044005342448753, "G_std": 0.1572104130561153, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.52232551574707, "grad_norm": 0.8457577228546143, "tokens_seen": 57843712, "tokens_per_sec": 14929.812631470622, "vram_gb": 6.442267648, "lambda": 0.27134962659329176, "lambda_min": 0.09313551336526871, "lambda_max": 0.5867331027984619, "S_mean": 0.3281046549479167, "G_mean": 0.35434069111943245, "G_std": 0.15750865746910372, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.4168713092803955, "grad_norm": 0.8003479242324829, "tokens_seen": 58007552, "tokens_per_sec": 14930.173271956857, "vram_gb": 6.442267648, "lambda": 0.2716083771859606, "lambda_min": 0.09310507029294968, "lambda_max": 0.5873910188674927, "S_mean": 0.3353474934895833, "G_mean": 0.3480169920561214, "G_std": 0.15837386607502899, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.559319734573364, "grad_norm": 0.8712272644042969, "tokens_seen": 58171392, "tokens_per_sec": 14930.595083070106, "vram_gb": 6.442267648, "lambda": 0.2718857803071539, "lambda_min": 0.0931440219283104, "lambda_max": 0.5881657004356384, "S_mean": 0.3328450520833333, "G_mean": 0.3462352875309686, "G_std": 0.15832139877602458, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.461482524871826, "grad_norm": 0.8156864047050476, "tokens_seen": 58335232, "tokens_per_sec": 14931.028614704934, "vram_gb": 6.442267648, "lambda": 0.2721764150386055, "lambda_min": 0.09313338994979858, "lambda_max": 0.5890774726867676, "S_mean": 0.335693359375, "G_mean": 0.34436867603411275, "G_std": 0.15771557111293077, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.395764112472534, "grad_norm": 0.8467737436294556, "tokens_seen": 58499072, "tokens_per_sec": 14931.584718474525, "vram_gb": 6.442267648, "lambda": 0.2724112216383219, "lambda_min": 0.09301432967185974, "lambda_max": 0.5901984572410583, "S_mean": 0.3358968098958333, "G_mean": 0.3394071236252785, "G_std": 0.15887910725238422, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.368766784667969, "grad_norm": 0.819206953048706, "tokens_seen": 58662912, "tokens_per_sec": 14932.26890054713, "vram_gb": 6.442267648, "lambda": 0.2726918300613761, "lambda_min": 0.09286247193813324, "lambda_max": 0.59130859375, "S_mean": 0.3575642903645833, "G_mean": 0.3426256822422147, "G_std": 0.15939914342015982, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.576282978057861, "grad_norm": 0.829356849193573, "tokens_seen": 58826752, "tokens_per_sec": 14932.20091672447, "vram_gb": 6.442267648, "lambda": 0.2729436152925094, "lambda_min": 0.09268462657928467, "lambda_max": 0.5920624732971191, "S_mean": 0.3393961588541667, "G_mean": 0.3485044447394709, "G_std": 0.16047772850530842, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.456357002258301, "grad_norm": 0.832150399684906, "tokens_seen": 58990592, "tokens_per_sec": 14931.779423883774, "vram_gb": 6.442267648, "lambda": 0.2732290243729949, "lambda_min": 0.0927295982837677, "lambda_max": 0.5926969647407532, "S_mean": 0.34698486328125, "G_mean": 0.3466080612658213, "G_std": 0.15690142956251898, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.550290822982788, "grad_norm": 0.9354361891746521, "tokens_seen": 59154432, "tokens_per_sec": 14930.428339013828, "vram_gb": 6.442267648, "lambda": 0.27355622965842485, "lambda_min": 0.09279616177082062, "lambda_max": 0.5930570960044861, "S_mean": 0.3386637369791667, "G_mean": 0.3470041834904502, "G_std": 0.1583349104039371, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.465380430221558, "grad_norm": 0.852089524269104, "tokens_seen": 59318272, "tokens_per_sec": 14929.31145945738, "vram_gb": 6.442267648, "lambda": 0.27387895838667947, "lambda_min": 0.09277332574129105, "lambda_max": 0.5937909483909607, "S_mean": 0.33721923828125, "G_mean": 0.3466790091091146, "G_std": 0.15882526982265213, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.508479118347168, "grad_norm": 0.8525053858757019, "tokens_seen": 59482112, "tokens_per_sec": 14928.097566281684, "vram_gb": 6.442267648, "lambda": 0.27417855244129896, "lambda_min": 0.09265968203544617, "lambda_max": 0.5946013331413269, "S_mean": 0.3482259114583333, "G_mean": 0.3487608864282568, "G_std": 0.15996919696529707, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.598617076873779, "grad_norm": 0.8217992186546326, "tokens_seen": 59645952, "tokens_per_sec": 14927.540518232961, "vram_gb": 6.442267648, "lambda": 0.27446882830311853, "lambda_min": 0.09269095957279205, "lambda_max": 0.5957288146018982, "S_mean": 0.3436991373697917, "G_mean": 0.34791149649148184, "G_std": 0.1561819432924191, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.405801773071289, "grad_norm": 0.826736569404602, "tokens_seen": 59809792, "tokens_per_sec": 14925.86529164389, "vram_gb": 6.442267648, "lambda": 0.2747661418591936, "lambda_min": 0.09262127429246902, "lambda_max": 0.5965187549591064, "S_mean": 0.333343505859375, "G_mean": 0.34430325240828097, "G_std": 0.1578502645716071, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.470573425292969, "grad_norm": 0.7832677364349365, "tokens_seen": 59973632, "tokens_per_sec": 14924.173001049969, "vram_gb": 6.442267648, "lambda": 0.2750614869097869, "lambda_min": 0.09254441410303116, "lambda_max": 0.5971704721450806, "S_mean": 0.3290303548177083, "G_mean": 0.3505839213418464, "G_std": 0.15913438618493578, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.49268364906311, "grad_norm": 0.8333630561828613, "tokens_seen": 60137472, "tokens_per_sec": 14923.384521524513, "vram_gb": 6.442267648, "lambda": 0.27532961933563155, "lambda_min": 0.09250792860984802, "lambda_max": 0.5980567336082458, "S_mean": 0.3440755208333333, "G_mean": 0.34687839603672427, "G_std": 0.1596141983754933, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.5015177726745605, "grad_norm": 0.8221184015274048, "tokens_seen": 60301312, "tokens_per_sec": 14922.816673522988, "vram_gb": 6.442267648, "lambda": 0.27558561973273754, "lambda_min": 0.09259950369596481, "lambda_max": 0.5989643335342407, "S_mean": 0.3381144205729167, "G_mean": 0.34004239753509563, "G_std": 0.15953069824414948, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.346953392028809, "grad_norm": 0.8667919635772705, "tokens_seen": 60465152, "tokens_per_sec": 14922.584007439476, "vram_gb": 6.442267648, "lambda": 0.2758992662032445, "lambda_min": 0.0925336554646492, "lambda_max": 0.6001026630401611, "S_mean": 0.3597615559895833, "G_mean": 0.34796322501885396, "G_std": 0.15726098938224217, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.367861270904541, "grad_norm": 0.8480340838432312, "tokens_seen": 60628992, "tokens_per_sec": 14923.10560190002, "vram_gb": 6.442267648, "lambda": 0.2761421278119087, "lambda_min": 0.0924084261059761, "lambda_max": 0.6008111238479614, "S_mean": 0.33917236328125, "G_mean": 0.338308632063369, "G_std": 0.1574006650286416, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.536217927932739, "grad_norm": 0.8719190359115601, "tokens_seen": 60792832, "tokens_per_sec": 14923.545639235419, "vram_gb": 6.442267648, "lambda": 0.2764232136930029, "lambda_min": 0.09252775460481644, "lambda_max": 0.6018645763397217, "S_mean": 0.343017578125, "G_mean": 0.34449541723976534, "G_std": 0.16132955175514022, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.564167261123657, "grad_norm": 0.8959869146347046, "tokens_seen": 60956672, "tokens_per_sec": 14924.26453158061, "vram_gb": 6.442267648, "lambda": 0.27672313526272774, "lambda_min": 0.09252282232046127, "lambda_max": 0.6027911305427551, "S_mean": 0.3493245442708333, "G_mean": 0.35585038394977647, "G_std": 0.1596744730292509, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.437676906585693, "grad_norm": 0.8718432188034058, "tokens_seen": 61120512, "tokens_per_sec": 14925.020458811421, "vram_gb": 6.442267648, "lambda": 0.2770028092587988, "lambda_min": 0.09258545935153961, "lambda_max": 0.6038758158683777, "S_mean": 0.3453572591145833, "G_mean": 0.34772513279070455, "G_std": 0.15822375278609493, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.678666830062866, "grad_norm": 0.8968720436096191, "tokens_seen": 61284352, "tokens_per_sec": 14925.728422735667, "vram_gb": 6.442267648, "lambda": 0.2772649625937144, "lambda_min": 0.09273816645145416, "lambda_max": 0.6046080589294434, "S_mean": 0.3423258463541667, "G_mean": 0.3457111962294827, "G_std": 0.1581421021837741, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.30437970161438, "grad_norm": 0.8147944808006287, "tokens_seen": 61448192, "tokens_per_sec": 14926.300572945423, "vram_gb": 6.442267648, "lambda": 0.2775505604222417, "lambda_min": 0.09273499995470047, "lambda_max": 0.6057263016700745, "S_mean": 0.34881591796875, "G_mean": 0.33973450027406216, "G_std": 0.1564243376099815, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.544728624820709, "val_ppl": 94.13487787347535, "lambda": 0.2775650418673952, "lambda_min": 0.0927327573299408, "lambda_max": 0.6057762503623962, "lambda_per_layer": [0.1699729561805725, 0.14171653985977173, 0.1435306966304779, 0.13735944032669067, 0.12183432281017303, 0.12057014554738998, 0.0927327573299408, 0.12832948565483093, 0.13306954503059387, 0.28426557779312134, 0.34232398867607117, 0.4626728296279907, 0.37999075651168823, 0.6057762503623962, 0.4554615616798401, 0.48486003279685974, 0.37541621923446655, 0.3801591098308563, 0.25384655594825745, 0.2543174624443054, 0.406940221786499, 0.36416536569595337, 0.22156667709350586, 0.20068250596523285], "S_mean": 0.3428853352864583, "G_mean": 0.3478049656841904, "G_std": 0.15853243763558567, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.535304069519043, "grad_norm": 0.8764663338661194, "tokens_seen": 61612032, "tokens_per_sec": 14911.929636498653, "vram_gb": 6.442267648, "lambda": 0.27782221293697756, "lambda_min": 0.09281246364116669, "lambda_max": 0.6064696311950684, "S_mean": 0.3311055501302083, "G_mean": 0.3388730324804783, "G_std": 0.15712921655115983, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.397222995758057, "grad_norm": 0.8375352621078491, "tokens_seen": 61775872, "tokens_per_sec": 14912.505438105429, "vram_gb": 6.442267648, "lambda": 0.27811516014238197, "lambda_min": 0.09286264330148697, "lambda_max": 0.607337474822998, "S_mean": 0.334197998046875, "G_mean": 0.33819003216922283, "G_std": 0.15748633653856814, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.422963857650757, "grad_norm": 0.9052349328994751, "tokens_seen": 61939712, "tokens_per_sec": 14913.339163249257, "vram_gb": 6.442267648, "lambda": 0.2783182244747877, "lambda_min": 0.09286585450172424, "lambda_max": 0.6079952120780945, "S_mean": 0.3296610514322917, "G_mean": 0.35280541811759275, "G_std": 0.16041095609155795, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.366327524185181, "grad_norm": 0.824134886264801, "tokens_seen": 62103552, "tokens_per_sec": 14913.862927663395, "vram_gb": 6.442267648, "lambda": 0.27858708581576747, "lambda_min": 0.09301295876502991, "lambda_max": 0.6086483001708984, "S_mean": 0.3448893229166667, "G_mean": 0.34702396268645924, "G_std": 0.1584937029207746, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.403933048248291, "grad_norm": 0.8271805644035339, "tokens_seen": 62267392, "tokens_per_sec": 14914.519064411801, "vram_gb": 6.442267648, "lambda": 0.2788164659092824, "lambda_min": 0.09305346757173538, "lambda_max": 0.6092666387557983, "S_mean": 0.3455708821614583, "G_mean": 0.34671472494180006, "G_std": 0.15973927437638244, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.387901306152344, "grad_norm": 0.8576989769935608, "tokens_seen": 62431232, "tokens_per_sec": 14915.310127015, "vram_gb": 6.442267648, "lambda": 0.2790191853418946, "lambda_min": 0.0930044874548912, "lambda_max": 0.6097972989082336, "S_mean": 0.3326009114583333, "G_mean": 0.3446747342435022, "G_std": 0.15715222378882268, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.411106824874878, "grad_norm": 0.8311066627502441, "tokens_seen": 62595072, "tokens_per_sec": 14916.082151503299, "vram_gb": 6.442267648, "lambda": 0.27925100705275935, "lambda_min": 0.09299866855144501, "lambda_max": 0.6106398105621338, "S_mean": 0.3353780110677083, "G_mean": 0.3431756311717133, "G_std": 0.15810949572672447, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.4907753467559814, "grad_norm": 0.8799846768379211, "tokens_seen": 62758912, "tokens_per_sec": 14916.949270659483, "vram_gb": 6.442267648, "lambda": 0.27953738532960415, "lambda_min": 0.09303112328052521, "lambda_max": 0.6111946702003479, "S_mean": 0.3415934244791667, "G_mean": 0.3548006162357827, "G_std": 0.15695235265108445, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.366718292236328, "grad_norm": 0.9089136123657227, "tokens_seen": 62922752, "tokens_per_sec": 14917.661529480973, "vram_gb": 6.442267648, "lambda": 0.27974383036295575, "lambda_min": 0.09304322302341461, "lambda_max": 0.6118252277374268, "S_mean": 0.347930908203125, "G_mean": 0.35047381355737645, "G_std": 0.15930465678684413, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.493201494216919, "grad_norm": 0.9066118597984314, "tokens_seen": 63086592, "tokens_per_sec": 14918.404070155511, "vram_gb": 6.442267648, "lambda": 0.2799709451695283, "lambda_min": 0.09307166934013367, "lambda_max": 0.6127403974533081, "S_mean": 0.3423055013020833, "G_mean": 0.3415645866965254, "G_std": 0.15845144477983317, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.521900177001953, "grad_norm": 0.9061043858528137, "tokens_seen": 63250432, "tokens_per_sec": 14919.005339155803, "vram_gb": 6.442267648, "lambda": 0.2801798991858959, "lambda_min": 0.09301938861608505, "lambda_max": 0.6137356758117676, "S_mean": 0.3318074544270833, "G_mean": 0.35005578383182484, "G_std": 0.1594124810459713, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.357175350189209, "grad_norm": 0.8531404733657837, "tokens_seen": 63414272, "tokens_per_sec": 14919.522160672157, "vram_gb": 6.442267648, "lambda": 0.28042268225302297, "lambda_min": 0.09302705526351929, "lambda_max": 0.6145948767662048, "S_mean": 0.3378702799479167, "G_mean": 0.33441605341310304, "G_std": 0.15384505107067525, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.567076206207275, "grad_norm": 0.9259220957756042, "tokens_seen": 63578112, "tokens_per_sec": 14920.254335071852, "vram_gb": 6.442267648, "lambda": 0.28063109548141557, "lambda_min": 0.09310941398143768, "lambda_max": 0.6152642965316772, "S_mean": 0.3494059244791667, "G_mean": 0.34774880995973945, "G_std": 0.15781061071902514, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.434901237487793, "grad_norm": 0.8684751987457275, "tokens_seen": 63741952, "tokens_per_sec": 14921.048059867344, "vram_gb": 6.442267648, "lambda": 0.2809249771138032, "lambda_min": 0.09315177798271179, "lambda_max": 0.616037905216217, "S_mean": 0.3623046875, "G_mean": 0.34316958324052393, "G_std": 0.1560211795537422, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.477313756942749, "grad_norm": 0.9121322631835938, "tokens_seen": 63905792, "tokens_per_sec": 14921.59874508974, "vram_gb": 6.442267648, "lambda": 0.28111171691368025, "lambda_min": 0.09313704073429108, "lambda_max": 0.6164135932922363, "S_mean": 0.3442586263020833, "G_mean": 0.35026451852172613, "G_std": 0.15982513331497708, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.358262062072754, "grad_norm": 0.8658533096313477, "tokens_seen": 64069632, "tokens_per_sec": 14922.430633570462, "vram_gb": 6.442267648, "lambda": 0.28134031873196363, "lambda_min": 0.09311777353286743, "lambda_max": 0.6175693273544312, "S_mean": 0.3480021158854167, "G_mean": 0.34219653873393935, "G_std": 0.15623813052661717, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.501925468444824, "grad_norm": 0.8913242220878601, "tokens_seen": 64233472, "tokens_per_sec": 14923.179311126263, "vram_gb": 6.442267648, "lambda": 0.2815718005100886, "lambda_min": 0.09311675280332565, "lambda_max": 0.6182768940925598, "S_mean": 0.3513997395833333, "G_mean": 0.34490649371097487, "G_std": 0.15704236940170327, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.430830478668213, "grad_norm": 0.945899486541748, "tokens_seen": 64397312, "tokens_per_sec": 14923.889199565845, "vram_gb": 6.442267648, "lambda": 0.28179345310976106, "lambda_min": 0.09308275580406189, "lambda_max": 0.6191670298576355, "S_mean": 0.3557942708333333, "G_mean": 0.3450868207340439, "G_std": 0.15772645524702966, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.385293960571289, "grad_norm": 0.9115813970565796, "tokens_seen": 64561152, "tokens_per_sec": 14924.427235904399, "vram_gb": 6.442267648, "lambda": 0.28200966740647954, "lambda_min": 0.09300399571657181, "lambda_max": 0.6194880604743958, "S_mean": 0.340057373046875, "G_mean": 0.34551286697387695, "G_std": 0.1568348112826546, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.5588860511779785, "grad_norm": 0.8994858264923096, "tokens_seen": 64724992, "tokens_per_sec": 14925.068126662587, "vram_gb": 6.442267648, "lambda": 0.2822326800475518, "lambda_min": 0.09309445321559906, "lambda_max": 0.6200725436210632, "S_mean": 0.35858154296875, "G_mean": 0.34515832302471, "G_std": 0.15746204267876843, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.420063018798828, "grad_norm": 0.941639244556427, "tokens_seen": 64888832, "tokens_per_sec": 14925.890904908387, "vram_gb": 6.442267648, "lambda": 0.2824596722299854, "lambda_min": 0.09304531663656235, "lambda_max": 0.6204421520233154, "S_mean": 0.3410847981770833, "G_mean": 0.33799145463854074, "G_std": 0.15605153872941932, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.469832897186279, "grad_norm": 0.8890786170959473, "tokens_seen": 65052672, "tokens_per_sec": 14926.628999245353, "vram_gb": 6.442267648, "lambda": 0.28266944953550893, "lambda_min": 0.09300357103347778, "lambda_max": 0.6208760738372803, "S_mean": 0.3415730794270833, "G_mean": 0.3475161101669073, "G_std": 0.15869941135557988, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.540445566177368, "grad_norm": 0.8800658583641052, "tokens_seen": 65216512, "tokens_per_sec": 14927.33593762832, "vram_gb": 6.442267648, "lambda": 0.2829163148999214, "lambda_min": 0.0929279550909996, "lambda_max": 0.6216716170310974, "S_mean": 0.3548787434895833, "G_mean": 0.3514249010477215, "G_std": 0.15913318850410482, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.252127647399902, "grad_norm": 0.9733749628067017, "tokens_seen": 65380352, "tokens_per_sec": 14927.972959000277, "vram_gb": 6.442267648, "lambda": 0.28313401776055497, "lambda_min": 0.09296073019504547, "lambda_max": 0.6221856474876404, "S_mean": 0.3495686848958333, "G_mean": 0.34654874727129936, "G_std": 0.15876236953772604, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.348716735839844, "grad_norm": 0.8893887400627136, "tokens_seen": 65544192, "tokens_per_sec": 14928.503676289281, "vram_gb": 6.442267648, "lambda": 0.28336625173687935, "lambda_min": 0.0929950475692749, "lambda_max": 0.6227977871894836, "S_mean": 0.3315836588541667, "G_mean": 0.34060760742674273, "G_std": 0.15568143827840686, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.535871839523315, "val_ppl": 93.30482668965786, "lambda": 0.28338137175887823, "lambda_min": 0.09299799054861069, "lambda_max": 0.6228481531143188, "lambda_per_layer": [0.17091158032417297, 0.1417865753173828, 0.14393872022628784, 0.13782699406147003, 0.12162186950445175, 0.12108495086431503, 0.09299799054861069, 0.1280686855316162, 0.13321126997470856, 0.2872048318386078, 0.34776031970977783, 0.47200071811676025, 0.38704875111579895, 0.6228481531143188, 0.46750447154045105, 0.5003424882888794, 0.3869067132472992, 0.392257958650589, 0.2595531642436981, 0.25904345512390137, 0.4185863733291626, 0.37684395909309387, 0.2273067981004715, 0.2044961303472519], "S_mean": 0.3671162923177083, "G_mean": 0.34787374967709184, "G_std": 0.15685468384375176, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.61940336227417, "grad_norm": 0.8756850361824036, "tokens_seen": 65708032, "tokens_per_sec": 14915.54927867721, "vram_gb": 6.442267648, "lambda": 0.28356551968803007, "lambda_min": 0.09295579791069031, "lambda_max": 0.623685896396637, "S_mean": 0.3477376302083333, "G_mean": 0.33989962299043935, "G_std": 0.15691855324742696, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.423712730407715, "grad_norm": 0.8822628259658813, "tokens_seen": 65871872, "tokens_per_sec": 14916.29236768297, "vram_gb": 6.442267648, "lambda": 0.28381693239013356, "lambda_min": 0.09294755011796951, "lambda_max": 0.6245275735855103, "S_mean": 0.3479207356770833, "G_mean": 0.33963298161203664, "G_std": 0.15457622908676663, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.303052186965942, "grad_norm": 0.9510588049888611, "tokens_seen": 66035712, "tokens_per_sec": 14916.692970485288, "vram_gb": 6.442267648, "lambda": 0.2840121618161599, "lambda_min": 0.09297502040863037, "lambda_max": 0.6253340840339661, "S_mean": 0.34173583984375, "G_mean": 0.34052017827828723, "G_std": 0.15691344354612133, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.3900556564331055, "grad_norm": 0.8646817803382874, "tokens_seen": 66199552, "tokens_per_sec": 14917.430841560637, "vram_gb": 6.442267648, "lambda": 0.2841923947756489, "lambda_min": 0.09295240789651871, "lambda_max": 0.6260951161384583, "S_mean": 0.338531494140625, "G_mean": 0.34592763311229646, "G_std": 0.15665768870773414, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.387407302856445, "grad_norm": 0.9012930393218994, "tokens_seen": 66363392, "tokens_per_sec": 14918.291518728069, "vram_gb": 6.442267648, "lambda": 0.2843751922870676, "lambda_min": 0.09297988563776016, "lambda_max": 0.6267963647842407, "S_mean": 0.3410441080729167, "G_mean": 0.3466095264690618, "G_std": 0.15814041540337107, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.447009563446045, "grad_norm": 0.8216201663017273, "tokens_seen": 66527232, "tokens_per_sec": 14919.235879501477, "vram_gb": 6.442267648, "lambda": 0.2845909384389718, "lambda_min": 0.09299599379301071, "lambda_max": 0.6275233030319214, "S_mean": 0.3313496907552083, "G_mean": 0.3473951699367414, "G_std": 0.1595308882339547, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.400808334350586, "grad_norm": 0.89640212059021, "tokens_seen": 66691072, "tokens_per_sec": 14920.399957740483, "vram_gb": 6.442267648, "lambda": 0.28481190496434766, "lambda_min": 0.09307341277599335, "lambda_max": 0.6281707882881165, "S_mean": 0.3532918294270833, "G_mean": 0.34228154163186747, "G_std": 0.15803469554521143, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.491588354110718, "grad_norm": 0.9322952032089233, "tokens_seen": 66854912, "tokens_per_sec": 14921.178688181817, "vram_gb": 6.442267648, "lambda": 0.28499374849100906, "lambda_min": 0.09318628162145615, "lambda_max": 0.6290043592453003, "S_mean": 0.3402913411458333, "G_mean": 0.34821502996298176, "G_std": 0.15800986904650927, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.443035840988159, "grad_norm": 0.8579691648483276, "tokens_seen": 67018752, "tokens_per_sec": 14921.761833057912, "vram_gb": 6.442267648, "lambda": 0.2852474596972267, "lambda_min": 0.09324546903371811, "lambda_max": 0.6296271681785583, "S_mean": 0.35205078125, "G_mean": 0.3383662384779503, "G_std": 0.15713459361965457, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.493460416793823, "grad_norm": 0.9549657106399536, "tokens_seen": 67182592, "tokens_per_sec": 14922.418730356718, "vram_gb": 6.442267648, "lambda": 0.28545837849378586, "lambda_min": 0.09316310286521912, "lambda_max": 0.6301694512367249, "S_mean": 0.3442281087239583, "G_mean": 0.33968524479617673, "G_std": 0.15596434396381179, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.37328028678894, "grad_norm": 0.8495403528213501, "tokens_seen": 67346432, "tokens_per_sec": 14923.073848857459, "vram_gb": 6.442267648, "lambda": 0.2856840845197439, "lambda_min": 0.09313680976629257, "lambda_max": 0.6309455037117004, "S_mean": 0.3375244140625, "G_mean": 0.33713511082654196, "G_std": 0.15729799346687892, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.430569648742676, "grad_norm": 0.9116666913032532, "tokens_seen": 67510272, "tokens_per_sec": 14923.586332953024, "vram_gb": 6.442267648, "lambda": 0.2859370264535149, "lambda_min": 0.09316182881593704, "lambda_max": 0.6317278146743774, "S_mean": 0.345947265625, "G_mean": 0.34765029783981544, "G_std": 0.15873170481063426, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.342037200927734, "grad_norm": 0.8925832509994507, "tokens_seen": 67674112, "tokens_per_sec": 14924.210603038107, "vram_gb": 6.442267648, "lambda": 0.2861492410302162, "lambda_min": 0.09318902343511581, "lambda_max": 0.6323824524879456, "S_mean": 0.3413899739583333, "G_mean": 0.33553352137096226, "G_std": 0.15609489815930525, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.415245532989502, "grad_norm": 0.8776534199714661, "tokens_seen": 67837952, "tokens_per_sec": 14924.620322436722, "vram_gb": 6.442267648, "lambda": 0.2863436782111724, "lambda_min": 0.09310941398143768, "lambda_max": 0.63270503282547, "S_mean": 0.3441060384114583, "G_mean": 0.34241331182420254, "G_std": 0.1565076201222837, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.582895517349243, "grad_norm": 0.9499868750572205, "tokens_seen": 68001792, "tokens_per_sec": 14925.006573386136, "vram_gb": 6.442267648, "lambda": 0.28654965820411843, "lambda_min": 0.09310061484575272, "lambda_max": 0.6332238912582397, "S_mean": 0.3375651041666667, "G_mean": 0.34562258422374725, "G_std": 0.15655811387114227, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.225627899169922, "grad_norm": 0.8693178296089172, "tokens_seen": 68165632, "tokens_per_sec": 14925.794613590962, "vram_gb": 6.442267648, "lambda": 0.2867387927447756, "lambda_min": 0.09309639781713486, "lambda_max": 0.6337780952453613, "S_mean": 0.3373616536458333, "G_mean": 0.34407483918281895, "G_std": 0.15661268859791258, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.418114185333252, "grad_norm": 0.9039289355278015, "tokens_seen": 68329472, "tokens_per_sec": 14926.538994421695, "vram_gb": 6.442267648, "lambda": 0.28688709406803053, "lambda_min": 0.09314433485269547, "lambda_max": 0.634405255317688, "S_mean": 0.3335978190104167, "G_mean": 0.3451058574331303, "G_std": 0.15662434052986404, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.404008388519287, "grad_norm": 0.9378255009651184, "tokens_seen": 68493312, "tokens_per_sec": 14927.282074261668, "vram_gb": 6.442267648, "lambda": 0.2870911819239457, "lambda_min": 0.09315700083971024, "lambda_max": 0.634814441204071, "S_mean": 0.3492228190104167, "G_mean": 0.3455642448582997, "G_std": 0.15700468114422014, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.36781644821167, "grad_norm": 0.834829568862915, "tokens_seen": 68657152, "tokens_per_sec": 14927.870021222103, "vram_gb": 6.442267648, "lambda": 0.2873180058474342, "lambda_min": 0.09312109649181366, "lambda_max": 0.6355202794075012, "S_mean": 0.3385823567708333, "G_mean": 0.3413767210052659, "G_std": 0.15539422588578114, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.5617406368255615, "grad_norm": 0.8695657849311829, "tokens_seen": 68820992, "tokens_per_sec": 14928.450713669128, "vram_gb": 6.442267648, "lambda": 0.28746493129680556, "lambda_min": 0.09309767186641693, "lambda_max": 0.6362132430076599, "S_mean": 0.3385416666666667, "G_mean": 0.34726010480274755, "G_std": 0.1568713376376157, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.370509624481201, "grad_norm": 0.8577411770820618, "tokens_seen": 68984832, "tokens_per_sec": 14929.12342559571, "vram_gb": 6.442267648, "lambda": 0.2876405840118726, "lambda_min": 0.09307782351970673, "lambda_max": 0.636407196521759, "S_mean": 0.34930419921875, "G_mean": 0.33901106705889106, "G_std": 0.1557102883622671, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.364018201828003, "grad_norm": 0.8573560118675232, "tokens_seen": 69148672, "tokens_per_sec": 14929.871064283318, "vram_gb": 6.442267648, "lambda": 0.28781865754475194, "lambda_min": 0.09308828413486481, "lambda_max": 0.63656085729599, "S_mean": 0.3344828287760417, "G_mean": 0.33993038449746865, "G_std": 0.15658417623490095, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.40442419052124, "grad_norm": 0.8482123017311096, "tokens_seen": 69312512, "tokens_per_sec": 14930.403878825658, "vram_gb": 6.442267648, "lambda": 0.2880376462514202, "lambda_min": 0.09309663623571396, "lambda_max": 0.6372667551040649, "S_mean": 0.3313191731770833, "G_mean": 0.33927446107069653, "G_std": 0.15438616081761816, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.531473159790039, "grad_norm": 0.9350166320800781, "tokens_seen": 69476352, "tokens_per_sec": 14930.874302966758, "vram_gb": 6.442267648, "lambda": 0.2882019927104314, "lambda_min": 0.09309780597686768, "lambda_max": 0.6377852559089661, "S_mean": 0.340576171875, "G_mean": 0.3433768747684856, "G_std": 0.15651302924379706, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.481328964233398, "grad_norm": 0.9225900769233704, "tokens_seen": 69640192, "tokens_per_sec": 14931.689976094633, "vram_gb": 6.442267648, "lambda": 0.2884171347444256, "lambda_min": 0.09310293197631836, "lambda_max": 0.6384561061859131, "S_mean": 0.35101318359375, "G_mean": 0.3420118213010331, "G_std": 0.1566459481837228, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.484929513931275, "val_ppl": 88.67070054359229, "lambda": 0.28842848042647046, "lambda_min": 0.09309998154640198, "lambda_max": 0.6384741067886353, "lambda_per_layer": [0.1716630607843399, 0.14232394099235535, 0.1444137841463089, 0.13820382952690125, 0.1211974248290062, 0.12104254215955734, 0.09309998154640198, 0.12821492552757263, 0.13319142162799835, 0.2901047468185425, 0.35120511054992676, 0.47924715280532837, 0.39274662733078003, 0.6384741067886353, 0.47807416319847107, 0.5130778551101685, 0.3957628607749939, 0.402540922164917, 0.2644674479961395, 0.2650430202484131, 0.42967620491981506, 0.38835713267326355, 0.2319578379392624, 0.2081974297761917], "S_mean": 0.3489786783854167, "G_mean": 0.33908477254832786, "G_std": 0.15614085046884915, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.411157608032227, "grad_norm": 0.907444953918457, "tokens_seen": 69804032, "tokens_per_sec": 14919.415439956092, "vram_gb": 6.442267648, "lambda": 0.28863336611539125, "lambda_min": 0.09308944642543793, "lambda_max": 0.639112114906311, "S_mean": 0.339874267578125, "G_mean": 0.34064470945547026, "G_std": 0.15622870294221988, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.378722667694092, "grad_norm": 0.8659120202064514, "tokens_seen": 69967872, "tokens_per_sec": 14919.891033048463, "vram_gb": 6.442267648, "lambda": 0.2887981891011198, "lambda_min": 0.09314893186092377, "lambda_max": 0.6398475170135498, "S_mean": 0.3467203776041667, "G_mean": 0.33992780186235905, "G_std": 0.1549269585714986, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.359395265579224, "grad_norm": 0.9393291473388672, "tokens_seen": 70131712, "tokens_per_sec": 14920.169890607938, "vram_gb": 6.442267648, "lambda": 0.28894921640555066, "lambda_min": 0.09318514168262482, "lambda_max": 0.6401887536048889, "S_mean": 0.3399759928385417, "G_mean": 0.34065695805475116, "G_std": 0.15644306068619093, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.498073101043701, "grad_norm": 0.9108629822731018, "tokens_seen": 70295552, "tokens_per_sec": 14920.86684580441, "vram_gb": 6.442267648, "lambda": 0.2891603596508503, "lambda_min": 0.09323767572641373, "lambda_max": 0.64076167345047, "S_mean": 0.3315022786458333, "G_mean": 0.34208472034273046, "G_std": 0.15656732899757722, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.544445276260376, "grad_norm": 0.9426627159118652, "tokens_seen": 70459392, "tokens_per_sec": 14921.5054360163, "vram_gb": 6.442267648, "lambda": 0.2894141674041748, "lambda_min": 0.09331571310758591, "lambda_max": 0.6415632963180542, "S_mean": 0.334716796875, "G_mean": 0.34263875102624297, "G_std": 0.15612295436828086, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.503887176513672, "grad_norm": 0.9027237892150879, "tokens_seen": 70623232, "tokens_per_sec": 14922.150438114033, "vram_gb": 6.442267648, "lambda": 0.2896295487880707, "lambda_min": 0.09340200573205948, "lambda_max": 0.6423794627189636, "S_mean": 0.3457132975260417, "G_mean": 0.3410064401881148, "G_std": 0.15626993612386286, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.397660732269287, "grad_norm": 0.9041110277175903, "tokens_seen": 70787072, "tokens_per_sec": 14922.789310880395, "vram_gb": 6.442267648, "lambda": 0.28979032083104056, "lambda_min": 0.09348879754543304, "lambda_max": 0.6429365873336792, "S_mean": 0.339691162109375, "G_mean": 0.3374485025803248, "G_std": 0.15436372389861694, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.435343503952026, "grad_norm": 0.8912280797958374, "tokens_seen": 70950912, "tokens_per_sec": 14923.396492242044, "vram_gb": 6.442267648, "lambda": 0.2899776327734192, "lambda_min": 0.09347334504127502, "lambda_max": 0.6437218189239502, "S_mean": 0.3421427408854167, "G_mean": 0.3356814038318892, "G_std": 0.1548841786958898, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.370511054992676, "grad_norm": 0.9264557361602783, "tokens_seen": 71114752, "tokens_per_sec": 14923.998529486882, "vram_gb": 6.442267648, "lambda": 0.2901350660249591, "lambda_min": 0.09348955750465393, "lambda_max": 0.6443753838539124, "S_mean": 0.331085205078125, "G_mean": 0.33909701618055504, "G_std": 0.15681207971647382, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.345489978790283, "grad_norm": 0.9482449293136597, "tokens_seen": 71278592, "tokens_per_sec": 14924.444161166273, "vram_gb": 6.442267648, "lambda": 0.2903583204994599, "lambda_min": 0.09359997510910034, "lambda_max": 0.6449413299560547, "S_mean": 0.3321024576822917, "G_mean": 0.3397424171368281, "G_std": 0.1542575512236605, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.43484902381897, "grad_norm": 0.8938546776771545, "tokens_seen": 71442432, "tokens_per_sec": 14924.919128582547, "vram_gb": 6.442267648, "lambda": 0.2905528728539745, "lambda_min": 0.09363717585802078, "lambda_max": 0.6454651951789856, "S_mean": 0.3617350260416667, "G_mean": 0.34344047160508734, "G_std": 0.1552811375974367, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.5669636726379395, "grad_norm": 0.8958938717842102, "tokens_seen": 71606272, "tokens_per_sec": 14925.580569327245, "vram_gb": 6.442267648, "lambda": 0.29072649621715146, "lambda_min": 0.09371259063482285, "lambda_max": 0.6461660861968994, "S_mean": 0.3296305338541667, "G_mean": 0.34288869503264624, "G_std": 0.1576944305949534, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.3853795528411865, "grad_norm": 0.9226843118667603, "tokens_seen": 71770112, "tokens_per_sec": 14926.27241859822, "vram_gb": 6.442267648, "lambda": 0.2908831536769867, "lambda_min": 0.0937238484621048, "lambda_max": 0.6467507481575012, "S_mean": 0.33837890625, "G_mean": 0.34687677848463255, "G_std": 0.15599932509940118, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.150863409042358, "grad_norm": 0.9029905796051025, "tokens_seen": 71933952, "tokens_per_sec": 14927.037367474535, "vram_gb": 6.442267648, "lambda": 0.2910599196329713, "lambda_min": 0.09373944252729416, "lambda_max": 0.6471125483512878, "S_mean": 0.355224609375, "G_mean": 0.3337502205589165, "G_std": 0.1554100428475067, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.415907382965088, "grad_norm": 0.9174730777740479, "tokens_seen": 72097792, "tokens_per_sec": 14927.603078887154, "vram_gb": 6.442267648, "lambda": 0.2912237076088786, "lambda_min": 0.09370113164186478, "lambda_max": 0.6476131677627563, "S_mean": 0.3454793294270833, "G_mean": 0.3469098535521577, "G_std": 0.15678623966717473, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.3893561363220215, "grad_norm": 0.9196536540985107, "tokens_seen": 72261632, "tokens_per_sec": 14928.193590178718, "vram_gb": 6.442267648, "lambda": 0.29135065836211044, "lambda_min": 0.09373873472213745, "lambda_max": 0.6481379866600037, "S_mean": 0.33978271484375, "G_mean": 0.3413136218053599, "G_std": 0.1559615437484657, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.236910104751587, "grad_norm": 0.9242839813232422, "tokens_seen": 72425472, "tokens_per_sec": 14928.615903153364, "vram_gb": 6.442267648, "lambda": 0.2915183479587237, "lambda_min": 0.0937214195728302, "lambda_max": 0.6487745046615601, "S_mean": 0.3442179361979167, "G_mean": 0.34311963253033656, "G_std": 0.15532436881524822, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.3792808055877686, "grad_norm": 0.9185293912887573, "tokens_seen": 72589312, "tokens_per_sec": 14929.087679210988, "vram_gb": 6.442267648, "lambda": 0.291716108409067, "lambda_min": 0.09371655434370041, "lambda_max": 0.6492765545845032, "S_mean": 0.346435546875, "G_mean": 0.33896452421322465, "G_std": 0.15552542641914138, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.503204822540283, "grad_norm": 0.9251797795295715, "tokens_seen": 72753152, "tokens_per_sec": 14929.650633611069, "vram_gb": 6.442267648, "lambda": 0.29186240521570045, "lambda_min": 0.09372960776090622, "lambda_max": 0.6496009826660156, "S_mean": 0.331207275390625, "G_mean": 0.34011702743979794, "G_std": 0.156351251876913, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.5486767292022705, "grad_norm": 0.9402858018875122, "tokens_seen": 72916992, "tokens_per_sec": 14930.34051903757, "vram_gb": 6.442267648, "lambda": 0.29197167977690697, "lambda_min": 0.09372105449438095, "lambda_max": 0.6498345732688904, "S_mean": 0.3343709309895833, "G_mean": 0.3456936933410664, "G_std": 0.1571352673539271, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.357720613479614, "grad_norm": 0.8732823133468628, "tokens_seen": 73080832, "tokens_per_sec": 14930.74212911276, "vram_gb": 6.442267648, "lambda": 0.29215598354736966, "lambda_min": 0.09379112720489502, "lambda_max": 0.6503708362579346, "S_mean": 0.3472086588541667, "G_mean": 0.3427434970314304, "G_std": 0.15718290756922215, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.3949596881866455, "grad_norm": 0.9115152955055237, "tokens_seen": 73244672, "tokens_per_sec": 14931.250518984327, "vram_gb": 6.442267648, "lambda": 0.29236572204778594, "lambda_min": 0.09383571892976761, "lambda_max": 0.6508325338363647, "S_mean": 0.3327738444010417, "G_mean": 0.343122077640146, "G_std": 0.1566310947528109, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.401913166046143, "grad_norm": 0.90683913230896, "tokens_seen": 73408512, "tokens_per_sec": 14931.95162335502, "vram_gb": 6.442267648, "lambda": 0.29251775424927473, "lambda_min": 0.09384574741125107, "lambda_max": 0.6510868072509766, "S_mean": 0.3369954427083333, "G_mean": 0.34454215434379876, "G_std": 0.15510825828338662, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.429496765136719, "grad_norm": 0.8858723640441895, "tokens_seen": 73572352, "tokens_per_sec": 14932.713285064596, "vram_gb": 6.442267648, "lambda": 0.29265144436309737, "lambda_min": 0.093882717192173, "lambda_max": 0.6514768600463867, "S_mean": 0.34307861328125, "G_mean": 0.33634375664405525, "G_std": 0.15641007172719887, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.415807008743286, "grad_norm": 0.9031572937965393, "tokens_seen": 73736192, "tokens_per_sec": 14933.433937820959, "vram_gb": 6.442267648, "lambda": 0.29282454525430995, "lambda_min": 0.09387306123971939, "lambda_max": 0.652010977268219, "S_mean": 0.3385213216145833, "G_mean": 0.33981903999422985, "G_std": 0.1564535885117948, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.457729065418244, "val_ppl": 86.29132446657533, "lambda": 0.2928325378646453, "lambda_min": 0.09387154132127762, "lambda_max": 0.6520532965660095, "lambda_per_layer": [0.17270798981189728, 0.142307311296463, 0.144866481423378, 0.13864760100841522, 0.12118793278932571, 0.120679572224617, 0.09387154132127762, 0.12805084884166718, 0.13370780646800995, 0.29208624362945557, 0.35430026054382324, 0.4853180944919586, 0.3966006636619568, 0.6520532965660095, 0.4864092171192169, 0.5249828696250916, 0.40459921956062317, 0.4118606448173523, 0.268604040145874, 0.2696773111820221, 0.43866023421287537, 0.3987135887145996, 0.23594066500663757, 0.21214747428894043], "S_mean": 0.3426717122395833, "G_mean": 0.34028665558435023, "G_std": 0.15661819298596433, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.4485063552856445, "grad_norm": 0.896520733833313, "tokens_seen": 73900032, "tokens_per_sec": 14921.701433906333, "vram_gb": 6.442267648, "lambda": 0.29298380482941866, "lambda_min": 0.09383848309516907, "lambda_max": 0.6526826620101929, "S_mean": 0.3357747395833333, "G_mean": 0.3389382710835586, "G_std": 0.15510649354352304, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.412085771560669, "grad_norm": 0.9541977047920227, "tokens_seen": 74063872, "tokens_per_sec": 14922.444548649177, "vram_gb": 6.442267648, "lambda": 0.29313348854581517, "lambda_min": 0.09383570402860641, "lambda_max": 0.6531543731689453, "S_mean": 0.349090576171875, "G_mean": 0.3328123997586469, "G_std": 0.1565231855493039, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.450541973114014, "grad_norm": 0.9031211733818054, "tokens_seen": 74227712, "tokens_per_sec": 14922.833552120435, "vram_gb": 6.442267648, "lambda": 0.2933033686131239, "lambda_min": 0.09388449043035507, "lambda_max": 0.6534423828125, "S_mean": 0.3415018717447917, "G_mean": 0.3366988787117104, "G_std": 0.1547851167852059, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.4299798011779785, "grad_norm": 0.9159786105155945, "tokens_seen": 74391552, "tokens_per_sec": 14923.357018461964, "vram_gb": 6.442267648, "lambda": 0.29349551318834227, "lambda_min": 0.09391067177057266, "lambda_max": 0.6540569067001343, "S_mean": 0.3351033528645833, "G_mean": 0.34200999955646694, "G_std": 0.1562501599546522, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.3490705490112305, "grad_norm": 0.9599686861038208, "tokens_seen": 74555392, "tokens_per_sec": 14923.926085907184, "vram_gb": 6.442267648, "lambda": 0.293619377228121, "lambda_min": 0.09391265362501144, "lambda_max": 0.6544483304023743, "S_mean": 0.3458658854166667, "G_mean": 0.34261049500977, "G_std": 0.15425823202046254, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.226641416549683, "grad_norm": 0.8940567970275879, "tokens_seen": 74719232, "tokens_per_sec": 14924.549809207509, "vram_gb": 6.442267648, "lambda": 0.29376209154725075, "lambda_min": 0.09390842169523239, "lambda_max": 0.654792308807373, "S_mean": 0.33673095703125, "G_mean": 0.3361261432679991, "G_std": 0.1556142813836535, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.479546308517456, "grad_norm": 0.9145958423614502, "tokens_seen": 74883072, "tokens_per_sec": 14925.116684593446, "vram_gb": 6.442267648, "lambda": 0.29394931967059773, "lambda_min": 0.09391575306653976, "lambda_max": 0.6553090214729309, "S_mean": 0.3474934895833333, "G_mean": 0.34007107249150675, "G_std": 0.15555037678374597, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.541511058807373, "grad_norm": 0.8634687662124634, "tokens_seen": 75046912, "tokens_per_sec": 14925.646785151539, "vram_gb": 6.442267648, "lambda": 0.2941182792807619, "lambda_min": 0.0939587876200676, "lambda_max": 0.6558763384819031, "S_mean": 0.34002685546875, "G_mean": 0.3459863947549214, "G_std": 0.1578207443235442, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.3404669761657715, "grad_norm": 0.949971616268158, "tokens_seen": 75210752, "tokens_per_sec": 14926.304452024768, "vram_gb": 6.442267648, "lambda": 0.29430665355175734, "lambda_min": 0.09395446628332138, "lambda_max": 0.6561278700828552, "S_mean": 0.3469441731770833, "G_mean": 0.3391829081811011, "G_std": 0.1542091170170655, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.459308624267578, "grad_norm": 0.9362788796424866, "tokens_seen": 75374592, "tokens_per_sec": 14926.815960159174, "vram_gb": 6.442267648, "lambda": 0.29446953255683184, "lambda_min": 0.093978151679039, "lambda_max": 0.6564785242080688, "S_mean": 0.338043212890625, "G_mean": 0.3425008775666356, "G_std": 0.15621602934940407, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.356881141662598, "grad_norm": 0.9759318828582764, "tokens_seen": 75538432, "tokens_per_sec": 14927.619685567493, "vram_gb": 6.442267648, "lambda": 0.2946350971857707, "lambda_min": 0.09406553208827972, "lambda_max": 0.6568685173988342, "S_mean": 0.3430277506510417, "G_mean": 0.340011556710427, "G_std": 0.1548775154321144, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.373432397842407, "grad_norm": 0.9082977175712585, "tokens_seen": 75702272, "tokens_per_sec": 14928.28122100684, "vram_gb": 6.442267648, "lambda": 0.2947903222714861, "lambda_min": 0.09415022283792496, "lambda_max": 0.6573172211647034, "S_mean": 0.34088134765625, "G_mean": 0.3398952594337364, "G_std": 0.15496066589063653, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.460382699966431, "grad_norm": 0.9384906888008118, "tokens_seen": 75866112, "tokens_per_sec": 14928.756156176603, "vram_gb": 6.442267648, "lambda": 0.29491288866847754, "lambda_min": 0.09415461868047714, "lambda_max": 0.6577894687652588, "S_mean": 0.34246826171875, "G_mean": 0.3459074574833115, "G_std": 0.1562779792196428, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.453553199768066, "grad_norm": 0.9040764570236206, "tokens_seen": 76029952, "tokens_per_sec": 14929.182558394841, "vram_gb": 6.442267648, "lambda": 0.29505225891868275, "lambda_min": 0.0941445454955101, "lambda_max": 0.6581253409385681, "S_mean": 0.3381449381510417, "G_mean": 0.33944013466437656, "G_std": 0.15601792799619338, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.53645658493042, "grad_norm": 1.046610951423645, "tokens_seen": 76193792, "tokens_per_sec": 14929.689773600281, "vram_gb": 6.442267648, "lambda": 0.2951832925900817, "lambda_min": 0.09419378638267517, "lambda_max": 0.6584573984146118, "S_mean": 0.3461405436197917, "G_mean": 0.34641445335000753, "G_std": 0.15608041388137886, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.286569118499756, "grad_norm": 0.8956454396247864, "tokens_seen": 76357632, "tokens_per_sec": 14930.266271098848, "vram_gb": 6.442267648, "lambda": 0.29534027073532343, "lambda_min": 0.0941961407661438, "lambda_max": 0.6590436697006226, "S_mean": 0.34710693359375, "G_mean": 0.33782087328533333, "G_std": 0.15572789447226873, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.280029773712158, "grad_norm": 0.9650254845619202, "tokens_seen": 76521472, "tokens_per_sec": 14930.82099235739, "vram_gb": 6.442267648, "lambda": 0.2955094895636042, "lambda_min": 0.09416727721691132, "lambda_max": 0.6597946882247925, "S_mean": 0.339935302734375, "G_mean": 0.33738527726382017, "G_std": 0.15614976384676993, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.468251705169678, "grad_norm": 0.9276087880134583, "tokens_seen": 76685312, "tokens_per_sec": 14931.29331054354, "vram_gb": 6.442267648, "lambda": 0.2956567853689194, "lambda_min": 0.09413707256317139, "lambda_max": 0.6601039171218872, "S_mean": 0.3379414876302083, "G_mean": 0.34047717197487753, "G_std": 0.15710285996707776, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.38978123664856, "grad_norm": 0.9779486656188965, "tokens_seen": 76849152, "tokens_per_sec": 14931.916838781302, "vram_gb": 6.442267648, "lambda": 0.2957918103784323, "lambda_min": 0.09412679076194763, "lambda_max": 0.6605513691902161, "S_mean": 0.35064697265625, "G_mean": 0.34681121236644685, "G_std": 0.15576530752393106, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.354689121246338, "grad_norm": 0.9554521441459656, "tokens_seen": 77012992, "tokens_per_sec": 14932.458137672997, "vram_gb": 6.442267648, "lambda": 0.2959379243354003, "lambda_min": 0.09416177123785019, "lambda_max": 0.6612764000892639, "S_mean": 0.3568929036458333, "G_mean": 0.34507056636114913, "G_std": 0.15728121134452522, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.242233991622925, "grad_norm": 0.9328673481941223, "tokens_seen": 77176832, "tokens_per_sec": 14933.145183081819, "vram_gb": 6.442267648, "lambda": 0.29608920713265735, "lambda_min": 0.09414268285036087, "lambda_max": 0.6616489887237549, "S_mean": 0.3348592122395833, "G_mean": 0.34414994926191866, "G_std": 0.15683374006766826, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.325241804122925, "grad_norm": 0.8951535820960999, "tokens_seen": 77340672, "tokens_per_sec": 14933.748429450181, "vram_gb": 6.442267648, "lambda": 0.2962161048005025, "lambda_min": 0.09414645284414291, "lambda_max": 0.6619619131088257, "S_mean": 0.343536376953125, "G_mean": 0.34234863286837935, "G_std": 0.15643781745651117, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.30618691444397, "grad_norm": 0.9228097796440125, "tokens_seen": 77504512, "tokens_per_sec": 14934.39065123639, "vram_gb": 6.442267648, "lambda": 0.29629969876259565, "lambda_min": 0.09414122998714447, "lambda_max": 0.6620470881462097, "S_mean": 0.3549702962239583, "G_mean": 0.34004356983738643, "G_std": 0.1562966755979384, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.3740293979644775, "grad_norm": 0.8960273861885071, "tokens_seen": 77668352, "tokens_per_sec": 14935.012999958464, "vram_gb": 6.442267648, "lambda": 0.29643203256030876, "lambda_min": 0.09410719573497772, "lambda_max": 0.6624281406402588, "S_mean": 0.3405965169270833, "G_mean": 0.34055806308363873, "G_std": 0.15588213751713434, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.363823652267456, "grad_norm": 0.9088011384010315, "tokens_seen": 77832192, "tokens_per_sec": 14935.505108504638, "vram_gb": 6.442267648, "lambda": 0.29654729800919694, "lambda_min": 0.09410837292671204, "lambda_max": 0.6626837253570557, "S_mean": 0.344970703125, "G_mean": 0.3484238611999899, "G_std": 0.15624978633907935, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.449713778495789, "val_ppl": 85.60243923654998, "lambda": 0.2965536577006181, "lambda_min": 0.09410681575536728, "lambda_max": 0.6627098321914673, "lambda_per_layer": [0.17372171580791473, 0.14225444197654724, 0.14545804262161255, 0.1390938013792038, 0.12114852666854858, 0.12076485902070999, 0.09410681575536728, 0.1279819756746292, 0.13359017670154572, 0.2941988408565521, 0.357408344745636, 0.4909340739250183, 0.39953234791755676, 0.6627098321914673, 0.4935649037361145, 0.5337433218955994, 0.4130803644657135, 0.4197838008403778, 0.27223506569862366, 0.274007648229599, 0.44713538885116577, 0.4072481691837311, 0.23945483565330505, 0.2141304910182953], "S_mean": 0.34375, "G_mean": 0.3423602002051969, "G_std": 0.15675876510795206, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.409050941467285, "grad_norm": 0.954928994178772, "tokens_seen": 77996032, "tokens_per_sec": 14924.499934439315, "vram_gb": 6.442267648, "lambda": 0.2967016731078426, "lambda_min": 0.09408268332481384, "lambda_max": 0.6632927656173706, "S_mean": 0.34161376953125, "G_mean": 0.34734264404202503, "G_std": 0.15720463128915677, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.285876750946045, "grad_norm": 0.9503238201141357, "tokens_seen": 78159872, "tokens_per_sec": 14924.94090626281, "vram_gb": 6.442267648, "lambda": 0.2968842424452305, "lambda_min": 0.09408397227525711, "lambda_max": 0.6638268232345581, "S_mean": 0.33203125, "G_mean": 0.34081506977478665, "G_std": 0.1554602294927463, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.402354717254639, "grad_norm": 0.9970703721046448, "tokens_seen": 78323712, "tokens_per_sec": 14925.544185634044, "vram_gb": 6.442267648, "lambda": 0.2969930050894618, "lambda_min": 0.09408852458000183, "lambda_max": 0.6640706658363342, "S_mean": 0.3411661783854167, "G_mean": 0.33961647904167575, "G_std": 0.15718531662908694, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.440065383911133, "grad_norm": 0.9733800292015076, "tokens_seen": 78487552, "tokens_per_sec": 14926.101316866781, "vram_gb": 6.442267648, "lambda": 0.29711972549557686, "lambda_min": 0.09410732984542847, "lambda_max": 0.6645560264587402, "S_mean": 0.33721923828125, "G_mean": 0.3427540115856876, "G_std": 0.1565513265474389, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.335299491882324, "grad_norm": 0.902706503868103, "tokens_seen": 78651392, "tokens_per_sec": 14926.607406185765, "vram_gb": 6.442267648, "lambda": 0.29729169482986134, "lambda_min": 0.09408088028430939, "lambda_max": 0.6648909449577332, "S_mean": 0.3491617838541667, "G_mean": 0.33877854763219756, "G_std": 0.15647958525611708, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.364617347717285, "grad_norm": 0.9746003746986389, "tokens_seen": 78815232, "tokens_per_sec": 14927.1062994856, "vram_gb": 6.442267648, "lambda": 0.29742314169804257, "lambda_min": 0.09410002827644348, "lambda_max": 0.6653509736061096, "S_mean": 0.3421529134114583, "G_mean": 0.33801450735578936, "G_std": 0.15592745854519308, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.269541025161743, "grad_norm": 0.9092373251914978, "tokens_seen": 78979072, "tokens_per_sec": 14927.75741517166, "vram_gb": 6.442267648, "lambda": 0.29755173654605943, "lambda_min": 0.09416116029024124, "lambda_max": 0.665748655796051, "S_mean": 0.3434855143229167, "G_mean": 0.3370431348060568, "G_std": 0.1557444205197195, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.521231174468994, "grad_norm": 0.9566341042518616, "tokens_seen": 79142912, "tokens_per_sec": 14928.378275149431, "vram_gb": 6.442267648, "lambda": 0.29764372824380797, "lambda_min": 0.09417083859443665, "lambda_max": 0.6659108996391296, "S_mean": 0.34918212890625, "G_mean": 0.3354852390475571, "G_std": 0.15450360373749086, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.324923992156982, "grad_norm": 0.8947659134864807, "tokens_seen": 79306752, "tokens_per_sec": 14929.05229244773, "vram_gb": 6.442267648, "lambda": 0.2977282904709379, "lambda_min": 0.09417808055877686, "lambda_max": 0.666290283203125, "S_mean": 0.3404947916666667, "G_mean": 0.339896086215352, "G_std": 0.1569779074828451, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.409160614013672, "grad_norm": 0.9395270943641663, "tokens_seen": 79470592, "tokens_per_sec": 14929.527483358044, "vram_gb": 6.442267648, "lambda": 0.2978308942789833, "lambda_min": 0.09422877430915833, "lambda_max": 0.666639506816864, "S_mean": 0.3370768229166667, "G_mean": 0.3441615041034917, "G_std": 0.1570589868739868, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.492318391799927, "grad_norm": 0.9576581120491028, "tokens_seen": 79634432, "tokens_per_sec": 14930.149719489213, "vram_gb": 6.442267648, "lambda": 0.2979542603716254, "lambda_min": 0.09419845789670944, "lambda_max": 0.667030930519104, "S_mean": 0.3389892578125, "G_mean": 0.3414705368534972, "G_std": 0.1549583519808948, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.226465225219727, "grad_norm": 0.9144774079322815, "tokens_seen": 79798272, "tokens_per_sec": 14930.717654356436, "vram_gb": 6.442267648, "lambda": 0.29808915468553704, "lambda_min": 0.09418074041604996, "lambda_max": 0.6673345565795898, "S_mean": 0.3456217447916667, "G_mean": 0.3430751849276324, "G_std": 0.15518488952269158, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.45874547958374, "grad_norm": 0.9581594467163086, "tokens_seen": 79962112, "tokens_per_sec": 14931.18409340754, "vram_gb": 6.442267648, "lambda": 0.29819946611920994, "lambda_min": 0.09416254609823227, "lambda_max": 0.6675744652748108, "S_mean": 0.3402099609375, "G_mean": 0.34178857536365587, "G_std": 0.15580120127803335, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.360068321228027, "grad_norm": 0.946341335773468, "tokens_seen": 80125952, "tokens_per_sec": 14931.78823745524, "vram_gb": 6.442267648, "lambda": 0.2983569260686636, "lambda_min": 0.09417353570461273, "lambda_max": 0.6681148409843445, "S_mean": 0.3270772298177083, "G_mean": 0.33922408032231033, "G_std": 0.1553312378237024, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.449268102645874, "grad_norm": 0.9416608214378357, "tokens_seen": 80289792, "tokens_per_sec": 14932.028614494182, "vram_gb": 6.442267648, "lambda": 0.29845513838032883, "lambda_min": 0.0941842719912529, "lambda_max": 0.6684383749961853, "S_mean": 0.3397115071614583, "G_mean": 0.3417251609886686, "G_std": 0.1557988755327339, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.277072191238403, "grad_norm": 0.9341790080070496, "tokens_seen": 80453632, "tokens_per_sec": 14932.568392106412, "vram_gb": 6.442267648, "lambda": 0.29855115339159966, "lambda_min": 0.09415172785520554, "lambda_max": 0.668938398361206, "S_mean": 0.3447672526041667, "G_mean": 0.3345434626874824, "G_std": 0.15420596511103213, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.483182907104492, "grad_norm": 1.06686270236969, "tokens_seen": 80617472, "tokens_per_sec": 14933.176871603357, "vram_gb": 6.442267648, "lambda": 0.2986542194460829, "lambda_min": 0.09415560215711594, "lambda_max": 0.6692141890525818, "S_mean": 0.3491617838541667, "G_mean": 0.3405516595424463, "G_std": 0.15533210647602877, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.513521671295166, "grad_norm": 1.033648133277893, "tokens_seen": 80781312, "tokens_per_sec": 14933.738369713426, "vram_gb": 6.442267648, "lambda": 0.29873531746367615, "lambda_min": 0.09419520199298859, "lambda_max": 0.6694401502609253, "S_mean": 0.3300577799479167, "G_mean": 0.34181329421699047, "G_std": 0.1558642538342004, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.342929363250732, "grad_norm": 0.9444940686225891, "tokens_seen": 80945152, "tokens_per_sec": 14934.315417888574, "vram_gb": 6.442267648, "lambda": 0.2988433415691058, "lambda_min": 0.09421089291572571, "lambda_max": 0.6697380542755127, "S_mean": 0.3414103190104167, "G_mean": 0.3430671164145072, "G_std": 0.15702977392356843, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.327055215835571, "grad_norm": 0.90279620885849, "tokens_seen": 81108992, "tokens_per_sec": 14934.969561617854, "vram_gb": 6.442267648, "lambda": 0.29895817550520104, "lambda_min": 0.09416559338569641, "lambda_max": 0.6699740886688232, "S_mean": 0.3359781901041667, "G_mean": 0.3322411383657406, "G_std": 0.15334298356901854, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.304943799972534, "grad_norm": 0.9676445722579956, "tokens_seen": 81272832, "tokens_per_sec": 14935.496073301856, "vram_gb": 6.442267648, "lambda": 0.2990503925830126, "lambda_min": 0.09411872178316116, "lambda_max": 0.6703228950500488, "S_mean": 0.3426005045572917, "G_mean": 0.3373091338823239, "G_std": 0.15506818801319847, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.26590371131897, "grad_norm": 0.9038296341896057, "tokens_seen": 81436672, "tokens_per_sec": 14935.945908144824, "vram_gb": 6.442267648, "lambda": 0.2991308659935991, "lambda_min": 0.09407632797956467, "lambda_max": 0.6706355214118958, "S_mean": 0.3521321614583333, "G_mean": 0.33665582824808854, "G_std": 0.15495826843349883, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.338632345199585, "grad_norm": 1.0484812259674072, "tokens_seen": 81600512, "tokens_per_sec": 14936.258672821154, "vram_gb": 6.442267648, "lambda": 0.29923958486566943, "lambda_min": 0.0940602496266365, "lambda_max": 0.6710255146026611, "S_mean": 0.3457234700520833, "G_mean": 0.33687299117445946, "G_std": 0.1543917516246438, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.393367290496826, "grad_norm": 0.9242567420005798, "tokens_seen": 81764352, "tokens_per_sec": 14936.746706545946, "vram_gb": 6.442267648, "lambda": 0.29932749706010026, "lambda_min": 0.09404738992452621, "lambda_max": 0.6714771389961243, "S_mean": 0.3278299967447917, "G_mean": 0.3354289364845802, "G_std": 0.15322831905602166, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.324761867523193, "grad_norm": 0.9318032264709473, "tokens_seen": 81928192, "tokens_per_sec": 14937.418687762578, "vram_gb": 6.442267648, "lambda": 0.29943383981784183, "lambda_min": 0.09404439479112625, "lambda_max": 0.671932578086853, "S_mean": 0.337127685546875, "G_mean": 0.3334744846603523, "G_std": 0.1545278997703766, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.415489661693573, "val_ppl": 82.72233695022565, "lambda": 0.2994380372886856, "lambda_min": 0.09404432773590088, "lambda_max": 0.671949565410614, "lambda_per_layer": [0.17420561611652374, 0.142252579331398, 0.14580200612545013, 0.13937564194202423, 0.12108823657035828, 0.12041874974966049, 0.09404432773590088, 0.1277526170015335, 0.13344621658325195, 0.29506078362464905, 0.3594039976596832, 0.4946514368057251, 0.40210437774658203, 0.671949565410614, 0.49945658445358276, 0.5420268774032593, 0.41799020767211914, 0.4260045886039734, 0.27525362372398376, 0.27725037932395935, 0.4534720182418823, 0.4152059555053711, 0.24187679588794708, 0.21641971170902252], "S_mean": 0.3440653483072917, "G_mean": 0.33599601973158616, "G_std": 0.15485980664379895, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.645721435546875, "grad_norm": 0.9993219375610352, "tokens_seen": 82092032, "tokens_per_sec": 14926.900273994608, "vram_gb": 6.442267648, "lambda": 0.2995408872763316, "lambda_min": 0.09404178708791733, "lambda_max": 0.6723024845123291, "S_mean": 0.34124755859375, "G_mean": 0.34047093490759534, "G_std": 0.15672027257581553, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.23856520652771, "grad_norm": 0.938989520072937, "tokens_seen": 82255872, "tokens_per_sec": 14927.222251301746, "vram_gb": 6.442267648, "lambda": 0.29959512564043206, "lambda_min": 0.09401305019855499, "lambda_max": 0.6724454760551453, "S_mean": 0.3538614908854167, "G_mean": 0.33106089492018026, "G_std": 0.15374788102538636, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.470928192138672, "grad_norm": 0.9454299211502075, "tokens_seen": 82419712, "tokens_per_sec": 14927.682542525152, "vram_gb": 6.442267648, "lambda": 0.29971005860716105, "lambda_min": 0.09395863115787506, "lambda_max": 0.6726465225219727, "S_mean": 0.343780517578125, "G_mean": 0.34144469009091455, "G_std": 0.15516569973745695, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.239022731781006, "grad_norm": 0.9523067474365234, "tokens_seen": 82583552, "tokens_per_sec": 14928.313989134382, "vram_gb": 6.442267648, "lambda": 0.2998379360263546, "lambda_min": 0.09394491463899612, "lambda_max": 0.6730488538742065, "S_mean": 0.3421223958333333, "G_mean": 0.3406158823830386, "G_std": 0.15515029341137657, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.28256630897522, "grad_norm": 0.9733583927154541, "tokens_seen": 82747392, "tokens_per_sec": 14928.698635303048, "vram_gb": 6.442267648, "lambda": 0.29997461506476003, "lambda_min": 0.09393171221017838, "lambda_max": 0.6733688116073608, "S_mean": 0.3431599934895833, "G_mean": 0.34258005674928427, "G_std": 0.1553571238570536, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.319440603256226, "grad_norm": 0.9292161464691162, "tokens_seen": 82911232, "tokens_per_sec": 14929.280392700235, "vram_gb": 6.442267648, "lambda": 0.30009393704434234, "lambda_min": 0.0938846617937088, "lambda_max": 0.6735929250717163, "S_mean": 0.3457845052083333, "G_mean": 0.33917338983155787, "G_std": 0.15688638866413385, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.266887426376343, "grad_norm": 0.9653306007385254, "tokens_seen": 83075072, "tokens_per_sec": 14929.826017874599, "vram_gb": 6.442267648, "lambda": 0.3001837246119976, "lambda_min": 0.09384505450725555, "lambda_max": 0.673830509185791, "S_mean": 0.3330078125, "G_mean": 0.3344895443879068, "G_std": 0.15476741377885142, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.369798183441162, "grad_norm": 0.9158118963241577, "tokens_seen": 83238912, "tokens_per_sec": 14930.368894371793, "vram_gb": 6.442267648, "lambda": 0.30028996833910543, "lambda_min": 0.09381222724914551, "lambda_max": 0.6740970611572266, "S_mean": 0.34246826171875, "G_mean": 0.3406371967867017, "G_std": 0.15555161656811833, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.2714831829071045, "grad_norm": 0.8914636373519897, "tokens_seen": 83402752, "tokens_per_sec": 14930.790403669524, "vram_gb": 6.442267648, "lambda": 0.30039458597699803, "lambda_min": 0.09378457814455032, "lambda_max": 0.6742639541625977, "S_mean": 0.3525797526041667, "G_mean": 0.3327654702588916, "G_std": 0.15439964116861424, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.460568189620972, "grad_norm": 1.0623669624328613, "tokens_seen": 83566592, "tokens_per_sec": 14931.227976451508, "vram_gb": 6.442267648, "lambda": 0.30047296173870564, "lambda_min": 0.09373653680086136, "lambda_max": 0.674344539642334, "S_mean": 0.3428548177083333, "G_mean": 0.3368753941419224, "G_std": 0.1548707621792952, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.374699115753174, "grad_norm": 0.9799293279647827, "tokens_seen": 83730432, "tokens_per_sec": 14931.770819275082, "vram_gb": 6.442267648, "lambda": 0.3005556231364608, "lambda_min": 0.0937589630484581, "lambda_max": 0.6745506525039673, "S_mean": 0.3440958658854167, "G_mean": 0.341253148081402, "G_std": 0.15691068830589452, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.3191046714782715, "grad_norm": 0.9660223722457886, "tokens_seen": 83894272, "tokens_per_sec": 14932.316552196035, "vram_gb": 6.442267648, "lambda": 0.3006566911935806, "lambda_min": 0.09377789497375488, "lambda_max": 0.6749304533004761, "S_mean": 0.3343505859375, "G_mean": 0.33171902170094353, "G_std": 0.15310027721958855, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.4344542026519775, "grad_norm": 0.9547362923622131, "tokens_seen": 84058112, "tokens_per_sec": 14932.934085890214, "vram_gb": 6.442267648, "lambda": 0.30073328129947186, "lambda_min": 0.09374210983514786, "lambda_max": 0.6751123070716858, "S_mean": 0.3334147135416667, "G_mean": 0.3398340597438316, "G_std": 0.15422400176369896, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.346273422241211, "grad_norm": 0.9351351857185364, "tokens_seen": 84221952, "tokens_per_sec": 14933.323194941408, "vram_gb": 6.442267648, "lambda": 0.30081363767385483, "lambda_min": 0.093744195997715, "lambda_max": 0.6754960417747498, "S_mean": 0.336822509765625, "G_mean": 0.33307005232200027, "G_std": 0.15401938802096993, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.253581523895264, "grad_norm": 0.9741402268409729, "tokens_seen": 84385792, "tokens_per_sec": 14933.763655695911, "vram_gb": 6.442267648, "lambda": 0.30089299753308296, "lambda_min": 0.09380330890417099, "lambda_max": 0.6759122014045715, "S_mean": 0.3420511881510417, "G_mean": 0.33023961385091144, "G_std": 0.15270732719606409, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.293280363082886, "grad_norm": 0.9490293264389038, "tokens_seen": 84549632, "tokens_per_sec": 14934.386928051628, "vram_gb": 6.442267648, "lambda": 0.3009818975503246, "lambda_min": 0.09381819516420364, "lambda_max": 0.676272451877594, "S_mean": 0.3465779622395833, "G_mean": 0.34072927323480445, "G_std": 0.15498555800877512, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.4628636837005615, "grad_norm": 1.0592477321624756, "tokens_seen": 84713472, "tokens_per_sec": 14935.023659118433, "vram_gb": 6.442267648, "lambda": 0.30108450135837, "lambda_min": 0.09379862248897552, "lambda_max": 0.6766549944877625, "S_mean": 0.3355000813802083, "G_mean": 0.33758410019800067, "G_std": 0.15446012381774685, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.341476202011108, "grad_norm": 0.9295258522033691, "tokens_seen": 84877312, "tokens_per_sec": 14935.465237024158, "vram_gb": 6.442267648, "lambda": 0.3011891804635525, "lambda_min": 0.09382359683513641, "lambda_max": 0.6769044399261475, "S_mean": 0.3378702799479167, "G_mean": 0.33347883688596386, "G_std": 0.1535851030688112, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.355471611022949, "grad_norm": 0.9808046221733093, "tokens_seen": 85041152, "tokens_per_sec": 14935.85121998282, "vram_gb": 6.442267648, "lambda": 0.30129477630058926, "lambda_min": 0.09381905198097229, "lambda_max": 0.6770684123039246, "S_mean": 0.33782958984375, "G_mean": 0.34035456118484336, "G_std": 0.15576175561485192, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.462846279144287, "grad_norm": 1.0741585493087769, "tokens_seen": 85204992, "tokens_per_sec": 14936.24767650435, "vram_gb": 6.442267648, "lambda": 0.30138971470296383, "lambda_min": 0.09384838491678238, "lambda_max": 0.6773272752761841, "S_mean": 0.3446553548177083, "G_mean": 0.339464097439001, "G_std": 0.15686365907701352, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.399160146713257, "grad_norm": 1.0261707305908203, "tokens_seen": 85368832, "tokens_per_sec": 14936.680105193169, "vram_gb": 6.442267648, "lambda": 0.3014794485643506, "lambda_min": 0.0938219353556633, "lambda_max": 0.6776118874549866, "S_mean": 0.3586018880208333, "G_mean": 0.34288667057019967, "G_std": 0.15642543191400668, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.423435926437378, "grad_norm": 1.0015788078308105, "tokens_seen": 85532672, "tokens_per_sec": 14937.246492724891, "vram_gb": 6.442267648, "lambda": 0.3015691541756193, "lambda_min": 0.09382539242506027, "lambda_max": 0.6778013110160828, "S_mean": 0.346282958984375, "G_mean": 0.33816341077908874, "G_std": 0.15607581473886967, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.246306657791138, "grad_norm": 0.9434471130371094, "tokens_seen": 85696512, "tokens_per_sec": 14937.651473781523, "vram_gb": 6.442267648, "lambda": 0.3016777755692601, "lambda_min": 0.09384181350469589, "lambda_max": 0.6781051754951477, "S_mean": 0.3497314453125, "G_mean": 0.3383414135314524, "G_std": 0.15585175657179207, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.335414409637451, "grad_norm": 0.9551554918289185, "tokens_seen": 85860352, "tokens_per_sec": 14938.11515546235, "vram_gb": 6.442267648, "lambda": 0.3017521755148967, "lambda_min": 0.09384995698928833, "lambda_max": 0.6783102750778198, "S_mean": 0.341796875, "G_mean": 0.3337688127843042, "G_std": 0.15466299493952343, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.51115083694458, "grad_norm": 1.2270331382751465, "tokens_seen": 86024192, "tokens_per_sec": 14938.532728648084, "vram_gb": 6.442267648, "lambda": 0.30183420578638714, "lambda_min": 0.09386590123176575, "lambda_max": 0.6786509156227112, "S_mean": 0.3401590983072917, "G_mean": 0.3411858434168001, "G_std": 0.15579143965927264, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.409650361537933, "val_ppl": 82.24070396352076, "lambda": 0.3018367492283384, "lambda_min": 0.09386471658945084, "lambda_max": 0.6786674857139587, "lambda_per_layer": [0.174759179353714, 0.1423782855272293, 0.14609235525131226, 0.1396339237689972, 0.12093829363584518, 0.12040045112371445, 0.09386471658945084, 0.12780454754829407, 0.13361693918704987, 0.2957431375980377, 0.36078816652297974, 0.4983282685279846, 0.4043641686439514, 0.6786674857139587, 0.5041345953941345, 0.5486405491828918, 0.42330190539360046, 0.43096521496772766, 0.27764949202537537, 0.28004539012908936, 0.4588973820209503, 0.42122575640678406, 0.24427129328250885, 0.2175704836845398], "S_mean": 0.3445536295572917, "G_mean": 0.3349849837832153, "G_std": 0.15453344874549657, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.354558229446411, "grad_norm": 1.0118298530578613, "tokens_seen": 86188032, "tokens_per_sec": 14928.527140111017, "vram_gb": 6.442267648, "lambda": 0.3019057046622038, "lambda_min": 0.09385207295417786, "lambda_max": 0.6788370609283447, "S_mean": 0.3415120442708333, "G_mean": 0.3370377699223657, "G_std": 0.15592554688919336, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.454797744750977, "grad_norm": 0.9736751914024353, "tokens_seen": 86351872, "tokens_per_sec": 14929.008570408385, "vram_gb": 6.442267648, "lambda": 0.30200791110595065, "lambda_min": 0.09385174512863159, "lambda_max": 0.6790484189987183, "S_mean": 0.3510538736979167, "G_mean": 0.3331927874435981, "G_std": 0.15468620109216621, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.422595500946045, "grad_norm": 0.9860777258872986, "tokens_seen": 86515712, "tokens_per_sec": 14929.470839436448, "vram_gb": 6.442267648, "lambda": 0.30211212548116845, "lambda_min": 0.0938287004828453, "lambda_max": 0.6793221831321716, "S_mean": 0.3312479654947917, "G_mean": 0.34042731773418683, "G_std": 0.15430629144733152, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.304254531860352, "grad_norm": 0.9627585411071777, "tokens_seen": 86679552, "tokens_per_sec": 14930.056422147143, "vram_gb": 6.442267648, "lambda": 0.3021982526406646, "lambda_min": 0.0938464030623436, "lambda_max": 0.6794958710670471, "S_mean": 0.329193115234375, "G_mean": 0.33403980180931586, "G_std": 0.15409313918401799, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.236014366149902, "grad_norm": 0.9724677801132202, "tokens_seen": 86843392, "tokens_per_sec": 14930.417646505406, "vram_gb": 6.442267648, "lambda": 0.30227868507305783, "lambda_min": 0.0939142182469368, "lambda_max": 0.6797395348548889, "S_mean": 0.3369344075520833, "G_mean": 0.33552397081318003, "G_std": 0.15514779083120325, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.330713748931885, "grad_norm": 1.0832579135894775, "tokens_seen": 87007232, "tokens_per_sec": 14930.725516610719, "vram_gb": 6.442267648, "lambda": 0.30237143735090893, "lambda_min": 0.09393954277038574, "lambda_max": 0.6800175309181213, "S_mean": 0.3444010416666667, "G_mean": 0.3409980114083737, "G_std": 0.15647655449962863, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.455790996551514, "grad_norm": 1.0482819080352783, "tokens_seen": 87171072, "tokens_per_sec": 14931.12175479593, "vram_gb": 6.442267648, "lambda": 0.3024684761961301, "lambda_min": 0.09396994858980179, "lambda_max": 0.6804964542388916, "S_mean": 0.3445638020833333, "G_mean": 0.34044065947333974, "G_std": 0.15411109485042593, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.259236812591553, "grad_norm": 0.9145346879959106, "tokens_seen": 87334912, "tokens_per_sec": 14931.589159545041, "vram_gb": 6.442267648, "lambda": 0.3025721811378996, "lambda_min": 0.09396173059940338, "lambda_max": 0.6809080243110657, "S_mean": 0.3563741048177083, "G_mean": 0.34107425234590966, "G_std": 0.15402753868450722, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.266319274902344, "grad_norm": 0.972868025302887, "tokens_seen": 87498752, "tokens_per_sec": 14932.096298217808, "vram_gb": 6.442267648, "lambda": 0.30267492060859996, "lambda_min": 0.09394697099924088, "lambda_max": 0.6811693906784058, "S_mean": 0.3602193196614583, "G_mean": 0.33487795627055067, "G_std": 0.154563615603062, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.3256683349609375, "grad_norm": 0.9177030920982361, "tokens_seen": 87662592, "tokens_per_sec": 14932.776146672646, "vram_gb": 6.442267648, "lambda": 0.30275496405859786, "lambda_min": 0.09393917769193649, "lambda_max": 0.6814695596694946, "S_mean": 0.3442179361979167, "G_mean": 0.33367965277284384, "G_std": 0.15381350570047894, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.349996089935303, "grad_norm": 1.0189820528030396, "tokens_seen": 87826432, "tokens_per_sec": 14933.378866863375, "vram_gb": 6.442267648, "lambda": 0.3028086358681321, "lambda_min": 0.09392533451318741, "lambda_max": 0.6816771030426025, "S_mean": 0.3458455403645833, "G_mean": 0.3409408323932439, "G_std": 0.15527999001399925, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.4051923751831055, "grad_norm": 1.0015795230865479, "tokens_seen": 87990272, "tokens_per_sec": 14933.876506768327, "vram_gb": 6.442267648, "lambda": 0.30290188267827034, "lambda_min": 0.09389311820268631, "lambda_max": 0.6819693446159363, "S_mean": 0.3417154947916667, "G_mean": 0.3410857390457143, "G_std": 0.1553135038508723, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.310626029968262, "grad_norm": 0.9396169185638428, "tokens_seen": 88154112, "tokens_per_sec": 14934.391647218476, "vram_gb": 6.442267648, "lambda": 0.30298712352911633, "lambda_min": 0.09385605901479721, "lambda_max": 0.6822060346603394, "S_mean": 0.3502197265625, "G_mean": 0.3348843969094257, "G_std": 0.1544894053367898, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.336966276168823, "grad_norm": 1.0335668325424194, "tokens_seen": 88317952, "tokens_per_sec": 14934.787719126036, "vram_gb": 6.442267648, "lambda": 0.30306046300878126, "lambda_min": 0.09386911243200302, "lambda_max": 0.6825258731842041, "S_mean": 0.339447021484375, "G_mean": 0.33898684607508284, "G_std": 0.15565164049621671, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.401880502700806, "grad_norm": 1.0354000329971313, "tokens_seen": 88481792, "tokens_per_sec": 14935.329657388495, "vram_gb": 6.442267648, "lambda": 0.3031277833506465, "lambda_min": 0.09384418278932571, "lambda_max": 0.6827008724212646, "S_mean": 0.3363749186197917, "G_mean": 0.3363374946638942, "G_std": 0.15488578666312, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.505280494689941, "grad_norm": 0.9883419275283813, "tokens_seen": 88645632, "tokens_per_sec": 14935.797518469786, "vram_gb": 6.442267648, "lambda": 0.3032203586772084, "lambda_min": 0.09384698420763016, "lambda_max": 0.6828886866569519, "S_mean": 0.3507181803385417, "G_mean": 0.33997396922980744, "G_std": 0.1560443063111355, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.191133737564087, "grad_norm": 0.9284513592720032, "tokens_seen": 88809472, "tokens_per_sec": 14936.34047803098, "vram_gb": 6.442267648, "lambda": 0.30329576258858043, "lambda_min": 0.09384307265281677, "lambda_max": 0.6830493211746216, "S_mean": 0.3392333984375, "G_mean": 0.33203533858371276, "G_std": 0.15324449760373682, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.368921518325806, "grad_norm": 0.9350549578666687, "tokens_seen": 88973312, "tokens_per_sec": 14936.724444073865, "vram_gb": 6.442267648, "lambda": 0.3033824913824598, "lambda_min": 0.09386421740055084, "lambda_max": 0.6832775473594666, "S_mean": 0.33892822265625, "G_mean": 0.33734686113893986, "G_std": 0.15253643272444606, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.214928388595581, "grad_norm": 0.9921126961708069, "tokens_seen": 89137152, "tokens_per_sec": 14937.306266672796, "vram_gb": 6.442267648, "lambda": 0.30348380530873936, "lambda_min": 0.09386667609214783, "lambda_max": 0.6835870146751404, "S_mean": 0.3398844401041667, "G_mean": 0.3345504488485555, "G_std": 0.15391882470188042, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.346899747848511, "grad_norm": 0.9610193371772766, "tokens_seen": 89300992, "tokens_per_sec": 14937.739640630669, "vram_gb": 6.442267648, "lambda": 0.3035652969653408, "lambda_min": 0.0938384160399437, "lambda_max": 0.6837693452835083, "S_mean": 0.3434041341145833, "G_mean": 0.33736404636874795, "G_std": 0.15498364821542054, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.347976207733154, "grad_norm": 0.9951085448265076, "tokens_seen": 89464832, "tokens_per_sec": 14938.312633997095, "vram_gb": 6.442267648, "lambda": 0.30365455833574134, "lambda_min": 0.09382542222738266, "lambda_max": 0.6840481162071228, "S_mean": 0.3447367350260417, "G_mean": 0.3356605110069116, "G_std": 0.1538749475730583, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.297002792358398, "grad_norm": 0.9627726078033447, "tokens_seen": 89628672, "tokens_per_sec": 14938.700063315997, "vram_gb": 6.442267648, "lambda": 0.3037222384785612, "lambda_min": 0.09380549937486649, "lambda_max": 0.6841060519218445, "S_mean": 0.34906005859375, "G_mean": 0.33223293457801145, "G_std": 0.1529973717406392, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.306100845336914, "grad_norm": 0.98227858543396, "tokens_seen": 89792512, "tokens_per_sec": 14939.29998284381, "vram_gb": 6.442267648, "lambda": 0.3038029440989097, "lambda_min": 0.09379667788743973, "lambda_max": 0.6843156218528748, "S_mean": 0.33331298828125, "G_mean": 0.33268481593889493, "G_std": 0.15397539812450609, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.496834754943848, "grad_norm": 1.021201252937317, "tokens_seen": 89956352, "tokens_per_sec": 14939.878886950808, "vram_gb": 6.442267648, "lambda": 0.303904262992243, "lambda_min": 0.09379525482654572, "lambda_max": 0.6846476197242737, "S_mean": 0.3391825358072917, "G_mean": 0.34465672137836617, "G_std": 0.1555502708458031, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.310189962387085, "grad_norm": 0.9704057574272156, "tokens_seen": 90120192, "tokens_per_sec": 14940.357461656455, "vram_gb": 6.442267648, "lambda": 0.30398521448175114, "lambda_min": 0.09379872679710388, "lambda_max": 0.6849756836891174, "S_mean": 0.3336181640625, "G_mean": 0.3398795675020665, "G_std": 0.15487452817615122, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.395095551013947, "val_ppl": 81.05237502935219, "lambda": 0.30398758004109067, "lambda_min": 0.09379842877388, "lambda_max": 0.6849870085716248, "lambda_per_layer": [0.17486244440078735, 0.14238399267196655, 0.14633892476558685, 0.13993608951568604, 0.12096955627202988, 0.1205492839217186, 0.09379842877388, 0.1276385337114334, 0.13339443504810333, 0.2967582643032074, 0.36224254965782166, 0.5008607506752014, 0.40606483817100525, 0.6849870085716248, 0.5080519318580627, 0.5548271536827087, 0.4278181791305542, 0.43636566400527954, 0.2794715166091919, 0.2827741503715515, 0.4637535810470581, 0.4264221787452698, 0.2461380958557129, 0.21929436922073364], "S_mean": 0.347686767578125, "G_mean": 0.334761279867962, "G_std": 0.15436289680656046, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.105283737182617, "grad_norm": 0.9570754766464233, "tokens_seen": 90284032, "tokens_per_sec": 14930.368898606424, "vram_gb": 6.442267648, "lambda": 0.3040475870172183, "lambda_min": 0.09377966076135635, "lambda_max": 0.6852011680603027, "S_mean": 0.344146728515625, "G_mean": 0.33876407992405194, "G_std": 0.1554784078228598, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.320252895355225, "grad_norm": 1.0587749481201172, "tokens_seen": 90447872, "tokens_per_sec": 14930.790212792184, "vram_gb": 6.442267648, "lambda": 0.3041220245261987, "lambda_min": 0.09377966076135635, "lambda_max": 0.6853207349777222, "S_mean": 0.3507893880208333, "G_mean": 0.33934111753478646, "G_std": 0.15577804995700717, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.3862388134002686, "grad_norm": 1.0398063659667969, "tokens_seen": 90611712, "tokens_per_sec": 14931.181030609094, "vram_gb": 6.442267648, "lambda": 0.30422475654631853, "lambda_min": 0.09379296749830246, "lambda_max": 0.6855359673500061, "S_mean": 0.3397216796875, "G_mean": 0.3382815996495386, "G_std": 0.154182597490338, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.388859987258911, "grad_norm": 0.9830520749092102, "tokens_seen": 90775552, "tokens_per_sec": 14931.599504903215, "vram_gb": 6.442267648, "lambda": 0.3043117892617981, "lambda_min": 0.09381668269634247, "lambda_max": 0.685694694519043, "S_mean": 0.34393310546875, "G_mean": 0.33478747998985153, "G_std": 0.15592514510111263, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.403968811035156, "grad_norm": 0.9742821455001831, "tokens_seen": 90939392, "tokens_per_sec": 14931.964605099665, "vram_gb": 6.442267648, "lambda": 0.304379681746165, "lambda_min": 0.09380820393562317, "lambda_max": 0.6858353614807129, "S_mean": 0.345245361328125, "G_mean": 0.33671689926025766, "G_std": 0.1531901047953094, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.385655879974365, "grad_norm": 0.985663890838623, "tokens_seen": 91103232, "tokens_per_sec": 14932.475693089611, "vram_gb": 6.442267648, "lambda": 0.3044529479617874, "lambda_min": 0.09385742992162704, "lambda_max": 0.6860862374305725, "S_mean": 0.36248779296875, "G_mean": 0.3414290250899891, "G_std": 0.1558387689292431, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.245863676071167, "grad_norm": 0.9843589067459106, "tokens_seen": 91267072, "tokens_per_sec": 14933.030432455913, "vram_gb": 6.442267648, "lambda": 0.3045361203451951, "lambda_min": 0.09387761354446411, "lambda_max": 0.6862536668777466, "S_mean": 0.3444163004557292, "G_mean": 0.33513817222168046, "G_std": 0.15447834289322296, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.283374547958374, "grad_norm": 0.9975724220275879, "tokens_seen": 91430912, "tokens_per_sec": 14933.62865367564, "vram_gb": 6.442267648, "lambda": 0.30461655867596465, "lambda_min": 0.09388244152069092, "lambda_max": 0.686516523361206, "S_mean": 0.3498433430989583, "G_mean": 0.33398131855453056, "G_std": 0.15421842655632645, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.300235033035278, "grad_norm": 0.9414021372795105, "tokens_seen": 91594752, "tokens_per_sec": 14934.085907818624, "vram_gb": 6.442267648, "lambda": 0.30468419070045155, "lambda_min": 0.09387697279453278, "lambda_max": 0.6866714954376221, "S_mean": 0.3450520833333333, "G_mean": 0.3358771277125925, "G_std": 0.15389608091209084, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.486553192138672, "grad_norm": 1.1040791273117065, "tokens_seen": 91758592, "tokens_per_sec": 14934.702785509084, "vram_gb": 6.442267648, "lambda": 0.3047658372670412, "lambda_min": 0.09386300295591354, "lambda_max": 0.6869040131568909, "S_mean": 0.35052490234375, "G_mean": 0.3363290848210454, "G_std": 0.15445982466917485, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.343919515609741, "grad_norm": 1.0270252227783203, "tokens_seen": 91922432, "tokens_per_sec": 14935.244976686521, "vram_gb": 6.442267648, "lambda": 0.30484327549735707, "lambda_min": 0.09388519823551178, "lambda_max": 0.6871203780174255, "S_mean": 0.342864990234375, "G_mean": 0.33613793497594696, "G_std": 0.15430876100435853, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.267004013061523, "grad_norm": 0.9596111178398132, "tokens_seen": 92086272, "tokens_per_sec": 14935.751152880133, "vram_gb": 6.442267648, "lambda": 0.3048963031421105, "lambda_min": 0.09387177973985672, "lambda_max": 0.6873399615287781, "S_mean": 0.3524169921875, "G_mean": 0.3335481498700877, "G_std": 0.15425074861074486, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.284451723098755, "grad_norm": 0.9954696893692017, "tokens_seen": 92250112, "tokens_per_sec": 14936.188480890374, "vram_gb": 6.442267648, "lambda": 0.3049839961652954, "lambda_min": 0.09389043599367142, "lambda_max": 0.6876910328865051, "S_mean": 0.34210205078125, "G_mean": 0.3378124154017617, "G_std": 0.15417791542131454, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.184192657470703, "grad_norm": 1.057458519935608, "tokens_seen": 92413952, "tokens_per_sec": 14936.417608181795, "vram_gb": 6.442267648, "lambda": 0.30507458187639713, "lambda_min": 0.09390851110219955, "lambda_max": 0.6879340410232544, "S_mean": 0.3476765950520833, "G_mean": 0.33554028424744803, "G_std": 0.1537270397723963, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.388285398483276, "grad_norm": 1.050795316696167, "tokens_seen": 92577792, "tokens_per_sec": 14936.89589686903, "vram_gb": 6.442267648, "lambda": 0.3051457942153017, "lambda_min": 0.09390249848365784, "lambda_max": 0.6880704164505005, "S_mean": 0.3483174641927083, "G_mean": 0.3417727934041371, "G_std": 0.15505913156084716, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.201393127441406, "grad_norm": 0.954869270324707, "tokens_seen": 92741632, "tokens_per_sec": 14937.109322969516, "vram_gb": 6.442267648, "lambda": 0.3052071801697214, "lambda_min": 0.0938853919506073, "lambda_max": 0.688233494758606, "S_mean": 0.36676025390625, "G_mean": 0.33881257148459554, "G_std": 0.15609618881717324, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.384293079376221, "grad_norm": 0.9835630655288696, "tokens_seen": 92905472, "tokens_per_sec": 14937.465060379966, "vram_gb": 6.442267648, "lambda": 0.3052649113039176, "lambda_min": 0.09384725242853165, "lambda_max": 0.6885461211204529, "S_mean": 0.3425496419270833, "G_mean": 0.3361024232581258, "G_std": 0.1551661347427095, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.2727744579315186, "grad_norm": 0.9744401574134827, "tokens_seen": 93069312, "tokens_per_sec": 14937.883720008396, "vram_gb": 6.442267648, "lambda": 0.3053384091084202, "lambda_min": 0.09380478411912918, "lambda_max": 0.6888332962989807, "S_mean": 0.343048095703125, "G_mean": 0.3341764595049123, "G_std": 0.15429358428809792, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.315613746643066, "grad_norm": 1.0321991443634033, "tokens_seen": 93233152, "tokens_per_sec": 14938.212742206579, "vram_gb": 6.442267648, "lambda": 0.30539992979417246, "lambda_min": 0.09379898756742477, "lambda_max": 0.6889733672142029, "S_mean": 0.3551025390625, "G_mean": 0.33833564491942525, "G_std": 0.1559890133406346, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.389629602432251, "grad_norm": 0.9738394021987915, "tokens_seen": 93396992, "tokens_per_sec": 14938.571961517726, "vram_gb": 6.442267648, "lambda": 0.305472475476563, "lambda_min": 0.09380114078521729, "lambda_max": 0.6890564560890198, "S_mean": 0.3383585611979167, "G_mean": 0.33463978317255777, "G_std": 0.1568928435056781, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.074204444885254, "grad_norm": 0.9881428480148315, "tokens_seen": 93560832, "tokens_per_sec": 14939.03186912644, "vram_gb": 6.442267648, "lambda": 0.3055427831908067, "lambda_min": 0.09378062933683395, "lambda_max": 0.6892334222793579, "S_mean": 0.3423360188802083, "G_mean": 0.3312970826712747, "G_std": 0.15339895151555538, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.187630653381348, "grad_norm": 0.9870569705963135, "tokens_seen": 93724672, "tokens_per_sec": 14939.47257969245, "vram_gb": 6.442267648, "lambda": 0.3056138039877017, "lambda_min": 0.09377076476812363, "lambda_max": 0.6895496249198914, "S_mean": 0.3487650553385417, "G_mean": 0.3308415880116324, "G_std": 0.15388548575962582, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.3174614906311035, "grad_norm": 1.0243229866027832, "tokens_seen": 93888512, "tokens_per_sec": 14939.827760631611, "vram_gb": 6.442267648, "lambda": 0.3056925230969985, "lambda_min": 0.09375452250242233, "lambda_max": 0.6898652911186218, "S_mean": 0.3371480305989583, "G_mean": 0.3361895938093464, "G_std": 0.15544404931521663, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.3737709522247314, "grad_norm": 0.9579787850379944, "tokens_seen": 94052352, "tokens_per_sec": 14940.200098936426, "vram_gb": 6.442267648, "lambda": 0.3057495007912318, "lambda_min": 0.09374532848596573, "lambda_max": 0.6900471448898315, "S_mean": 0.3426310221354167, "G_mean": 0.3343528201803565, "G_std": 0.15423960828532776, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.504175901412964, "grad_norm": 1.0536632537841797, "tokens_seen": 94216192, "tokens_per_sec": 14940.699443935167, "vram_gb": 6.442267648, "lambda": 0.3058134081463019, "lambda_min": 0.09373607486486435, "lambda_max": 0.6902294158935547, "S_mean": 0.34417724609375, "G_mean": 0.33373715456885594, "G_std": 0.1538211596586431, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.387491691112518, "val_ppl": 80.43840136806239, "lambda": 0.3058166274180015, "lambda_min": 0.09373526275157928, "lambda_max": 0.6902421116828918, "lambda_per_layer": [0.1751909703016281, 0.1423848420381546, 0.14652618765830994, 0.14014197885990143, 0.12089821696281433, 0.12026557326316833, 0.09373526275157928, 0.12760037183761597, 0.13356821238994598, 0.2976582944393158, 0.3630855977535248, 0.5024304389953613, 0.40754032135009766, 0.6902421116828918, 0.5109379887580872, 0.5608271360397339, 0.43170374631881714, 0.4407833516597748, 0.28135207295417786, 0.2850656807422638, 0.46788617968559265, 0.4315091371536255, 0.24764762818813324, 0.22061775624752045], "S_mean": 0.34979248046875, "G_mean": 0.3379689680101971, "G_std": 0.15551061117245504, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.2996134757995605, "grad_norm": 0.9791490435600281, "tokens_seen": 94380032, "tokens_per_sec": 14930.980194892485, "vram_gb": 6.442267648, "lambda": 0.30590026701490086, "lambda_min": 0.09374766051769257, "lambda_max": 0.6904698014259338, "S_mean": 0.3398844401041667, "G_mean": 0.3364600057248026, "G_std": 0.1551150221688052, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.195216178894043, "grad_norm": 1.036594271659851, "tokens_seen": 94543872, "tokens_per_sec": 14931.444465040147, "vram_gb": 6.442267648, "lambda": 0.3059872568895419, "lambda_min": 0.09377072006464005, "lambda_max": 0.6907430291175842, "S_mean": 0.3432515462239583, "G_mean": 0.33031873777508736, "G_std": 0.15311540282952288, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.328946113586426, "grad_norm": 1.051435947418213, "tokens_seen": 94707712, "tokens_per_sec": 14931.768587103217, "vram_gb": 6.442267648, "lambda": 0.30604644026607275, "lambda_min": 0.09377225488424301, "lambda_max": 0.6909395456314087, "S_mean": 0.3369547526041667, "G_mean": 0.3335089859707902, "G_std": 0.15459346348264566, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.360336542129517, "grad_norm": 0.9814910888671875, "tokens_seen": 94871552, "tokens_per_sec": 14932.154990368961, "vram_gb": 6.442267648, "lambda": 0.3060865756124258, "lambda_min": 0.09374108910560608, "lambda_max": 0.6910414099693298, "S_mean": 0.3419901529947917, "G_mean": 0.33480301103554666, "G_std": 0.15344799015050134, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.1657867431640625, "grad_norm": 0.9770437479019165, "tokens_seen": 95035392, "tokens_per_sec": 14932.53826540966, "vram_gb": 6.442267648, "lambda": 0.306142745539546, "lambda_min": 0.09374772012233734, "lambda_max": 0.6912041902542114, "S_mean": 0.34039306640625, "G_mean": 0.33385506831109524, "G_std": 0.15374889170440534, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.424900054931641, "grad_norm": 1.030259132385254, "tokens_seen": 95199232, "tokens_per_sec": 14933.0452414193, "vram_gb": 6.442267648, "lambda": 0.3062308228885134, "lambda_min": 0.09373047202825546, "lambda_max": 0.6913896203041077, "S_mean": 0.3412984212239583, "G_mean": 0.33734663265446824, "G_std": 0.15470807060288885, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.35712742805481, "grad_norm": 1.0151249170303345, "tokens_seen": 95363072, "tokens_per_sec": 14933.612764575913, "vram_gb": 6.442267648, "lambda": 0.30630997226883966, "lambda_min": 0.09377019107341766, "lambda_max": 0.6916354894638062, "S_mean": 0.3464558919270833, "G_mean": 0.33328206464648247, "G_std": 0.15476829182201376, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.3632776737213135, "grad_norm": 1.0088129043579102, "tokens_seen": 95526912, "tokens_per_sec": 14934.073451252752, "vram_gb": 6.442267648, "lambda": 0.3063824021567901, "lambda_min": 0.09377368539571762, "lambda_max": 0.6918641924858093, "S_mean": 0.33770751953125, "G_mean": 0.3354764223719637, "G_std": 0.15387375346229723, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.265809774398804, "grad_norm": 0.9767937064170837, "tokens_seen": 95690752, "tokens_per_sec": 14934.488503222756, "vram_gb": 6.442267648, "lambda": 0.3064426699032386, "lambda_min": 0.09373880922794342, "lambda_max": 0.6919170618057251, "S_mean": 0.3571370442708333, "G_mean": 0.33739031789203483, "G_std": 0.15501480510768792, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.301992893218994, "grad_norm": 0.960076093673706, "tokens_seen": 95854592, "tokens_per_sec": 14934.758910015595, "vram_gb": 6.442267648, "lambda": 0.30652635730803013, "lambda_min": 0.09373873472213745, "lambda_max": 0.6921103596687317, "S_mean": 0.34332275390625, "G_mean": 0.33484403354426223, "G_std": 0.15566136261137822, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.177984952926636, "grad_norm": 0.985136866569519, "tokens_seen": 96018432, "tokens_per_sec": 14935.191097660672, "vram_gb": 6.442267648, "lambda": 0.306582841090858, "lambda_min": 0.09374712407588959, "lambda_max": 0.6923027634620667, "S_mean": 0.34832763671875, "G_mean": 0.33021775384744007, "G_std": 0.1542280881355206, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.383031845092773, "grad_norm": 1.0049360990524292, "tokens_seen": 96182272, "tokens_per_sec": 14935.724063723424, "vram_gb": 6.442267648, "lambda": 0.3066388402755062, "lambda_min": 0.09378242492675781, "lambda_max": 0.6923472881317139, "S_mean": 0.3436787923177083, "G_mean": 0.34156534215435386, "G_std": 0.1556029338389635, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.266032695770264, "grad_norm": 1.00051748752594, "tokens_seen": 96346112, "tokens_per_sec": 14936.124011768197, "vram_gb": 6.442267648, "lambda": 0.3066916608562072, "lambda_min": 0.09381093829870224, "lambda_max": 0.6924671530723572, "S_mean": 0.3409220377604167, "G_mean": 0.3384451263894637, "G_std": 0.15507607938100895, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.263361930847168, "grad_norm": 1.0014923810958862, "tokens_seen": 96509952, "tokens_per_sec": 14936.441457941462, "vram_gb": 6.442267648, "lambda": 0.3067607646808028, "lambda_min": 0.09382528066635132, "lambda_max": 0.6925358772277832, "S_mean": 0.3388264973958333, "G_mean": 0.3284249833474557, "G_std": 0.15373371879104525, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.230196475982666, "grad_norm": 0.9634944200515747, "tokens_seen": 96673792, "tokens_per_sec": 14936.549854864474, "vram_gb": 6.442267648, "lambda": 0.3068497485170762, "lambda_min": 0.09384240955114365, "lambda_max": 0.6927365064620972, "S_mean": 0.336029052734375, "G_mean": 0.33474206971004605, "G_std": 0.15440178747909764, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.313230991363525, "grad_norm": 0.9672502279281616, "tokens_seen": 96837632, "tokens_per_sec": 14936.972175740762, "vram_gb": 6.442267648, "lambda": 0.3069329826782147, "lambda_min": 0.09385986626148224, "lambda_max": 0.6930207014083862, "S_mean": 0.3358154296875, "G_mean": 0.33670608171572286, "G_std": 0.1554788243956864, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.289872646331787, "grad_norm": 1.0091803073883057, "tokens_seen": 97001472, "tokens_per_sec": 14937.456299136149, "vram_gb": 6.442267648, "lambda": 0.30698336971302825, "lambda_min": 0.0938757136464119, "lambda_max": 0.6931686401367188, "S_mean": 0.3412068684895833, "G_mean": 0.33943870834385353, "G_std": 0.15383322746492922, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.299001693725586, "grad_norm": 1.048848032951355, "tokens_seen": 97165312, "tokens_per_sec": 14937.846183492275, "vram_gb": 6.442267648, "lambda": 0.30705410055816174, "lambda_min": 0.09387911111116409, "lambda_max": 0.6934057474136353, "S_mean": 0.3494873046875, "G_mean": 0.3348457107786089, "G_std": 0.15464291158908358, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.257605791091919, "grad_norm": 1.000069499015808, "tokens_seen": 97329152, "tokens_per_sec": 14938.146033433444, "vram_gb": 6.442267648, "lambda": 0.3071059910580516, "lambda_min": 0.09385334700345993, "lambda_max": 0.6935995817184448, "S_mean": 0.3504536946614583, "G_mean": 0.33525876033430296, "G_std": 0.15513021941296756, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.394107818603516, "grad_norm": 1.0914443731307983, "tokens_seen": 97492992, "tokens_per_sec": 14938.53312488796, "vram_gb": 6.442267648, "lambda": 0.3071805763368805, "lambda_min": 0.09386026859283447, "lambda_max": 0.6937470436096191, "S_mean": 0.3410237630208333, "G_mean": 0.33361885766498744, "G_std": 0.15499235743967196, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.247368812561035, "grad_norm": 1.024673581123352, "tokens_seen": 97656832, "tokens_per_sec": 14938.756613025887, "vram_gb": 6.442267648, "lambda": 0.30724938493222, "lambda_min": 0.09386765956878662, "lambda_max": 0.6938807964324951, "S_mean": 0.350616455078125, "G_mean": 0.3293047013382117, "G_std": 0.1534051235842829, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.313060998916626, "grad_norm": 1.0457574129104614, "tokens_seen": 97820672, "tokens_per_sec": 14939.131783163228, "vram_gb": 6.442267648, "lambda": 0.30732027099778253, "lambda_min": 0.09388013184070587, "lambda_max": 0.6941522359848022, "S_mean": 0.3430887858072917, "G_mean": 0.3317298356754084, "G_std": 0.1534626161446795, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.218209266662598, "grad_norm": 1.0314892530441284, "tokens_seen": 97984512, "tokens_per_sec": 14939.570999488526, "vram_gb": 6.442267648, "lambda": 0.30739027882615727, "lambda_min": 0.09388221055269241, "lambda_max": 0.6943408250808716, "S_mean": 0.3507080078125, "G_mean": 0.3333937887412806, "G_std": 0.15241702382142344, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.30188250541687, "grad_norm": 0.9794838428497314, "tokens_seen": 98148352, "tokens_per_sec": 14939.817970119864, "vram_gb": 6.442267648, "lambda": 0.3074745563790202, "lambda_min": 0.09390667080879211, "lambda_max": 0.694563627243042, "S_mean": 0.3485819498697917, "G_mean": 0.3332850156972806, "G_std": 0.15346733775610724, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.382353591918945, "val_ppl": 80.02616085611275, "lambda": 0.3075284864753485, "lambda_min": 0.09392528235912323, "lambda_max": 0.6946620941162109, "lambda_per_layer": [0.17552679777145386, 0.14261183142662048, 0.14674584567546844, 0.14037945866584778, 0.12091401219367981, 0.1200467050075531, 0.09392528235912323, 0.12750451266765594, 0.1335289627313614, 0.2981714904308319, 0.3644765317440033, 0.5049618482589722, 0.4087373614311218, 0.6946620941162109, 0.5144373178482056, 0.5656660795211792, 0.4359172284603119, 0.44458475708961487, 0.2826550304889679, 0.28706851601600647, 0.4713136553764343, 0.43620753288269043, 0.24883079528808594, 0.22181002795696259], "S_mean": 0.3513387044270833, "G_mean": 0.33881601070364314, "G_std": 0.1547946820889289, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 6583.949243545532, "nan_failures": 0} diff --git a/full_s2/config.json b/full_s2/config.json new file mode 100644 index 0000000000000000000000000000000000000000..8dd8250b0e4ee2b23f891b9c169c672f7cadb69b --- /dev/null +++ b/full_s2/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 2, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s2", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 2001343, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/full_s2/final.pt b/full_s2/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..a0de9eabf654617569ef24592026bf6b865398a1 --- /dev/null +++ b/full_s2/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ffbdf35a45d7b455806ed6ef360340c606c1a9c1ca78eff6811b679009e180c0 +size 256003111 diff --git a/full_s2/metrics.jsonl b/full_s2/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..b97f4e19cf203fd8363fd8d48eade465cdb2080f --- /dev/null +++ b/full_s2/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 2, "config": {"variant": "full", "seed": 2, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s2", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 2001343, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.854904174804688, "grad_norm": 9.82266616821289, "tokens_seen": 8192, "tokens_per_sec": 7949.072424096885, "vram_gb": 5.172759552, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.1842219034830729, "G_mean": 0.48413818205396336, "G_std": 0.08791288267821074, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.860017776489258, "grad_norm": 1.9153634309768677, "tokens_seen": 172032, "tokens_per_sec": 16894.5405073786, "vram_gb": 5.684933632, "lambda": 0.12692545354366302, "lambda_min": 0.12689146399497986, "lambda_max": 0.126954585313797, "S_mean": 0.4583333333333333, "G_mean": 0.4522758461534977, "G_std": 0.06555448744135599, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.090084075927734, "grad_norm": 1.7883098125457764, "tokens_seen": 335872, "tokens_per_sec": 17467.168316411342, "vram_gb": 5.684933632, "lambda": 0.1268333693345388, "lambda_min": 0.1267632097005844, "lambda_max": 0.1269422471523285, "S_mean": 0.4685465494791667, "G_mean": 0.43023352945844334, "G_std": 0.07426608144305646, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.17108154296875, "grad_norm": 1.4939039945602417, "tokens_seen": 499712, "tokens_per_sec": 17765.238450684737, "vram_gb": 5.684933632, "lambda": 0.1267441939562559, "lambda_min": 0.12661194801330566, "lambda_max": 0.1269465982913971, "S_mean": 0.4690450032552083, "G_mean": 0.42800211906433105, "G_std": 0.08704299386590719, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.545831680297852, "grad_norm": 0.8365325331687927, "tokens_seen": 663552, "tokens_per_sec": 17918.365781092012, "vram_gb": 5.684933632, "lambda": 0.12663992308080196, "lambda_min": 0.1264103651046753, "lambda_max": 0.12705954909324646, "S_mean": 0.42108154296875, "G_mean": 0.4271373637020588, "G_std": 0.09993291413411498, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.231682777404785, "grad_norm": 0.7165818214416504, "tokens_seen": 827392, "tokens_per_sec": 18035.922363742673, "vram_gb": 5.684933632, "lambda": 0.12654135872920355, "lambda_min": 0.126191645860672, "lambda_max": 0.12709146738052368, "S_mean": 0.2967427571614583, "G_mean": 0.415961633125941, "G_std": 0.11553410657991965, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.137962818145752, "grad_norm": 0.6679629683494568, "tokens_seen": 991232, "tokens_per_sec": 18104.1633279377, "vram_gb": 5.684933632, "lambda": 0.1264414396136999, "lambda_min": 0.12592540681362152, "lambda_max": 0.12729471921920776, "S_mean": 0.3251749674479167, "G_mean": 0.41266874223947525, "G_std": 0.12496475735679269, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 7.025486707687378, "grad_norm": 0.6786813139915466, "tokens_seen": 1155072, "tokens_per_sec": 18150.23010913587, "vram_gb": 5.684933632, "lambda": 0.12638316924373308, "lambda_min": 0.12577863037586212, "lambda_max": 0.12725278735160828, "S_mean": 0.2895914713541667, "G_mean": 0.419362660497427, "G_std": 0.13452056205521026, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.739368438720703, "grad_norm": 0.6209365129470825, "tokens_seen": 1318912, "tokens_per_sec": 18176.72701258103, "vram_gb": 5.684933632, "lambda": 0.12631283203760782, "lambda_min": 0.12559325993061066, "lambda_max": 0.12732844054698944, "S_mean": 0.3003336588541667, "G_mean": 0.40420519933104515, "G_std": 0.1345534281184276, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.847113132476807, "grad_norm": 0.49714717268943787, "tokens_seen": 1482752, "tokens_per_sec": 18191.623246518262, "vram_gb": 5.684933632, "lambda": 0.12634840918083987, "lambda_min": 0.125461608171463, "lambda_max": 0.12740711867809296, "S_mean": 0.28948974609375, "G_mean": 0.40053849294781685, "G_std": 0.14381275760630766, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.6233179569244385, "grad_norm": 0.5631446838378906, "tokens_seen": 1646592, "tokens_per_sec": 18209.08950106837, "vram_gb": 5.684933632, "lambda": 0.12632321938872337, "lambda_min": 0.1253025233745575, "lambda_max": 0.12752945721149445, "S_mean": 0.31695556640625, "G_mean": 0.3915938114126523, "G_std": 0.14701770711690187, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.633415222167969, "grad_norm": 0.5961374640464783, "tokens_seen": 1810432, "tokens_per_sec": 18236.655740354094, "vram_gb": 5.684933632, "lambda": 0.1262961132451892, "lambda_min": 0.12487708777189255, "lambda_max": 0.1275729089975357, "S_mean": 0.2930094401041667, "G_mean": 0.39850909635424614, "G_std": 0.15258629154413939, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.584829807281494, "grad_norm": 0.5155202746391296, "tokens_seen": 1974272, "tokens_per_sec": 18260.56338364688, "vram_gb": 5.684933632, "lambda": 0.12628011809041104, "lambda_min": 0.12485725432634354, "lambda_max": 0.12766033411026, "S_mean": 0.3095499674479167, "G_mean": 0.38567479078968364, "G_std": 0.15267316189905009, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.420532464981079, "grad_norm": 0.49189460277557373, "tokens_seen": 2138112, "tokens_per_sec": 18284.720774665933, "vram_gb": 5.684933632, "lambda": 0.1262688816835483, "lambda_min": 0.12475881725549698, "lambda_max": 0.12779965996742249, "S_mean": 0.2797749837239583, "G_mean": 0.40669455751776695, "G_std": 0.16170571806530157, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.505403518676758, "grad_norm": 0.48894423246383667, "tokens_seen": 2301952, "tokens_per_sec": 18304.643401114015, "vram_gb": 5.684933632, "lambda": 0.12628869184603295, "lambda_min": 0.12482307106256485, "lambda_max": 0.1278594732284546, "S_mean": 0.3190104166666667, "G_mean": 0.4135252746442954, "G_std": 0.17206255129228035, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.642948627471924, "grad_norm": 0.46331071853637695, "tokens_seen": 2465792, "tokens_per_sec": 18320.552946598524, "vram_gb": 5.684933632, "lambda": 0.12627174612134695, "lambda_min": 0.12465717643499374, "lambda_max": 0.12796764075756073, "S_mean": 0.2758992513020833, "G_mean": 0.41215543697277707, "G_std": 0.17860139875362316, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.592201232910156, "grad_norm": 0.48334455490112305, "tokens_seen": 2629632, "tokens_per_sec": 18327.639765386757, "vram_gb": 5.684933632, "lambda": 0.12624872537950674, "lambda_min": 0.12458408623933792, "lambda_max": 0.12809821963310242, "S_mean": 0.3280029296875, "G_mean": 0.4080927607913812, "G_std": 0.1787333389123281, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.454668283462524, "grad_norm": 0.46755385398864746, "tokens_seen": 2793472, "tokens_per_sec": 18348.538417495118, "vram_gb": 5.684933632, "lambda": 0.12630452867597342, "lambda_min": 0.1246858760714531, "lambda_max": 0.1285051703453064, "S_mean": 0.2953898111979167, "G_mean": 0.41941652198632556, "G_std": 0.18747743653754392, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.402219772338867, "grad_norm": 0.5138446688652039, "tokens_seen": 2957312, "tokens_per_sec": 18357.685441732192, "vram_gb": 5.684933632, "lambda": 0.12636436987668276, "lambda_min": 0.1247170940041542, "lambda_max": 0.12882986664772034, "S_mean": 0.3053995768229167, "G_mean": 0.41444091126322746, "G_std": 0.19427094608545303, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.269313812255859, "grad_norm": 0.4890187680721283, "tokens_seen": 3121152, "tokens_per_sec": 18367.18369142019, "vram_gb": 5.684933632, "lambda": 0.12644809919099012, "lambda_min": 0.12503023445606232, "lambda_max": 0.12909150123596191, "S_mean": 0.30718994140625, "G_mean": 0.44231489673256874, "G_std": 0.1996519242723783, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.463404893875122, "grad_norm": 0.494167298078537, "tokens_seen": 3284992, "tokens_per_sec": 18360.49361107695, "vram_gb": 5.684933632, "lambda": 0.1264700721949339, "lambda_min": 0.1250123530626297, "lambda_max": 0.1294553130865097, "S_mean": 0.3629557291666667, "G_mean": 0.4171920989950498, "G_std": 0.1959075198198358, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.2934441566467285, "grad_norm": 0.4740597903728485, "tokens_seen": 3448832, "tokens_per_sec": 18352.730040609593, "vram_gb": 5.684933632, "lambda": 0.12653880566358566, "lambda_min": 0.12502574920654297, "lambda_max": 0.12993602454662323, "S_mean": 0.2877705891927083, "G_mean": 0.44463082775473595, "G_std": 0.20718572226663431, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.299933910369873, "grad_norm": 0.48090052604675293, "tokens_seen": 3612672, "tokens_per_sec": 18368.834637432177, "vram_gb": 5.684933632, "lambda": 0.12665680578599373, "lambda_min": 0.12495414167642593, "lambda_max": 0.1304105967283249, "S_mean": 0.3266805013020833, "G_mean": 0.4387928210198879, "G_std": 0.20816946712632975, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.242947578430176, "grad_norm": 0.4725148379802704, "tokens_seen": 3776512, "tokens_per_sec": 18382.98624703906, "vram_gb": 5.684933632, "lambda": 0.1267462329318126, "lambda_min": 0.1248808205127716, "lambda_max": 0.13096854090690613, "S_mean": 0.3063863118489583, "G_mean": 0.442952507485946, "G_std": 0.21057702787220478, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.483664512634277, "grad_norm": 0.5051085948944092, "tokens_seen": 3940352, "tokens_per_sec": 18387.472110787272, "vram_gb": 5.684933632, "lambda": 0.12689094307521978, "lambda_min": 0.12490679323673248, "lambda_max": 0.13144122064113617, "S_mean": 0.34307861328125, "G_mean": 0.4348972775042057, "G_std": 0.20737076488633951, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.3186023235321045, "grad_norm": 0.4745250940322876, "tokens_seen": 4104192, "tokens_per_sec": 18393.748243192007, "vram_gb": 5.684933632, "lambda": 0.12710919479529062, "lambda_min": 0.12506213784217834, "lambda_max": 0.13164539635181427, "S_mean": 0.3445841471354167, "G_mean": 0.4438770276804765, "G_std": 0.2142142647256454, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.343235492706299, "grad_norm": 0.4708714783191681, "tokens_seen": 4268032, "tokens_per_sec": 18389.66198838239, "vram_gb": 5.684933632, "lambda": 0.1273205168545246, "lambda_min": 0.12522432208061218, "lambda_max": 0.13191327452659607, "S_mean": 0.3226725260416667, "G_mean": 0.46138598769903183, "G_std": 0.21915021042029062, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.088433027267456, "grad_norm": 0.45517921447753906, "tokens_seen": 4431872, "tokens_per_sec": 18396.15474045094, "vram_gb": 5.684933632, "lambda": 0.12752549350261688, "lambda_min": 0.12529242038726807, "lambda_max": 0.13255539536476135, "S_mean": 0.3934326171875, "G_mean": 0.49114174271623295, "G_std": 0.21848365850746632, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.052109718322754, "grad_norm": 0.49719077348709106, "tokens_seen": 4595712, "tokens_per_sec": 18401.776764750808, "vram_gb": 5.684933632, "lambda": 0.12772539630532265, "lambda_min": 0.12532100081443787, "lambda_max": 0.13315477967262268, "S_mean": 0.3782145182291667, "G_mean": 0.4747086713711421, "G_std": 0.21810557538022599, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 5.916567325592041, "grad_norm": 0.5081382989883423, "tokens_seen": 4759552, "tokens_per_sec": 18406.628963825115, "vram_gb": 5.684933632, "lambda": 0.1279321194936832, "lambda_min": 0.12536142766475677, "lambda_max": 0.1336439996957779, "S_mean": 0.3675130208333333, "G_mean": 0.48434366037448245, "G_std": 0.2160866033906738, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.061230659484863, "grad_norm": 0.5097229480743408, "tokens_seen": 4923392, "tokens_per_sec": 18416.46915955604, "vram_gb": 5.684933632, "lambda": 0.1281968541443348, "lambda_min": 0.12569159269332886, "lambda_max": 0.1339481770992279, "S_mean": 0.3642985026041667, "G_mean": 0.5085552694896857, "G_std": 0.2172487893452247, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.221756458282471, "grad_norm": 0.47119203209877014, "tokens_seen": 5087232, "tokens_per_sec": 18420.241212633417, "vram_gb": 5.684933632, "lambda": 0.12840498921771845, "lambda_min": 0.1258346289396286, "lambda_max": 0.13396382331848145, "S_mean": 0.3610331217447917, "G_mean": 0.5146757401525974, "G_std": 0.21680130871633688, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.131063938140869, "grad_norm": 0.4646916091442108, "tokens_seen": 5251072, "tokens_per_sec": 18419.21055027219, "vram_gb": 5.684933632, "lambda": 0.12866881676018238, "lambda_min": 0.1257028579711914, "lambda_max": 0.13442572951316833, "S_mean": 0.32452392578125, "G_mean": 0.4856528143088023, "G_std": 0.21459721649686495, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.116045713424683, "grad_norm": 0.49498119950294495, "tokens_seen": 5414912, "tokens_per_sec": 18424.014852899916, "vram_gb": 5.684933632, "lambda": 0.12898446805775166, "lambda_min": 0.12583991885185242, "lambda_max": 0.13473033905029297, "S_mean": 0.3666585286458333, "G_mean": 0.525129958987236, "G_std": 0.20943152904510498, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.061836004257202, "grad_norm": 0.4292519986629486, "tokens_seen": 5578752, "tokens_per_sec": 18422.883434253883, "vram_gb": 5.684933632, "lambda": 0.12929840572178364, "lambda_min": 0.12595854699611664, "lambda_max": 0.13572783768177032, "S_mean": 0.35955810546875, "G_mean": 0.5323292103906473, "G_std": 0.2030294177432855, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.965781927108765, "grad_norm": 0.47674238681793213, "tokens_seen": 5742592, "tokens_per_sec": 18424.236252578212, "vram_gb": 5.684933632, "lambda": 0.12961199941734472, "lambda_min": 0.1259245127439499, "lambda_max": 0.13713142275810242, "S_mean": 0.3599141438802083, "G_mean": 0.5174335166811943, "G_std": 0.20765727882583937, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.033849239349365, "grad_norm": 0.5864001512527466, "tokens_seen": 5906432, "tokens_per_sec": 18427.689522087265, "vram_gb": 5.684933632, "lambda": 0.13001847142974535, "lambda_min": 0.1261983960866928, "lambda_max": 0.13855621218681335, "S_mean": 0.3694254557291667, "G_mean": 0.5206350771089395, "G_std": 0.2100229517867168, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.010033130645752, "grad_norm": 0.5099093317985535, "tokens_seen": 6070272, "tokens_per_sec": 18423.76092717239, "vram_gb": 5.684933632, "lambda": 0.1303439848124981, "lambda_min": 0.1261942982673645, "lambda_max": 0.13996188342571259, "S_mean": 0.3293965657552083, "G_mean": 0.5434011556208134, "G_std": 0.20747819604972997, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.020395517349243, "grad_norm": 0.4883263111114502, "tokens_seen": 6234112, "tokens_per_sec": 18422.392255088966, "vram_gb": 5.684933632, "lambda": 0.13071177092691263, "lambda_min": 0.1261218637228012, "lambda_max": 0.14124473929405212, "S_mean": 0.3828531901041667, "G_mean": 0.5497094007829825, "G_std": 0.1960687261695663, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.788971900939941, "grad_norm": 0.45284727215766907, "tokens_seen": 6397952, "tokens_per_sec": 18426.17919881845, "vram_gb": 5.684933632, "lambda": 0.1311031791071097, "lambda_min": 0.12610335648059845, "lambda_max": 0.1425572782754898, "S_mean": 0.4096272786458333, "G_mean": 0.5489365843435129, "G_std": 0.19368440844118595, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.8960301876068115, "grad_norm": 0.5183782577514648, "tokens_seen": 6561792, "tokens_per_sec": 18429.165629971278, "vram_gb": 5.684933632, "lambda": 0.13150799833238125, "lambda_min": 0.1261657476425171, "lambda_max": 0.14385713636875153, "S_mean": 0.4027506510416667, "G_mean": 0.5431501778463522, "G_std": 0.19599455470840135, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.840850830078125, "grad_norm": 0.46044978499412537, "tokens_seen": 6725632, "tokens_per_sec": 18433.99195624774, "vram_gb": 5.684933632, "lambda": 0.13191263501842818, "lambda_min": 0.1260666847229004, "lambda_max": 0.1451754868030548, "S_mean": 0.3944905598958333, "G_mean": 0.5251021397610506, "G_std": 0.19584180818249783, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.915983200073242, "grad_norm": 0.5112923383712769, "tokens_seen": 6889472, "tokens_per_sec": 18433.461019189024, "vram_gb": 5.684933632, "lambda": 0.13229165226221085, "lambda_min": 0.12595665454864502, "lambda_max": 0.14653180539608002, "S_mean": 0.4242757161458333, "G_mean": 0.5420791395008564, "G_std": 0.19648190028965473, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.885740756988525, "grad_norm": 0.4783810079097748, "tokens_seen": 7053312, "tokens_per_sec": 18436.189621754285, "vram_gb": 5.684933632, "lambda": 0.13265409568945566, "lambda_min": 0.12562115490436554, "lambda_max": 0.1476440727710724, "S_mean": 0.4002482096354167, "G_mean": 0.5364958656330904, "G_std": 0.19828016279886165, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.982444763183594, "grad_norm": 0.5566452145576477, "tokens_seen": 7217152, "tokens_per_sec": 18439.434367944927, "vram_gb": 5.684933632, "lambda": 0.13300994535287222, "lambda_min": 0.12538932263851166, "lambda_max": 0.14886674284934998, "S_mean": 0.4126993815104167, "G_mean": 0.5545470807701349, "G_std": 0.19139659932504097, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.76087212562561, "grad_norm": 0.4781349003314972, "tokens_seen": 7380992, "tokens_per_sec": 18442.184091033432, "vram_gb": 5.684933632, "lambda": 0.13343547781308493, "lambda_min": 0.12516415119171143, "lambda_max": 0.15006490051746368, "S_mean": 0.4122517903645833, "G_mean": 0.5333253517746925, "G_std": 0.19505445938557386, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.920749664306641, "grad_norm": 0.4829954504966736, "tokens_seen": 7544832, "tokens_per_sec": 18444.697922012547, "vram_gb": 5.684933632, "lambda": 0.13383407766620317, "lambda_min": 0.12458336353302002, "lambda_max": 0.15135370194911957, "S_mean": 0.4136962890625, "G_mean": 0.5261904081950585, "G_std": 0.1933561454837521, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.839652061462402, "grad_norm": 0.491666316986084, "tokens_seen": 7708672, "tokens_per_sec": 18449.04641302783, "vram_gb": 5.684933632, "lambda": 0.1342735110471646, "lambda_min": 0.12432686984539032, "lambda_max": 0.15246737003326416, "S_mean": 0.42437744140625, "G_mean": 0.5237048634638389, "G_std": 0.186365798736612, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.728305816650391, "grad_norm": 0.5277457237243652, "tokens_seen": 7872512, "tokens_per_sec": 18453.06423511686, "vram_gb": 5.684933632, "lambda": 0.13469342483828464, "lambda_min": 0.12419193238019943, "lambda_max": 0.15358558297157288, "S_mean": 0.3939208984375, "G_mean": 0.5491224192082882, "G_std": 0.19177487616737685, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.81601095199585, "grad_norm": 0.48973020911216736, "tokens_seen": 8036352, "tokens_per_sec": 18456.599041900507, "vram_gb": 5.684933632, "lambda": 0.13506792889287075, "lambda_min": 0.12398787587881088, "lambda_max": 0.1547316610813141, "S_mean": 0.40618896484375, "G_mean": 0.5385550862799088, "G_std": 0.18924829053382078, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.730866432189941, "grad_norm": 0.5203363299369812, "tokens_seen": 8200192, "tokens_per_sec": 18458.10430267284, "vram_gb": 5.684933632, "lambda": 0.13545610196888447, "lambda_min": 0.12355636060237885, "lambda_max": 0.15603692829608917, "S_mean": 0.4051310221354167, "G_mean": 0.5289684981107712, "G_std": 0.19187330019970736, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.725707912445069, "val_ppl": 306.65026996652256, "lambda": 0.13547919349124035, "lambda_min": 0.1235489472746849, "lambda_max": 0.1561095267534256, "lambda_per_layer": [0.13851191103458405, 0.13442787528038025, 0.13202355802059174, 0.12814563512802124, 0.12685872614383698, 0.12681907415390015, 0.1235489472746849, 0.12835600972175598, 0.12593476474285126, 0.12860094010829926, 0.1561095267534256, 0.1534213125705719, 0.1532880663871765, 0.14359629154205322, 0.14284656941890717, 0.13532066345214844, 0.14993347227573395, 0.13122951984405518, 0.13407279551029205, 0.1296561062335968, 0.13245825469493866, 0.13572193682193756, 0.13030889630317688, 0.1303097903728485], "S_mean": 0.4349772135416667, "G_mean": 0.5236137757698694, "G_std": 0.19444023010631403, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.677178859710693, "grad_norm": 0.5162785053253174, "tokens_seen": 8364032, "tokens_per_sec": 18392.88503196964, "vram_gb": 5.684933632, "lambda": 0.13589525253822407, "lambda_min": 0.12327861040830612, "lambda_max": 0.1574268639087677, "S_mean": 0.3877766927083333, "G_mean": 0.5185140880445639, "G_std": 0.19373635978748402, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.82243275642395, "grad_norm": 0.5153461694717407, "tokens_seen": 8527872, "tokens_per_sec": 18389.135350668766, "vram_gb": 5.684933632, "lambda": 0.13630167953670025, "lambda_min": 0.12281054258346558, "lambda_max": 0.15854254364967346, "S_mean": 0.4241536458333333, "G_mean": 0.5284766412029663, "G_std": 0.18693189757565656, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.712300777435303, "grad_norm": 0.8453338146209717, "tokens_seen": 8691712, "tokens_per_sec": 18390.899888175532, "vram_gb": 5.684933632, "lambda": 0.13668747867147127, "lambda_min": 0.12246647477149963, "lambda_max": 0.1598307490348816, "S_mean": 0.4436442057291667, "G_mean": 0.5174616674582163, "G_std": 0.19367211125791073, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.774377107620239, "grad_norm": 0.5237844586372375, "tokens_seen": 8855552, "tokens_per_sec": 18391.555592049946, "vram_gb": 5.684933632, "lambda": 0.1370680301139752, "lambda_min": 0.1219986230134964, "lambda_max": 0.16123807430267334, "S_mean": 0.4228922526041667, "G_mean": 0.5161223740627369, "G_std": 0.18476252568264803, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.5989460945129395, "grad_norm": 0.5238717198371887, "tokens_seen": 9019392, "tokens_per_sec": 18393.832598443663, "vram_gb": 5.684933632, "lambda": 0.137446830359598, "lambda_min": 0.12159185856580734, "lambda_max": 0.16250982880592346, "S_mean": 0.4029744466145833, "G_mean": 0.5228482174376646, "G_std": 0.18357865822811922, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.703449010848999, "grad_norm": 0.5702036023139954, "tokens_seen": 9183232, "tokens_per_sec": 18397.62237443185, "vram_gb": 5.684933632, "lambda": 0.13789547079553208, "lambda_min": 0.1214069053530693, "lambda_max": 0.16379624605178833, "S_mean": 0.403076171875, "G_mean": 0.507317022110025, "G_std": 0.18885252376397452, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.742313385009766, "grad_norm": 0.5662906169891357, "tokens_seen": 9347072, "tokens_per_sec": 18397.98134880277, "vram_gb": 5.684933632, "lambda": 0.13832512373725572, "lambda_min": 0.12091876566410065, "lambda_max": 0.16496098041534424, "S_mean": 0.42840576171875, "G_mean": 0.5107714664191008, "G_std": 0.1918076705187559, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.659882068634033, "grad_norm": 0.5179038047790527, "tokens_seen": 9510912, "tokens_per_sec": 18398.94500855986, "vram_gb": 5.684933632, "lambda": 0.1387224905192852, "lambda_min": 0.1207188218832016, "lambda_max": 0.1658809930086136, "S_mean": 0.3973388671875, "G_mean": 0.5247598265608152, "G_std": 0.18918837793171406, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.503299951553345, "grad_norm": 0.514677107334137, "tokens_seen": 9674752, "tokens_per_sec": 18398.422045987463, "vram_gb": 5.684933632, "lambda": 0.13918971673895916, "lambda_min": 0.12057160586118698, "lambda_max": 0.16686655580997467, "S_mean": 0.37884521484375, "G_mean": 0.5210008726765712, "G_std": 0.18953893892467022, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.546677589416504, "grad_norm": 0.47911131381988525, "tokens_seen": 9838592, "tokens_per_sec": 18397.85709436976, "vram_gb": 5.684933632, "lambda": 0.13964535668492317, "lambda_min": 0.12050957977771759, "lambda_max": 0.16771483421325684, "S_mean": 0.3899943033854167, "G_mean": 0.500509966785709, "G_std": 0.18709403897325197, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.596323728561401, "grad_norm": 0.48529675602912903, "tokens_seen": 10002432, "tokens_per_sec": 18399.988678292633, "vram_gb": 5.684933632, "lambda": 0.14003067122151455, "lambda_min": 0.12019788473844528, "lambda_max": 0.16883429884910583, "S_mean": 0.39691162109375, "G_mean": 0.5094861071556807, "G_std": 0.1885319355254372, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.5314719676971436, "grad_norm": 0.5206640958786011, "tokens_seen": 10166272, "tokens_per_sec": 18399.121301581526, "vram_gb": 5.684933632, "lambda": 0.14037068064014116, "lambda_min": 0.11984908580780029, "lambda_max": 0.16982397437095642, "S_mean": 0.3963216145833333, "G_mean": 0.49680651413897675, "G_std": 0.18824219548453888, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.480214357376099, "grad_norm": 0.5552561283111572, "tokens_seen": 10330112, "tokens_per_sec": 18402.219378781156, "vram_gb": 5.684933632, "lambda": 0.14069891100128493, "lambda_min": 0.11957522481679916, "lambda_max": 0.17081256210803986, "S_mean": 0.39483642578125, "G_mean": 0.49864096318682033, "G_std": 0.18724037831028303, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.469577312469482, "grad_norm": 0.5388451218605042, "tokens_seen": 10493952, "tokens_per_sec": 18402.240786419185, "vram_gb": 5.684933632, "lambda": 0.14109345184018215, "lambda_min": 0.11915387958288193, "lambda_max": 0.17186595499515533, "S_mean": 0.4150797526041667, "G_mean": 0.49228996845583123, "G_std": 0.19201775702337423, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.597417831420898, "grad_norm": 0.5082797408103943, "tokens_seen": 10657792, "tokens_per_sec": 18403.208803506303, "vram_gb": 5.684933632, "lambda": 0.14156868805487952, "lambda_min": 0.11887900531291962, "lambda_max": 0.1730031818151474, "S_mean": 0.4161173502604167, "G_mean": 0.4901578289767106, "G_std": 0.19125629992534718, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.4459614753723145, "grad_norm": 0.5287191867828369, "tokens_seen": 10821632, "tokens_per_sec": 18405.148257042092, "vram_gb": 5.684933632, "lambda": 0.14196639135479927, "lambda_min": 0.11866430938243866, "lambda_max": 0.173911914229393, "S_mean": 0.4045206705729167, "G_mean": 0.4994811595728, "G_std": 0.18612833227962255, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.660367965698242, "grad_norm": 0.5884348750114441, "tokens_seen": 10985472, "tokens_per_sec": 18406.729118380932, "vram_gb": 5.684933632, "lambda": 0.1423983027537664, "lambda_min": 0.11832369118928909, "lambda_max": 0.17524845898151398, "S_mean": 0.3874918619791667, "G_mean": 0.48677730311950046, "G_std": 0.18520283419638872, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.46584677696228, "grad_norm": 0.5665557980537415, "tokens_seen": 11149312, "tokens_per_sec": 18409.073271040234, "vram_gb": 5.684933632, "lambda": 0.14282935267935196, "lambda_min": 0.11798200011253357, "lambda_max": 0.17674165964126587, "S_mean": 0.3905232747395833, "G_mean": 0.4950997866690159, "G_std": 0.18537519096086422, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.4969587326049805, "grad_norm": 0.6155384182929993, "tokens_seen": 11313152, "tokens_per_sec": 18411.407955509425, "vram_gb": 5.684933632, "lambda": 0.14321468956768513, "lambda_min": 0.11775818467140198, "lambda_max": 0.17786802351474762, "S_mean": 0.3951416015625, "G_mean": 0.49573741666972637, "G_std": 0.1872368191058437, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.389250040054321, "grad_norm": 0.5629193186759949, "tokens_seen": 11476992, "tokens_per_sec": 18413.398522682026, "vram_gb": 5.684933632, "lambda": 0.14363887626677752, "lambda_min": 0.11724447458982468, "lambda_max": 0.1790458858013153, "S_mean": 0.4007161458333333, "G_mean": 0.49526443021992844, "G_std": 0.18716347745309272, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.451704978942871, "grad_norm": 0.5375266671180725, "tokens_seen": 11640832, "tokens_per_sec": 18415.042310840814, "vram_gb": 5.684933632, "lambda": 0.1440575054536263, "lambda_min": 0.11677185446023941, "lambda_max": 0.18042585253715515, "S_mean": 0.3995768229166667, "G_mean": 0.4655254849543174, "G_std": 0.18372367322444916, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.5097336769104, "grad_norm": 0.5438635945320129, "tokens_seen": 11804672, "tokens_per_sec": 18416.45713879121, "vram_gb": 5.684933632, "lambda": 0.1445297353590528, "lambda_min": 0.11643416434526443, "lambda_max": 0.18165495991706848, "S_mean": 0.36932373046875, "G_mean": 0.47922661527991295, "G_std": 0.18393157298366228, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.420355319976807, "grad_norm": 0.5694162845611572, "tokens_seen": 11968512, "tokens_per_sec": 18419.67043509997, "vram_gb": 5.684933632, "lambda": 0.14494937906662622, "lambda_min": 0.11619266122579575, "lambda_max": 0.18284033238887787, "S_mean": 0.3683878580729167, "G_mean": 0.47419710208972293, "G_std": 0.18711551185697317, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.364103078842163, "grad_norm": 0.5341063141822815, "tokens_seen": 12132352, "tokens_per_sec": 18422.798140730476, "vram_gb": 5.684933632, "lambda": 0.1453350962450107, "lambda_min": 0.11608871817588806, "lambda_max": 0.18394120037555695, "S_mean": 0.3715413411458333, "G_mean": 0.47464812360703945, "G_std": 0.18236775929108262, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.616244792938232, "grad_norm": 0.6011621356010437, "tokens_seen": 12296192, "tokens_per_sec": 18425.58790215845, "vram_gb": 5.684933632, "lambda": 0.14579517673701048, "lambda_min": 0.11587008088827133, "lambda_max": 0.18528029322624207, "S_mean": 0.3860880533854167, "G_mean": 0.47905650983254117, "G_std": 0.17920459325735769, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.395395994186401, "grad_norm": 0.5605080127716064, "tokens_seen": 12460032, "tokens_per_sec": 18427.714859310927, "vram_gb": 5.684933632, "lambda": 0.1461938265711069, "lambda_min": 0.11548486351966858, "lambda_max": 0.18660971522331238, "S_mean": 0.39349365234375, "G_mean": 0.46732063715656597, "G_std": 0.18190580032144985, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.396644592285156, "grad_norm": 0.5321729183197021, "tokens_seen": 12623872, "tokens_per_sec": 18431.15410652809, "vram_gb": 5.684933632, "lambda": 0.14668183152874312, "lambda_min": 0.11513461172580719, "lambda_max": 0.18795493245124817, "S_mean": 0.36529541015625, "G_mean": 0.4651111603404085, "G_std": 0.18213641550391912, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.6022210121154785, "grad_norm": 0.5669626593589783, "tokens_seen": 12787712, "tokens_per_sec": 18432.704336410057, "vram_gb": 5.684933632, "lambda": 0.14709010782341161, "lambda_min": 0.11491045355796814, "lambda_max": 0.18926621973514557, "S_mean": 0.3819376627604167, "G_mean": 0.47138171053181094, "G_std": 0.18132305533314744, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.493300676345825, "grad_norm": 0.5549096465110779, "tokens_seen": 12951552, "tokens_per_sec": 18429.208094437745, "vram_gb": 5.684933632, "lambda": 0.1475543569152554, "lambda_min": 0.11487597972154617, "lambda_max": 0.19064316153526306, "S_mean": 0.3899332682291667, "G_mean": 0.4618686428293586, "G_std": 0.1782499199422697, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.40377140045166, "grad_norm": 0.5691052079200745, "tokens_seen": 13115392, "tokens_per_sec": 18430.944911075323, "vram_gb": 5.684933632, "lambda": 0.14800871443003416, "lambda_min": 0.11472859233617783, "lambda_max": 0.19203226268291473, "S_mean": 0.39532470703125, "G_mean": 0.46421256692459184, "G_std": 0.18395018205046654, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.378726005554199, "grad_norm": 0.5555028319358826, "tokens_seen": 13279232, "tokens_per_sec": 18430.240482178415, "vram_gb": 5.684933632, "lambda": 0.14845969714224339, "lambda_min": 0.11452744156122208, "lambda_max": 0.1935141384601593, "S_mean": 0.377197265625, "G_mean": 0.4632339822128415, "G_std": 0.18078137406458458, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.440428972244263, "grad_norm": 0.5774223804473877, "tokens_seen": 13443072, "tokens_per_sec": 18431.424015059223, "vram_gb": 5.684933632, "lambda": 0.14885643031448126, "lambda_min": 0.1143089160323143, "lambda_max": 0.1951252967119217, "S_mean": 0.3784586588541667, "G_mean": 0.47566917911171913, "G_std": 0.18610904676218828, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.168713808059692, "grad_norm": 0.5418930053710938, "tokens_seen": 13606912, "tokens_per_sec": 18431.330819335788, "vram_gb": 5.684933632, "lambda": 0.14929607852051655, "lambda_min": 0.11375034600496292, "lambda_max": 0.1962881088256836, "S_mean": 0.3886515299479167, "G_mean": 0.44603611901402473, "G_std": 0.17987271212041378, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.439802885055542, "grad_norm": 0.5706246495246887, "tokens_seen": 13770752, "tokens_per_sec": 18430.597082102708, "vram_gb": 5.684933632, "lambda": 0.14969619922339916, "lambda_min": 0.1132986769080162, "lambda_max": 0.19728821516036987, "S_mean": 0.37603759765625, "G_mean": 0.4659826283653577, "G_std": 0.18272213249777755, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.515917062759399, "grad_norm": 0.6044225692749023, "tokens_seen": 13934592, "tokens_per_sec": 18431.78450191499, "vram_gb": 5.684933632, "lambda": 0.15010455095519623, "lambda_min": 0.11287111043930054, "lambda_max": 0.19849549233913422, "S_mean": 0.37725830078125, "G_mean": 0.47073886419336003, "G_std": 0.18315821513533592, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.298243045806885, "grad_norm": 0.5533999800682068, "tokens_seen": 14098432, "tokens_per_sec": 18431.350136599667, "vram_gb": 5.684933632, "lambda": 0.15055705265452465, "lambda_min": 0.11273472756147385, "lambda_max": 0.1997445523738861, "S_mean": 0.3757120768229167, "G_mean": 0.47026906193544465, "G_std": 0.18316978107516965, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.369457244873047, "grad_norm": 0.5788376331329346, "tokens_seen": 14262272, "tokens_per_sec": 18433.566281802807, "vram_gb": 5.684933632, "lambda": 0.15100570799161991, "lambda_min": 0.11286360025405884, "lambda_max": 0.20084868371486664, "S_mean": 0.39227294921875, "G_mean": 0.4625313753883044, "G_std": 0.18251570981616774, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.24001932144165, "grad_norm": 0.5337008833885193, "tokens_seen": 14426112, "tokens_per_sec": 18435.136913348924, "vram_gb": 5.684933632, "lambda": 0.15146891369173923, "lambda_min": 0.11271945387125015, "lambda_max": 0.20202580094337463, "S_mean": 0.382080078125, "G_mean": 0.4495198515554269, "G_std": 0.18378950003534555, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.272395133972168, "grad_norm": 0.5673756003379822, "tokens_seen": 14589952, "tokens_per_sec": 18435.98211420916, "vram_gb": 5.684933632, "lambda": 0.1519279001901547, "lambda_min": 0.11282923072576523, "lambda_max": 0.20349621772766113, "S_mean": 0.3642374674479167, "G_mean": 0.45763474112997454, "G_std": 0.18474024829144278, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.43968653678894, "grad_norm": 0.5772559642791748, "tokens_seen": 14753792, "tokens_per_sec": 18438.07654535139, "vram_gb": 5.684933632, "lambda": 0.15239717345684767, "lambda_min": 0.11292236298322678, "lambda_max": 0.20489472150802612, "S_mean": 0.38623046875, "G_mean": 0.45396521128714085, "G_std": 0.1824041958898306, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.489909410476685, "grad_norm": 0.5751405358314514, "tokens_seen": 14917632, "tokens_per_sec": 18438.233273817586, "vram_gb": 5.684933632, "lambda": 0.15286842888842025, "lambda_min": 0.11292733252048492, "lambda_max": 0.20615778863430023, "S_mean": 0.3860270182291667, "G_mean": 0.4512081081047654, "G_std": 0.1794468774460256, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.425823926925659, "grad_norm": 0.5811431407928467, "tokens_seen": 15081472, "tokens_per_sec": 18440.479015375713, "vram_gb": 5.684933632, "lambda": 0.15337270312011242, "lambda_min": 0.11291643232107162, "lambda_max": 0.20773553848266602, "S_mean": 0.3660888671875, "G_mean": 0.4454848974322279, "G_std": 0.18137816401819387, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.430876970291138, "grad_norm": 0.5940179824829102, "tokens_seen": 15245312, "tokens_per_sec": 18442.421498938438, "vram_gb": 5.684933632, "lambda": 0.15397522784769535, "lambda_min": 0.11303000152111053, "lambda_max": 0.2091735154390335, "S_mean": 0.3658447265625, "G_mean": 0.4515508736173312, "G_std": 0.17886255277941623, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.176948547363281, "grad_norm": 0.5740511417388916, "tokens_seen": 15409152, "tokens_per_sec": 18442.366328510754, "vram_gb": 5.684933632, "lambda": 0.15446000453084707, "lambda_min": 0.11265753954648972, "lambda_max": 0.21026429533958435, "S_mean": 0.3436279296875, "G_mean": 0.45178980721781653, "G_std": 0.1813681578884522, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.232155084609985, "grad_norm": 0.5892778038978577, "tokens_seen": 15572992, "tokens_per_sec": 18444.094851965376, "vram_gb": 5.684933632, "lambda": 0.15490084451933703, "lambda_min": 0.11259496212005615, "lambda_max": 0.21144254505634308, "S_mean": 0.3763020833333333, "G_mean": 0.4526829474295179, "G_std": 0.1787535441108048, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.430644750595093, "grad_norm": 0.5954188704490662, "tokens_seen": 15736832, "tokens_per_sec": 18445.654166627293, "vram_gb": 5.684933632, "lambda": 0.15538312184313932, "lambda_min": 0.11247498542070389, "lambda_max": 0.2124594748020172, "S_mean": 0.3651326497395833, "G_mean": 0.44620835439612466, "G_std": 0.17827944923192263, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.5489418506622314, "grad_norm": 0.6721384525299072, "tokens_seen": 15900672, "tokens_per_sec": 18446.650537487054, "vram_gb": 5.684933632, "lambda": 0.15589597324530283, "lambda_min": 0.1122267097234726, "lambda_max": 0.21394827961921692, "S_mean": 0.3935139973958333, "G_mean": 0.4416217254474759, "G_std": 0.18176984243715802, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.173271894454956, "grad_norm": 0.5848382115364075, "tokens_seen": 16064512, "tokens_per_sec": 18448.582719204394, "vram_gb": 5.684933632, "lambda": 0.15629517752677202, "lambda_min": 0.11188039183616638, "lambda_max": 0.21510803699493408, "S_mean": 0.3615519205729167, "G_mean": 0.4462760289510091, "G_std": 0.17934118040526906, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.156810998916626, "grad_norm": 0.6036354303359985, "tokens_seen": 16228352, "tokens_per_sec": 18449.789669625814, "vram_gb": 5.684933632, "lambda": 0.1567376066620151, "lambda_min": 0.1114204004406929, "lambda_max": 0.21618080139160156, "S_mean": 0.3819376627604167, "G_mean": 0.43893749949832755, "G_std": 0.1819151327945292, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.109173059463501, "grad_norm": 0.5832589268684387, "tokens_seen": 16392192, "tokens_per_sec": 18450.967056027777, "vram_gb": 5.684933632, "lambda": 0.15720953016231456, "lambda_min": 0.11110986769199371, "lambda_max": 0.2174759805202484, "S_mean": 0.393310546875, "G_mean": 0.4413282200694084, "G_std": 0.17899861450617513, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.2818265676498415, "val_ppl": 196.7288859923149, "lambda": 0.15723472647368908, "lambda_min": 0.11110488325357437, "lambda_max": 0.21751245856285095, "lambda_per_layer": [0.14859208464622498, 0.1387222856283188, 0.13351620733737946, 0.12883906066417694, 0.12979212403297424, 0.12633764743804932, 0.11110488325357437, 0.11282972991466522, 0.12392591685056686, 0.1322687268257141, 0.20096993446350098, 0.20889510214328766, 0.21751245856285095, 0.1890082210302353, 0.20590002834796906, 0.1578947901725769, 0.21344546973705292, 0.14888760447502136, 0.1870819330215454, 0.15442276000976562, 0.1539630889892578, 0.16354331374168396, 0.14680573344230652, 0.13937433063983917], "S_mean": 0.3939005533854167, "G_mean": 0.43081952538341284, "G_std": 0.17498544386277595, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.194804430007935, "grad_norm": 0.6796712279319763, "tokens_seen": 16556032, "tokens_per_sec": 18417.23244000214, "vram_gb": 5.684933632, "lambda": 0.1577280911927422, "lambda_min": 0.11125942319631577, "lambda_max": 0.21859797835350037, "S_mean": 0.3646036783854167, "G_mean": 0.4364671589185794, "G_std": 0.18238056482126316, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.29899263381958, "grad_norm": 0.56915283203125, "tokens_seen": 16719872, "tokens_per_sec": 18419.557184715708, "vram_gb": 5.684933632, "lambda": 0.1582211727897326, "lambda_min": 0.11109297722578049, "lambda_max": 0.21990595757961273, "S_mean": 0.36865234375, "G_mean": 0.4461989753569166, "G_std": 0.18119817273691297, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.346614360809326, "grad_norm": 0.6076067090034485, "tokens_seen": 16883712, "tokens_per_sec": 18420.926402773017, "vram_gb": 5.684933632, "lambda": 0.15872606386741003, "lambda_min": 0.11037855595350266, "lambda_max": 0.2217078059911728, "S_mean": 0.3574015299479167, "G_mean": 0.434550487746795, "G_std": 0.18026542632530132, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.382883071899414, "grad_norm": 0.6013332009315491, "tokens_seen": 17047552, "tokens_per_sec": 18422.815940871715, "vram_gb": 5.684933632, "lambda": 0.15926411903152862, "lambda_min": 0.11019890010356903, "lambda_max": 0.2230255901813507, "S_mean": 0.3570760091145833, "G_mean": 0.4344435107583801, "G_std": 0.18158457521349192, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.199599027633667, "grad_norm": 0.6122947335243225, "tokens_seen": 17211392, "tokens_per_sec": 18425.20340328005, "vram_gb": 5.684933632, "lambda": 0.15979217582692704, "lambda_min": 0.11040414124727249, "lambda_max": 0.22403115034103394, "S_mean": 0.3575032552083333, "G_mean": 0.4274547640234232, "G_std": 0.17964237881824374, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.344148635864258, "grad_norm": 0.6105227470397949, "tokens_seen": 17375232, "tokens_per_sec": 18425.90095028979, "vram_gb": 5.684933632, "lambda": 0.16027451865375042, "lambda_min": 0.11045318096876144, "lambda_max": 0.22514063119888306, "S_mean": 0.3576456705729167, "G_mean": 0.42834157838175696, "G_std": 0.18301199950898686, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.131174802780151, "grad_norm": 0.5627541542053223, "tokens_seen": 17539072, "tokens_per_sec": 18427.38928450109, "vram_gb": 5.684933632, "lambda": 0.1607913759847482, "lambda_min": 0.11019578576087952, "lambda_max": 0.22655241191387177, "S_mean": 0.37628173828125, "G_mean": 0.421819521424671, "G_std": 0.17789902600149313, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.17346715927124, "grad_norm": 0.5714813470840454, "tokens_seen": 17702912, "tokens_per_sec": 18428.326339899868, "vram_gb": 5.684933632, "lambda": 0.16130892280489206, "lambda_min": 0.11024530231952667, "lambda_max": 0.2277265042066574, "S_mean": 0.3615315755208333, "G_mean": 0.4264714699238539, "G_std": 0.18204395504047474, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.295729875564575, "grad_norm": 0.6329736709594727, "tokens_seen": 17866752, "tokens_per_sec": 18429.545369198448, "vram_gb": 5.684933632, "lambda": 0.16189563553780317, "lambda_min": 0.11046033352613449, "lambda_max": 0.22924970090389252, "S_mean": 0.3684285481770833, "G_mean": 0.4254876160994172, "G_std": 0.18014850529531637, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.070456504821777, "grad_norm": 0.6782873272895813, "tokens_seen": 18030592, "tokens_per_sec": 18431.692505243056, "vram_gb": 5.684933632, "lambda": 0.16239047547181448, "lambda_min": 0.11053051054477692, "lambda_max": 0.23042815923690796, "S_mean": 0.3939615885416667, "G_mean": 0.4118834890735646, "G_std": 0.178482577825586, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.150299787521362, "grad_norm": 0.561820924282074, "tokens_seen": 18194432, "tokens_per_sec": 18431.60847062017, "vram_gb": 5.684933632, "lambda": 0.16283243149518967, "lambda_min": 0.11086472123861313, "lambda_max": 0.2314062863588333, "S_mean": 0.3499552408854167, "G_mean": 0.4342050813138485, "G_std": 0.18228813338403901, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.292851448059082, "grad_norm": 0.6664631366729736, "tokens_seen": 18358272, "tokens_per_sec": 18433.072336832138, "vram_gb": 5.684933632, "lambda": 0.16335143117854992, "lambda_min": 0.11067330837249756, "lambda_max": 0.23273135721683502, "S_mean": 0.36328125, "G_mean": 0.4264349148919185, "G_std": 0.1781194331124425, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.078594923019409, "grad_norm": 0.6372174024581909, "tokens_seen": 18522112, "tokens_per_sec": 18432.99657360675, "vram_gb": 5.684933632, "lambda": 0.16392874903976917, "lambda_min": 0.11053191125392914, "lambda_max": 0.23418660461902618, "S_mean": 0.3714599609375, "G_mean": 0.42783731361851096, "G_std": 0.1764940652064979, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.117345094680786, "grad_norm": 0.6194882392883301, "tokens_seen": 18685952, "tokens_per_sec": 18434.155116901173, "vram_gb": 5.684933632, "lambda": 0.16449401434510946, "lambda_min": 0.11065839231014252, "lambda_max": 0.2355729639530182, "S_mean": 0.3575439453125, "G_mean": 0.430241489627709, "G_std": 0.18134488180900613, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.325234413146973, "grad_norm": 0.6062797904014587, "tokens_seen": 18849792, "tokens_per_sec": 18435.11219586056, "vram_gb": 5.684933632, "lambda": 0.1649830381696423, "lambda_min": 0.11101806908845901, "lambda_max": 0.23711644113063812, "S_mean": 0.3583984375, "G_mean": 0.41450281891350943, "G_std": 0.1824146603854994, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.268121004104614, "grad_norm": 0.644294261932373, "tokens_seen": 19013632, "tokens_per_sec": 18434.684262772655, "vram_gb": 5.684933632, "lambda": 0.16551449801772833, "lambda_min": 0.11079313606023788, "lambda_max": 0.23882395029067993, "S_mean": 0.361419677734375, "G_mean": 0.4235620096636315, "G_std": 0.1767481374554336, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.112839460372925, "grad_norm": 0.6286070942878723, "tokens_seen": 19177472, "tokens_per_sec": 18436.038295854534, "vram_gb": 5.684933632, "lambda": 0.16596304842581353, "lambda_min": 0.11044911295175552, "lambda_max": 0.24030748009681702, "S_mean": 0.3483683268229167, "G_mean": 0.4244716231090327, "G_std": 0.17713109652201334, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.107539892196655, "grad_norm": 0.6048677563667297, "tokens_seen": 19341312, "tokens_per_sec": 18439.37419589897, "vram_gb": 5.684933632, "lambda": 0.16649243297676244, "lambda_min": 0.11020894348621368, "lambda_max": 0.24185222387313843, "S_mean": 0.3447265625, "G_mean": 0.4092130420419077, "G_std": 0.18021737085655332, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.100576162338257, "grad_norm": 0.6363562941551208, "tokens_seen": 19505152, "tokens_per_sec": 18440.867798533658, "vram_gb": 5.684933632, "lambda": 0.16700528499980768, "lambda_min": 0.11019366979598999, "lambda_max": 0.243587926030159, "S_mean": 0.3292032877604167, "G_mean": 0.42368073513110477, "G_std": 0.18019810070594153, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.081074237823486, "grad_norm": 0.60907381772995, "tokens_seen": 19668992, "tokens_per_sec": 18439.40689377543, "vram_gb": 5.684933632, "lambda": 0.16756803635507822, "lambda_min": 0.11022961884737015, "lambda_max": 0.2453833818435669, "S_mean": 0.3567301432291667, "G_mean": 0.4141194128120939, "G_std": 0.1801056961218516, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.236386060714722, "grad_norm": 0.6714658737182617, "tokens_seen": 19832832, "tokens_per_sec": 18437.117970167343, "vram_gb": 5.684933632, "lambda": 0.168087774887681, "lambda_min": 0.11005902290344238, "lambda_max": 0.24688319861888885, "S_mean": 0.34552001953125, "G_mean": 0.4349857857450843, "G_std": 0.18130497494712472, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.118074417114258, "grad_norm": 0.6271987557411194, "tokens_seen": 19996672, "tokens_per_sec": 18435.860157697563, "vram_gb": 5.684933632, "lambda": 0.16862649905184904, "lambda_min": 0.10988052189350128, "lambda_max": 0.24832572042942047, "S_mean": 0.3292643229166667, "G_mean": 0.4139840779826045, "G_std": 0.18157824563483396, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.035667896270752, "grad_norm": 0.6081103682518005, "tokens_seen": 20160512, "tokens_per_sec": 18435.066878761994, "vram_gb": 5.684933632, "lambda": 0.1691224565729499, "lambda_min": 0.109624482691288, "lambda_max": 0.24962568283081055, "S_mean": 0.3310953776041667, "G_mean": 0.41543683502823114, "G_std": 0.17940076952800155, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.044111967086792, "grad_norm": 0.6556463241577148, "tokens_seen": 20324352, "tokens_per_sec": 18433.86330619409, "vram_gb": 5.684933632, "lambda": 0.16971852785597244, "lambda_min": 0.1094806119799614, "lambda_max": 0.25142911076545715, "S_mean": 0.3281656901041667, "G_mean": 0.42961839788282913, "G_std": 0.1814554021693766, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.142707347869873, "grad_norm": 0.6017922759056091, "tokens_seen": 20488192, "tokens_per_sec": 18432.129256089105, "vram_gb": 5.684933632, "lambda": 0.17016773981352648, "lambda_min": 0.10916733741760254, "lambda_max": 0.2529795169830322, "S_mean": 0.3404541015625, "G_mean": 0.41371549883236486, "G_std": 0.18037915439344943, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 4.965353727340698, "grad_norm": 0.6116042137145996, "tokens_seen": 20652032, "tokens_per_sec": 18431.45954254211, "vram_gb": 5.684933632, "lambda": 0.17066154287507138, "lambda_min": 0.10897539556026459, "lambda_max": 0.25442954897880554, "S_mean": 0.3612263997395833, "G_mean": 0.4112963353594144, "G_std": 0.17580032519375285, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.056196451187134, "grad_norm": 0.6633033752441406, "tokens_seen": 20815872, "tokens_per_sec": 18430.5455547667, "vram_gb": 5.684933632, "lambda": 0.17125032128145298, "lambda_min": 0.10899113118648529, "lambda_max": 0.2561221420764923, "S_mean": 0.3447265625, "G_mean": 0.4135508197359741, "G_std": 0.17891465065379938, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.157141923904419, "grad_norm": 0.7735832333564758, "tokens_seen": 20979712, "tokens_per_sec": 18428.580922435467, "vram_gb": 5.684933632, "lambda": 0.1718167985479037, "lambda_min": 0.10890401154756546, "lambda_max": 0.25785696506500244, "S_mean": 0.3497314453125, "G_mean": 0.41368107000986737, "G_std": 0.18041143286973238, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.153190851211548, "grad_norm": 0.6853125691413879, "tokens_seen": 21143552, "tokens_per_sec": 18427.802416139417, "vram_gb": 5.684933632, "lambda": 0.17230215078840652, "lambda_min": 0.1087852418422699, "lambda_max": 0.2594205141067505, "S_mean": 0.3492228190104167, "G_mean": 0.42312095252176124, "G_std": 0.1788416071018825, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.028350591659546, "grad_norm": 0.6242074966430664, "tokens_seen": 21307392, "tokens_per_sec": 18427.67429527872, "vram_gb": 5.684933632, "lambda": 0.1728131820758184, "lambda_min": 0.10861464589834213, "lambda_max": 0.2609703540802002, "S_mean": 0.3333536783854167, "G_mean": 0.4155926622139911, "G_std": 0.1801092689856887, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.964184045791626, "grad_norm": 0.6673594117164612, "tokens_seen": 21471232, "tokens_per_sec": 18427.110469160998, "vram_gb": 5.684933632, "lambda": 0.17337300473203263, "lambda_min": 0.10862365365028381, "lambda_max": 0.26286429166793823, "S_mean": 0.3348795572916667, "G_mean": 0.4151487424969673, "G_std": 0.1751816344136993, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.088481187820435, "grad_norm": 0.6835013031959534, "tokens_seen": 21635072, "tokens_per_sec": 18427.11251966304, "vram_gb": 5.684933632, "lambda": 0.17382766089091697, "lambda_min": 0.10832780599594116, "lambda_max": 0.264112651348114, "S_mean": 0.3500162760416667, "G_mean": 0.4139624477053682, "G_std": 0.18122423952445388, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.14517068862915, "grad_norm": 0.6734297275543213, "tokens_seen": 21798912, "tokens_per_sec": 18425.85722114501, "vram_gb": 5.684933632, "lambda": 0.17434489292403063, "lambda_min": 0.10810642689466476, "lambda_max": 0.2659313380718231, "S_mean": 0.3496297200520833, "G_mean": 0.40837875303501886, "G_std": 0.17427066286715368, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.092753648757935, "grad_norm": 0.6425706148147583, "tokens_seen": 21962752, "tokens_per_sec": 18425.187539661783, "vram_gb": 5.684933632, "lambda": 0.17484164300064245, "lambda_min": 0.1079811379313469, "lambda_max": 0.267659991979599, "S_mean": 0.3412272135416667, "G_mean": 0.40649866793925565, "G_std": 0.1768753908885022, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.117919921875, "grad_norm": 0.6641344428062439, "tokens_seen": 22126592, "tokens_per_sec": 18424.009979490842, "vram_gb": 5.684933632, "lambda": 0.17536256772776446, "lambda_min": 0.10830418020486832, "lambda_max": 0.2694409191608429, "S_mean": 0.33306884765625, "G_mean": 0.41110881650820374, "G_std": 0.17764814555024108, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 4.992223739624023, "grad_norm": 0.6618537306785583, "tokens_seen": 22290432, "tokens_per_sec": 18422.894583725218, "vram_gb": 5.684933632, "lambda": 0.17586318496614695, "lambda_min": 0.10829934477806091, "lambda_max": 0.2708345651626587, "S_mean": 0.3345743815104167, "G_mean": 0.4039522058640917, "G_std": 0.17738204325238863, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.063323497772217, "grad_norm": 0.6263458728790283, "tokens_seen": 22454272, "tokens_per_sec": 18422.24883270739, "vram_gb": 5.684933632, "lambda": 0.17638398396472135, "lambda_min": 0.10846512764692307, "lambda_max": 0.2722143828868866, "S_mean": 0.3424886067708333, "G_mean": 0.40223020532478887, "G_std": 0.17600106758375964, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.001258134841919, "grad_norm": 0.6445072293281555, "tokens_seen": 22618112, "tokens_per_sec": 18422.150588555178, "vram_gb": 5.684933632, "lambda": 0.17691764483849207, "lambda_min": 0.10875249654054642, "lambda_max": 0.2738584578037262, "S_mean": 0.3223063151041667, "G_mean": 0.4051111680455506, "G_std": 0.17641976204079887, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 4.941710710525513, "grad_norm": 0.6370978951454163, "tokens_seen": 22781952, "tokens_per_sec": 18421.49915225417, "vram_gb": 5.684933632, "lambda": 0.1773693005864819, "lambda_min": 0.10868401825428009, "lambda_max": 0.2754160165786743, "S_mean": 0.32928466796875, "G_mean": 0.39169166640688974, "G_std": 0.17734874311524132, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.987155199050903, "grad_norm": 0.6868960857391357, "tokens_seen": 22945792, "tokens_per_sec": 18419.482780969232, "vram_gb": 5.684933632, "lambda": 0.17784373989949623, "lambda_min": 0.10843224823474884, "lambda_max": 0.2767491936683655, "S_mean": 0.3129069010416667, "G_mean": 0.40321701345965266, "G_std": 0.17840974242426455, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.963497877120972, "grad_norm": 0.6623120307922363, "tokens_seen": 23109632, "tokens_per_sec": 18417.14395471863, "vram_gb": 5.684933632, "lambda": 0.1784131700793902, "lambda_min": 0.1086282730102539, "lambda_max": 0.27842187881469727, "S_mean": 0.330535888671875, "G_mean": 0.39375390236576396, "G_std": 0.1785436806579431, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.118469476699829, "grad_norm": 0.6642457842826843, "tokens_seen": 23273472, "tokens_per_sec": 18414.658173205346, "vram_gb": 5.684933632, "lambda": 0.17893077867726484, "lambda_min": 0.10886819660663605, "lambda_max": 0.27995434403419495, "S_mean": 0.30975341796875, "G_mean": 0.4047230863943696, "G_std": 0.17897017990859845, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.852043628692627, "grad_norm": 0.6525393128395081, "tokens_seen": 23437312, "tokens_per_sec": 18412.080225659683, "vram_gb": 5.684933632, "lambda": 0.17937469203025103, "lambda_min": 0.10907954722642899, "lambda_max": 0.2812708616256714, "S_mean": 0.3323160807291667, "G_mean": 0.3962149474148949, "G_std": 0.17583300289697945, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.021032333374023, "grad_norm": 0.6263832449913025, "tokens_seen": 23601152, "tokens_per_sec": 18411.864209426934, "vram_gb": 5.684933632, "lambda": 0.1798654356971383, "lambda_min": 0.1091575026512146, "lambda_max": 0.2830303907394409, "S_mean": 0.33056640625, "G_mean": 0.4030799643757443, "G_std": 0.17583639469618598, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.008398532867432, "grad_norm": 0.692312479019165, "tokens_seen": 23764992, "tokens_per_sec": 18410.317975473306, "vram_gb": 5.684933632, "lambda": 0.18034114222973585, "lambda_min": 0.10952426493167877, "lambda_max": 0.28421133756637573, "S_mean": 0.31298828125, "G_mean": 0.4049566965550184, "G_std": 0.17788212172066173, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.050968170166016, "grad_norm": 0.6696919798851013, "tokens_seen": 23928832, "tokens_per_sec": 18409.506747275325, "vram_gb": 5.684933632, "lambda": 0.18082317368437847, "lambda_min": 0.10925791412591934, "lambda_max": 0.2857508361339569, "S_mean": 0.3265177408854167, "G_mean": 0.39658166623363894, "G_std": 0.17685355967842042, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.1431214809417725, "grad_norm": 0.7350555658340454, "tokens_seen": 24092672, "tokens_per_sec": 18409.258629414206, "vram_gb": 5.684933632, "lambda": 0.18131747903923193, "lambda_min": 0.10932078212499619, "lambda_max": 0.28687137365341187, "S_mean": 0.3338216145833333, "G_mean": 0.40029849562173087, "G_std": 0.17929710781512162, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.879191637039185, "grad_norm": 0.6492951512336731, "tokens_seen": 24256512, "tokens_per_sec": 18407.740412184598, "vram_gb": 5.684933632, "lambda": 0.18177484907209873, "lambda_min": 0.10923430323600769, "lambda_max": 0.28827229142189026, "S_mean": 0.3316853841145833, "G_mean": 0.3961283323975901, "G_std": 0.174750056816265, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.966959238052368, "grad_norm": 0.6395127773284912, "tokens_seen": 24420352, "tokens_per_sec": 18407.073554055307, "vram_gb": 5.684933632, "lambda": 0.1822403014327089, "lambda_min": 0.10910649597644806, "lambda_max": 0.28964412212371826, "S_mean": 0.3256327311197917, "G_mean": 0.393532975576818, "G_std": 0.17767092570041618, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.879403114318848, "grad_norm": 0.6436803340911865, "tokens_seen": 24584192, "tokens_per_sec": 18405.98275953428, "vram_gb": 5.684933632, "lambda": 0.18269335757941008, "lambda_min": 0.10885429382324219, "lambda_max": 0.2911350727081299, "S_mean": 0.3259480794270833, "G_mean": 0.4037764500826597, "G_std": 0.17324736698841056, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.035415816307068, "val_ppl": 153.76351669825792, "lambda": 0.1827171165496111, "lambda_min": 0.1088484674692154, "lambda_max": 0.29120299220085144, "lambda_per_layer": [0.15509051084518433, 0.14094963669776917, 0.13458192348480225, 0.12792269885540009, 0.13094410300254822, 0.12636308372020721, 0.10987874120473862, 0.1088484674692154, 0.12262172251939774, 0.13717949390411377, 0.23170949518680573, 0.26337507367134094, 0.2746555805206299, 0.24881355464458466, 0.28227731585502625, 0.18986180424690247, 0.29120299220085144, 0.1785767823457718, 0.26231518387794495, 0.18381060659885406, 0.1792365312576294, 0.19507341086864471, 0.16365888714790344, 0.14626319706439972], "S_mean": 0.3200785319010417, "G_mean": 0.4012245857156813, "G_std": 0.17462929935815433, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.046597242355347, "grad_norm": 0.7036339640617371, "tokens_seen": 24748032, "tokens_per_sec": 18381.94192865226, "vram_gb": 5.684933632, "lambda": 0.1831766627728939, "lambda_min": 0.1089453175663948, "lambda_max": 0.29282185435295105, "S_mean": 0.3340250651041667, "G_mean": 0.41135511733591557, "G_std": 0.1784301334992051, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.027196407318115, "grad_norm": 0.6971616148948669, "tokens_seen": 24911872, "tokens_per_sec": 18380.016249293672, "vram_gb": 5.684933632, "lambda": 0.1837120500082771, "lambda_min": 0.10910408943891525, "lambda_max": 0.2943202257156372, "S_mean": 0.33245849609375, "G_mean": 0.3988512436238428, "G_std": 0.17551435033480325, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.966259241104126, "grad_norm": 0.7120621204376221, "tokens_seen": 25075712, "tokens_per_sec": 18380.29629047452, "vram_gb": 5.684933632, "lambda": 0.18421843368560076, "lambda_min": 0.1094682514667511, "lambda_max": 0.29553577303886414, "S_mean": 0.3499959309895833, "G_mean": 0.38992744653175276, "G_std": 0.17370670513870815, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.840972661972046, "grad_norm": 0.6323248744010925, "tokens_seen": 25239552, "tokens_per_sec": 18380.58969834196, "vram_gb": 5.684933632, "lambda": 0.1848160351316134, "lambda_min": 0.10950670391321182, "lambda_max": 0.29744383692741394, "S_mean": 0.3315836588541667, "G_mean": 0.3933831875522931, "G_std": 0.17607608848872283, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.984758138656616, "grad_norm": 0.6807710528373718, "tokens_seen": 25403392, "tokens_per_sec": 18380.514000895997, "vram_gb": 5.684933632, "lambda": 0.18530254159122705, "lambda_min": 0.10942403227090836, "lambda_max": 0.29897814989089966, "S_mean": 0.3350423177083333, "G_mean": 0.38876636264224845, "G_std": 0.17748961768423518, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.7244298458099365, "grad_norm": 0.6734124422073364, "tokens_seen": 25567232, "tokens_per_sec": 18378.404156400742, "vram_gb": 5.684933632, "lambda": 0.1858147345483303, "lambda_min": 0.10928605496883392, "lambda_max": 0.3005553185939789, "S_mean": 0.3256937662760417, "G_mean": 0.39413539599627256, "G_std": 0.1774014685458193, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.879835605621338, "grad_norm": 0.6309425830841064, "tokens_seen": 25731072, "tokens_per_sec": 18378.28540060924, "vram_gb": 5.684933632, "lambda": 0.18633918619404236, "lambda_min": 0.10919681191444397, "lambda_max": 0.30234092473983765, "S_mean": 0.333984375, "G_mean": 0.3939918363466859, "G_std": 0.17544885096140206, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.842808485031128, "grad_norm": 0.6866129040718079, "tokens_seen": 25894912, "tokens_per_sec": 18378.55952051185, "vram_gb": 5.684933632, "lambda": 0.18685126273582378, "lambda_min": 0.10924957692623138, "lambda_max": 0.3037863075733185, "S_mean": 0.33331298828125, "G_mean": 0.39178433222696185, "G_std": 0.1758557315915823, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.939096450805664, "grad_norm": 0.6231144666671753, "tokens_seen": 26058752, "tokens_per_sec": 18377.90354481065, "vram_gb": 5.684933632, "lambda": 0.1873847550402085, "lambda_min": 0.10935454815626144, "lambda_max": 0.30520614981651306, "S_mean": 0.3395487467447917, "G_mean": 0.3922201964693765, "G_std": 0.17613526441467306, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.891336917877197, "grad_norm": 0.6632130742073059, "tokens_seen": 26222592, "tokens_per_sec": 18377.652719645186, "vram_gb": 5.684933632, "lambda": 0.18788201113541922, "lambda_min": 0.10952837765216827, "lambda_max": 0.3067655861377716, "S_mean": 0.3332926432291667, "G_mean": 0.3853043174992005, "G_std": 0.17262502930437526, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.92768406867981, "grad_norm": 0.7245835661888123, "tokens_seen": 26386432, "tokens_per_sec": 18376.14903227934, "vram_gb": 5.684933632, "lambda": 0.18842553750922283, "lambda_min": 0.10964436084032059, "lambda_max": 0.3083699643611908, "S_mean": 0.3598531087239583, "G_mean": 0.39062416056791943, "G_std": 0.17433351006669304, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.980017900466919, "grad_norm": 0.7184387445449829, "tokens_seen": 26550272, "tokens_per_sec": 18376.123791667567, "vram_gb": 5.684933632, "lambda": 0.18892436133076748, "lambda_min": 0.10951919108629227, "lambda_max": 0.3096102476119995, "S_mean": 0.336151123046875, "G_mean": 0.3880583730836709, "G_std": 0.17345120028282204, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.069700241088867, "grad_norm": 0.7337720990180969, "tokens_seen": 26714112, "tokens_per_sec": 18374.81666349256, "vram_gb": 5.684933632, "lambda": 0.1894298124437531, "lambda_min": 0.10937457531690598, "lambda_max": 0.31142139434814453, "S_mean": 0.3338419596354167, "G_mean": 0.4001246471889317, "G_std": 0.17636990073757866, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.8567986488342285, "grad_norm": 0.9344682693481445, "tokens_seen": 26877952, "tokens_per_sec": 18375.16986453616, "vram_gb": 5.684933632, "lambda": 0.1899496034408609, "lambda_min": 0.10939310491085052, "lambda_max": 0.3129042088985443, "S_mean": 0.3509318033854167, "G_mean": 0.38592807451883954, "G_std": 0.17412709219691655, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.7542219161987305, "grad_norm": 0.6655998826026917, "tokens_seen": 27041792, "tokens_per_sec": 18374.623050037768, "vram_gb": 5.684933632, "lambda": 0.19044240284711123, "lambda_min": 0.10955080389976501, "lambda_max": 0.3142969012260437, "S_mean": 0.3288065592447917, "G_mean": 0.38255268211166066, "G_std": 0.1738617743831128, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.866291761398315, "grad_norm": 0.824974775314331, "tokens_seen": 27205632, "tokens_per_sec": 18373.81321275909, "vram_gb": 5.684933632, "lambda": 0.19088284516086182, "lambda_min": 0.10975848138332367, "lambda_max": 0.3156134784221649, "S_mean": 0.330810546875, "G_mean": 0.38907974772155285, "G_std": 0.17486325092613697, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.895113468170166, "grad_norm": 0.7022296786308289, "tokens_seen": 27369472, "tokens_per_sec": 18373.382316275955, "vram_gb": 5.684933632, "lambda": 0.1913753580302, "lambda_min": 0.10993003100156784, "lambda_max": 0.3170755207538605, "S_mean": 0.3058573404947917, "G_mean": 0.394565490540117, "G_std": 0.17512176286739609, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.839140176773071, "grad_norm": 0.7234605550765991, "tokens_seen": 27533312, "tokens_per_sec": 18372.680937833324, "vram_gb": 5.684933632, "lambda": 0.19190064569314322, "lambda_min": 0.10988762974739075, "lambda_max": 0.31841686367988586, "S_mean": 0.323455810546875, "G_mean": 0.3847434848236541, "G_std": 0.17405934732717773, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.9555206298828125, "grad_norm": 0.7221221923828125, "tokens_seen": 27697152, "tokens_per_sec": 18372.812457026925, "vram_gb": 5.684933632, "lambda": 0.19236861417690912, "lambda_min": 0.10969972610473633, "lambda_max": 0.3201132118701935, "S_mean": 0.3428548177083333, "G_mean": 0.3877182236562173, "G_std": 0.17200532493491968, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.866522789001465, "grad_norm": 0.6862291693687439, "tokens_seen": 27860992, "tokens_per_sec": 18372.00291550912, "vram_gb": 5.684933632, "lambda": 0.1928472894554337, "lambda_min": 0.10954055935144424, "lambda_max": 0.3220037519931793, "S_mean": 0.3085530598958333, "G_mean": 0.39847093882660073, "G_std": 0.17540632793679833, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.991650104522705, "grad_norm": 0.6894790530204773, "tokens_seen": 28024832, "tokens_per_sec": 18371.92374147622, "vram_gb": 5.684933632, "lambda": 0.19331851601600647, "lambda_min": 0.10950344055891037, "lambda_max": 0.3234070837497711, "S_mean": 0.3223368326822917, "G_mean": 0.3863439983688295, "G_std": 0.17665808422801396, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.9192588329315186, "grad_norm": 0.7280336618423462, "tokens_seen": 28188672, "tokens_per_sec": 18371.42219586145, "vram_gb": 5.684933632, "lambda": 0.19383815148224434, "lambda_min": 0.10940424352884293, "lambda_max": 0.32511067390441895, "S_mean": 0.3135172526041667, "G_mean": 0.3923082761466503, "G_std": 0.17567757423967123, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.90695858001709, "grad_norm": 0.704423189163208, "tokens_seen": 28352512, "tokens_per_sec": 18370.801242620764, "vram_gb": 5.684933632, "lambda": 0.19435821970303854, "lambda_min": 0.10952910780906677, "lambda_max": 0.3271767795085907, "S_mean": 0.311065673828125, "G_mean": 0.3862011462139587, "G_std": 0.17494752172691128, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.9365553855896, "grad_norm": 0.7253395915031433, "tokens_seen": 28516352, "tokens_per_sec": 18370.190166111446, "vram_gb": 5.684933632, "lambda": 0.1948233644167582, "lambda_min": 0.10985030978918076, "lambda_max": 0.32836654782295227, "S_mean": 0.32666015625, "G_mean": 0.3873302536085248, "G_std": 0.17162273343031606, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.835371732711792, "grad_norm": 0.6924338936805725, "tokens_seen": 28680192, "tokens_per_sec": 18368.502263557235, "vram_gb": 5.684933632, "lambda": 0.1953278916577498, "lambda_min": 0.1098368912935257, "lambda_max": 0.33009955286979675, "S_mean": 0.3192647298177083, "G_mean": 0.38518177702402073, "G_std": 0.1758746843164166, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.91135311126709, "grad_norm": 0.705729603767395, "tokens_seen": 28844032, "tokens_per_sec": 18366.53187239729, "vram_gb": 5.684933632, "lambda": 0.19589741869519153, "lambda_min": 0.10974675416946411, "lambda_max": 0.33178114891052246, "S_mean": 0.3364461263020833, "G_mean": 0.38527073928465444, "G_std": 0.1713131278132399, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.8636815547943115, "grad_norm": 0.6619192957878113, "tokens_seen": 29007872, "tokens_per_sec": 18366.813106110567, "vram_gb": 5.684933632, "lambda": 0.19641887893279394, "lambda_min": 0.10953472554683685, "lambda_max": 0.33342811465263367, "S_mean": 0.3108317057291667, "G_mean": 0.3838514550589025, "G_std": 0.17392676246042052, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.800853252410889, "grad_norm": 0.7516964077949524, "tokens_seen": 29171712, "tokens_per_sec": 18366.628464455775, "vram_gb": 5.684933632, "lambda": 0.19689542582879463, "lambda_min": 0.10980426520109177, "lambda_max": 0.334334135055542, "S_mean": 0.34637451171875, "G_mean": 0.3865942684933543, "G_std": 0.17316612438298762, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.877652645111084, "grad_norm": 0.6848257780075073, "tokens_seen": 29335552, "tokens_per_sec": 18366.35770465549, "vram_gb": 5.684933632, "lambda": 0.19738874894877276, "lambda_min": 0.11009523272514343, "lambda_max": 0.3355673551559448, "S_mean": 0.325164794921875, "G_mean": 0.38344471994787455, "G_std": 0.1723681069755306, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.743008852005005, "grad_norm": 0.7986724376678467, "tokens_seen": 29499392, "tokens_per_sec": 18366.078635613358, "vram_gb": 5.684933632, "lambda": 0.197815821506083, "lambda_min": 0.110256627202034, "lambda_max": 0.3367938697338104, "S_mean": 0.3358256022135417, "G_mean": 0.3862852974173923, "G_std": 0.17596980494757494, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.846733570098877, "grad_norm": 0.8038134574890137, "tokens_seen": 29663232, "tokens_per_sec": 18365.560732726663, "vram_gb": 5.684933632, "lambda": 0.19834799381593862, "lambda_min": 0.11034417152404785, "lambda_max": 0.3385161757469177, "S_mean": 0.3229166666666667, "G_mean": 0.3789879215570788, "G_std": 0.17337136312077442, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.78402304649353, "grad_norm": 0.7952026128768921, "tokens_seen": 29827072, "tokens_per_sec": 18366.156045786425, "vram_gb": 5.684933632, "lambda": 0.198832582992812, "lambda_min": 0.11028983443975449, "lambda_max": 0.340319961309433, "S_mean": 0.3316141764322917, "G_mean": 0.38778681276987, "G_std": 0.175561593612656, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.843961954116821, "grad_norm": 0.713870108127594, "tokens_seen": 29990912, "tokens_per_sec": 18366.90869923673, "vram_gb": 5.684933632, "lambda": 0.19932392860452333, "lambda_min": 0.11029070615768433, "lambda_max": 0.34159040451049805, "S_mean": 0.3266499837239583, "G_mean": 0.3850980217879017, "G_std": 0.17362549544001618, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.884620666503906, "grad_norm": 0.7095836400985718, "tokens_seen": 30154752, "tokens_per_sec": 18367.282134370857, "vram_gb": 5.684933632, "lambda": 0.19980428367853165, "lambda_min": 0.11002428829669952, "lambda_max": 0.34282130002975464, "S_mean": 0.3243306477864583, "G_mean": 0.3861343474127352, "G_std": 0.17152668737495938, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.796576261520386, "grad_norm": 0.6976439356803894, "tokens_seen": 30318592, "tokens_per_sec": 18367.938066969633, "vram_gb": 5.684933632, "lambda": 0.20031365348647037, "lambda_min": 0.1098901778459549, "lambda_max": 0.3442494571208954, "S_mean": 0.315032958984375, "G_mean": 0.38018740294501185, "G_std": 0.16933120228350163, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.961455821990967, "grad_norm": 0.7135117650032043, "tokens_seen": 30482432, "tokens_per_sec": 18368.29198896977, "vram_gb": 5.684933632, "lambda": 0.20072645942370096, "lambda_min": 0.10998454689979553, "lambda_max": 0.34525421261787415, "S_mean": 0.3320210774739583, "G_mean": 0.37767988350242376, "G_std": 0.17304838332347572, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.855337858200073, "grad_norm": 0.7145443558692932, "tokens_seen": 30646272, "tokens_per_sec": 18368.69580159647, "vram_gb": 5.684933632, "lambda": 0.20118917804211378, "lambda_min": 0.10974128544330597, "lambda_max": 0.346856951713562, "S_mean": 0.3286031087239583, "G_mean": 0.37415416243796545, "G_std": 0.17198632168583572, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.804053783416748, "grad_norm": 0.6705689430236816, "tokens_seen": 30810112, "tokens_per_sec": 18369.122207757777, "vram_gb": 5.684933632, "lambda": 0.20177840751906237, "lambda_min": 0.10989158600568771, "lambda_max": 0.34870585799217224, "S_mean": 0.3236897786458333, "G_mean": 0.3798754387535155, "G_std": 0.1712709714192897, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.7846150398254395, "grad_norm": 0.7128115892410278, "tokens_seen": 30973952, "tokens_per_sec": 18369.170025257106, "vram_gb": 5.684933632, "lambda": 0.2023206309725841, "lambda_min": 0.11030581593513489, "lambda_max": 0.35055723786354065, "S_mean": 0.3172200520833333, "G_mean": 0.37871278449893, "G_std": 0.16969286696985364, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.891379356384277, "grad_norm": 0.6926180720329285, "tokens_seen": 31137792, "tokens_per_sec": 18369.28272960839, "vram_gb": 5.684933632, "lambda": 0.20276674100508293, "lambda_min": 0.11035490781068802, "lambda_max": 0.35178375244140625, "S_mean": 0.31402587890625, "G_mean": 0.3825483511512478, "G_std": 0.17335953291816017, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.838066101074219, "grad_norm": 0.6499356627464294, "tokens_seen": 31301632, "tokens_per_sec": 18369.505112959676, "vram_gb": 5.684933632, "lambda": 0.2032441608607769, "lambda_min": 0.11047980189323425, "lambda_max": 0.3534048795700073, "S_mean": 0.3155314127604167, "G_mean": 0.37459241412580013, "G_std": 0.1721199662424624, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 5.00715446472168, "grad_norm": 0.6964709162712097, "tokens_seen": 31465472, "tokens_per_sec": 18369.38852431183, "vram_gb": 5.684933632, "lambda": 0.2037030402570963, "lambda_min": 0.1104787066578865, "lambda_max": 0.35462144017219543, "S_mean": 0.32525634765625, "G_mean": 0.3785966210222493, "G_std": 0.17237664378869036, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.647920608520508, "grad_norm": 0.7058426737785339, "tokens_seen": 31629312, "tokens_per_sec": 18369.007318449814, "vram_gb": 5.684933632, "lambda": 0.20427408752342066, "lambda_min": 0.1104745864868164, "lambda_max": 0.35575583577156067, "S_mean": 0.30462646484375, "G_mean": 0.37602947598012787, "G_std": 0.17251780536025763, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.7571611404418945, "grad_norm": 0.7758068442344666, "tokens_seen": 31793152, "tokens_per_sec": 18368.315127280333, "vram_gb": 5.684933632, "lambda": 0.20472345563272634, "lambda_min": 0.11060862243175507, "lambda_max": 0.3568832576274872, "S_mean": 0.33013916015625, "G_mean": 0.3808244798177232, "G_std": 0.17205151403322816, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.768296003341675, "grad_norm": 0.7045725584030151, "tokens_seen": 31956992, "tokens_per_sec": 18367.37318764309, "vram_gb": 5.684933632, "lambda": 0.20519114627192417, "lambda_min": 0.11087387800216675, "lambda_max": 0.358615517616272, "S_mean": 0.3342692057291667, "G_mean": 0.3668788247741759, "G_std": 0.17184907849878073, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.796726703643799, "grad_norm": 0.6879357695579529, "tokens_seen": 32120832, "tokens_per_sec": 18367.282853229328, "vram_gb": 5.684933632, "lambda": 0.20564960315823555, "lambda_min": 0.11115311086177826, "lambda_max": 0.3602295517921448, "S_mean": 0.3356730143229167, "G_mean": 0.3729277398282041, "G_std": 0.17305143138704202, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.931597471237183, "grad_norm": 0.7128216624259949, "tokens_seen": 32284672, "tokens_per_sec": 18366.985794952587, "vram_gb": 5.684933632, "lambda": 0.20609594322741032, "lambda_min": 0.11112898588180542, "lambda_max": 0.3611152470111847, "S_mean": 0.3075358072916667, "G_mean": 0.3854743697059651, "G_std": 0.17617763159796596, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.719607353210449, "grad_norm": 0.7492715716362, "tokens_seen": 32448512, "tokens_per_sec": 18366.86013892748, "vram_gb": 5.684933632, "lambda": 0.20658881527682146, "lambda_min": 0.11103074997663498, "lambda_max": 0.36281847953796387, "S_mean": 0.306182861328125, "G_mean": 0.379374358492593, "G_std": 0.1739499776934584, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.635461807250977, "grad_norm": 0.7028664350509644, "tokens_seen": 32612352, "tokens_per_sec": 18366.610819540296, "vram_gb": 5.684933632, "lambda": 0.2070237503697475, "lambda_min": 0.11094591021537781, "lambda_max": 0.3641178607940674, "S_mean": 0.3388671875, "G_mean": 0.3729872376813243, "G_std": 0.17080358183011413, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.8703413009643555, "grad_norm": 0.7077152132987976, "tokens_seen": 32776192, "tokens_per_sec": 18366.501846451407, "vram_gb": 5.684933632, "lambda": 0.20749107841402292, "lambda_min": 0.1108972504734993, "lambda_max": 0.36552947759628296, "S_mean": 0.3096415201822917, "G_mean": 0.3746379624741773, "G_std": 0.1712254632730037, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.856380915641784, "val_ppl": 128.55809658205334, "lambda": 0.20751028694212437, "lambda_min": 0.11089001595973969, "lambda_max": 0.36558935046195984, "lambda_per_layer": [0.15903136134147644, 0.14325520396232605, 0.13568362593650818, 0.125973641872406, 0.13325394690036774, 0.12696748971939087, 0.11089001595973969, 0.11163417249917984, 0.12132533639669418, 0.14352013170719147, 0.25126442313194275, 0.3078417181968689, 0.32526615262031555, 0.3006496727466583, 0.36422887444496155, 0.22642798721790314, 0.36558935046195984, 0.21339896321296692, 0.33281517028808594, 0.21076244115829468, 0.20407456159591675, 0.2314363569021225, 0.18169835209846497, 0.15325793623924255], "S_mean": 0.3095601399739583, "G_mean": 0.3754263712714116, "G_std": 0.1699632767898341, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.889860153198242, "grad_norm": 0.7195908427238464, "tokens_seen": 32940032, "tokens_per_sec": 18349.335762540937, "vram_gb": 5.684933632, "lambda": 0.20798185374587774, "lambda_min": 0.11083637923002243, "lambda_max": 0.3672894537448883, "S_mean": 0.3206888834635417, "G_mean": 0.37987362042379874, "G_std": 0.17324420545871058, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.656934976577759, "grad_norm": 0.7024455666542053, "tokens_seen": 33103872, "tokens_per_sec": 18347.900687162568, "vram_gb": 5.684933632, "lambda": 0.2084938862050573, "lambda_min": 0.11090496927499771, "lambda_max": 0.36878904700279236, "S_mean": 0.34130859375, "G_mean": 0.37919288019960123, "G_std": 0.17114971034849683, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.707941770553589, "grad_norm": 0.7422006726264954, "tokens_seen": 33267712, "tokens_per_sec": 18344.312470907462, "vram_gb": 5.684933632, "lambda": 0.2089303166915973, "lambda_min": 0.1110100969672203, "lambda_max": 0.37055736780166626, "S_mean": 0.323333740234375, "G_mean": 0.37904641567729414, "G_std": 0.171304083817328, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.831609725952148, "grad_norm": 0.7235727906227112, "tokens_seen": 33431552, "tokens_per_sec": 18343.944543364112, "vram_gb": 5.684933632, "lambda": 0.20939962783207497, "lambda_min": 0.1109507754445076, "lambda_max": 0.3719809353351593, "S_mean": 0.3293050130208333, "G_mean": 0.3734715984513362, "G_std": 0.1702093141308675, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.722265720367432, "grad_norm": 0.7256105542182922, "tokens_seen": 33595392, "tokens_per_sec": 18344.009461627327, "vram_gb": 5.684933632, "lambda": 0.20991316996514797, "lambda_min": 0.11097852140665054, "lambda_max": 0.37333497405052185, "S_mean": 0.3300882975260417, "G_mean": 0.36823732778429985, "G_std": 0.16972347938766083, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.908880710601807, "grad_norm": 0.7410311698913574, "tokens_seen": 33759232, "tokens_per_sec": 18344.110857078536, "vram_gb": 5.684933632, "lambda": 0.21041056948403516, "lambda_min": 0.11101491004228592, "lambda_max": 0.3751862943172455, "S_mean": 0.3290812174479167, "G_mean": 0.3751599001698196, "G_std": 0.17174381638566652, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.816783666610718, "grad_norm": 0.7378029227256775, "tokens_seen": 33923072, "tokens_per_sec": 18344.42089700625, "vram_gb": 5.684933632, "lambda": 0.21092920874555907, "lambda_min": 0.11088258773088455, "lambda_max": 0.37629109621047974, "S_mean": 0.3054351806640625, "G_mean": 0.3734304079165061, "G_std": 0.17103569745086133, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.74686598777771, "grad_norm": 0.6750489473342896, "tokens_seen": 34086912, "tokens_per_sec": 18344.60754099639, "vram_gb": 5.684933632, "lambda": 0.21138408749053875, "lambda_min": 0.11079195886850357, "lambda_max": 0.37771135568618774, "S_mean": 0.31683349609375, "G_mean": 0.3752025291323662, "G_std": 0.1731685639048616, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.821976661682129, "grad_norm": 0.6910550594329834, "tokens_seen": 34250752, "tokens_per_sec": 18344.47656699646, "vram_gb": 5.684933632, "lambda": 0.2118880252043406, "lambda_min": 0.11085108667612076, "lambda_max": 0.3793494999408722, "S_mean": 0.3116556803385417, "G_mean": 0.36622643082713086, "G_std": 0.17119396734051406, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.662708282470703, "grad_norm": 0.7223303914070129, "tokens_seen": 34414592, "tokens_per_sec": 18344.427436017257, "vram_gb": 5.684933632, "lambda": 0.21241609131296477, "lambda_min": 0.11077234148979187, "lambda_max": 0.38118407130241394, "S_mean": 0.3360087076822917, "G_mean": 0.36605495602513355, "G_std": 0.16924962067666152, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.703820705413818, "grad_norm": 0.6828016638755798, "tokens_seen": 34578432, "tokens_per_sec": 18344.39194545572, "vram_gb": 5.684933632, "lambda": 0.2128771562129259, "lambda_min": 0.11082107573747635, "lambda_max": 0.3830416202545166, "S_mean": 0.3299357096354167, "G_mean": 0.3791825131823619, "G_std": 0.1708677114608387, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.736978054046631, "grad_norm": 0.7245003581047058, "tokens_seen": 34742272, "tokens_per_sec": 18344.29352114628, "vram_gb": 5.684933632, "lambda": 0.21334155400594076, "lambda_min": 0.11085544526576996, "lambda_max": 0.3845580816268921, "S_mean": 0.3183186848958333, "G_mean": 0.3726536488781373, "G_std": 0.16860456485301256, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.619559288024902, "grad_norm": 0.735026478767395, "tokens_seen": 34906112, "tokens_per_sec": 18343.999763467687, "vram_gb": 5.684933632, "lambda": 0.21385138357679048, "lambda_min": 0.11086935549974442, "lambda_max": 0.38613757491111755, "S_mean": 0.338287353515625, "G_mean": 0.37003951938822865, "G_std": 0.17082930852969488, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.704063892364502, "grad_norm": 0.7220703363418579, "tokens_seen": 35069952, "tokens_per_sec": 18344.377381992388, "vram_gb": 5.684933632, "lambda": 0.21429701211551824, "lambda_min": 0.11081729829311371, "lambda_max": 0.38764679431915283, "S_mean": 0.3386739095052083, "G_mean": 0.3701711723891397, "G_std": 0.17098489232982197, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.7515528202056885, "grad_norm": 0.7976961731910706, "tokens_seen": 35233792, "tokens_per_sec": 18344.018255776704, "vram_gb": 5.684933632, "lambda": 0.21476611432929835, "lambda_min": 0.11106407642364502, "lambda_max": 0.38951388001441956, "S_mean": 0.339996337890625, "G_mean": 0.3680025717864434, "G_std": 0.1697942641718934, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.839765787124634, "grad_norm": 0.7728347182273865, "tokens_seen": 35397632, "tokens_per_sec": 18344.053798689518, "vram_gb": 5.684933632, "lambda": 0.2153128059580922, "lambda_min": 0.11102185398340225, "lambda_max": 0.3917522728443146, "S_mean": 0.3250528971354167, "G_mean": 0.37151685915887356, "G_std": 0.171906642926236, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.779895782470703, "grad_norm": 0.7071683406829834, "tokens_seen": 35561472, "tokens_per_sec": 18344.06937741269, "vram_gb": 5.684933632, "lambda": 0.21580229544391236, "lambda_min": 0.11073339730501175, "lambda_max": 0.39321431517601013, "S_mean": 0.3208719889322917, "G_mean": 0.3638923742497961, "G_std": 0.16894134498822191, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.691041469573975, "grad_norm": 0.7656663656234741, "tokens_seen": 35725312, "tokens_per_sec": 18343.9466304911, "vram_gb": 5.684933632, "lambda": 0.21623171865940094, "lambda_min": 0.11031491309404373, "lambda_max": 0.39437127113342285, "S_mean": 0.32501220703125, "G_mean": 0.371763720177114, "G_std": 0.1707175603757302, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.648616790771484, "grad_norm": 0.6936368942260742, "tokens_seen": 35889152, "tokens_per_sec": 18343.88939257742, "vram_gb": 5.684933632, "lambda": 0.21671825492133698, "lambda_min": 0.11011931300163269, "lambda_max": 0.39577987790107727, "S_mean": 0.3219401041666667, "G_mean": 0.36193791606153053, "G_std": 0.17071290337480605, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.720289707183838, "grad_norm": 0.7645454406738281, "tokens_seen": 36052992, "tokens_per_sec": 18343.596109667575, "vram_gb": 5.684933632, "lambda": 0.21719104579339424, "lambda_min": 0.11019862443208694, "lambda_max": 0.39729222655296326, "S_mean": 0.33135986328125, "G_mean": 0.3707248494805147, "G_std": 0.16921170870773494, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.737690687179565, "grad_norm": 0.7176875472068787, "tokens_seen": 36216832, "tokens_per_sec": 18343.810870861922, "vram_gb": 5.684933632, "lambda": 0.21760284279783568, "lambda_min": 0.110556960105896, "lambda_max": 0.398610919713974, "S_mean": 0.3369852701822917, "G_mean": 0.37194110996400315, "G_std": 0.1685051692960163, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.735971450805664, "grad_norm": 0.7472202777862549, "tokens_seen": 36380672, "tokens_per_sec": 18343.219424524705, "vram_gb": 5.684933632, "lambda": 0.21803683818628392, "lambda_min": 0.11097829043865204, "lambda_max": 0.4002392292022705, "S_mean": 0.324310302734375, "G_mean": 0.3762660894232492, "G_std": 0.17140475722650686, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.883273124694824, "grad_norm": 0.7957772612571716, "tokens_seen": 36544512, "tokens_per_sec": 18343.09925985281, "vram_gb": 5.684933632, "lambda": 0.21844663905600706, "lambda_min": 0.11120085418224335, "lambda_max": 0.40189212560653687, "S_mean": 0.3518473307291667, "G_mean": 0.36829678419356543, "G_std": 0.1693207798525691, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.8281450271606445, "grad_norm": 0.732804000377655, "tokens_seen": 36708352, "tokens_per_sec": 18343.289508813552, "vram_gb": 5.684933632, "lambda": 0.21886573328326145, "lambda_min": 0.11135759204626083, "lambda_max": 0.4026963710784912, "S_mean": 0.3318277994791667, "G_mean": 0.373716742809241, "G_std": 0.17100194407006106, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.679991006851196, "grad_norm": 0.7464026808738708, "tokens_seen": 36872192, "tokens_per_sec": 18342.578604013444, "vram_gb": 5.684933632, "lambda": 0.2192375414694349, "lambda_min": 0.11152318120002747, "lambda_max": 0.4036891758441925, "S_mean": 0.32080078125, "G_mean": 0.3700681682676077, "G_std": 0.17047238388719657, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.796224117279053, "grad_norm": 0.746731162071228, "tokens_seen": 37036032, "tokens_per_sec": 18342.79585638962, "vram_gb": 5.684933632, "lambda": 0.21969680146624646, "lambda_min": 0.111502505838871, "lambda_max": 0.4052669107913971, "S_mean": 0.3225504557291667, "G_mean": 0.37437101911442977, "G_std": 0.16869828687049448, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.7517101764678955, "grad_norm": 0.7102594375610352, "tokens_seen": 37199872, "tokens_per_sec": 18343.0958067988, "vram_gb": 5.684933632, "lambda": 0.22013943828642368, "lambda_min": 0.11143764853477478, "lambda_max": 0.40690407156944275, "S_mean": 0.322113037109375, "G_mean": 0.36656052983986837, "G_std": 0.16877422505058348, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.763530492782593, "grad_norm": 0.7329270243644714, "tokens_seen": 37363712, "tokens_per_sec": 18342.214867631614, "vram_gb": 5.684933632, "lambda": 0.22059112321585417, "lambda_min": 0.11145494133234024, "lambda_max": 0.4087468385696411, "S_mean": 0.316436767578125, "G_mean": 0.36933529990104336, "G_std": 0.16798906253340343, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.591773986816406, "grad_norm": 0.7348660230636597, "tokens_seen": 37527552, "tokens_per_sec": 18342.317533666726, "vram_gb": 5.684933632, "lambda": 0.22097623565544686, "lambda_min": 0.11140450835227966, "lambda_max": 0.40965601801872253, "S_mean": 0.3194478352864583, "G_mean": 0.3670616849946479, "G_std": 0.16830137449627122, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.605813980102539, "grad_norm": 0.7271320819854736, "tokens_seen": 37691392, "tokens_per_sec": 18341.5586283948, "vram_gb": 5.684933632, "lambda": 0.22144688945263624, "lambda_min": 0.11142157763242722, "lambda_max": 0.4111691117286682, "S_mean": 0.3150278727213542, "G_mean": 0.37005076456504565, "G_std": 0.17090548520597318, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.6722941398620605, "grad_norm": 0.7439782619476318, "tokens_seen": 37855232, "tokens_per_sec": 18341.45229252623, "vram_gb": 5.684933632, "lambda": 0.2219452199836572, "lambda_min": 0.11150970309972763, "lambda_max": 0.41301360726356506, "S_mean": 0.3158060709635417, "G_mean": 0.3624509124395748, "G_std": 0.1683202403752754, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.700278997421265, "grad_norm": 0.7683107852935791, "tokens_seen": 38019072, "tokens_per_sec": 18341.439106735932, "vram_gb": 5.684933632, "lambda": 0.22240000311285257, "lambda_min": 0.11156589537858963, "lambda_max": 0.4145940840244293, "S_mean": 0.317474365234375, "G_mean": 0.355315966764465, "G_std": 0.16914287015485266, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.871840715408325, "grad_norm": 0.7281843423843384, "tokens_seen": 38182912, "tokens_per_sec": 18341.37432032535, "vram_gb": 5.684933632, "lambda": 0.22276649562021097, "lambda_min": 0.11174751818180084, "lambda_max": 0.415869802236557, "S_mean": 0.3148091634114583, "G_mean": 0.3647414025229712, "G_std": 0.17107819501931468, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.865746974945068, "grad_norm": 0.7770081162452698, "tokens_seen": 38346752, "tokens_per_sec": 18341.990624059123, "vram_gb": 5.684933632, "lambda": 0.22325715453674397, "lambda_min": 0.11175801604986191, "lambda_max": 0.4171387851238251, "S_mean": 0.3102823893229167, "G_mean": 0.36839777169128257, "G_std": 0.17011824087239802, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.702103614807129, "grad_norm": 0.7671533823013306, "tokens_seen": 38510592, "tokens_per_sec": 18342.779790655408, "vram_gb": 5.684933632, "lambda": 0.22376455211391053, "lambda_min": 0.11187461018562317, "lambda_max": 0.4189852178096771, "S_mean": 0.3009236653645833, "G_mean": 0.3655489800342669, "G_std": 0.16970839040974775, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.563380002975464, "grad_norm": 0.8063391447067261, "tokens_seen": 38674432, "tokens_per_sec": 18342.99869844228, "vram_gb": 5.684933632, "lambda": 0.22411862884958586, "lambda_min": 0.11200819909572601, "lambda_max": 0.42023617029190063, "S_mean": 0.3099568684895833, "G_mean": 0.36582632541346055, "G_std": 0.16684660439689955, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.884422540664673, "grad_norm": 0.7386531233787537, "tokens_seen": 38838272, "tokens_per_sec": 18343.967515362416, "vram_gb": 5.684933632, "lambda": 0.2245661892617742, "lambda_min": 0.11226534098386765, "lambda_max": 0.42194119095802307, "S_mean": 0.3119150797526042, "G_mean": 0.36656110376740497, "G_std": 0.1699252532562241, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.52537202835083, "grad_norm": 0.7985311150550842, "tokens_seen": 39002112, "tokens_per_sec": 18344.354166077854, "vram_gb": 5.684933632, "lambda": 0.22499683126807213, "lambda_min": 0.11220431327819824, "lambda_max": 0.42290377616882324, "S_mean": 0.318939208984375, "G_mean": 0.3614852003908406, "G_std": 0.16726646580112478, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.50473165512085, "grad_norm": 0.7415153980255127, "tokens_seen": 39165952, "tokens_per_sec": 18345.019875095022, "vram_gb": 5.684933632, "lambda": 0.22545017457256714, "lambda_min": 0.11195889860391617, "lambda_max": 0.4247332215309143, "S_mean": 0.3283182779947917, "G_mean": 0.36245282537614304, "G_std": 0.16765887274717292, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.7722203731536865, "grad_norm": 0.7334048748016357, "tokens_seen": 39329792, "tokens_per_sec": 18344.495010121514, "vram_gb": 5.684933632, "lambda": 0.22590030543506145, "lambda_min": 0.11181065440177917, "lambda_max": 0.4262715280056, "S_mean": 0.3043212890625, "G_mean": 0.36767158440003794, "G_std": 0.17079216909284392, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.672344923019409, "grad_norm": 0.7626912593841553, "tokens_seen": 39493632, "tokens_per_sec": 18344.218677158817, "vram_gb": 5.684933632, "lambda": 0.22634083156784376, "lambda_min": 0.11185657232999802, "lambda_max": 0.42794069647789, "S_mean": 0.3144429524739583, "G_mean": 0.3642715894772361, "G_std": 0.16925106958175698, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.7345616817474365, "grad_norm": 0.7072625160217285, "tokens_seen": 39657472, "tokens_per_sec": 18344.32704735269, "vram_gb": 5.684933632, "lambda": 0.22682149770359197, "lambda_min": 0.11169864982366562, "lambda_max": 0.4299866557121277, "S_mean": 0.3136342366536458, "G_mean": 0.35907149376968545, "G_std": 0.1673320970342805, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.764049530029297, "grad_norm": 0.7755438685417175, "tokens_seen": 39821312, "tokens_per_sec": 18344.131360213487, "vram_gb": 5.684933632, "lambda": 0.2272549153616031, "lambda_min": 0.11142351478338242, "lambda_max": 0.43187519907951355, "S_mean": 0.3136494954427083, "G_mean": 0.37219169611732167, "G_std": 0.16816465296627334, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.619915723800659, "grad_norm": 0.7673128843307495, "tokens_seen": 39985152, "tokens_per_sec": 18344.131879642242, "vram_gb": 5.684933632, "lambda": 0.22768030098328987, "lambda_min": 0.11129722744226456, "lambda_max": 0.43336161971092224, "S_mean": 0.3109639485677083, "G_mean": 0.3636318159600099, "G_std": 0.16749934937494496, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.6797401905059814, "grad_norm": 0.770844578742981, "tokens_seen": 40148992, "tokens_per_sec": 18344.287537844055, "vram_gb": 5.684933632, "lambda": 0.22808641909311214, "lambda_min": 0.11103283613920212, "lambda_max": 0.43492522835731506, "S_mean": 0.3070780436197917, "G_mean": 0.3708611101222535, "G_std": 0.16866638767533004, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.637969493865967, "grad_norm": 0.7459375858306885, "tokens_seen": 40312832, "tokens_per_sec": 18344.243898538356, "vram_gb": 5.684933632, "lambda": 0.22853189085920653, "lambda_min": 0.11104588210582733, "lambda_max": 0.43648406863212585, "S_mean": 0.3109334309895833, "G_mean": 0.3683882237722476, "G_std": 0.16775869065895677, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.653827667236328, "grad_norm": 0.7608476877212524, "tokens_seen": 40476672, "tokens_per_sec": 18343.76601053704, "vram_gb": 5.684933632, "lambda": 0.2289446129774054, "lambda_min": 0.11099306493997574, "lambda_max": 0.43766647577285767, "S_mean": 0.3175048828125, "G_mean": 0.3626647622634967, "G_std": 0.17003592220135033, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.64014196395874, "grad_norm": 0.726241946220398, "tokens_seen": 40640512, "tokens_per_sec": 18343.978412743516, "vram_gb": 5.684933632, "lambda": 0.22938456603636345, "lambda_min": 0.11071620136499405, "lambda_max": 0.4393172860145569, "S_mean": 0.3129069010416667, "G_mean": 0.3609300156434377, "G_std": 0.1695797199693819, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.6223461627960205, "grad_norm": 0.7567439675331116, "tokens_seen": 40804352, "tokens_per_sec": 18343.87107281245, "vram_gb": 5.684933632, "lambda": 0.2299287881081303, "lambda_min": 0.11066241562366486, "lambda_max": 0.44126537442207336, "S_mean": 0.317840576171875, "G_mean": 0.3685564718519648, "G_std": 0.16860432329121977, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.527128219604492, "grad_norm": 0.7230198383331299, "tokens_seen": 40968192, "tokens_per_sec": 18343.555409863293, "vram_gb": 5.684933632, "lambda": 0.23039571971942982, "lambda_min": 0.11074883490800858, "lambda_max": 0.4429238438606262, "S_mean": 0.3185017903645833, "G_mean": 0.3614051959011704, "G_std": 0.16972956076885262, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.731330108642578, "val_ppl": 113.44635802038653, "lambda": 0.230415233100454, "lambda_min": 0.11073039472103119, "lambda_max": 0.4429914653301239, "lambda_per_layer": [0.16218845546245575, 0.1436181366443634, 0.13690444827079773, 0.12514184415340424, 0.1354047656059265, 0.12693171203136444, 0.11073039472103119, 0.11493545025587082, 0.12044181674718857, 0.14750321209430695, 0.26637715101242065, 0.3466918468475342, 0.36772987246513367, 0.3423878848552704, 0.4429914653301239, 0.26129424571990967, 0.43480291962623596, 0.2452409267425537, 0.3988909423351288, 0.24005712568759918, 0.2334708571434021, 0.26519110798835754, 0.200325146317482, 0.16071386635303497], "S_mean": 0.31915283203125, "G_mean": 0.3722440559261789, "G_std": 0.16833433466187367, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.806277751922607, "grad_norm": 0.7781277894973755, "tokens_seen": 41132032, "tokens_per_sec": 18329.29949899905, "vram_gb": 5.684933632, "lambda": 0.23078136704862118, "lambda_min": 0.1106669083237648, "lambda_max": 0.4446078836917877, "S_mean": 0.3348897298177083, "G_mean": 0.36648691119626164, "G_std": 0.16734205783965686, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.611755609512329, "grad_norm": 0.7996699213981628, "tokens_seen": 41295872, "tokens_per_sec": 18328.516448096514, "vram_gb": 5.684933632, "lambda": 0.23125379936148724, "lambda_min": 0.11064550280570984, "lambda_max": 0.446239173412323, "S_mean": 0.326202392578125, "G_mean": 0.36731131731842953, "G_std": 0.1710789401549846, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.6352598667144775, "grad_norm": 0.7738931179046631, "tokens_seen": 41459712, "tokens_per_sec": 18328.342938518825, "vram_gb": 5.684933632, "lambda": 0.23172950775672993, "lambda_min": 0.11063151806592941, "lambda_max": 0.44812655448913574, "S_mean": 0.3280283610026042, "G_mean": 0.35900086257606745, "G_std": 0.16683382998841503, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.541438341140747, "grad_norm": 0.7606974244117737, "tokens_seen": 41623552, "tokens_per_sec": 18328.229929418114, "vram_gb": 5.684933632, "lambda": 0.23209853656589985, "lambda_min": 0.11075708270072937, "lambda_max": 0.44954222440719604, "S_mean": 0.3169047037760417, "G_mean": 0.35634246841073036, "G_std": 0.16867054191728434, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.6732306480407715, "grad_norm": 0.8006278872489929, "tokens_seen": 41787392, "tokens_per_sec": 18326.229446305686, "vram_gb": 5.684933632, "lambda": 0.23248967000593743, "lambda_min": 0.11101115494966507, "lambda_max": 0.4503997266292572, "S_mean": 0.315887451171875, "G_mean": 0.36003898790416616, "G_std": 0.16906746659272662, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.507554054260254, "grad_norm": 0.7874524593353271, "tokens_seen": 41951232, "tokens_per_sec": 18325.220496242946, "vram_gb": 5.684933632, "lambda": 0.23290867265313864, "lambda_min": 0.11100293695926666, "lambda_max": 0.4518808126449585, "S_mean": 0.3237559000651042, "G_mean": 0.36572482793902356, "G_std": 0.16919304931070656, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.771813154220581, "grad_norm": 0.8484532833099365, "tokens_seen": 42115072, "tokens_per_sec": 18324.800350220663, "vram_gb": 5.684933632, "lambda": 0.23327579628676176, "lambda_min": 0.11098736524581909, "lambda_max": 0.45274287462234497, "S_mean": 0.3371378580729167, "G_mean": 0.3646634053438902, "G_std": 0.16750008907789984, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.839520692825317, "grad_norm": 0.8747958540916443, "tokens_seen": 42278912, "tokens_per_sec": 18324.57603022516, "vram_gb": 5.684933632, "lambda": 0.23367973789572716, "lambda_min": 0.11076528578996658, "lambda_max": 0.4542904198169708, "S_mean": 0.325897216796875, "G_mean": 0.3683308439018826, "G_std": 0.17046555783599615, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.617646217346191, "grad_norm": 0.7239834070205688, "tokens_seen": 42442752, "tokens_per_sec": 18323.50767800667, "vram_gb": 5.684933632, "lambda": 0.23416248646875223, "lambda_min": 0.1105920672416687, "lambda_max": 0.4557724595069885, "S_mean": 0.3203837076822917, "G_mean": 0.35976670884216827, "G_std": 0.16605249586670348, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.642350196838379, "grad_norm": 0.786734938621521, "tokens_seen": 42606592, "tokens_per_sec": 18322.888100301458, "vram_gb": 5.684933632, "lambda": 0.23463715302447477, "lambda_min": 0.11059542000293732, "lambda_max": 0.45729735493659973, "S_mean": 0.3171895345052083, "G_mean": 0.35749017881850403, "G_std": 0.1678152425059428, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.718000888824463, "grad_norm": 0.8244031667709351, "tokens_seen": 42770432, "tokens_per_sec": 18323.597248592156, "vram_gb": 5.684933632, "lambda": 0.23508761016031107, "lambda_min": 0.11086272448301315, "lambda_max": 0.4588511288166046, "S_mean": 0.32244873046875, "G_mean": 0.36218861141242087, "G_std": 0.16743783625618866, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.5908308029174805, "grad_norm": 0.735645592212677, "tokens_seen": 42934272, "tokens_per_sec": 18324.22848294945, "vram_gb": 5.684933632, "lambda": 0.23551624920219183, "lambda_min": 0.1109774187207222, "lambda_max": 0.46038588881492615, "S_mean": 0.324951171875, "G_mean": 0.3550149424312015, "G_std": 0.16474623046815395, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.455601692199707, "grad_norm": 0.798892617225647, "tokens_seen": 43098112, "tokens_per_sec": 18324.103739302383, "vram_gb": 5.684933632, "lambda": 0.2359797777608037, "lambda_min": 0.11094684898853302, "lambda_max": 0.4617641568183899, "S_mean": 0.3160502115885417, "G_mean": 0.3588457959704101, "G_std": 0.16778698206568757, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.792470932006836, "grad_norm": 0.8087335228919983, "tokens_seen": 43261952, "tokens_per_sec": 18324.42256003338, "vram_gb": 5.684933632, "lambda": 0.23636363012095293, "lambda_min": 0.11117362231016159, "lambda_max": 0.4635390043258667, "S_mean": 0.306427001953125, "G_mean": 0.37189727183431387, "G_std": 0.16795298899523914, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.683740139007568, "grad_norm": 0.7743595838546753, "tokens_seen": 43425792, "tokens_per_sec": 18324.877294305013, "vram_gb": 5.684933632, "lambda": 0.2366569982841611, "lambda_min": 0.11129086464643478, "lambda_max": 0.4650460183620453, "S_mean": 0.3150227864583333, "G_mean": 0.3652012792105476, "G_std": 0.16959451713288823, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.722020387649536, "grad_norm": 0.7939242720603943, "tokens_seen": 43589632, "tokens_per_sec": 18324.481672677455, "vram_gb": 5.684933632, "lambda": 0.2370383289332191, "lambda_min": 0.11140308529138565, "lambda_max": 0.4661214351654053, "S_mean": 0.324554443359375, "G_mean": 0.35622096558411914, "G_std": 0.1674972987578561, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.466689825057983, "grad_norm": 0.7499772906303406, "tokens_seen": 43753472, "tokens_per_sec": 18324.554819509, "vram_gb": 5.684933632, "lambda": 0.2374259065836668, "lambda_min": 0.11159130930900574, "lambda_max": 0.4670066237449646, "S_mean": 0.3178812662760417, "G_mean": 0.3587189270959546, "G_std": 0.1695661216896648, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.709376096725464, "grad_norm": 0.8239237666130066, "tokens_seen": 43917312, "tokens_per_sec": 18324.486495470457, "vram_gb": 5.684933632, "lambda": 0.2378961956128478, "lambda_min": 0.11177447438240051, "lambda_max": 0.467692494392395, "S_mean": 0.31488037109375, "G_mean": 0.36667414599408704, "G_std": 0.1671691076286758, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.492398023605347, "grad_norm": 0.8256344199180603, "tokens_seen": 44081152, "tokens_per_sec": 18324.523796372774, "vram_gb": 5.684933632, "lambda": 0.23831748279432455, "lambda_min": 0.11159071326255798, "lambda_max": 0.4686262607574463, "S_mean": 0.3243509928385417, "G_mean": 0.3562206371376912, "G_std": 0.16599835672726235, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.541135311126709, "grad_norm": 0.7919809222221375, "tokens_seen": 44244992, "tokens_per_sec": 18324.449708027532, "vram_gb": 5.684933632, "lambda": 0.2387167252600193, "lambda_min": 0.11145371198654175, "lambda_max": 0.47025740146636963, "S_mean": 0.3130696614583333, "G_mean": 0.3605493443707625, "G_std": 0.1673056847260644, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.695796966552734, "grad_norm": 0.8036615252494812, "tokens_seen": 44408832, "tokens_per_sec": 18324.66566549784, "vram_gb": 5.684933632, "lambda": 0.23914241635551056, "lambda_min": 0.11147463321685791, "lambda_max": 0.47172150015830994, "S_mean": 0.3102315266927083, "G_mean": 0.3691505810711533, "G_std": 0.1676844028600802, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.535985946655273, "grad_norm": 0.762037992477417, "tokens_seen": 44572672, "tokens_per_sec": 18324.67726287297, "vram_gb": 5.684933632, "lambda": 0.23948462773114443, "lambda_min": 0.1115865558385849, "lambda_max": 0.4732505679130554, "S_mean": 0.3237152099609375, "G_mean": 0.3569627126368384, "G_std": 0.16632702484882125, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.750191926956177, "grad_norm": 0.790401816368103, "tokens_seen": 44736512, "tokens_per_sec": 18325.1131930611, "vram_gb": 5.684933632, "lambda": 0.23992769109706083, "lambda_min": 0.11173310875892639, "lambda_max": 0.4746929407119751, "S_mean": 0.3211466471354167, "G_mean": 0.3621095049505432, "G_std": 0.16756238016144684, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.531227111816406, "grad_norm": 0.7367933988571167, "tokens_seen": 44900352, "tokens_per_sec": 18325.08506765149, "vram_gb": 5.684933632, "lambda": 0.24027853272855282, "lambda_min": 0.11180566251277924, "lambda_max": 0.4757727384567261, "S_mean": 0.3187052408854167, "G_mean": 0.3600333489788075, "G_std": 0.16696162874965617, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.88590407371521, "grad_norm": 0.8015565276145935, "tokens_seen": 45064192, "tokens_per_sec": 18325.257337150473, "vram_gb": 5.684933632, "lambda": 0.24063249056537947, "lambda_min": 0.11163444072008133, "lambda_max": 0.4771035313606262, "S_mean": 0.3232828776041667, "G_mean": 0.3626681532866011, "G_std": 0.1666933783562854, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.587677478790283, "grad_norm": 0.8355839848518372, "tokens_seen": 45228032, "tokens_per_sec": 18325.041315282906, "vram_gb": 5.684933632, "lambda": 0.24102115258574486, "lambda_min": 0.11164489388465881, "lambda_max": 0.4784543514251709, "S_mean": 0.3217976888020833, "G_mean": 0.36627815337851644, "G_std": 0.16580026880061874, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.579493045806885, "grad_norm": 0.8069216012954712, "tokens_seen": 45391872, "tokens_per_sec": 18324.513718367463, "vram_gb": 5.684933632, "lambda": 0.2414532263452808, "lambda_min": 0.11141353845596313, "lambda_max": 0.47982358932495117, "S_mean": 0.3348592122395833, "G_mean": 0.35865988675504923, "G_std": 0.16572815245793512, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.6303229331970215, "grad_norm": 0.7922699451446533, "tokens_seen": 45555712, "tokens_per_sec": 18324.76266973021, "vram_gb": 5.684933632, "lambda": 0.2418029671534896, "lambda_min": 0.11139596998691559, "lambda_max": 0.4810506999492645, "S_mean": 0.3141072591145833, "G_mean": 0.36070890026167035, "G_std": 0.16904919447066882, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.575681686401367, "grad_norm": 0.764124870300293, "tokens_seen": 45719552, "tokens_per_sec": 18324.455906117495, "vram_gb": 5.684933632, "lambda": 0.2422080272808671, "lambda_min": 0.11163421720266342, "lambda_max": 0.48235923051834106, "S_mean": 0.3070882161458333, "G_mean": 0.35591344147299725, "G_std": 0.16564275367030254, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.74993371963501, "grad_norm": 0.760315477848053, "tokens_seen": 45883392, "tokens_per_sec": 18324.696273619662, "vram_gb": 5.684933632, "lambda": 0.24267926843216023, "lambda_min": 0.11189767718315125, "lambda_max": 0.48433881998062134, "S_mean": 0.3045094807942708, "G_mean": 0.35002347896806896, "G_std": 0.16551033492820957, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.660975694656372, "grad_norm": 0.84068763256073, "tokens_seen": 46047232, "tokens_per_sec": 18324.765430342824, "vram_gb": 5.684933632, "lambda": 0.2430457528680563, "lambda_min": 0.11200527846813202, "lambda_max": 0.4855974316596985, "S_mean": 0.3033803304036458, "G_mean": 0.3599376818941285, "G_std": 0.16814165158818165, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.507852554321289, "grad_norm": 0.7621597647666931, "tokens_seen": 46211072, "tokens_per_sec": 18323.191647328473, "vram_gb": 5.684933632, "lambda": 0.24339958435545364, "lambda_min": 0.11200017482042313, "lambda_max": 0.48665010929107666, "S_mean": 0.3100077311197917, "G_mean": 0.3587719136849046, "G_std": 0.169670496097145, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.534568548202515, "grad_norm": 0.7635360360145569, "tokens_seen": 46374912, "tokens_per_sec": 18323.12132490451, "vram_gb": 5.684933632, "lambda": 0.24386021650085846, "lambda_min": 0.11201990395784378, "lambda_max": 0.4883299469947815, "S_mean": 0.31182861328125, "G_mean": 0.3577356638076405, "G_std": 0.16815195981568345, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.637898921966553, "grad_norm": 0.7961224913597107, "tokens_seen": 46538752, "tokens_per_sec": 18323.381205209884, "vram_gb": 5.684933632, "lambda": 0.24425046953062216, "lambda_min": 0.11185119301080704, "lambda_max": 0.4899158477783203, "S_mean": 0.316436767578125, "G_mean": 0.3595401063406219, "G_std": 0.16799354929632196, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.552515268325806, "grad_norm": 0.8054670095443726, "tokens_seen": 46702592, "tokens_per_sec": 18323.360645116336, "vram_gb": 5.684933632, "lambda": 0.2446083746229609, "lambda_min": 0.11199140548706055, "lambda_max": 0.49113911390304565, "S_mean": 0.3400777180989583, "G_mean": 0.3507873003060619, "G_std": 0.16558609495405108, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.521943092346191, "grad_norm": 0.8088148236274719, "tokens_seen": 46866432, "tokens_per_sec": 18323.754407332213, "vram_gb": 5.684933632, "lambda": 0.24502059817314148, "lambda_min": 0.11222871392965317, "lambda_max": 0.4917791783809662, "S_mean": 0.3223876953125, "G_mean": 0.3544035234954208, "G_std": 0.16755278614194444, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.547201633453369, "grad_norm": 0.756773054599762, "tokens_seen": 47030272, "tokens_per_sec": 18324.069930191345, "vram_gb": 5.684933632, "lambda": 0.24539149397363266, "lambda_min": 0.11229481548070908, "lambda_max": 0.492756187915802, "S_mean": 0.326019287109375, "G_mean": 0.35764799538689357, "G_std": 0.16618170115786293, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.329510688781738, "grad_norm": 0.7489163875579834, "tokens_seen": 47194112, "tokens_per_sec": 18324.205965603152, "vram_gb": 5.684933632, "lambda": 0.24579982490589222, "lambda_min": 0.11228762567043304, "lambda_max": 0.49454087018966675, "S_mean": 0.3115234375, "G_mean": 0.34527440726136166, "G_std": 0.16704362879196802, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.547924995422363, "grad_norm": 0.8337866067886353, "tokens_seen": 47357952, "tokens_per_sec": 18324.572325839967, "vram_gb": 5.684933632, "lambda": 0.24613815701256195, "lambda_min": 0.11260657012462616, "lambda_max": 0.4961147606372833, "S_mean": 0.3205159505208333, "G_mean": 0.3523742681524406, "G_std": 0.16586922446731478, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.620680093765259, "grad_norm": 0.7771402597427368, "tokens_seen": 47521792, "tokens_per_sec": 18324.82194484791, "vram_gb": 5.684933632, "lambda": 0.24657330506791672, "lambda_min": 0.11274037510156631, "lambda_max": 0.4976191222667694, "S_mean": 0.3112538655598958, "G_mean": 0.3569521813187748, "G_std": 0.16681223001796752, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.5723652839660645, "grad_norm": 0.8142629861831665, "tokens_seen": 47685632, "tokens_per_sec": 18325.070149107203, "vram_gb": 5.684933632, "lambda": 0.2470536750430862, "lambda_min": 0.11282840371131897, "lambda_max": 0.49894532561302185, "S_mean": 0.3119099934895833, "G_mean": 0.35477494083655375, "G_std": 0.1668911357410252, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.591609954833984, "grad_norm": 0.7748205661773682, "tokens_seen": 47849472, "tokens_per_sec": 18325.472491533164, "vram_gb": 5.684933632, "lambda": 0.2474379101768136, "lambda_min": 0.11282352358102798, "lambda_max": 0.4998268187046051, "S_mean": 0.321807861328125, "G_mean": 0.35730172010759514, "G_std": 0.16378145959849158, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.62332010269165, "grad_norm": 0.802817165851593, "tokens_seen": 48013312, "tokens_per_sec": 18325.57963919376, "vram_gb": 5.684933632, "lambda": 0.24784611475964388, "lambda_min": 0.1130177453160286, "lambda_max": 0.500826895236969, "S_mean": 0.3197021484375, "G_mean": 0.35157510071682435, "G_std": 0.16656824414773533, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.642140626907349, "grad_norm": 0.8554878830909729, "tokens_seen": 48177152, "tokens_per_sec": 18326.19986402511, "vram_gb": 5.684933632, "lambda": 0.24823339097201824, "lambda_min": 0.11327999830245972, "lambda_max": 0.5020186305046082, "S_mean": 0.308349609375, "G_mean": 0.3571040197275579, "G_std": 0.16621703344086805, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.7834508419036865, "grad_norm": 0.8063092231750488, "tokens_seen": 48340992, "tokens_per_sec": 18325.7748596628, "vram_gb": 5.684933632, "lambda": 0.24857573583722115, "lambda_min": 0.11314935982227325, "lambda_max": 0.503147304058075, "S_mean": 0.3026326497395833, "G_mean": 0.3591122531797737, "G_std": 0.16453800447440395, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.619385242462158, "grad_norm": 0.8230732083320618, "tokens_seen": 48504832, "tokens_per_sec": 18326.088696226052, "vram_gb": 5.684933632, "lambda": 0.24897401966154575, "lambda_min": 0.11338192969560623, "lambda_max": 0.5045660734176636, "S_mean": 0.311126708984375, "G_mean": 0.35313502644809586, "G_std": 0.16518777799016485, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.585985898971558, "grad_norm": 0.8161322474479675, "tokens_seen": 48668672, "tokens_per_sec": 18326.386996548677, "vram_gb": 5.684933632, "lambda": 0.2493547728906075, "lambda_min": 0.11357877403497696, "lambda_max": 0.5055725574493408, "S_mean": 0.2955525716145833, "G_mean": 0.35611751636800665, "G_std": 0.16606372641399503, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.641727447509766, "grad_norm": 0.7428655624389648, "tokens_seen": 48832512, "tokens_per_sec": 18326.0272225429, "vram_gb": 5.684933632, "lambda": 0.2497344557195902, "lambda_min": 0.11376165598630905, "lambda_max": 0.5068237781524658, "S_mean": 0.3086954752604167, "G_mean": 0.35023068031296134, "G_std": 0.16402443343152603, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.488976001739502, "grad_norm": 0.7976290583610535, "tokens_seen": 48996352, "tokens_per_sec": 18326.2759026495, "vram_gb": 5.684933632, "lambda": 0.2501613786444068, "lambda_min": 0.11384223401546478, "lambda_max": 0.5082229375839233, "S_mean": 0.3073018391927083, "G_mean": 0.35462310118600726, "G_std": 0.16599736956413835, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.514764785766602, "grad_norm": 0.7626950144767761, "tokens_seen": 49160192, "tokens_per_sec": 18326.502139332908, "vram_gb": 5.684933632, "lambda": 0.25054719516386587, "lambda_min": 0.11384163796901703, "lambda_max": 0.5094919204711914, "S_mean": 0.2976786295572917, "G_mean": 0.3519288348034024, "G_std": 0.16449783012891808, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.615075850486756, "val_ppl": 100.99548879894533, "lambda": 0.25056739275654155, "lambda_min": 0.11384102702140808, "lambda_max": 0.5095624327659607, "lambda_per_layer": [0.16591285169124603, 0.14443466067314148, 0.13793379068374634, 0.12507803738117218, 0.1365952342748642, 0.12715235352516174, 0.11384102702140808, 0.11527083814144135, 0.11932428181171417, 0.15138015151023865, 0.27944156527519226, 0.3773668110370636, 0.4016319513320923, 0.38121217489242554, 0.5095624327659607, 0.2945917248725891, 0.49629831314086914, 0.27207937836647034, 0.45881903171539307, 0.2657192647457123, 0.2591273784637451, 0.2957824766635895, 0.2174234688282013, 0.16763822734355927], "S_mean": 0.308990478515625, "G_mean": 0.35381572615976137, "G_std": 0.16438272785550603, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.449131965637207, "grad_norm": 0.7911531329154968, "tokens_seen": 49324032, "tokens_per_sec": 18314.46012707095, "vram_gb": 5.684933632, "lambda": 0.2508945145333807, "lambda_min": 0.11400000005960464, "lambda_max": 0.5105955600738525, "S_mean": 0.3173319498697917, "G_mean": 0.34866391378454864, "G_std": 0.16522799688391387, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.596409797668457, "grad_norm": 0.7752252817153931, "tokens_seen": 49487872, "tokens_per_sec": 18314.50671335757, "vram_gb": 5.684933632, "lambda": 0.2513109513868888, "lambda_min": 0.11413067579269409, "lambda_max": 0.5119528770446777, "S_mean": 0.3098246256510417, "G_mean": 0.35723599046468735, "G_std": 0.16611979537022611, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.504250526428223, "grad_norm": 0.8166911602020264, "tokens_seen": 49651712, "tokens_per_sec": 18314.185575743235, "vram_gb": 5.684933632, "lambda": 0.25163984516014654, "lambda_min": 0.11411400884389877, "lambda_max": 0.5129300355911255, "S_mean": 0.300567626953125, "G_mean": 0.36254176722529036, "G_std": 0.16706358288259557, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.595298767089844, "grad_norm": 0.8144213557243347, "tokens_seen": 49815552, "tokens_per_sec": 18314.102465007185, "vram_gb": 5.684933632, "lambda": 0.2519926804428299, "lambda_min": 0.11403805762529373, "lambda_max": 0.5141575932502747, "S_mean": 0.32196044921875, "G_mean": 0.35580452958432335, "G_std": 0.16405081706276783, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.552089691162109, "grad_norm": 0.8229228258132935, "tokens_seen": 49979392, "tokens_per_sec": 18314.201935462686, "vram_gb": 5.684933632, "lambda": 0.2524294729034106, "lambda_min": 0.11391580104827881, "lambda_max": 0.5156664848327637, "S_mean": 0.3075408935546875, "G_mean": 0.35547514143399894, "G_std": 0.16821106822074702, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.451720714569092, "grad_norm": 0.7631535530090332, "tokens_seen": 50143232, "tokens_per_sec": 18314.09057003333, "vram_gb": 5.684933632, "lambda": 0.2528292102118333, "lambda_min": 0.11399679630994797, "lambda_max": 0.5165224671363831, "S_mean": 0.3143971761067708, "G_mean": 0.35154489598547417, "G_std": 0.16695524577517062, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.51317572593689, "grad_norm": 0.8318537473678589, "tokens_seen": 50307072, "tokens_per_sec": 18313.972288740497, "vram_gb": 5.684933632, "lambda": 0.2531834427888195, "lambda_min": 0.11400123685598373, "lambda_max": 0.5178478360176086, "S_mean": 0.3021036783854167, "G_mean": 0.3497035066441943, "G_std": 0.1656762718145425, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.630081653594971, "grad_norm": 0.8474770784378052, "tokens_seen": 50470912, "tokens_per_sec": 18314.28007470022, "vram_gb": 5.684933632, "lambda": 0.2535893510406216, "lambda_min": 0.11387836188077927, "lambda_max": 0.5193527936935425, "S_mean": 0.3025054931640625, "G_mean": 0.3611762940417975, "G_std": 0.16557575987341502, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.510631561279297, "grad_norm": 0.8391297459602356, "tokens_seen": 50634752, "tokens_per_sec": 18314.325825836706, "vram_gb": 5.684933632, "lambda": 0.2539746171484391, "lambda_min": 0.1139214038848877, "lambda_max": 0.5207139849662781, "S_mean": 0.3143310546875, "G_mean": 0.3534837254943947, "G_std": 0.1655942757303516, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.587310075759888, "grad_norm": 0.8135585188865662, "tokens_seen": 50798592, "tokens_per_sec": 18314.29643960683, "vram_gb": 5.684933632, "lambda": 0.25430611707270145, "lambda_min": 0.1139434352517128, "lambda_max": 0.5217722058296204, "S_mean": 0.322418212890625, "G_mean": 0.35488781162227195, "G_std": 0.16622434017093232, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.4781458377838135, "grad_norm": 0.8127007484436035, "tokens_seen": 50962432, "tokens_per_sec": 18314.377972895967, "vram_gb": 5.684933632, "lambda": 0.25466801784932613, "lambda_min": 0.1139756292104721, "lambda_max": 0.5231765508651733, "S_mean": 0.311614990234375, "G_mean": 0.35047577368095517, "G_std": 0.16756975821529826, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.446071624755859, "grad_norm": 0.8009553551673889, "tokens_seen": 51126272, "tokens_per_sec": 18314.44524780143, "vram_gb": 5.684933632, "lambda": 0.25506712806721527, "lambda_min": 0.11383358389139175, "lambda_max": 0.5243517160415649, "S_mean": 0.3166249593098958, "G_mean": 0.3448609738455464, "G_std": 0.16424057655967772, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.431025981903076, "grad_norm": 0.9279693365097046, "tokens_seen": 51290112, "tokens_per_sec": 18314.292269906913, "vram_gb": 5.684933632, "lambda": 0.25548212602734566, "lambda_min": 0.11383887380361557, "lambda_max": 0.5256092548370361, "S_mean": 0.3039347330729167, "G_mean": 0.3521242570908119, "G_std": 0.16740688452652344, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.733680248260498, "grad_norm": 0.8093475103378296, "tokens_seen": 51453952, "tokens_per_sec": 18314.723661325876, "vram_gb": 5.684933632, "lambda": 0.25578777057429153, "lambda_min": 0.1138659119606018, "lambda_max": 0.5262277722358704, "S_mean": 0.3144632975260417, "G_mean": 0.35100746310005587, "G_std": 0.16522852603035668, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.723020076751709, "grad_norm": 0.854012131690979, "tokens_seen": 51617792, "tokens_per_sec": 18315.06618224304, "vram_gb": 5.684933632, "lambda": 0.2561467581739028, "lambda_min": 0.11389245092868805, "lambda_max": 0.5275143384933472, "S_mean": 0.3099263509114583, "G_mean": 0.34998311459397274, "G_std": 0.16634873153331378, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.4768900871276855, "grad_norm": 0.8189149498939514, "tokens_seen": 51781632, "tokens_per_sec": 18315.66865334123, "vram_gb": 5.684933632, "lambda": 0.25649868107090396, "lambda_min": 0.11386582255363464, "lambda_max": 0.5289296507835388, "S_mean": 0.3184611002604167, "G_mean": 0.3466288612689823, "G_std": 0.16517856647260487, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.628717422485352, "grad_norm": 0.8116748332977295, "tokens_seen": 51945472, "tokens_per_sec": 18316.170385138714, "vram_gb": 5.684933632, "lambda": 0.2568798279389739, "lambda_min": 0.1140507236123085, "lambda_max": 0.5301774740219116, "S_mean": 0.3170064290364583, "G_mean": 0.34797933565763134, "G_std": 0.16466692179286233, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.501227140426636, "grad_norm": 0.7763937711715698, "tokens_seen": 52109312, "tokens_per_sec": 18316.320041290557, "vram_gb": 5.684933632, "lambda": 0.2572236234943072, "lambda_min": 0.11412113159894943, "lambda_max": 0.5309147834777832, "S_mean": 0.3173319498697917, "G_mean": 0.3391026667474459, "G_std": 0.1657625779043883, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.528851509094238, "grad_norm": 0.7675957083702087, "tokens_seen": 52273152, "tokens_per_sec": 18316.628793290845, "vram_gb": 5.684933632, "lambda": 0.257581099246939, "lambda_min": 0.11406087130308151, "lambda_max": 0.531886100769043, "S_mean": 0.325897216796875, "G_mean": 0.34649092812711996, "G_std": 0.16438232955988497, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.3637309074401855, "grad_norm": 0.8929879665374756, "tokens_seen": 52436992, "tokens_per_sec": 18316.918168569908, "vram_gb": 5.684933632, "lambda": 0.25803817994892597, "lambda_min": 0.11411208659410477, "lambda_max": 0.5335701704025269, "S_mean": 0.3186543782552083, "G_mean": 0.3426804624808331, "G_std": 0.16371150345851979, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.5722575187683105, "grad_norm": 0.8395776152610779, "tokens_seen": 52600832, "tokens_per_sec": 18317.379421722642, "vram_gb": 5.684933632, "lambda": 0.25837179242322844, "lambda_min": 0.11410457640886307, "lambda_max": 0.534850537776947, "S_mean": 0.334014892578125, "G_mean": 0.35516571734721464, "G_std": 0.16405146413793167, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.719301223754883, "grad_norm": 0.8907283544540405, "tokens_seen": 52764672, "tokens_per_sec": 18316.967834935647, "vram_gb": 5.684933632, "lambda": 0.2586974569906791, "lambda_min": 0.11410415917634964, "lambda_max": 0.5358672142028809, "S_mean": 0.3278706868489583, "G_mean": 0.34903534354331595, "G_std": 0.16618506125329682, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.514435052871704, "grad_norm": 0.8246885538101196, "tokens_seen": 52928512, "tokens_per_sec": 18316.671323807244, "vram_gb": 5.684933632, "lambda": 0.25904644653201103, "lambda_min": 0.11411894857883453, "lambda_max": 0.5368737578392029, "S_mean": 0.3228047688802083, "G_mean": 0.3505143219760309, "G_std": 0.16532062843907624, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.565034627914429, "grad_norm": 0.9467251896858215, "tokens_seen": 53092352, "tokens_per_sec": 18316.903313537357, "vram_gb": 5.684933632, "lambda": 0.2594121241321166, "lambda_min": 0.11402039974927902, "lambda_max": 0.5383058786392212, "S_mean": 0.3638509114583333, "G_mean": 0.3448467819641034, "G_std": 0.1630883680578942, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.393195629119873, "grad_norm": 0.8040415048599243, "tokens_seen": 53256192, "tokens_per_sec": 18317.055659642167, "vram_gb": 5.684933632, "lambda": 0.25970530106375617, "lambda_min": 0.11408012360334396, "lambda_max": 0.5388789176940918, "S_mean": 0.3337910970052083, "G_mean": 0.3492836563382298, "G_std": 0.16384722047951072, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.647537708282471, "grad_norm": 0.8543932437896729, "tokens_seen": 53420032, "tokens_per_sec": 18317.00771045075, "vram_gb": 5.684933632, "lambda": 0.2600031749655803, "lambda_min": 0.11421141773462296, "lambda_max": 0.5397521257400513, "S_mean": 0.3298543294270833, "G_mean": 0.3541179499588907, "G_std": 0.1647783361064891, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.558269500732422, "grad_norm": 0.8545401692390442, "tokens_seen": 53583872, "tokens_per_sec": 18316.849789871136, "vram_gb": 5.684933632, "lambda": 0.2603443854798873, "lambda_min": 0.1142260953783989, "lambda_max": 0.5408366918563843, "S_mean": 0.324462890625, "G_mean": 0.35399328971592087, "G_std": 0.16647308451744416, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.560261011123657, "grad_norm": 0.8658432364463806, "tokens_seen": 53747712, "tokens_per_sec": 18316.343426968248, "vram_gb": 5.684933632, "lambda": 0.26064069972683984, "lambda_min": 0.1140621080994606, "lambda_max": 0.5420885682106018, "S_mean": 0.333953857421875, "G_mean": 0.3536200257949531, "G_std": 0.16708494059275836, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.433517694473267, "grad_norm": 0.8134875297546387, "tokens_seen": 53911552, "tokens_per_sec": 18316.447174039466, "vram_gb": 5.684933632, "lambda": 0.2609589636946718, "lambda_min": 0.11401655524969101, "lambda_max": 0.543184757232666, "S_mean": 0.3275146484375, "G_mean": 0.35654911228145164, "G_std": 0.166363653804486, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.6110146045684814, "grad_norm": 0.8311744332313538, "tokens_seen": 54075392, "tokens_per_sec": 18316.263590083803, "vram_gb": 5.684933632, "lambda": 0.2612045304849744, "lambda_min": 0.11379017680883408, "lambda_max": 0.5440013408660889, "S_mean": 0.3422139485677083, "G_mean": 0.3551192869587491, "G_std": 0.16692357137799263, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.458227872848511, "grad_norm": 0.7926127314567566, "tokens_seen": 54239232, "tokens_per_sec": 18315.59317450151, "vram_gb": 5.684933632, "lambda": 0.26152102028330165, "lambda_min": 0.11370769888162613, "lambda_max": 0.5449813604354858, "S_mean": 0.312469482421875, "G_mean": 0.34749683861931163, "G_std": 0.1656739974860102, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.56843900680542, "grad_norm": 0.8202399015426636, "tokens_seen": 54403072, "tokens_per_sec": 18315.89648062973, "vram_gb": 5.684933632, "lambda": 0.26186503004282713, "lambda_min": 0.1136116161942482, "lambda_max": 0.5458647012710571, "S_mean": 0.3236083984375, "G_mean": 0.3510279416417082, "G_std": 0.16341593593824655, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.66080117225647, "grad_norm": 0.841748058795929, "tokens_seen": 54566912, "tokens_per_sec": 18315.559477987856, "vram_gb": 5.684933632, "lambda": 0.2621956942602992, "lambda_min": 0.11364536732435226, "lambda_max": 0.5470096468925476, "S_mean": 0.3250223795572917, "G_mean": 0.35527290791894, "G_std": 0.16576094351088008, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.4863739013671875, "grad_norm": 0.8323931694030762, "tokens_seen": 54730752, "tokens_per_sec": 18315.679025538288, "vram_gb": 5.684933632, "lambda": 0.26259634271264076, "lambda_min": 0.11366734653711319, "lambda_max": 0.5483115911483765, "S_mean": 0.3122456868489583, "G_mean": 0.3510572847444564, "G_std": 0.1659263219529142, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.588007688522339, "grad_norm": 0.8123099207878113, "tokens_seen": 54894592, "tokens_per_sec": 18315.639570027553, "vram_gb": 5.684933632, "lambda": 0.26286860182881355, "lambda_min": 0.11359269917011261, "lambda_max": 0.5494000315666199, "S_mean": 0.3129170735677083, "G_mean": 0.3537772623045991, "G_std": 0.16378649999387562, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.586742639541626, "grad_norm": 0.8142202496528625, "tokens_seen": 55058432, "tokens_per_sec": 18315.284833851198, "vram_gb": 5.684933632, "lambda": 0.26317142850408953, "lambda_min": 0.1134655550122261, "lambda_max": 0.5503736734390259, "S_mean": 0.3117879231770833, "G_mean": 0.3513659493376811, "G_std": 0.16607015296661606, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.562175512313843, "grad_norm": 0.7841270565986633, "tokens_seen": 55222272, "tokens_per_sec": 18315.289332658846, "vram_gb": 5.684933632, "lambda": 0.2634890132273237, "lambda_min": 0.11341526359319687, "lambda_max": 0.5509458184242249, "S_mean": 0.325439453125, "G_mean": 0.34843026466357213, "G_std": 0.1654535949540635, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.590551376342773, "grad_norm": 0.8127267360687256, "tokens_seen": 55386112, "tokens_per_sec": 18315.460677824834, "vram_gb": 5.684933632, "lambda": 0.2638515178114176, "lambda_min": 0.11331547051668167, "lambda_max": 0.5521224141120911, "S_mean": 0.3227437337239583, "G_mean": 0.34799542735951644, "G_std": 0.16466355824377388, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.5394978523254395, "grad_norm": 0.7920504212379456, "tokens_seen": 55549952, "tokens_per_sec": 18315.26300587358, "vram_gb": 5.684933632, "lambda": 0.26413755770772696, "lambda_min": 0.11327264457941055, "lambda_max": 0.5531415939331055, "S_mean": 0.332733154296875, "G_mean": 0.35010140396965045, "G_std": 0.16351555470221987, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.56787633895874, "grad_norm": 0.8719427585601807, "tokens_seen": 55713792, "tokens_per_sec": 18315.433852291673, "vram_gb": 5.684933632, "lambda": 0.264411261305213, "lambda_min": 0.11313431710004807, "lambda_max": 0.5540128350257874, "S_mean": 0.3243306477864583, "G_mean": 0.34468358017814654, "G_std": 0.16619920292093107, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.525229215621948, "grad_norm": 0.8171858787536621, "tokens_seen": 55877632, "tokens_per_sec": 18315.547474029845, "vram_gb": 5.684933632, "lambda": 0.26471197543044883, "lambda_min": 0.11326746642589569, "lambda_max": 0.5547348260879517, "S_mean": 0.3356831868489583, "G_mean": 0.35014793323352933, "G_std": 0.16361847861359516, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.594271659851074, "grad_norm": 0.8305523991584778, "tokens_seen": 56041472, "tokens_per_sec": 18315.843474906724, "vram_gb": 5.684933632, "lambda": 0.2649932544057568, "lambda_min": 0.11335974931716919, "lambda_max": 0.5556422472000122, "S_mean": 0.32379150390625, "G_mean": 0.3480816565764447, "G_std": 0.16500297064582506, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.571757793426514, "grad_norm": 0.8726531863212585, "tokens_seen": 56205312, "tokens_per_sec": 18316.194217752072, "vram_gb": 5.684933632, "lambda": 0.2653119992464781, "lambda_min": 0.11348345875740051, "lambda_max": 0.556609034538269, "S_mean": 0.32110595703125, "G_mean": 0.35150448031102616, "G_std": 0.16675813236118606, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.46379542350769, "grad_norm": 0.8756328821182251, "tokens_seen": 56369152, "tokens_per_sec": 18316.429456021742, "vram_gb": 5.684933632, "lambda": 0.26557062411059934, "lambda_min": 0.11352080851793289, "lambda_max": 0.5580390691757202, "S_mean": 0.3365885416666667, "G_mean": 0.3512707483799507, "G_std": 0.16481580290322503, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.558157444000244, "grad_norm": 0.8557299971580505, "tokens_seen": 56532992, "tokens_per_sec": 18317.099041005415, "vram_gb": 5.684933632, "lambda": 0.2658611920972665, "lambda_min": 0.11341656744480133, "lambda_max": 0.55894535779953, "S_mean": 0.3274739583333333, "G_mean": 0.3417081370328863, "G_std": 0.16399449455396584, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.392046213150024, "grad_norm": 0.8124165534973145, "tokens_seen": 56696832, "tokens_per_sec": 18317.31314559267, "vram_gb": 5.684933632, "lambda": 0.2661539052302639, "lambda_min": 0.11336307227611542, "lambda_max": 0.55954509973526, "S_mean": 0.31756591796875, "G_mean": 0.34680186902793747, "G_std": 0.16489920928142965, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.737052917480469, "grad_norm": 0.8944672346115112, "tokens_seen": 56860672, "tokens_per_sec": 18317.185069401014, "vram_gb": 5.684933632, "lambda": 0.26639222415784997, "lambda_min": 0.1134234145283699, "lambda_max": 0.560462236404419, "S_mean": 0.3256632486979167, "G_mean": 0.3537677451968193, "G_std": 0.16504059499129653, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.505312919616699, "grad_norm": 0.8255881071090698, "tokens_seen": 57024512, "tokens_per_sec": 18317.58728074231, "vram_gb": 5.684933632, "lambda": 0.2667310054724415, "lambda_min": 0.11353585124015808, "lambda_max": 0.5616027116775513, "S_mean": 0.3285929361979167, "G_mean": 0.3496264221612364, "G_std": 0.16321062746768197, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.445613622665405, "grad_norm": 0.827376127243042, "tokens_seen": 57188352, "tokens_per_sec": 18317.48518555623, "vram_gb": 5.684933632, "lambda": 0.2670760548984011, "lambda_min": 0.11352182924747467, "lambda_max": 0.563128650188446, "S_mean": 0.3160603841145833, "G_mean": 0.3417915853206068, "G_std": 0.16352152929175645, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.558589935302734, "grad_norm": 0.8336998224258423, "tokens_seen": 57352192, "tokens_per_sec": 18317.516993706304, "vram_gb": 5.684933632, "lambda": 0.2673562038689852, "lambda_min": 0.11355015635490417, "lambda_max": 0.5640270113945007, "S_mean": 0.3346761067708333, "G_mean": 0.3417685638802747, "G_std": 0.16342069356081387, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.563444447517395, "val_ppl": 95.9132797731003, "lambda": 0.2673744199176629, "lambda_min": 0.11354700475931168, "lambda_max": 0.5640806555747986, "lambda_per_layer": [0.16855774819850922, 0.14571323990821838, 0.13839419186115265, 0.12499495595693588, 0.13755804300308228, 0.1273602694272995, 0.11354700475931168, 0.11739686876535416, 0.11859574168920517, 0.15490461885929108, 0.2888690233230591, 0.3980198800563812, 0.4292864203453064, 0.4121720790863037, 0.5640806555747986, 0.3228822350502014, 0.5473698973655701, 0.29595863819122314, 0.5093029737472534, 0.28949955105781555, 0.28060024976730347, 0.32341885566711426, 0.23451058566570282, 0.1739923506975174], "S_mean": 0.3318379720052083, "G_mean": 0.35009022972856957, "G_std": 0.16255030691778907, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.461688041687012, "grad_norm": 0.8640387058258057, "tokens_seen": 57516032, "tokens_per_sec": 18308.11645391511, "vram_gb": 5.684933632, "lambda": 0.26773539930582047, "lambda_min": 0.11372233927249908, "lambda_max": 0.5651726722717285, "S_mean": 0.31976318359375, "G_mean": 0.3509233488390843, "G_std": 0.1650376815426474, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.5659520626068115, "grad_norm": 0.8953574895858765, "tokens_seen": 57679872, "tokens_per_sec": 18308.06312110526, "vram_gb": 5.684933632, "lambda": 0.267977526721855, "lambda_min": 0.11362505704164505, "lambda_max": 0.5663177371025085, "S_mean": 0.3256785074869792, "G_mean": 0.35230124679704505, "G_std": 0.16454945326161882, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.596237659454346, "grad_norm": 0.8971176147460938, "tokens_seen": 57843712, "tokens_per_sec": 18307.486991897957, "vram_gb": 5.684933632, "lambda": 0.2681980977455775, "lambda_min": 0.113593690097332, "lambda_max": 0.5667585134506226, "S_mean": 0.3289235432942708, "G_mean": 0.3519947857906421, "G_std": 0.16495359758846462, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.464958667755127, "grad_norm": 0.8549544215202332, "tokens_seen": 58007552, "tokens_per_sec": 18307.759985295474, "vram_gb": 5.684933632, "lambda": 0.2684271289035678, "lambda_min": 0.11359146982431412, "lambda_max": 0.5670768618583679, "S_mean": 0.3253377278645833, "G_mean": 0.34930820561324555, "G_std": 0.1619722293689847, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.467929840087891, "grad_norm": 0.8263993859291077, "tokens_seen": 58171392, "tokens_per_sec": 18307.83106763879, "vram_gb": 5.684933632, "lambda": 0.2687070118263364, "lambda_min": 0.11361845582723618, "lambda_max": 0.5680193305015564, "S_mean": 0.3253936767578125, "G_mean": 0.34349508156689507, "G_std": 0.1630776651048412, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.42181396484375, "grad_norm": 0.882777214050293, "tokens_seen": 58335232, "tokens_per_sec": 18307.75810824389, "vram_gb": 5.684933632, "lambda": 0.26900801062583923, "lambda_min": 0.11361750960350037, "lambda_max": 0.5691274404525757, "S_mean": 0.320556640625, "G_mean": 0.3450092206088205, "G_std": 0.1648543986181418, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.545012712478638, "grad_norm": 0.8198006749153137, "tokens_seen": 58499072, "tokens_per_sec": 18308.14257250334, "vram_gb": 5.684933632, "lambda": 0.26931104219208163, "lambda_min": 0.11360598355531693, "lambda_max": 0.5699942708015442, "S_mean": 0.3269246419270833, "G_mean": 0.34349807801966864, "G_std": 0.16475731960963458, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.4232048988342285, "grad_norm": 0.8098025918006897, "tokens_seen": 58662912, "tokens_per_sec": 18308.126296716808, "vram_gb": 5.684933632, "lambda": 0.269560053323706, "lambda_min": 0.11361505091190338, "lambda_max": 0.5707308053970337, "S_mean": 0.33880615234375, "G_mean": 0.345428672619164, "G_std": 0.16375706542748958, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.490327835083008, "grad_norm": 0.8556560277938843, "tokens_seen": 58826752, "tokens_per_sec": 18308.331281122046, "vram_gb": 5.684933632, "lambda": 0.26989066414535046, "lambda_min": 0.1137247160077095, "lambda_max": 0.5717910528182983, "S_mean": 0.3245951334635417, "G_mean": 0.34445443594207364, "G_std": 0.16341917604828873, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.533039808273315, "grad_norm": 0.8646711111068726, "tokens_seen": 58990592, "tokens_per_sec": 18308.27919924816, "vram_gb": 5.684933632, "lambda": 0.270141807384789, "lambda_min": 0.11375132203102112, "lambda_max": 0.5727308988571167, "S_mean": 0.3174641927083333, "G_mean": 0.34280206814097863, "G_std": 0.16408912236026177, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.574103832244873, "grad_norm": 0.8675534129142761, "tokens_seen": 59154432, "tokens_per_sec": 18308.19614885262, "vram_gb": 5.684933632, "lambda": 0.2704648192351063, "lambda_min": 0.113746777176857, "lambda_max": 0.5738272070884705, "S_mean": 0.3205362955729167, "G_mean": 0.3518008942094942, "G_std": 0.16462746200462183, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.383671522140503, "grad_norm": 0.8825657963752747, "tokens_seen": 59318272, "tokens_per_sec": 18308.147913526584, "vram_gb": 5.684933632, "lambda": 0.27075411037852365, "lambda_min": 0.11380024254322052, "lambda_max": 0.5746175050735474, "S_mean": 0.3207295735677083, "G_mean": 0.3458780182215075, "G_std": 0.165247360477224, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.576082468032837, "grad_norm": 0.9112436175346375, "tokens_seen": 59482112, "tokens_per_sec": 18307.995630776317, "vram_gb": 5.684933632, "lambda": 0.2710619615390897, "lambda_min": 0.1138707846403122, "lambda_max": 0.5758807063102722, "S_mean": 0.3198191324869792, "G_mean": 0.34576650025943917, "G_std": 0.16252523777075112, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.4439005851745605, "grad_norm": 0.8835352659225464, "tokens_seen": 59645952, "tokens_per_sec": 18308.12919898883, "vram_gb": 5.684933632, "lambda": 0.2713607791811228, "lambda_min": 0.11385886371135712, "lambda_max": 0.5769028067588806, "S_mean": 0.3223978678385417, "G_mean": 0.3417631190580626, "G_std": 0.1631949671621745, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.449857950210571, "grad_norm": 0.8735646605491638, "tokens_seen": 59809792, "tokens_per_sec": 18308.33891160302, "vram_gb": 5.684933632, "lambda": 0.2716145838300387, "lambda_min": 0.11387628316879272, "lambda_max": 0.5780619382858276, "S_mean": 0.3345947265625, "G_mean": 0.3456472070732464, "G_std": 0.1640594482111434, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.597323894500732, "grad_norm": 0.8673101663589478, "tokens_seen": 59973632, "tokens_per_sec": 18308.233642139796, "vram_gb": 5.684933632, "lambda": 0.27185906500866014, "lambda_min": 0.11394067853689194, "lambda_max": 0.5788353681564331, "S_mean": 0.3301289876302083, "G_mean": 0.35261366454263526, "G_std": 0.16468346662198505, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.382306814193726, "grad_norm": 0.8066856861114502, "tokens_seen": 60137472, "tokens_per_sec": 18308.272911856075, "vram_gb": 5.684933632, "lambda": 0.2721202426279585, "lambda_min": 0.11395450681447983, "lambda_max": 0.5792979598045349, "S_mean": 0.3315938313802083, "G_mean": 0.34390215047945577, "G_std": 0.16344820288941264, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.343100070953369, "grad_norm": 0.8664368391036987, "tokens_seen": 60301312, "tokens_per_sec": 18308.47531275303, "vram_gb": 5.684933632, "lambda": 0.27236748735109967, "lambda_min": 0.11396033316850662, "lambda_max": 0.5799933671951294, "S_mean": 0.3429158528645833, "G_mean": 0.3472267137064288, "G_std": 0.16332448029424995, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.5737669467926025, "grad_norm": 0.8278705477714539, "tokens_seen": 60465152, "tokens_per_sec": 18308.465996390863, "vram_gb": 5.684933632, "lambda": 0.2726407693699002, "lambda_min": 0.11396972835063934, "lambda_max": 0.5810927152633667, "S_mean": 0.3337046305338542, "G_mean": 0.3504553808985899, "G_std": 0.1612116877998536, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.441937446594238, "grad_norm": 0.8371890187263489, "tokens_seen": 60628992, "tokens_per_sec": 18308.87825126592, "vram_gb": 5.684933632, "lambda": 0.27289222956945497, "lambda_min": 0.11397463083267212, "lambda_max": 0.5815757513046265, "S_mean": 0.3301595052083333, "G_mean": 0.3411280196160078, "G_std": 0.1635266765176008, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.6552979946136475, "grad_norm": 0.8429610729217529, "tokens_seen": 60792832, "tokens_per_sec": 18309.0939664139, "vram_gb": 5.684933632, "lambda": 0.2731316403175394, "lambda_min": 0.11401496082544327, "lambda_max": 0.5822544693946838, "S_mean": 0.3342692057291667, "G_mean": 0.3511813074971239, "G_std": 0.16415888257324696, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.427738666534424, "grad_norm": 0.8465979099273682, "tokens_seen": 60956672, "tokens_per_sec": 18309.208513263387, "vram_gb": 5.684933632, "lambda": 0.27346060518175364, "lambda_min": 0.11408253759145737, "lambda_max": 0.5831547379493713, "S_mean": 0.32513427734375, "G_mean": 0.3474676991657664, "G_std": 0.16387221690577766, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.5304059982299805, "grad_norm": 0.8897098898887634, "tokens_seen": 61120512, "tokens_per_sec": 18309.315795317572, "vram_gb": 5.684933632, "lambda": 0.27370642808576423, "lambda_min": 0.11404892057180405, "lambda_max": 0.5837045311927795, "S_mean": 0.3352864583333333, "G_mean": 0.3403428123953442, "G_std": 0.16375272115692496, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.512777090072632, "grad_norm": 0.8456519842147827, "tokens_seen": 61284352, "tokens_per_sec": 18309.35332858155, "vram_gb": 5.684933632, "lambda": 0.273967313580215, "lambda_min": 0.11412841081619263, "lambda_max": 0.584714949131012, "S_mean": 0.3222757975260417, "G_mean": 0.3409225351642817, "G_std": 0.16417862304175893, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.45744776725769, "grad_norm": 0.8602230548858643, "tokens_seen": 61448192, "tokens_per_sec": 18309.49670762958, "vram_gb": 5.684933632, "lambda": 0.27417553402483463, "lambda_min": 0.11407583206892014, "lambda_max": 0.5856832265853882, "S_mean": 0.3198191324869792, "G_mean": 0.34461744587558013, "G_std": 0.1624052165231357, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.343456029891968, "grad_norm": 0.8719291687011719, "tokens_seen": 61612032, "tokens_per_sec": 18309.483358150428, "vram_gb": 5.684933632, "lambda": 0.27432938509931165, "lambda_min": 0.11405840516090393, "lambda_max": 0.5865827202796936, "S_mean": 0.3259073893229167, "G_mean": 0.34491568446780246, "G_std": 0.16446113916269192, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.4452805519104, "grad_norm": 0.8705009818077087, "tokens_seen": 61775872, "tokens_per_sec": 18309.43377378159, "vram_gb": 5.684933632, "lambda": 0.2745937841633956, "lambda_min": 0.11414214223623276, "lambda_max": 0.5873732566833496, "S_mean": 0.3174235026041667, "G_mean": 0.34407577019495267, "G_std": 0.16337317957853278, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.580087900161743, "grad_norm": 0.8731511831283569, "tokens_seen": 61939712, "tokens_per_sec": 18309.604448048285, "vram_gb": 5.684933632, "lambda": 0.27488923724740744, "lambda_min": 0.1140364408493042, "lambda_max": 0.5883341431617737, "S_mean": 0.3251241048177083, "G_mean": 0.34745575363437337, "G_std": 0.16477778763510287, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.485240459442139, "grad_norm": 0.8695411682128906, "tokens_seen": 62103552, "tokens_per_sec": 18309.941337379925, "vram_gb": 5.684933632, "lambda": 0.275138265453279, "lambda_min": 0.11389511823654175, "lambda_max": 0.5886814594268799, "S_mean": 0.321563720703125, "G_mean": 0.34007783024571836, "G_std": 0.16517942173716924, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.472201347351074, "grad_norm": 0.8390256762504578, "tokens_seen": 62267392, "tokens_per_sec": 18309.831461127218, "vram_gb": 5.684933632, "lambda": 0.2753533609211445, "lambda_min": 0.11383084952831268, "lambda_max": 0.5893043875694275, "S_mean": 0.3190816243489583, "G_mean": 0.34357141936197877, "G_std": 0.16270933591295034, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.698488235473633, "grad_norm": 0.9539046287536621, "tokens_seen": 62431232, "tokens_per_sec": 18310.301646963107, "vram_gb": 5.684933632, "lambda": 0.27554303128272295, "lambda_min": 0.11382976919412613, "lambda_max": 0.5903827548027039, "S_mean": 0.3230031331380208, "G_mean": 0.3465550330777963, "G_std": 0.16303860838524997, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.426868677139282, "grad_norm": 0.9065605998039246, "tokens_seen": 62595072, "tokens_per_sec": 18310.094798937826, "vram_gb": 5.684933632, "lambda": 0.27578023355454206, "lambda_min": 0.11383264511823654, "lambda_max": 0.5909004807472229, "S_mean": 0.3203684488932292, "G_mean": 0.34294593582550686, "G_std": 0.16284248671339205, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.369337797164917, "grad_norm": 0.912496030330658, "tokens_seen": 62758912, "tokens_per_sec": 18309.91207262305, "vram_gb": 5.684933632, "lambda": 0.27607018190125626, "lambda_min": 0.11377501487731934, "lambda_max": 0.5915574431419373, "S_mean": 0.317901611328125, "G_mean": 0.34929618790435296, "G_std": 0.1638752610888332, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.323535203933716, "grad_norm": 0.8612839579582214, "tokens_seen": 62922752, "tokens_per_sec": 18310.19724586617, "vram_gb": 5.684933632, "lambda": 0.2763404094924529, "lambda_min": 0.11374475806951523, "lambda_max": 0.5926886796951294, "S_mean": 0.316925048828125, "G_mean": 0.3509070803411305, "G_std": 0.16294095226718733, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.460654973983765, "grad_norm": 0.8583738803863525, "tokens_seen": 63086592, "tokens_per_sec": 18310.37801892172, "vram_gb": 5.684933632, "lambda": 0.276559518960615, "lambda_min": 0.11349263787269592, "lambda_max": 0.5938400030136108, "S_mean": 0.345611572265625, "G_mean": 0.34671878232620656, "G_std": 0.16046730991608152, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.513431549072266, "grad_norm": 0.857861340045929, "tokens_seen": 63250432, "tokens_per_sec": 18310.656218864322, "vram_gb": 5.684933632, "lambda": 0.27682762779295444, "lambda_min": 0.11338090896606445, "lambda_max": 0.5947079658508301, "S_mean": 0.333160400390625, "G_mean": 0.3445802017425497, "G_std": 0.16305217608654252, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.409846305847168, "grad_norm": 0.914784848690033, "tokens_seen": 63414272, "tokens_per_sec": 18310.664582356883, "vram_gb": 5.684933632, "lambda": 0.27704560042669374, "lambda_min": 0.1132739782333374, "lambda_max": 0.594902753829956, "S_mean": 0.3204345703125, "G_mean": 0.34736425064814586, "G_std": 0.1649066396833708, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.545191049575806, "grad_norm": 0.9398573040962219, "tokens_seen": 63578112, "tokens_per_sec": 18310.797433497126, "vram_gb": 5.684933632, "lambda": 0.27727884457757074, "lambda_min": 0.11337976157665253, "lambda_max": 0.5957674384117126, "S_mean": 0.3195343017578125, "G_mean": 0.3515725485049188, "G_std": 0.1634280801129838, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.5954203605651855, "grad_norm": 0.9030051231384277, "tokens_seen": 63741952, "tokens_per_sec": 18311.116521776938, "vram_gb": 5.684933632, "lambda": 0.2775564383094509, "lambda_min": 0.1134127527475357, "lambda_max": 0.5966980457305908, "S_mean": 0.3270467122395833, "G_mean": 0.33264025393873453, "G_std": 0.16200712400798997, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.488631248474121, "grad_norm": 0.8780511617660522, "tokens_seen": 63905792, "tokens_per_sec": 18311.354521603287, "vram_gb": 5.684933632, "lambda": 0.27780077420175076, "lambda_min": 0.11349714547395706, "lambda_max": 0.5974187254905701, "S_mean": 0.325897216796875, "G_mean": 0.3463479704999675, "G_std": 0.16224328653576472, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.43180775642395, "grad_norm": 0.8515444993972778, "tokens_seen": 64069632, "tokens_per_sec": 18311.492353968355, "vram_gb": 5.684933632, "lambda": 0.27803632368644077, "lambda_min": 0.1134013757109642, "lambda_max": 0.597611665725708, "S_mean": 0.3244832356770833, "G_mean": 0.3506323369219899, "G_std": 0.16329276864416897, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.334449529647827, "grad_norm": 0.8161323666572571, "tokens_seen": 64233472, "tokens_per_sec": 18311.046162165167, "vram_gb": 5.684933632, "lambda": 0.278230636070172, "lambda_min": 0.11332020163536072, "lambda_max": 0.5981635451316833, "S_mean": 0.33551025390625, "G_mean": 0.34209092054516077, "G_std": 0.16233149787876755, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.408618211746216, "grad_norm": 0.908902108669281, "tokens_seen": 64397312, "tokens_per_sec": 18310.84690771024, "vram_gb": 5.684933632, "lambda": 0.27840364755441743, "lambda_min": 0.11342118680477142, "lambda_max": 0.5986472368240356, "S_mean": 0.3262379964192708, "G_mean": 0.34498072632898885, "G_std": 0.16241681145038456, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.474586486816406, "grad_norm": 0.9503316283226013, "tokens_seen": 64561152, "tokens_per_sec": 18311.143991530917, "vram_gb": 5.684933632, "lambda": 0.2786326926822464, "lambda_min": 0.11338141560554504, "lambda_max": 0.5996681451797485, "S_mean": 0.3389892578125, "G_mean": 0.3454666421748698, "G_std": 0.16386095329653472, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.444415807723999, "grad_norm": 0.8491029143333435, "tokens_seen": 64724992, "tokens_per_sec": 18311.05847675853, "vram_gb": 5.684933632, "lambda": 0.27885624797393876, "lambda_min": 0.11341426521539688, "lambda_max": 0.6004441976547241, "S_mean": 0.3182729085286458, "G_mean": 0.3414739913617571, "G_std": 0.16327770551045737, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.523522138595581, "grad_norm": 0.8698946237564087, "tokens_seen": 64888832, "tokens_per_sec": 18311.40223898455, "vram_gb": 5.684933632, "lambda": 0.2790412101894617, "lambda_min": 0.11348184198141098, "lambda_max": 0.6012282371520996, "S_mean": 0.3249918619791667, "G_mean": 0.34968565109496313, "G_std": 0.16411363620621464, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.433583736419678, "grad_norm": 0.9256979823112488, "tokens_seen": 65052672, "tokens_per_sec": 18311.56399471999, "vram_gb": 5.684933632, "lambda": 0.2792527923981349, "lambda_min": 0.11345066875219345, "lambda_max": 0.6018796563148499, "S_mean": 0.3375651041666667, "G_mean": 0.346094433683902, "G_std": 0.16069844900630414, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.492583513259888, "grad_norm": 0.8702196478843689, "tokens_seen": 65216512, "tokens_per_sec": 18311.94735356888, "vram_gb": 5.684933632, "lambda": 0.27943616081029177, "lambda_min": 0.11343409866094589, "lambda_max": 0.6023731231689453, "S_mean": 0.3283640543619792, "G_mean": 0.3391097190324217, "G_std": 0.16149987163953483, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.4707934856414795, "grad_norm": 0.8620835542678833, "tokens_seen": 65380352, "tokens_per_sec": 18312.234719921067, "vram_gb": 5.684933632, "lambda": 0.27962516620755196, "lambda_min": 0.1134989857673645, "lambda_max": 0.6033461093902588, "S_mean": 0.325653076171875, "G_mean": 0.33773344111007947, "G_std": 0.16183195049719265, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.423211574554443, "grad_norm": 0.8839797973632812, "tokens_seen": 65544192, "tokens_per_sec": 18312.414812626837, "vram_gb": 5.684933632, "lambda": 0.2798181405911843, "lambda_min": 0.11349427700042725, "lambda_max": 0.6040685176849365, "S_mean": 0.3370869954427083, "G_mean": 0.341323188583677, "G_std": 0.16160004345389703, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.461314749717713, "val_ppl": 86.60129330698858, "lambda": 0.27983039741714794, "lambda_min": 0.11349616944789886, "lambda_max": 0.6040905117988586, "lambda_per_layer": [0.17081336677074432, 0.14570900797843933, 0.13945326209068298, 0.12443449348211288, 0.13821165263652802, 0.12736789882183075, 0.11349616944789886, 0.11837153136730194, 0.11788817495107651, 0.15767288208007812, 0.29447418451309204, 0.4129263758659363, 0.4499932527542114, 0.43401584029197693, 0.6040905117988586, 0.34437188506126404, 0.5886150598526001, 0.3123815357685089, 0.5475848317146301, 0.3049620985984802, 0.2975400984287262, 0.3437737226486206, 0.2491237074136734, 0.17865799367427826], "S_mean": 0.34576416015625, "G_mean": 0.3478023901892205, "G_std": 0.1641885726712644, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.483109951019287, "grad_norm": 0.8756827712059021, "tokens_seen": 65708032, "tokens_per_sec": 18304.17820385937, "vram_gb": 5.684933632, "lambda": 0.2800451160098116, "lambda_min": 0.1135382354259491, "lambda_max": 0.6048858761787415, "S_mean": 0.3202718098958333, "G_mean": 0.3455191315151751, "G_std": 0.16244023114753267, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.510156631469727, "grad_norm": 0.8772358298301697, "tokens_seen": 65871872, "tokens_per_sec": 18304.14996205503, "vram_gb": 5.684933632, "lambda": 0.2802237846578161, "lambda_min": 0.11351440846920013, "lambda_max": 0.6054936051368713, "S_mean": 0.3208719889322917, "G_mean": 0.33901863413242, "G_std": 0.16315601230598986, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.3731958866119385, "grad_norm": 0.8571100234985352, "tokens_seen": 66035712, "tokens_per_sec": 18303.985284974653, "vram_gb": 5.684933632, "lambda": 0.2804586937030156, "lambda_min": 0.11361277103424072, "lambda_max": 0.6060099005699158, "S_mean": 0.3238016764322917, "G_mean": 0.34125416663785774, "G_std": 0.16225703918219855, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.443458557128906, "grad_norm": 0.8461475372314453, "tokens_seen": 66199552, "tokens_per_sec": 18301.83348608565, "vram_gb": 5.684933632, "lambda": 0.2807151038820545, "lambda_min": 0.11371053755283356, "lambda_max": 0.6065605282783508, "S_mean": 0.3342692057291667, "G_mean": 0.34719248100494343, "G_std": 0.16171848572169742, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.474703788757324, "grad_norm": 0.887431263923645, "tokens_seen": 66363392, "tokens_per_sec": 18301.562034982166, "vram_gb": 5.684933632, "lambda": 0.2809012910972039, "lambda_min": 0.113754503428936, "lambda_max": 0.6071889400482178, "S_mean": 0.3226064046223958, "G_mean": 0.34576702358511585, "G_std": 0.16333547607064247, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.522934913635254, "grad_norm": 0.8817707896232605, "tokens_seen": 66527232, "tokens_per_sec": 18301.4257509449, "vram_gb": 5.684933632, "lambda": 0.2811124635239442, "lambda_min": 0.11376599222421646, "lambda_max": 0.6079167723655701, "S_mean": 0.3347371419270833, "G_mean": 0.3453328316099942, "G_std": 0.16318006340103844, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.418008327484131, "grad_norm": 0.8298039436340332, "tokens_seen": 66691072, "tokens_per_sec": 18301.729005855086, "vram_gb": 5.684933632, "lambda": 0.28133445885032415, "lambda_min": 0.11377262324094772, "lambda_max": 0.6082577109336853, "S_mean": 0.3094889322916667, "G_mean": 0.34428204378734034, "G_std": 0.16245077965625873, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.553967475891113, "grad_norm": 0.863016664981842, "tokens_seen": 66854912, "tokens_per_sec": 18301.3696543675, "vram_gb": 5.684933632, "lambda": 0.28156424841533106, "lambda_min": 0.11370567232370377, "lambda_max": 0.6084107756614685, "S_mean": 0.3187510172526042, "G_mean": 0.3436093144118786, "G_std": 0.16188524360768497, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.532091379165649, "grad_norm": 0.9361913204193115, "tokens_seen": 67018752, "tokens_per_sec": 18301.58134492828, "vram_gb": 5.684933632, "lambda": 0.2817673208191991, "lambda_min": 0.11365456879138947, "lambda_max": 0.6091267466545105, "S_mean": 0.3391316731770833, "G_mean": 0.3430606619610141, "G_std": 0.16426182631403208, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.487440824508667, "grad_norm": 0.8649386763572693, "tokens_seen": 67182592, "tokens_per_sec": 18301.853198515702, "vram_gb": 5.684933632, "lambda": 0.2819822688276569, "lambda_min": 0.11358944326639175, "lambda_max": 0.6100109815597534, "S_mean": 0.3230489095052083, "G_mean": 0.34271760533253354, "G_std": 0.16309669183101505, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.5120155811309814, "grad_norm": 0.9856757521629333, "tokens_seen": 67346432, "tokens_per_sec": 18302.236037746185, "vram_gb": 5.684933632, "lambda": 0.2822049530223012, "lambda_min": 0.11355844885110855, "lambda_max": 0.6106610894203186, "S_mean": 0.3250223795572917, "G_mean": 0.34834318530435365, "G_std": 0.1629119652789086, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.409685850143433, "grad_norm": 0.951300323009491, "tokens_seen": 67510272, "tokens_per_sec": 18302.288465151738, "vram_gb": 5.684933632, "lambda": 0.28237522145112354, "lambda_min": 0.11360245198011398, "lambda_max": 0.6112634539604187, "S_mean": 0.3283284505208333, "G_mean": 0.3425897490233183, "G_std": 0.1631835107691586, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.449405670166016, "grad_norm": 0.8804646730422974, "tokens_seen": 67674112, "tokens_per_sec": 18302.47015592654, "vram_gb": 5.684933632, "lambda": 0.2825614387790362, "lambda_min": 0.11362731456756592, "lambda_max": 0.6119053363800049, "S_mean": 0.3328959147135417, "G_mean": 0.34251963269586366, "G_std": 0.16253970144316554, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.442147254943848, "grad_norm": 0.8800201416015625, "tokens_seen": 67837952, "tokens_per_sec": 18302.574004357717, "vram_gb": 5.684933632, "lambda": 0.2827739864587784, "lambda_min": 0.11366160959005356, "lambda_max": 0.6126548051834106, "S_mean": 0.3355611165364583, "G_mean": 0.3436471870324264, "G_std": 0.16301634923244515, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.492730379104614, "grad_norm": 0.9391375184059143, "tokens_seen": 68001792, "tokens_per_sec": 18302.36153183223, "vram_gb": 5.684933632, "lambda": 0.2829879376416405, "lambda_min": 0.1135336235165596, "lambda_max": 0.6133534908294678, "S_mean": 0.3332722981770833, "G_mean": 0.34521379228681326, "G_std": 0.16309270286001265, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.415581941604614, "grad_norm": 0.908284068107605, "tokens_seen": 68165632, "tokens_per_sec": 18302.437271174553, "vram_gb": 5.684933632, "lambda": 0.2831706767901778, "lambda_min": 0.11356225609779358, "lambda_max": 0.6139476299285889, "S_mean": 0.3259073893229167, "G_mean": 0.3452549030383428, "G_std": 0.16433781975259384, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.469374179840088, "grad_norm": 0.8650596141815186, "tokens_seen": 68329472, "tokens_per_sec": 18302.749508528126, "vram_gb": 5.684933632, "lambda": 0.2833177875727415, "lambda_min": 0.11359068006277084, "lambda_max": 0.6147544384002686, "S_mean": 0.3216501871744792, "G_mean": 0.33958334289491177, "G_std": 0.16114373276165375, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.476213216781616, "grad_norm": 0.868923008441925, "tokens_seen": 68493312, "tokens_per_sec": 18302.668242540636, "vram_gb": 5.684933632, "lambda": 0.28351906811197597, "lambda_min": 0.11362040787935257, "lambda_max": 0.6151278614997864, "S_mean": 0.3163401285807292, "G_mean": 0.34123490176474053, "G_std": 0.16223661116479585, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.415838241577148, "grad_norm": 0.8691690564155579, "tokens_seen": 68657152, "tokens_per_sec": 18302.945494251842, "vram_gb": 5.684933632, "lambda": 0.28374673519283533, "lambda_min": 0.11358851939439774, "lambda_max": 0.6157373189926147, "S_mean": 0.3297322591145833, "G_mean": 0.3427600240490089, "G_std": 0.16129260626621544, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.381198406219482, "grad_norm": 0.9214006662368774, "tokens_seen": 68820992, "tokens_per_sec": 18302.919894392777, "vram_gb": 5.684933632, "lambda": 0.28394926711916924, "lambda_min": 0.11364870518445969, "lambda_max": 0.616358757019043, "S_mean": 0.3345438639322917, "G_mean": 0.34632193025512, "G_std": 0.16180062751906613, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.556975841522217, "grad_norm": 0.9081931114196777, "tokens_seen": 68984832, "tokens_per_sec": 18302.870080859993, "vram_gb": 5.684933632, "lambda": 0.2841445741554101, "lambda_min": 0.11371218413114548, "lambda_max": 0.6170262098312378, "S_mean": 0.322418212890625, "G_mean": 0.3454383021841447, "G_std": 0.162917559617199, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.449560642242432, "grad_norm": 0.9173803329467773, "tokens_seen": 69148672, "tokens_per_sec": 18302.84347948575, "vram_gb": 5.684933632, "lambda": 0.28432086296379566, "lambda_min": 0.11365549266338348, "lambda_max": 0.6177599430084229, "S_mean": 0.324493408203125, "G_mean": 0.3401517115222911, "G_std": 0.16357757698278874, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.40029239654541, "grad_norm": 0.8982884883880615, "tokens_seen": 69312512, "tokens_per_sec": 18303.282187592795, "vram_gb": 5.684933632, "lambda": 0.28454557651033, "lambda_min": 0.11361292004585266, "lambda_max": 0.6181635856628418, "S_mean": 0.3327229817708333, "G_mean": 0.3425490638862054, "G_std": 0.16391996596939862, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.3572564125061035, "grad_norm": 0.8454310894012451, "tokens_seen": 69476352, "tokens_per_sec": 18303.514135462494, "vram_gb": 5.684933632, "lambda": 0.2847661593308051, "lambda_min": 0.11359920352697372, "lambda_max": 0.6186743378639221, "S_mean": 0.3251851399739583, "G_mean": 0.33732015592977405, "G_std": 0.1600413373671472, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.3746538162231445, "grad_norm": 0.8640390038490295, "tokens_seen": 69640192, "tokens_per_sec": 18303.740652977292, "vram_gb": 5.684933632, "lambda": 0.2849052483215928, "lambda_min": 0.1135992780327797, "lambda_max": 0.6189064383506775, "S_mean": 0.318695068359375, "G_mean": 0.3368231837327282, "G_std": 0.16208745446056128, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.5093464851379395, "grad_norm": 0.9045097231864929, "tokens_seen": 69804032, "tokens_per_sec": 18303.93518620324, "vram_gb": 5.684933632, "lambda": 0.2850850997492671, "lambda_min": 0.11362975090742111, "lambda_max": 0.6193546652793884, "S_mean": 0.3251444498697917, "G_mean": 0.3454651453066617, "G_std": 0.16331426023195186, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.573728084564209, "grad_norm": 1.080008625984192, "tokens_seen": 69967872, "tokens_per_sec": 18303.86661521794, "vram_gb": 5.684933632, "lambda": 0.28532285677889985, "lambda_min": 0.1137026771903038, "lambda_max": 0.6201719045639038, "S_mean": 0.333282470703125, "G_mean": 0.3492842603009194, "G_std": 0.16382385115139186, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.301100015640259, "grad_norm": 0.9448833465576172, "tokens_seen": 70131712, "tokens_per_sec": 18303.91107720981, "vram_gb": 5.684933632, "lambda": 0.28551208848754567, "lambda_min": 0.1137518435716629, "lambda_max": 0.6209668517112732, "S_mean": 0.3353729248046875, "G_mean": 0.3420008389900128, "G_std": 0.16269725134285787, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.471673488616943, "grad_norm": 0.866918683052063, "tokens_seen": 70295552, "tokens_per_sec": 18303.639431609627, "vram_gb": 5.684933632, "lambda": 0.28568725877751905, "lambda_min": 0.11369120329618454, "lambda_max": 0.6212416887283325, "S_mean": 0.3206787109375, "G_mean": 0.3427431220188737, "G_std": 0.1616846399459367, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.363693714141846, "grad_norm": 0.902588963508606, "tokens_seen": 70459392, "tokens_per_sec": 18304.032963645957, "vram_gb": 5.684933632, "lambda": 0.28584979691853124, "lambda_min": 0.11368819326162338, "lambda_max": 0.6215593218803406, "S_mean": 0.3396809895833333, "G_mean": 0.34264243106978637, "G_std": 0.1614094361041983, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.3837199211120605, "grad_norm": 0.8802580833435059, "tokens_seen": 70623232, "tokens_per_sec": 18303.703746857, "vram_gb": 5.684933632, "lambda": 0.28602315795918304, "lambda_min": 0.11368390172719955, "lambda_max": 0.6218884587287903, "S_mean": 0.3318074544270833, "G_mean": 0.3388632466085255, "G_std": 0.16172228881623596, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.37570333480835, "grad_norm": 0.9114189147949219, "tokens_seen": 70787072, "tokens_per_sec": 18303.566590626397, "vram_gb": 5.684933632, "lambda": 0.2862177149703105, "lambda_min": 0.11373566091060638, "lambda_max": 0.6224699020385742, "S_mean": 0.3166046142578125, "G_mean": 0.3360674226035674, "G_std": 0.16166073099399605, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.435862064361572, "grad_norm": 0.9091727137565613, "tokens_seen": 70950912, "tokens_per_sec": 18303.802030754854, "vram_gb": 5.684933632, "lambda": 0.28639540790269774, "lambda_min": 0.11378367990255356, "lambda_max": 0.6228268146514893, "S_mean": 0.3288675944010417, "G_mean": 0.34096252320644754, "G_std": 0.1620210357553636, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.307260990142822, "grad_norm": 0.919672966003418, "tokens_seen": 71114752, "tokens_per_sec": 18303.69007212249, "vram_gb": 5.684933632, "lambda": 0.28655648169418174, "lambda_min": 0.11389072239398956, "lambda_max": 0.6232487559318542, "S_mean": 0.335052490234375, "G_mean": 0.3462958197730283, "G_std": 0.161753670234854, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.187216520309448, "grad_norm": 0.8728259801864624, "tokens_seen": 71278592, "tokens_per_sec": 18303.94980087737, "vram_gb": 5.684933632, "lambda": 0.2867042403668165, "lambda_min": 0.11397712677717209, "lambda_max": 0.6241298913955688, "S_mean": 0.3253936767578125, "G_mean": 0.3338698341976851, "G_std": 0.16237147633607188, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.366339683532715, "grad_norm": 0.9321828484535217, "tokens_seen": 71442432, "tokens_per_sec": 18304.335448071233, "vram_gb": 5.684933632, "lambda": 0.28689033165574074, "lambda_min": 0.11401662975549698, "lambda_max": 0.6245800256729126, "S_mean": 0.3347880045572917, "G_mean": 0.33544097864069045, "G_std": 0.16032502811867744, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.425334453582764, "grad_norm": 0.891732931137085, "tokens_seen": 71606272, "tokens_per_sec": 18304.440507950654, "vram_gb": 5.684933632, "lambda": 0.2870753730336825, "lambda_min": 0.11395881325006485, "lambda_max": 0.6254270076751709, "S_mean": 0.3239542643229167, "G_mean": 0.3406755831092596, "G_std": 0.16161984073308608, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.521371126174927, "grad_norm": 0.9075690507888794, "tokens_seen": 71770112, "tokens_per_sec": 18304.922730245707, "vram_gb": 5.684933632, "lambda": 0.2872346993535757, "lambda_min": 0.11393603682518005, "lambda_max": 0.6257383227348328, "S_mean": 0.3191731770833333, "G_mean": 0.3465053917995344, "G_std": 0.16324763551043966, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.531110525131226, "grad_norm": 0.9262644648551941, "tokens_seen": 71933952, "tokens_per_sec": 18304.950910633117, "vram_gb": 5.684933632, "lambda": 0.2873885929584503, "lambda_min": 0.11397518962621689, "lambda_max": 0.6260592341423035, "S_mean": 0.3252156575520833, "G_mean": 0.33731718532120186, "G_std": 0.16157403402030468, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.340263366699219, "grad_norm": 0.893042802810669, "tokens_seen": 72097792, "tokens_per_sec": 18305.192089817174, "vram_gb": 5.684933632, "lambda": 0.28755657747387886, "lambda_min": 0.11403324455022812, "lambda_max": 0.6266574859619141, "S_mean": 0.3289693196614583, "G_mean": 0.33812927291728556, "G_std": 0.16206323308870196, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.374868392944336, "grad_norm": 0.9786997437477112, "tokens_seen": 72261632, "tokens_per_sec": 18305.26169644228, "vram_gb": 5.684933632, "lambda": 0.2877129102125764, "lambda_min": 0.11411146074533463, "lambda_max": 0.6271061897277832, "S_mean": 0.33251953125, "G_mean": 0.3362420320821305, "G_std": 0.16114116758884242, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.502784967422485, "grad_norm": 0.8667539358139038, "tokens_seen": 72425472, "tokens_per_sec": 18305.279754416362, "vram_gb": 5.684933632, "lambda": 0.287870019984742, "lambda_min": 0.11411493271589279, "lambda_max": 0.6275680065155029, "S_mean": 0.3290812174479167, "G_mean": 0.335821316887935, "G_std": 0.16217199127034596, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.380628824234009, "grad_norm": 0.894866406917572, "tokens_seen": 72589312, "tokens_per_sec": 18305.607072422947, "vram_gb": 5.684933632, "lambda": 0.28801581387718517, "lambda_min": 0.11414738744497299, "lambda_max": 0.6280460953712463, "S_mean": 0.3213094075520833, "G_mean": 0.3378468573403855, "G_std": 0.16118082555476576, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.395124435424805, "grad_norm": 0.9044979214668274, "tokens_seen": 72753152, "tokens_per_sec": 18305.736628735278, "vram_gb": 5.684933632, "lambda": 0.2881877450272441, "lambda_min": 0.11413386464118958, "lambda_max": 0.6287338733673096, "S_mean": 0.3253428141276042, "G_mean": 0.33469846736018855, "G_std": 0.16093025379814208, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.458606958389282, "grad_norm": 0.9075219035148621, "tokens_seen": 72916992, "tokens_per_sec": 18305.749830381235, "vram_gb": 5.684933632, "lambda": 0.28834663207332295, "lambda_min": 0.11414337158203125, "lambda_max": 0.629194438457489, "S_mean": 0.3243204752604167, "G_mean": 0.3401527948056658, "G_std": 0.16027774836402386, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.364114999771118, "grad_norm": 0.9163205027580261, "tokens_seen": 73080832, "tokens_per_sec": 18305.91327269661, "vram_gb": 5.684933632, "lambda": 0.28851402923464775, "lambda_min": 0.11414242535829544, "lambda_max": 0.6295562386512756, "S_mean": 0.3311258951822917, "G_mean": 0.33773778681643307, "G_std": 0.16163912027453384, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.437330722808838, "grad_norm": 0.8803932070732117, "tokens_seen": 73244672, "tokens_per_sec": 18306.06506974988, "vram_gb": 5.684933632, "lambda": 0.2887246481453379, "lambda_min": 0.11416911333799362, "lambda_max": 0.6301714181900024, "S_mean": 0.3280385335286458, "G_mean": 0.33842016554748017, "G_std": 0.16187836497556418, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.40770959854126, "grad_norm": 0.9307950735092163, "tokens_seen": 73408512, "tokens_per_sec": 18306.47211867866, "vram_gb": 5.684933632, "lambda": 0.28890900282810134, "lambda_min": 0.11418285220861435, "lambda_max": 0.6308135986328125, "S_mean": 0.3141886393229167, "G_mean": 0.34036789066158235, "G_std": 0.16210928667957583, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.317036867141724, "grad_norm": 0.8867979049682617, "tokens_seen": 73572352, "tokens_per_sec": 18306.7852132875, "vram_gb": 5.684933632, "lambda": 0.28903149751325447, "lambda_min": 0.11423196643590927, "lambda_max": 0.6311702728271484, "S_mean": 0.3254292805989583, "G_mean": 0.34201516350731254, "G_std": 0.16240525889831284, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.350869178771973, "grad_norm": 0.9256718754768372, "tokens_seen": 73736192, "tokens_per_sec": 18307.0478791545, "vram_gb": 5.684933632, "lambda": 0.28915111379077035, "lambda_min": 0.1142454445362091, "lambda_max": 0.6317964196205139, "S_mean": 0.3338724772135417, "G_mean": 0.3391826833443095, "G_std": 0.1614746544510126, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.4913547992706295, "val_ppl": 89.24226937584467, "lambda": 0.2891586081435283, "lambda_min": 0.11424443870782852, "lambda_max": 0.6318284869194031, "lambda_per_layer": [0.1722801923751831, 0.14654728770256042, 0.1400623470544815, 0.1245713010430336, 0.1390732079744339, 0.12735052406787872, 0.11424443870782852, 0.11858312785625458, 0.11744076013565063, 0.15982645750045776, 0.29868999123573303, 0.42122364044189453, 0.46140211820602417, 0.45171236991882324, 0.6318284869194031, 0.361054390668869, 0.6165632605552673, 0.3252534568309784, 0.5762724280357361, 0.3198678493499756, 0.3112622797489166, 0.36181071400642395, 0.26052796840667725, 0.1823579967021942], "S_mean": 0.33642578125, "G_mean": 0.34129032865166664, "G_std": 0.16357715835329145, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.367238283157349, "grad_norm": 0.9093368649482727, "tokens_seen": 73900032, "tokens_per_sec": 18300.01350782108, "vram_gb": 5.684933632, "lambda": 0.28927144470314187, "lambda_min": 0.11417198181152344, "lambda_max": 0.632455587387085, "S_mean": 0.3372701009114583, "G_mean": 0.33862423004272085, "G_std": 0.16242493440707526, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.524964332580566, "grad_norm": 0.9472259879112244, "tokens_seen": 74063872, "tokens_per_sec": 18300.97673148249, "vram_gb": 5.684933632, "lambda": 0.2893817185734709, "lambda_min": 0.1141115128993988, "lambda_max": 0.6327232122421265, "S_mean": 0.3267618815104167, "G_mean": 0.34088653915872175, "G_std": 0.16167171664225558, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.3942649364471436, "grad_norm": 0.9216307401657104, "tokens_seen": 74227712, "tokens_per_sec": 18300.871047131455, "vram_gb": 5.684933632, "lambda": 0.2895372382675608, "lambda_min": 0.1141158863902092, "lambda_max": 0.6332277059555054, "S_mean": 0.3260854085286458, "G_mean": 0.34071164710136753, "G_std": 0.16333121332960823, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.307258367538452, "grad_norm": 0.8684093356132507, "tokens_seen": 74391552, "tokens_per_sec": 18300.50875020829, "vram_gb": 5.684933632, "lambda": 0.28968385327607393, "lambda_min": 0.11415998637676239, "lambda_max": 0.6334114670753479, "S_mean": 0.3262125651041667, "G_mean": 0.3396356809729089, "G_std": 0.16076700994744897, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.447652339935303, "grad_norm": 1.0110864639282227, "tokens_seen": 74555392, "tokens_per_sec": 18300.583649149015, "vram_gb": 5.684933632, "lambda": 0.28984434126565856, "lambda_min": 0.11419417709112167, "lambda_max": 0.6336227655410767, "S_mean": 0.3373463948567708, "G_mean": 0.3415426362771541, "G_std": 0.1617933272694548, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.363403797149658, "grad_norm": 0.9229015111923218, "tokens_seen": 74719232, "tokens_per_sec": 18300.763749256577, "vram_gb": 5.684933632, "lambda": 0.28998504144450027, "lambda_min": 0.11419116705656052, "lambda_max": 0.6340071558952332, "S_mean": 0.3435770670572917, "G_mean": 0.34391885142152506, "G_std": 0.1600830099390199, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.292315244674683, "grad_norm": 0.9122387766838074, "tokens_seen": 74883072, "tokens_per_sec": 18301.15204685212, "vram_gb": 5.684933632, "lambda": 0.2901645516976714, "lambda_min": 0.11423617601394653, "lambda_max": 0.6346235275268555, "S_mean": 0.3361612955729167, "G_mean": 0.33488942504239577, "G_std": 0.16103709256276488, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.43886399269104, "grad_norm": 0.9433947801589966, "tokens_seen": 75046912, "tokens_per_sec": 18301.278999913997, "vram_gb": 5.684933632, "lambda": 0.2903410258392493, "lambda_min": 0.11420734971761703, "lambda_max": 0.6351687908172607, "S_mean": 0.34033203125, "G_mean": 0.3386434703910102, "G_std": 0.16170846189682683, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.3573198318481445, "grad_norm": 1.0251346826553345, "tokens_seen": 75210752, "tokens_per_sec": 18301.462593848566, "vram_gb": 5.684933632, "lambda": 0.29047109310825664, "lambda_min": 0.11422722786664963, "lambda_max": 0.6357074975967407, "S_mean": 0.3357035319010417, "G_mean": 0.3450638285527627, "G_std": 0.16152816479249546, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.468571424484253, "grad_norm": 0.9500181674957275, "tokens_seen": 75374592, "tokens_per_sec": 18301.189089169147, "vram_gb": 5.684933632, "lambda": 0.29060574527829885, "lambda_min": 0.1141866073012352, "lambda_max": 0.6361148953437805, "S_mean": 0.330841064453125, "G_mean": 0.3408366416115314, "G_std": 0.16102249334411076, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.314281940460205, "grad_norm": 0.9708465337753296, "tokens_seen": 75538432, "tokens_per_sec": 18300.942667573716, "vram_gb": 5.684933632, "lambda": 0.290707743105789, "lambda_min": 0.11421546339988708, "lambda_max": 0.6361236572265625, "S_mean": 0.3745930989583333, "G_mean": 0.33629184992363054, "G_std": 0.16022941749542952, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.355479717254639, "grad_norm": 0.9596006274223328, "tokens_seen": 75702272, "tokens_per_sec": 18300.973937089708, "vram_gb": 5.684933632, "lambda": 0.29084125130126876, "lambda_min": 0.11429928243160248, "lambda_max": 0.6364753246307373, "S_mean": 0.3298848470052083, "G_mean": 0.3358357850617419, "G_std": 0.16112796822562814, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.349143028259277, "grad_norm": 0.8840278387069702, "tokens_seen": 75866112, "tokens_per_sec": 18301.141126146606, "vram_gb": 5.684933632, "lambda": 0.29100487505396205, "lambda_min": 0.11435738205909729, "lambda_max": 0.6369671821594238, "S_mean": 0.3367563883463542, "G_mean": 0.3334297842035691, "G_std": 0.16041616381456456, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.218531608581543, "grad_norm": 0.9329987168312073, "tokens_seen": 76029952, "tokens_per_sec": 18300.883207732455, "vram_gb": 5.684933632, "lambda": 0.29116071357081336, "lambda_min": 0.11441399902105331, "lambda_max": 0.6372325420379639, "S_mean": 0.3277333577473958, "G_mean": 0.3412368368978302, "G_std": 0.16174372021729747, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.458051443099976, "grad_norm": 1.0093456506729126, "tokens_seen": 76193792, "tokens_per_sec": 18301.187758036787, "vram_gb": 5.684933632, "lambda": 0.29130803669492406, "lambda_min": 0.11436179280281067, "lambda_max": 0.6375943422317505, "S_mean": 0.3348846435546875, "G_mean": 0.3452843878573428, "G_std": 0.16309294469344118, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.330323219299316, "grad_norm": 0.9421985745429993, "tokens_seen": 76357632, "tokens_per_sec": 18301.166579170662, "vram_gb": 5.684933632, "lambda": 0.2914233735452096, "lambda_min": 0.11433680355548859, "lambda_max": 0.6380197405815125, "S_mean": 0.3410441080729167, "G_mean": 0.3435021953191608, "G_std": 0.16174764248232046, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.350874900817871, "grad_norm": 0.9199778437614441, "tokens_seen": 76521472, "tokens_per_sec": 18301.183857247186, "vram_gb": 5.684933632, "lambda": 0.2915653583283226, "lambda_min": 0.11436529457569122, "lambda_max": 0.6384995579719543, "S_mean": 0.3248087565104167, "G_mean": 0.33519791966925067, "G_std": 0.16266747075133026, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.100013017654419, "grad_norm": 0.8985541462898254, "tokens_seen": 76685312, "tokens_per_sec": 18301.190714907763, "vram_gb": 5.684933632, "lambda": 0.2916759125267466, "lambda_min": 0.11440327763557434, "lambda_max": 0.6389003992080688, "S_mean": 0.3400777180989583, "G_mean": 0.3326109075763573, "G_std": 0.15929986233823, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.498483419418335, "grad_norm": 1.003814935684204, "tokens_seen": 76849152, "tokens_per_sec": 18301.402851245886, "vram_gb": 5.684933632, "lambda": 0.2918123643224438, "lambda_min": 0.11436506360769272, "lambda_max": 0.6394022703170776, "S_mean": 0.332611083984375, "G_mean": 0.3454054951046904, "G_std": 0.16133100163036337, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.423384666442871, "grad_norm": 0.9119575619697571, "tokens_seen": 77012992, "tokens_per_sec": 18301.33110522993, "vram_gb": 5.684933632, "lambda": 0.29192141722887754, "lambda_min": 0.11439462751150131, "lambda_max": 0.6398098468780518, "S_mean": 0.33917236328125, "G_mean": 0.3417293028129886, "G_std": 0.16070789048292985, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.356496572494507, "grad_norm": 0.9783031344413757, "tokens_seen": 77176832, "tokens_per_sec": 18301.185188947038, "vram_gb": 5.684933632, "lambda": 0.29202832467854023, "lambda_min": 0.11447539925575256, "lambda_max": 0.6399737596511841, "S_mean": 0.33935546875, "G_mean": 0.34317640005610883, "G_std": 0.1616378865437582, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.271456480026245, "grad_norm": 0.9199460744857788, "tokens_seen": 77340672, "tokens_per_sec": 18301.40917815183, "vram_gb": 5.684933632, "lambda": 0.2921571529780825, "lambda_min": 0.11451836675405502, "lambda_max": 0.6403587460517883, "S_mean": 0.335113525390625, "G_mean": 0.33100863088232774, "G_std": 0.1602012509635339, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.435890436172485, "grad_norm": 0.8741987347602844, "tokens_seen": 77504512, "tokens_per_sec": 18301.51647282974, "vram_gb": 5.684933632, "lambda": 0.2922652779767911, "lambda_min": 0.11453954875469208, "lambda_max": 0.6407881379127502, "S_mean": 0.347686767578125, "G_mean": 0.33791570202447474, "G_std": 0.160769082683449, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.252845287322998, "grad_norm": 1.0004546642303467, "tokens_seen": 77668352, "tokens_per_sec": 18301.75690110379, "vram_gb": 5.684933632, "lambda": 0.29236923065036535, "lambda_min": 0.11456838995218277, "lambda_max": 0.6412393450737, "S_mean": 0.3306884765625, "G_mean": 0.3376257074220727, "G_std": 0.16175050614401698, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.421686172485352, "grad_norm": 0.9353110790252686, "tokens_seen": 77832192, "tokens_per_sec": 18301.825321952114, "vram_gb": 5.684933632, "lambda": 0.2925151623785496, "lambda_min": 0.11461702734231949, "lambda_max": 0.6416314840316772, "S_mean": 0.3392130533854167, "G_mean": 0.3401383471985658, "G_std": 0.16234675236046314, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.2761900424957275, "grad_norm": 0.9133639931678772, "tokens_seen": 77996032, "tokens_per_sec": 18301.71779846117, "vram_gb": 5.684933632, "lambda": 0.29266442855199176, "lambda_min": 0.11462309956550598, "lambda_max": 0.6418830156326294, "S_mean": 0.3262532552083333, "G_mean": 0.3382982169277966, "G_std": 0.16083147931688777, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.253673076629639, "grad_norm": 0.9387442469596863, "tokens_seen": 78159872, "tokens_per_sec": 18302.00435581536, "vram_gb": 5.684933632, "lambda": 0.2928137170771758, "lambda_min": 0.11458858102560043, "lambda_max": 0.6422836780548096, "S_mean": 0.329742431640625, "G_mean": 0.33977019755790633, "G_std": 0.1622970807754124, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.291926622390747, "grad_norm": 0.9126749038696289, "tokens_seen": 78323712, "tokens_per_sec": 18302.290428836088, "vram_gb": 5.684933632, "lambda": 0.2929294751957059, "lambda_min": 0.11452137678861618, "lambda_max": 0.6426612734794617, "S_mean": 0.3488667805989583, "G_mean": 0.33709967974573374, "G_std": 0.15995111200027168, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.2066686153411865, "grad_norm": 0.8809450268745422, "tokens_seen": 78487552, "tokens_per_sec": 18302.736682850446, "vram_gb": 5.684933632, "lambda": 0.29305023948351544, "lambda_min": 0.11446040123701096, "lambda_max": 0.643025815486908, "S_mean": 0.3294016520182292, "G_mean": 0.3303916146978736, "G_std": 0.15954786341171712, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.3627800941467285, "grad_norm": 0.9575546979904175, "tokens_seen": 78651392, "tokens_per_sec": 18302.682243626157, "vram_gb": 5.684933632, "lambda": 0.29315228822330636, "lambda_min": 0.11446914076805115, "lambda_max": 0.6431084275245667, "S_mean": 0.3303426106770833, "G_mean": 0.3390725862700492, "G_std": 0.16124684732252112, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.4498090744018555, "grad_norm": 0.9017840027809143, "tokens_seen": 78815232, "tokens_per_sec": 18302.81575088233, "vram_gb": 5.684933632, "lambda": 0.29326609211663407, "lambda_min": 0.11450033634901047, "lambda_max": 0.6433915495872498, "S_mean": 0.3433837890625, "G_mean": 0.33715248891773325, "G_std": 0.15968531377923986, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.385221242904663, "grad_norm": 0.8962880373001099, "tokens_seen": 78979072, "tokens_per_sec": 18303.032888290498, "vram_gb": 5.684933632, "lambda": 0.293414739271005, "lambda_min": 0.11454196274280548, "lambda_max": 0.6439452767372131, "S_mean": 0.3355611165364583, "G_mean": 0.3335470393455277, "G_std": 0.16117389550587782, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.244266033172607, "grad_norm": 0.9677070379257202, "tokens_seen": 79142912, "tokens_per_sec": 18303.184803533146, "vram_gb": 5.684933632, "lambda": 0.2935737905402978, "lambda_min": 0.11452579498291016, "lambda_max": 0.6444349884986877, "S_mean": 0.328399658203125, "G_mean": 0.3376707124213378, "G_std": 0.1595427270124977, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.208378791809082, "grad_norm": 0.9632114171981812, "tokens_seen": 79306752, "tokens_per_sec": 18303.44101013128, "vram_gb": 5.684933632, "lambda": 0.29369101269791525, "lambda_min": 0.11444037407636642, "lambda_max": 0.6447771787643433, "S_mean": 0.32763671875, "G_mean": 0.33507958784078556, "G_std": 0.16014176529521743, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.227559328079224, "grad_norm": 0.967364490032196, "tokens_seen": 79470592, "tokens_per_sec": 18302.77082250039, "vram_gb": 5.684933632, "lambda": 0.2938101164375742, "lambda_min": 0.11448565870523453, "lambda_max": 0.645225465297699, "S_mean": 0.338470458984375, "G_mean": 0.33479424302155775, "G_std": 0.1605558063990126, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.232109546661377, "grad_norm": 0.9559404850006104, "tokens_seen": 79634432, "tokens_per_sec": 18302.763760596023, "vram_gb": 5.684933632, "lambda": 0.29392986247936886, "lambda_min": 0.11449555307626724, "lambda_max": 0.6453949809074402, "S_mean": 0.3383382161458333, "G_mean": 0.3371958912660678, "G_std": 0.16137335519306362, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.346513509750366, "grad_norm": 0.9845459461212158, "tokens_seen": 79798272, "tokens_per_sec": 18302.832702392392, "vram_gb": 5.684933632, "lambda": 0.29402144129077595, "lambda_min": 0.11449728161096573, "lambda_max": 0.6456889510154724, "S_mean": 0.3388824462890625, "G_mean": 0.3368426085604976, "G_std": 0.15987971987730512, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.203978776931763, "grad_norm": 0.8928399085998535, "tokens_seen": 79962112, "tokens_per_sec": 18302.76068870639, "vram_gb": 5.684933632, "lambda": 0.2941578694929679, "lambda_min": 0.11449795216321945, "lambda_max": 0.6459231376647949, "S_mean": 0.3281962076822917, "G_mean": 0.33265756047330797, "G_std": 0.160996791246968, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.35025691986084, "grad_norm": 0.9735608696937561, "tokens_seen": 80125952, "tokens_per_sec": 18302.946314831446, "vram_gb": 5.684933632, "lambda": 0.2942352027942737, "lambda_min": 0.11449658870697021, "lambda_max": 0.6461488008499146, "S_mean": 0.3232421875, "G_mean": 0.3320882262196392, "G_std": 0.16130001218213388, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.388446092605591, "grad_norm": 0.9593302011489868, "tokens_seen": 80289792, "tokens_per_sec": 18303.10750224103, "vram_gb": 5.684933632, "lambda": 0.29434534596900147, "lambda_min": 0.11449728161096573, "lambda_max": 0.6465306282043457, "S_mean": 0.3392130533854167, "G_mean": 0.33472171099856496, "G_std": 0.16051593042599657, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.426323175430298, "grad_norm": 0.9386768341064453, "tokens_seen": 80453632, "tokens_per_sec": 18303.1103942646, "vram_gb": 5.684933632, "lambda": 0.2944585792720318, "lambda_min": 0.11451420933008194, "lambda_max": 0.6466996073722839, "S_mean": 0.3430277506510417, "G_mean": 0.33602952988197404, "G_std": 0.15991140715777874, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.16821813583374, "grad_norm": 0.9679516553878784, "tokens_seen": 80617472, "tokens_per_sec": 18303.412331123363, "vram_gb": 5.684933632, "lambda": 0.2945928235227863, "lambda_min": 0.1145254522562027, "lambda_max": 0.6471407413482666, "S_mean": 0.3315938313802083, "G_mean": 0.3351038912466417, "G_std": 0.16234245679030815, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.429563760757446, "grad_norm": 0.9623845219612122, "tokens_seen": 80781312, "tokens_per_sec": 18303.337774381293, "vram_gb": 5.684933632, "lambda": 0.29472578534235555, "lambda_min": 0.11453680694103241, "lambda_max": 0.647325873374939, "S_mean": 0.3417104085286458, "G_mean": 0.33624815203559894, "G_std": 0.160032733848008, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.513549089431763, "grad_norm": 0.9513931274414062, "tokens_seen": 80945152, "tokens_per_sec": 18303.415705519394, "vram_gb": 5.684933632, "lambda": 0.29482037015259266, "lambda_min": 0.11460161954164505, "lambda_max": 0.6475863456726074, "S_mean": 0.3267008463541667, "G_mean": 0.33789721317589283, "G_std": 0.16147254248304913, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.31284761428833, "grad_norm": 0.970393180847168, "tokens_seen": 81108992, "tokens_per_sec": 18303.383208032967, "vram_gb": 5.684933632, "lambda": 0.2949146314834555, "lambda_min": 0.11461327970027924, "lambda_max": 0.6478762626647949, "S_mean": 0.346527099609375, "G_mean": 0.33645067488153774, "G_std": 0.16012431149526188, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.397211313247681, "grad_norm": 0.9391934871673584, "tokens_seen": 81272832, "tokens_per_sec": 18303.100998080692, "vram_gb": 5.684933632, "lambda": 0.2950349214176337, "lambda_min": 0.11456429213285446, "lambda_max": 0.6481633186340332, "S_mean": 0.3387349446614583, "G_mean": 0.33672376939405996, "G_std": 0.16089964646380395, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.331122636795044, "grad_norm": 0.8819867968559265, "tokens_seen": 81436672, "tokens_per_sec": 18303.374767357735, "vram_gb": 5.684933632, "lambda": 0.2951022122676174, "lambda_min": 0.11454769223928452, "lambda_max": 0.6484102010726929, "S_mean": 0.341461181640625, "G_mean": 0.33262418761538964, "G_std": 0.16160503363547227, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.184585094451904, "grad_norm": 0.9137320518493652, "tokens_seen": 81600512, "tokens_per_sec": 18303.400746541807, "vram_gb": 5.684933632, "lambda": 0.29520911288758117, "lambda_min": 0.11454416066408157, "lambda_max": 0.6487032771110535, "S_mean": 0.3337504069010417, "G_mean": 0.33283226176475483, "G_std": 0.1609576599439606, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.3211448192596436, "grad_norm": 0.9404392242431641, "tokens_seen": 81764352, "tokens_per_sec": 18303.668460139295, "vram_gb": 5.684933632, "lambda": 0.2953006786604722, "lambda_min": 0.11453433334827423, "lambda_max": 0.6487590670585632, "S_mean": 0.3406473795572917, "G_mean": 0.3369789677672088, "G_std": 0.15883965832957378, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.211376905441284, "grad_norm": 0.8846038579940796, "tokens_seen": 81928192, "tokens_per_sec": 18303.692306952245, "vram_gb": 5.684933632, "lambda": 0.2954214770967762, "lambda_min": 0.11449019610881805, "lambda_max": 0.6488254070281982, "S_mean": 0.334228515625, "G_mean": 0.3333827465151747, "G_std": 0.16170682133330652, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.40084228515625, "val_ppl": 81.51950242314847, "lambda": 0.29542670864611864, "lambda_min": 0.11449024826288223, "lambda_max": 0.6488329768180847, "lambda_per_layer": [0.17352795600891113, 0.14678099751472473, 0.14047349989414215, 0.12483104318380356, 0.13965348899364471, 0.12735998630523682, 0.11449024826288223, 0.11857514828443527, 0.11726918816566467, 0.16097432374954224, 0.301432728767395, 0.42763620615005493, 0.4694485664367676, 0.4628853499889374, 0.6488329768180847, 0.3717869222164154, 0.6359699368476868, 0.3336731195449829, 0.5972959995269775, 0.3280580937862396, 0.3216843903064728, 0.37408557534217834, 0.26860398054122925, 0.18491128087043762], "S_mean": 0.3358154296875, "G_mean": 0.3387032875325531, "G_std": 0.16037169774062932, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.494457006454468, "grad_norm": 0.9441652894020081, "tokens_seen": 82092032, "tokens_per_sec": 18297.109127873005, "vram_gb": 5.684933632, "lambda": 0.29554028374453384, "lambda_min": 0.1144636869430542, "lambda_max": 0.6490956544876099, "S_mean": 0.3460693359375, "G_mean": 0.33878051637051004, "G_std": 0.15999136352911592, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.518745183944702, "grad_norm": 1.0158753395080566, "tokens_seen": 82255872, "tokens_per_sec": 18297.200111974096, "vram_gb": 5.684933632, "lambda": 0.2956211045384407, "lambda_min": 0.11447807401418686, "lambda_max": 0.6494983434677124, "S_mean": 0.342193603515625, "G_mean": 0.335706892888993, "G_std": 0.1603184949296216, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.507289886474609, "grad_norm": 0.9920344948768616, "tokens_seen": 82419712, "tokens_per_sec": 18297.004699625184, "vram_gb": 5.684933632, "lambda": 0.295744691354533, "lambda_min": 0.11451137065887451, "lambda_max": 0.6498463153839111, "S_mean": 0.3412373860677083, "G_mean": 0.3379207417989771, "G_std": 0.16104409967859587, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.347923755645752, "grad_norm": 0.9524886608123779, "tokens_seen": 82583552, "tokens_per_sec": 18297.283400045344, "vram_gb": 5.684933632, "lambda": 0.2958727565904458, "lambda_min": 0.11454810947179794, "lambda_max": 0.6502238512039185, "S_mean": 0.3228047688802083, "G_mean": 0.32970282970927656, "G_std": 0.15968121813299754, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.47417426109314, "grad_norm": 0.9803518652915955, "tokens_seen": 82747392, "tokens_per_sec": 18297.624687197018, "vram_gb": 5.684933632, "lambda": 0.29597582078228396, "lambda_min": 0.11458645761013031, "lambda_max": 0.6507058143615723, "S_mean": 0.3401590983072917, "G_mean": 0.33863294854139286, "G_std": 0.16107464620533088, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.281745910644531, "grad_norm": 0.9703969955444336, "tokens_seen": 82911232, "tokens_per_sec": 18297.99788824587, "vram_gb": 5.684933632, "lambda": 0.2960841543972492, "lambda_min": 0.11458931863307953, "lambda_max": 0.651088297367096, "S_mean": 0.33526611328125, "G_mean": 0.33341941563412547, "G_std": 0.16141957289073616, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.278881072998047, "grad_norm": 0.9363967776298523, "tokens_seen": 83075072, "tokens_per_sec": 18298.052100654342, "vram_gb": 5.684933632, "lambda": 0.2961673429235816, "lambda_min": 0.11451709270477295, "lambda_max": 0.6513792872428894, "S_mean": 0.3414815266927083, "G_mean": 0.33104372140951455, "G_std": 0.15919936553109437, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.372420787811279, "grad_norm": 1.0033891201019287, "tokens_seen": 83238912, "tokens_per_sec": 18298.05434584186, "vram_gb": 5.684933632, "lambda": 0.296258218276004, "lambda_min": 0.11451704800128937, "lambda_max": 0.6517495512962341, "S_mean": 0.3415018717447917, "G_mean": 0.334837014398848, "G_std": 0.16020894803417227, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.379830360412598, "grad_norm": 1.0168853998184204, "tokens_seen": 83402752, "tokens_per_sec": 18298.4174186121, "vram_gb": 5.684933632, "lambda": 0.2963631597037117, "lambda_min": 0.1145290955901146, "lambda_max": 0.6521099805831909, "S_mean": 0.343414306640625, "G_mean": 0.33466524886898696, "G_std": 0.15923477596758553, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.282938480377197, "grad_norm": 0.9603071808815002, "tokens_seen": 83566592, "tokens_per_sec": 18298.56791441546, "vram_gb": 5.684933632, "lambda": 0.2964528252681096, "lambda_min": 0.11450383067131042, "lambda_max": 0.6524679064750671, "S_mean": 0.3273213704427083, "G_mean": 0.3319472138925145, "G_std": 0.15924094807511815, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.379225492477417, "grad_norm": 0.941716194152832, "tokens_seen": 83730432, "tokens_per_sec": 18298.10025662443, "vram_gb": 5.684933632, "lambda": 0.29653269456078607, "lambda_min": 0.11447913199663162, "lambda_max": 0.6527936458587646, "S_mean": 0.3337453206380208, "G_mean": 0.3360574174827586, "G_std": 0.16044179939975342, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.301035165786743, "grad_norm": 0.9269334673881531, "tokens_seen": 83894272, "tokens_per_sec": 18298.16757845096, "vram_gb": 5.684933632, "lambda": 0.2966192088400324, "lambda_min": 0.1144888773560524, "lambda_max": 0.6528663635253906, "S_mean": 0.3389078776041667, "G_mean": 0.331117530586198, "G_std": 0.1601878566046556, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.311074733734131, "grad_norm": 0.9384282231330872, "tokens_seen": 84058112, "tokens_per_sec": 18298.153860517912, "vram_gb": 5.684933632, "lambda": 0.2967294656361143, "lambda_min": 0.1145118772983551, "lambda_max": 0.653044581413269, "S_mean": 0.3372294108072917, "G_mean": 0.3397163654056688, "G_std": 0.16059874122341475, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.456803321838379, "grad_norm": 0.9585292339324951, "tokens_seen": 84221952, "tokens_per_sec": 18297.947249039127, "vram_gb": 5.684933632, "lambda": 0.29682743549346924, "lambda_min": 0.11453670263290405, "lambda_max": 0.6533459424972534, "S_mean": 0.3513590494791667, "G_mean": 0.3370238135879238, "G_std": 0.15977722445192435, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.289989233016968, "grad_norm": 1.013595461845398, "tokens_seen": 84385792, "tokens_per_sec": 18298.03700590039, "vram_gb": 5.684933632, "lambda": 0.296941928875943, "lambda_min": 0.11453314125537872, "lambda_max": 0.653714656829834, "S_mean": 0.3316243489583333, "G_mean": 0.33729294082149863, "G_std": 0.16054032288957387, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.487519264221191, "grad_norm": 0.9792861342430115, "tokens_seen": 84549632, "tokens_per_sec": 18298.92950556602, "vram_gb": 5.684933632, "lambda": 0.29703755987187225, "lambda_min": 0.114507295191288, "lambda_max": 0.6539789438247681, "S_mean": 0.3472493489583333, "G_mean": 0.34162324977417785, "G_std": 0.16038766755567244, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.298107624053955, "grad_norm": 0.9412373900413513, "tokens_seen": 84713472, "tokens_per_sec": 18299.835375500996, "vram_gb": 5.684933632, "lambda": 0.29712083221723634, "lambda_min": 0.11453042924404144, "lambda_max": 0.6541967391967773, "S_mean": 0.327056884765625, "G_mean": 0.33505570151222247, "G_std": 0.15962769424853226, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.3197104930877686, "grad_norm": 0.9652338624000549, "tokens_seen": 84877312, "tokens_per_sec": 18300.04437151199, "vram_gb": 5.684933632, "lambda": 0.2971818794806798, "lambda_min": 0.11452236026525497, "lambda_max": 0.6543780565261841, "S_mean": 0.336669921875, "G_mean": 0.3349855304695666, "G_std": 0.1606039167769874, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.425763130187988, "grad_norm": 1.0547688007354736, "tokens_seen": 85041152, "tokens_per_sec": 18299.95541818707, "vram_gb": 5.684933632, "lambda": 0.2972594431291024, "lambda_min": 0.11449617147445679, "lambda_max": 0.6546632051467896, "S_mean": 0.3363545735677083, "G_mean": 0.33489597293858725, "G_std": 0.15957625109391907, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.4406867027282715, "grad_norm": 0.9772782325744629, "tokens_seen": 85204992, "tokens_per_sec": 18300.38659147791, "vram_gb": 5.684933632, "lambda": 0.2973357926433285, "lambda_min": 0.11444694548845291, "lambda_max": 0.6548567414283752, "S_mean": 0.3362325032552083, "G_mean": 0.34063150531922776, "G_std": 0.16185825422871858, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.258662939071655, "grad_norm": 0.9680190086364746, "tokens_seen": 85368832, "tokens_per_sec": 18300.559622464363, "vram_gb": 5.684933632, "lambda": 0.2974061307807763, "lambda_min": 0.11441539227962494, "lambda_max": 0.6551107168197632, "S_mean": 0.351959228515625, "G_mean": 0.33310824042807025, "G_std": 0.16028673675221702, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.209959268569946, "grad_norm": 1.0796191692352295, "tokens_seen": 85532672, "tokens_per_sec": 18300.636998375445, "vram_gb": 5.684933632, "lambda": 0.29749381138632697, "lambda_min": 0.11438464373350143, "lambda_max": 0.6552073955535889, "S_mean": 0.3485514322916667, "G_mean": 0.33501911008109647, "G_std": 0.16089551437956592, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.359525680541992, "grad_norm": 1.021314024925232, "tokens_seen": 85696512, "tokens_per_sec": 18300.976461613718, "vram_gb": 5.684933632, "lambda": 0.29757903578380746, "lambda_min": 0.11438462138175964, "lambda_max": 0.6553659439086914, "S_mean": 0.336578369140625, "G_mean": 0.3378456477851917, "G_std": 0.16051275147280344, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.373137712478638, "grad_norm": 0.9551416635513306, "tokens_seen": 85860352, "tokens_per_sec": 18300.975585562817, "vram_gb": 5.684933632, "lambda": 0.29769213559726876, "lambda_min": 0.11435811221599579, "lambda_max": 0.6557420492172241, "S_mean": 0.351409912109375, "G_mean": 0.3355237925425172, "G_std": 0.15904903365299106, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.26921272277832, "grad_norm": 0.9598186016082764, "tokens_seen": 86024192, "tokens_per_sec": 18301.31124074489, "vram_gb": 5.684933632, "lambda": 0.2977768012012045, "lambda_min": 0.11437993496656418, "lambda_max": 0.6559026837348938, "S_mean": 0.33697509765625, "G_mean": 0.3309869532628606, "G_std": 0.15902653123096874, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.218538284301758, "grad_norm": 0.9564220905303955, "tokens_seen": 86188032, "tokens_per_sec": 18301.500282840218, "vram_gb": 5.684933632, "lambda": 0.29787065119793016, "lambda_min": 0.11440525949001312, "lambda_max": 0.6559059023857117, "S_mean": 0.3348388671875, "G_mean": 0.33339215551192564, "G_std": 0.1588239842094481, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.097026824951172, "grad_norm": 0.977676272392273, "tokens_seen": 86351872, "tokens_per_sec": 18301.815427292677, "vram_gb": 5.684933632, "lambda": 0.2979574119672179, "lambda_min": 0.11443601548671722, "lambda_max": 0.656082808971405, "S_mean": 0.3409322102864583, "G_mean": 0.330084431450814, "G_std": 0.16062587359920144, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.347348213195801, "grad_norm": 0.9851080179214478, "tokens_seen": 86515712, "tokens_per_sec": 18302.144863768368, "vram_gb": 5.684933632, "lambda": 0.29804212382684153, "lambda_min": 0.11443658173084259, "lambda_max": 0.6564435362815857, "S_mean": 0.3362070719401042, "G_mean": 0.3382441345602274, "G_std": 0.15975364029873163, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.405350208282471, "grad_norm": 1.0215035676956177, "tokens_seen": 86679552, "tokens_per_sec": 18301.901788330586, "vram_gb": 5.684933632, "lambda": 0.29812996586163837, "lambda_min": 0.11445305496454239, "lambda_max": 0.656616747379303, "S_mean": 0.3445638020833333, "G_mean": 0.3381600435823202, "G_std": 0.15899706073105335, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.443716526031494, "grad_norm": 0.9979812502861023, "tokens_seen": 86843392, "tokens_per_sec": 18302.054026014244, "vram_gb": 5.684933632, "lambda": 0.2982260699694355, "lambda_min": 0.11446899175643921, "lambda_max": 0.6568352580070496, "S_mean": 0.3424530029296875, "G_mean": 0.33483957685530186, "G_std": 0.1603300527591879, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.381146669387817, "grad_norm": 0.9968048930168152, "tokens_seen": 87007232, "tokens_per_sec": 18302.426329241393, "vram_gb": 5.684933632, "lambda": 0.29827895543227595, "lambda_min": 0.1145016998052597, "lambda_max": 0.657053530216217, "S_mean": 0.342803955078125, "G_mean": 0.3335218519593279, "G_std": 0.15936680557206273, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.362797021865845, "grad_norm": 0.9318727850914001, "tokens_seen": 87171072, "tokens_per_sec": 18302.61367297868, "vram_gb": 5.684933632, "lambda": 0.29834595633049804, "lambda_min": 0.1145472303032875, "lambda_max": 0.6572187542915344, "S_mean": 0.3382364908854167, "G_mean": 0.3355657681822777, "G_std": 0.16032823213996986, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.339734077453613, "grad_norm": 0.9901126623153687, "tokens_seen": 87334912, "tokens_per_sec": 18302.833410047075, "vram_gb": 5.684933632, "lambda": 0.29840943186233443, "lambda_min": 0.11457538604736328, "lambda_max": 0.6573284268379211, "S_mean": 0.33319091796875, "G_mean": 0.33601428552841145, "G_std": 0.16022347604545453, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.418370723724365, "grad_norm": 0.9387347102165222, "tokens_seen": 87498752, "tokens_per_sec": 18303.030503947102, "vram_gb": 5.684933632, "lambda": 0.2984905168414116, "lambda_min": 0.1146208718419075, "lambda_max": 0.6575756072998047, "S_mean": 0.3319498697916667, "G_mean": 0.3320116763934493, "G_std": 0.15903217970238379, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.4438722133636475, "grad_norm": 0.9554469585418701, "tokens_seen": 87662592, "tokens_per_sec": 18303.285136944978, "vram_gb": 5.684933632, "lambda": 0.2985793311769764, "lambda_min": 0.11464501172304153, "lambda_max": 0.6578395962715149, "S_mean": 0.347625732421875, "G_mean": 0.33309401882191497, "G_std": 0.1602531939667339, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.314390182495117, "grad_norm": 0.9354898929595947, "tokens_seen": 87826432, "tokens_per_sec": 18303.442156105855, "vram_gb": 5.684933632, "lambda": 0.29866899953534204, "lambda_min": 0.11471422016620636, "lambda_max": 0.6580091118812561, "S_mean": 0.3366495768229167, "G_mean": 0.3293830219190568, "G_std": 0.15961665411790213, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.196840286254883, "grad_norm": 0.9207687377929688, "tokens_seen": 87990272, "tokens_per_sec": 18303.525553883086, "vram_gb": 5.684933632, "lambda": 0.29875212348997593, "lambda_min": 0.1147475391626358, "lambda_max": 0.6582183241844177, "S_mean": 0.3384806315104167, "G_mean": 0.33091396605595946, "G_std": 0.15939297309766212, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.324772357940674, "grad_norm": 0.9928910136222839, "tokens_seen": 88154112, "tokens_per_sec": 18303.401165740142, "vram_gb": 5.684933632, "lambda": 0.29883819352835417, "lambda_min": 0.11474484205245972, "lambda_max": 0.6583670973777771, "S_mean": 0.3516743977864583, "G_mean": 0.3360634380951524, "G_std": 0.15844860331465802, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.4318132400512695, "grad_norm": 0.9787935614585876, "tokens_seen": 88317952, "tokens_per_sec": 18303.693791184087, "vram_gb": 5.684933632, "lambda": 0.2989079315836231, "lambda_min": 0.11477728933095932, "lambda_max": 0.6586380004882812, "S_mean": 0.3385009765625, "G_mean": 0.338360452093184, "G_std": 0.16170217112327614, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.384175777435303, "grad_norm": 1.166783094406128, "tokens_seen": 88481792, "tokens_per_sec": 18303.945828945434, "vram_gb": 5.684933632, "lambda": 0.2989699486643076, "lambda_min": 0.1147647500038147, "lambda_max": 0.658871054649353, "S_mean": 0.3376871744791667, "G_mean": 0.33223221303584677, "G_std": 0.1613045739941299, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.325561761856079, "grad_norm": 1.032138466835022, "tokens_seen": 88645632, "tokens_per_sec": 18304.169692026364, "vram_gb": 5.684933632, "lambda": 0.2990274153028925, "lambda_min": 0.11472863703966141, "lambda_max": 0.6589810848236084, "S_mean": 0.3479512532552083, "G_mean": 0.3372191662589709, "G_std": 0.16046694429436079, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.315182447433472, "grad_norm": 1.0010327100753784, "tokens_seen": 88809472, "tokens_per_sec": 18304.146822766437, "vram_gb": 5.684933632, "lambda": 0.29912532183031243, "lambda_min": 0.1147177666425705, "lambda_max": 0.6593714356422424, "S_mean": 0.336700439453125, "G_mean": 0.3354375382574896, "G_std": 0.1604094117647037, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.304867506027222, "grad_norm": 0.9863932132720947, "tokens_seen": 88973312, "tokens_per_sec": 18303.702954560904, "vram_gb": 5.684933632, "lambda": 0.29917903058230877, "lambda_min": 0.11469913274049759, "lambda_max": 0.659537672996521, "S_mean": 0.3450826009114583, "G_mean": 0.3341499847980837, "G_std": 0.16115547949448228, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.430673360824585, "grad_norm": 1.0193030834197998, "tokens_seen": 89137152, "tokens_per_sec": 18303.344905222046, "vram_gb": 5.684933632, "lambda": 0.29924711895485717, "lambda_min": 0.11472926288843155, "lambda_max": 0.6596758365631104, "S_mean": 0.328155517578125, "G_mean": 0.3305690231888245, "G_std": 0.16123932495247573, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.493624448776245, "grad_norm": 0.9765960574150085, "tokens_seen": 89300992, "tokens_per_sec": 18303.39521072069, "vram_gb": 5.684933632, "lambda": 0.2993392863621314, "lambda_min": 0.11473789066076279, "lambda_max": 0.6600205898284912, "S_mean": 0.3390909830729167, "G_mean": 0.33885904937051237, "G_std": 0.16066581290215254, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.519110441207886, "grad_norm": 1.1485693454742432, "tokens_seen": 89464832, "tokens_per_sec": 18303.567876794892, "vram_gb": 5.684933632, "lambda": 0.2994183097034693, "lambda_min": 0.11475619673728943, "lambda_max": 0.6602990031242371, "S_mean": 0.3358815511067708, "G_mean": 0.3364360462874174, "G_std": 0.1591326657216996, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.515133380889893, "grad_norm": 0.9684486389160156, "tokens_seen": 89628672, "tokens_per_sec": 18303.73602388937, "vram_gb": 5.684933632, "lambda": 0.29951002666105825, "lambda_min": 0.11473193019628525, "lambda_max": 0.660468339920044, "S_mean": 0.3509419759114583, "G_mean": 0.33465244015678763, "G_std": 0.15946373098995537, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.297977447509766, "grad_norm": 0.9418538808822632, "tokens_seen": 89792512, "tokens_per_sec": 18303.868978778028, "vram_gb": 5.684933632, "lambda": 0.29959105141460896, "lambda_min": 0.1147080808877945, "lambda_max": 0.6605238914489746, "S_mean": 0.34124755859375, "G_mean": 0.3320754912371437, "G_std": 0.15850785731648406, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.167317867279053, "grad_norm": 0.9615747928619385, "tokens_seen": 89956352, "tokens_per_sec": 18303.922115568574, "vram_gb": 5.684933632, "lambda": 0.2996717893208067, "lambda_min": 0.11470919847488403, "lambda_max": 0.6607507467269897, "S_mean": 0.3339742024739583, "G_mean": 0.3328418214805424, "G_std": 0.15887430147267878, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.397610425949097, "grad_norm": 0.9893903732299805, "tokens_seen": 90120192, "tokens_per_sec": 18304.12168894613, "vram_gb": 5.684933632, "lambda": 0.29974689210454625, "lambda_min": 0.11469290405511856, "lambda_max": 0.6609135270118713, "S_mean": 0.3415018717447917, "G_mean": 0.33909621454464894, "G_std": 0.16000713466200978, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.400774669647217, "val_ppl": 81.5139906268396, "lambda": 0.29974835366010666, "lambda_min": 0.11469300836324692, "lambda_max": 0.660916805267334, "lambda_per_layer": [0.17449769377708435, 0.14720384776592255, 0.1408756971359253, 0.12480100989341736, 0.13990965485572815, 0.12736812233924866, 0.11469300836324692, 0.11864502727985382, 0.11710140854120255, 0.16172415018081665, 0.3029109537601471, 0.43055808544158936, 0.47448211908340454, 0.47117024660110474, 0.660916805267334, 0.3791702091693878, 0.6493703126907349, 0.3395620882511139, 0.6115767359733582, 0.335424542427063, 0.32827889919281006, 0.3820599615573883, 0.2743665874004364, 0.18729332089424133], "S_mean": 0.3400980631510417, "G_mean": 0.33338943449780345, "G_std": 0.16058224133060625, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.339426755905151, "grad_norm": 1.0636805295944214, "tokens_seen": 90284032, "tokens_per_sec": 18298.042247025547, "vram_gb": 5.684933632, "lambda": 0.2998280708367626, "lambda_min": 0.11468800157308578, "lambda_max": 0.661101758480072, "S_mean": 0.3738606770833333, "G_mean": 0.33585909722993773, "G_std": 0.16078883269801736, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.2726616859436035, "grad_norm": 1.0248723030090332, "tokens_seen": 90447872, "tokens_per_sec": 18298.2942088541, "vram_gb": 5.684933632, "lambda": 0.2999063804745674, "lambda_min": 0.11469236016273499, "lambda_max": 0.661219596862793, "S_mean": 0.3436228434244792, "G_mean": 0.3344186300722261, "G_std": 0.15955947273566076, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.3881988525390625, "grad_norm": 0.998538613319397, "tokens_seen": 90611712, "tokens_per_sec": 18298.3378910222, "vram_gb": 5.684933632, "lambda": 0.29997246836622554, "lambda_min": 0.11472880095243454, "lambda_max": 0.6612858772277832, "S_mean": 0.3492635091145833, "G_mean": 0.3404817064292729, "G_std": 0.16082414775155485, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.2957422733306885, "grad_norm": 1.1063721179962158, "tokens_seen": 90775552, "tokens_per_sec": 18298.07198719664, "vram_gb": 5.684933632, "lambda": 0.3000399845962723, "lambda_min": 0.11476445943117142, "lambda_max": 0.6614391803741455, "S_mean": 0.3424886067708333, "G_mean": 0.33245221005442244, "G_std": 0.15989287376093367, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.446012020111084, "grad_norm": 0.9411303997039795, "tokens_seen": 90939392, "tokens_per_sec": 18298.411613554483, "vram_gb": 5.684933632, "lambda": 0.3001103528464834, "lambda_min": 0.11479799449443817, "lambda_max": 0.6615458130836487, "S_mean": 0.3568623860677083, "G_mean": 0.3366591820959002, "G_std": 0.16011887788772583, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.179198741912842, "grad_norm": 0.9529906511306763, "tokens_seen": 91103232, "tokens_per_sec": 18298.504609146, "vram_gb": 5.684933632, "lambda": 0.3001735821987192, "lambda_min": 0.11481842398643494, "lambda_max": 0.6616130471229553, "S_mean": 0.3373514811197917, "G_mean": 0.3322420703091969, "G_std": 0.1589634792568783, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.324622392654419, "grad_norm": 0.9449768662452698, "tokens_seen": 91267072, "tokens_per_sec": 18298.738039139364, "vram_gb": 5.684933632, "lambda": 0.30025535232077044, "lambda_min": 0.1148805320262909, "lambda_max": 0.6618222594261169, "S_mean": 0.34088134765625, "G_mean": 0.3340846349019557, "G_std": 0.1571286154600481, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.358956813812256, "grad_norm": 0.9759833216667175, "tokens_seen": 91430912, "tokens_per_sec": 18298.924047811375, "vram_gb": 5.684933632, "lambda": 0.3003443709264199, "lambda_min": 0.11489507555961609, "lambda_max": 0.6619969010353088, "S_mean": 0.3344217936197917, "G_mean": 0.3310563878621906, "G_std": 0.1592829144016529, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.3412559032440186, "grad_norm": 1.005688190460205, "tokens_seen": 91594752, "tokens_per_sec": 18298.78893285955, "vram_gb": 5.684933632, "lambda": 0.3004110446199775, "lambda_min": 0.11491888761520386, "lambda_max": 0.6621326804161072, "S_mean": 0.3359476725260417, "G_mean": 0.3342722758340339, "G_std": 0.15852053001678237, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.178097724914551, "grad_norm": 0.9618853330612183, "tokens_seen": 91758592, "tokens_per_sec": 18298.83238169296, "vram_gb": 5.684933632, "lambda": 0.30048211570829153, "lambda_min": 0.1148788332939148, "lambda_max": 0.6623698472976685, "S_mean": 0.348968505859375, "G_mean": 0.3339957483112812, "G_std": 0.15788512498450777, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.124012231826782, "grad_norm": 0.9652330875396729, "tokens_seen": 91922432, "tokens_per_sec": 18298.837337946083, "vram_gb": 5.684933632, "lambda": 0.3005703358600537, "lambda_min": 0.11486610025167465, "lambda_max": 0.662570595741272, "S_mean": 0.3272501627604167, "G_mean": 0.3322759289294481, "G_std": 0.1585802952370917, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.341446876525879, "grad_norm": 1.051569938659668, "tokens_seen": 92086272, "tokens_per_sec": 18299.001593292272, "vram_gb": 5.684933632, "lambda": 0.3006470849116643, "lambda_min": 0.11484877020120621, "lambda_max": 0.6628332138061523, "S_mean": 0.3421122233072917, "G_mean": 0.3359257663444926, "G_std": 0.15987082361243665, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.358169794082642, "grad_norm": 1.0423332452774048, "tokens_seen": 92250112, "tokens_per_sec": 18298.899153397433, "vram_gb": 5.684933632, "lambda": 0.3007193918650349, "lambda_min": 0.11485057324171066, "lambda_max": 0.6630902290344238, "S_mean": 0.3338673909505208, "G_mean": 0.33518690429627895, "G_std": 0.1603124690009281, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.319365978240967, "grad_norm": 0.9693359136581421, "tokens_seen": 92413952, "tokens_per_sec": 18299.099270109662, "vram_gb": 5.684933632, "lambda": 0.3007976400355498, "lambda_min": 0.11487004905939102, "lambda_max": 0.6633457541465759, "S_mean": 0.3336079915364583, "G_mean": 0.3354551646237572, "G_std": 0.16050375955334553, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.3560919761657715, "grad_norm": 1.0470939874649048, "tokens_seen": 92577792, "tokens_per_sec": 18299.287798681584, "vram_gb": 5.684933632, "lambda": 0.3008703788121541, "lambda_min": 0.1149098202586174, "lambda_max": 0.6636372208595276, "S_mean": 0.3393910725911458, "G_mean": 0.32774160988628864, "G_std": 0.1590350920256848, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.217267036437988, "grad_norm": 0.9748811721801758, "tokens_seen": 92741632, "tokens_per_sec": 18299.601831285374, "vram_gb": 5.684933632, "lambda": 0.30093539350976545, "lambda_min": 0.11495637148618698, "lambda_max": 0.6638873815536499, "S_mean": 0.3392181396484375, "G_mean": 0.3309194231405854, "G_std": 0.16063627790814886, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.39889669418335, "grad_norm": 1.043174386024475, "tokens_seen": 92905472, "tokens_per_sec": 18299.68376236195, "vram_gb": 5.684933632, "lambda": 0.3010226304953297, "lambda_min": 0.11498221755027771, "lambda_max": 0.6641063094139099, "S_mean": 0.33551025390625, "G_mean": 0.33545717131346464, "G_std": 0.16202487715054303, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.313206195831299, "grad_norm": 0.9954037070274353, "tokens_seen": 93069312, "tokens_per_sec": 18299.922502206424, "vram_gb": 5.684933632, "lambda": 0.3010765478635828, "lambda_min": 0.11496803164482117, "lambda_max": 0.6643026471138, "S_mean": 0.3377685546875, "G_mean": 0.3337508357750873, "G_std": 0.16051575366873294, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.331543922424316, "grad_norm": 0.9892162084579468, "tokens_seen": 93233152, "tokens_per_sec": 18299.808723525242, "vram_gb": 5.684933632, "lambda": 0.30114170846839744, "lambda_min": 0.11495769023895264, "lambda_max": 0.6644406318664551, "S_mean": 0.3272806803385417, "G_mean": 0.3364961484136681, "G_std": 0.16145167235905925, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.3581321239471436, "grad_norm": 1.1598377227783203, "tokens_seen": 93396992, "tokens_per_sec": 18299.819318056652, "vram_gb": 5.684933632, "lambda": 0.30120555125176907, "lambda_min": 0.11494777351617813, "lambda_max": 0.6644868850708008, "S_mean": 0.3450724283854167, "G_mean": 0.3350443052283178, "G_std": 0.15914278279524297, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.331273078918457, "grad_norm": 1.0105501413345337, "tokens_seen": 93560832, "tokens_per_sec": 18299.941431968684, "vram_gb": 5.684933632, "lambda": 0.3012923700734973, "lambda_min": 0.11495349556207657, "lambda_max": 0.664680004119873, "S_mean": 0.337310791015625, "G_mean": 0.3355515001652141, "G_std": 0.16024791123345494, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.202687978744507, "grad_norm": 1.014878511428833, "tokens_seen": 93724672, "tokens_per_sec": 18300.07032683764, "vram_gb": 5.684933632, "lambda": 0.30137043446302414, "lambda_min": 0.11497323215007782, "lambda_max": 0.6646715402603149, "S_mean": 0.3459574381510417, "G_mean": 0.3367900756808619, "G_std": 0.1614934163323293, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.264427661895752, "grad_norm": 1.0104656219482422, "tokens_seen": 93888512, "tokens_per_sec": 18300.20175697087, "vram_gb": 5.684933632, "lambda": 0.3014207547530532, "lambda_min": 0.11496689170598984, "lambda_max": 0.664679229259491, "S_mean": 0.3319295247395833, "G_mean": 0.3333430814867218, "G_std": 0.16050185814189413, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.237182140350342, "grad_norm": 0.9871423840522766, "tokens_seen": 94052352, "tokens_per_sec": 18300.492410394425, "vram_gb": 5.684933632, "lambda": 0.30149094946682453, "lambda_min": 0.11493252962827682, "lambda_max": 0.6649429798126221, "S_mean": 0.346771240234375, "G_mean": 0.3290424144361168, "G_std": 0.1599293303443119, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.374102592468262, "grad_norm": 1.0480964183807373, "tokens_seen": 94216192, "tokens_per_sec": 18300.380791183547, "vram_gb": 5.684933632, "lambda": 0.3015477191656828, "lambda_min": 0.11492631584405899, "lambda_max": 0.6651272177696228, "S_mean": 0.3539326985677083, "G_mean": 0.33636049134656787, "G_std": 0.16049358516465873, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.432260274887085, "grad_norm": 0.997864842414856, "tokens_seen": 94380032, "tokens_per_sec": 18300.470019796692, "vram_gb": 5.684933632, "lambda": 0.3016110733151436, "lambda_min": 0.11492222547531128, "lambda_max": 0.6651429533958435, "S_mean": 0.3321990966796875, "G_mean": 0.33577623140687746, "G_std": 0.16010085035425922, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.323203086853027, "grad_norm": 1.0681946277618408, "tokens_seen": 94543872, "tokens_per_sec": 18300.46199154802, "vram_gb": 5.684933632, "lambda": 0.3016840474059184, "lambda_min": 0.11489011347293854, "lambda_max": 0.6652683615684509, "S_mean": 0.3403116861979167, "G_mean": 0.3353229062631726, "G_std": 0.15915595988432565, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.270740032196045, "grad_norm": 1.0326095819473267, "tokens_seen": 94707712, "tokens_per_sec": 18300.483153074685, "vram_gb": 5.684933632, "lambda": 0.30174472214033204, "lambda_min": 0.11491381376981735, "lambda_max": 0.6654284596443176, "S_mean": 0.35888671875, "G_mean": 0.33598918669546646, "G_std": 0.1584037896245718, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.487384080886841, "grad_norm": 1.1923161745071411, "tokens_seen": 94871552, "tokens_per_sec": 18300.57150374456, "vram_gb": 5.684933632, "lambda": 0.30178547588487464, "lambda_min": 0.11492768675088882, "lambda_max": 0.665605366230011, "S_mean": 0.3397369384765625, "G_mean": 0.33549159734199446, "G_std": 0.1588036223935584, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.2109293937683105, "grad_norm": 0.9893594980239868, "tokens_seen": 95035392, "tokens_per_sec": 18300.561694335367, "vram_gb": 5.684933632, "lambda": 0.3018618741383155, "lambda_min": 0.1149124726653099, "lambda_max": 0.665813148021698, "S_mean": 0.34637451171875, "G_mean": 0.33840199327096343, "G_std": 0.15851924335584044, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.322298288345337, "grad_norm": 1.0263413190841675, "tokens_seen": 95199232, "tokens_per_sec": 18300.773127859535, "vram_gb": 5.684933632, "lambda": 0.30193774309009314, "lambda_min": 0.11488223820924759, "lambda_max": 0.6660123467445374, "S_mean": 0.3487040201822917, "G_mean": 0.3322816089882205, "G_std": 0.16006387991365045, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.229787111282349, "grad_norm": 0.9151020646095276, "tokens_seen": 95363072, "tokens_per_sec": 18301.03934721232, "vram_gb": 5.684933632, "lambda": 0.3020254758497079, "lambda_min": 0.11488699913024902, "lambda_max": 0.6662822365760803, "S_mean": 0.3392333984375, "G_mean": 0.33092236580948037, "G_std": 0.15841673671578368, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.267922878265381, "grad_norm": 1.0158768892288208, "tokens_seen": 95526912, "tokens_per_sec": 18301.25248729806, "vram_gb": 5.684933632, "lambda": 0.3020991949985425, "lambda_min": 0.1148870587348938, "lambda_max": 0.6664142608642578, "S_mean": 0.3513387044270833, "G_mean": 0.3331262009839217, "G_std": 0.15918074191237488, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.303467512130737, "grad_norm": 0.9622659087181091, "tokens_seen": 95690752, "tokens_per_sec": 18301.54808792485, "vram_gb": 5.684933632, "lambda": 0.30217099376022816, "lambda_min": 0.11488669365644455, "lambda_max": 0.6666827201843262, "S_mean": 0.3372344970703125, "G_mean": 0.33133028556282323, "G_std": 0.15863326424732804, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.346234321594238, "grad_norm": 1.0369611978530884, "tokens_seen": 95854592, "tokens_per_sec": 18301.886247415205, "vram_gb": 5.684933632, "lambda": 0.30223641358315945, "lambda_min": 0.11492305994033813, "lambda_max": 0.666922390460968, "S_mean": 0.3488871256510417, "G_mean": 0.33266834588721395, "G_std": 0.16024897732616714, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.300654888153076, "grad_norm": 1.0032899379730225, "tokens_seen": 96018432, "tokens_per_sec": 18302.237375970602, "vram_gb": 5.684933632, "lambda": 0.30230084465195733, "lambda_min": 0.11490209400653839, "lambda_max": 0.6669367551803589, "S_mean": 0.339752197265625, "G_mean": 0.332668022873501, "G_std": 0.1588445275944347, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.3439085483551025, "grad_norm": 1.1075217723846436, "tokens_seen": 96182272, "tokens_per_sec": 18302.412297787276, "vram_gb": 5.684933632, "lambda": 0.30235824082046747, "lambda_min": 0.11489380896091461, "lambda_max": 0.6670559048652649, "S_mean": 0.3413798014322917, "G_mean": 0.33805657100553316, "G_std": 0.1611820838879794, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.224354267120361, "grad_norm": 0.9955230951309204, "tokens_seen": 96346112, "tokens_per_sec": 18302.17948265916, "vram_gb": 5.684933632, "lambda": 0.30240761302411556, "lambda_min": 0.11489015817642212, "lambda_max": 0.6672632694244385, "S_mean": 0.3362325032552083, "G_mean": 0.32713424678271014, "G_std": 0.15989679699608436, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.417724132537842, "grad_norm": 1.0249660015106201, "tokens_seen": 96509952, "tokens_per_sec": 18302.284704445454, "vram_gb": 5.684933632, "lambda": 0.30246808255712193, "lambda_min": 0.11489313095808029, "lambda_max": 0.6673226952552795, "S_mean": 0.3454691569010417, "G_mean": 0.3366701540847619, "G_std": 0.15948548602561155, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.322502136230469, "grad_norm": 0.9714866876602173, "tokens_seen": 96673792, "tokens_per_sec": 18302.52505334409, "vram_gb": 5.684933632, "lambda": 0.30254902100811404, "lambda_min": 0.11489081382751465, "lambda_max": 0.6675107479095459, "S_mean": 0.3348541259765625, "G_mean": 0.3355330795360108, "G_std": 0.1582099283890178, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.487121105194092, "grad_norm": 0.9837056398391724, "tokens_seen": 96837632, "tokens_per_sec": 18302.966486312922, "vram_gb": 5.684933632, "lambda": 0.30261956869314116, "lambda_min": 0.11487367749214172, "lambda_max": 0.6677167415618896, "S_mean": 0.3366546630859375, "G_mean": 0.3343801381221662, "G_std": 0.1588149401747311, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.2584147453308105, "grad_norm": 1.004274845123291, "tokens_seen": 97001472, "tokens_per_sec": 18303.065715348912, "vram_gb": 5.684933632, "lambda": 0.3026861995458603, "lambda_min": 0.1148667186498642, "lambda_max": 0.6679018139839172, "S_mean": 0.347320556640625, "G_mean": 0.33055491792038083, "G_std": 0.15877475310117006, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.223903179168701, "grad_norm": 1.0604736804962158, "tokens_seen": 97165312, "tokens_per_sec": 18303.36198023759, "vram_gb": 5.684933632, "lambda": 0.3027473653977116, "lambda_min": 0.11488121002912521, "lambda_max": 0.6679624915122986, "S_mean": 0.3506876627604167, "G_mean": 0.3318434583488852, "G_std": 0.15872420754749328, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.331086158752441, "grad_norm": 1.1418510675430298, "tokens_seen": 97329152, "tokens_per_sec": 18303.314092142104, "vram_gb": 5.684933632, "lambda": 0.30281743717690307, "lambda_min": 0.11489415168762207, "lambda_max": 0.6681203246116638, "S_mean": 0.3430379231770833, "G_mean": 0.33413421533380944, "G_std": 0.15824863543578735, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.29667329788208, "grad_norm": 0.9637771844863892, "tokens_seen": 97492992, "tokens_per_sec": 18303.499697903004, "vram_gb": 5.684933632, "lambda": 0.3028759406879544, "lambda_min": 0.11489319056272507, "lambda_max": 0.6681361794471741, "S_mean": 0.3415120442708333, "G_mean": 0.3323155481678744, "G_std": 0.15909214802862456, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.20845365524292, "grad_norm": 1.0798174142837524, "tokens_seen": 97656832, "tokens_per_sec": 18303.449029817963, "vram_gb": 5.684933632, "lambda": 0.3029349548742175, "lambda_min": 0.11487986147403717, "lambda_max": 0.6683857440948486, "S_mean": 0.3377888997395833, "G_mean": 0.33602407574653625, "G_std": 0.15851465713543197, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.195626258850098, "grad_norm": 1.1194273233413696, "tokens_seen": 97820672, "tokens_per_sec": 18303.372849411982, "vram_gb": 5.684933632, "lambda": 0.30299328671147424, "lambda_min": 0.11487869918346405, "lambda_max": 0.6685100197792053, "S_mean": 0.3481597900390625, "G_mean": 0.3265834611374885, "G_std": 0.15879796782974154, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.294053554534912, "grad_norm": 1.0297762155532837, "tokens_seen": 97984512, "tokens_per_sec": 18303.56435363366, "vram_gb": 5.684933632, "lambda": 0.30304825988908607, "lambda_min": 0.1148543506860733, "lambda_max": 0.6686635613441467, "S_mean": 0.3406982421875, "G_mean": 0.3326101692703863, "G_std": 0.15883280887889364, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.2694268226623535, "grad_norm": 1.025038719177246, "tokens_seen": 98148352, "tokens_per_sec": 18303.551111159875, "vram_gb": 5.684933632, "lambda": 0.30312894998739165, "lambda_min": 0.11484593898057938, "lambda_max": 0.6688721179962158, "S_mean": 0.3407389322916667, "G_mean": 0.33056403052372235, "G_std": 0.15872544038575143, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.397656035423279, "val_ppl": 81.26017429181104, "lambda": 0.30320383980870247, "lambda_min": 0.11483360826969147, "lambda_max": 0.6690832376480103, "lambda_per_layer": [0.17556753754615784, 0.14752598106861115, 0.14106102287769318, 0.12479514628648758, 0.14015400409698486, 0.1273481249809265, 0.11483360826969147, 0.11893660575151443, 0.11699232459068298, 0.16223755478858948, 0.3040373623371124, 0.4331870675086975, 0.4787544310092926, 0.47652706503868103, 0.6690832376480103, 0.38499143719673157, 0.6605284214019775, 0.3439040780067444, 0.6233647465705872, 0.34063223004341125, 0.33438730239868164, 0.3889225125312805, 0.2798336446285248, 0.1892867088317871], "S_mean": 0.3710734049479167, "G_mean": 0.33595271113639075, "G_std": 0.16295996525635323, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5372.456871032715, "nan_failures": 0} diff --git a/full_s3/config.json b/full_s3/config.json new file mode 100644 index 0000000000000000000000000000000000000000..b15210956554e43b0785ab47dd0859918c7862e0 --- /dev/null +++ b/full_s3/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 3, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s3", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 3001346, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/full_s3/final.pt b/full_s3/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..bff69f164386086b502671eb115410235016fa28 --- /dev/null +++ b/full_s3/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:484ac9a39316cb184ca5e19e871ca977fa8529d4c3bafaa6613ee6e3aeef2eef +size 256003111 diff --git a/full_s3/metrics.jsonl b/full_s3/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..4c549411d8227b2a495c4d4c15898aa5ff2b068d --- /dev/null +++ b/full_s3/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 3, "config": {"variant": "full", "seed": 3, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s3", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 3001346, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.922065734863281, "grad_norm": 11.558189392089844, "tokens_seen": 8192, "tokens_per_sec": 7926.3778687200975, "vram_gb": 5.172759552, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.1702117919921875, "G_mean": 0.4967380625506242, "G_std": 0.0861214076479276, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.836324691772461, "grad_norm": 1.9053367376327515, "tokens_seen": 172032, "tokens_per_sec": 17542.03578990561, "vram_gb": 5.684933632, "lambda": 0.12691842702527842, "lambda_min": 0.12688212096691132, "lambda_max": 0.1269623339176178, "S_mean": 0.46319580078125, "G_mean": 0.45481953024864197, "G_std": 0.06870699386733274, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.096193313598633, "grad_norm": 1.722855806350708, "tokens_seen": 335872, "tokens_per_sec": 18045.27686987305, "vram_gb": 5.684933632, "lambda": 0.12683101060489813, "lambda_min": 0.1267586201429367, "lambda_max": 0.1269371211528778, "S_mean": 0.4689687093098958, "G_mean": 0.43924688175320625, "G_std": 0.07003289585312207, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.173812866210938, "grad_norm": 1.5849695205688477, "tokens_seen": 499712, "tokens_per_sec": 18218.686914519454, "vram_gb": 5.684933632, "lambda": 0.12671193418403467, "lambda_min": 0.12658728659152985, "lambda_max": 0.12692049145698547, "S_mean": 0.4872639973958333, "G_mean": 0.44859347864985466, "G_std": 0.07698930171318352, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.518906116485596, "grad_norm": 0.8003024458885193, "tokens_seen": 663552, "tokens_per_sec": 18313.954364204612, "vram_gb": 5.684933632, "lambda": 0.12658455595374107, "lambda_min": 0.12631343305110931, "lambda_max": 0.12695574760437012, "S_mean": 0.3827616373697917, "G_mean": 0.4408634069065253, "G_std": 0.08753298766290148, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.093411922454834, "grad_norm": 0.6154975891113281, "tokens_seen": 827392, "tokens_per_sec": 18379.97918827916, "vram_gb": 5.684933632, "lambda": 0.12652622039119402, "lambda_min": 0.12606103718280792, "lambda_max": 0.12724287807941437, "S_mean": 0.2587890625, "G_mean": 0.43364975104729336, "G_std": 0.10298183312018712, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.186219215393066, "grad_norm": 0.8128833174705505, "tokens_seen": 991232, "tokens_per_sec": 18424.01693418622, "vram_gb": 5.684933632, "lambda": 0.12645522008339563, "lambda_min": 0.12583255767822266, "lambda_max": 0.12743347883224487, "S_mean": 0.28424072265625, "G_mean": 0.4304923477272193, "G_std": 0.11629939870908856, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.893113136291504, "grad_norm": 0.6174357533454895, "tokens_seen": 1155072, "tokens_per_sec": 18434.66463985636, "vram_gb": 5.684933632, "lambda": 0.12640980879465738, "lambda_min": 0.1257643699645996, "lambda_max": 0.12744824588298798, "S_mean": 0.26348876953125, "G_mean": 0.4268057569861412, "G_std": 0.12002680497244, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.866682291030884, "grad_norm": 0.7101200819015503, "tokens_seen": 1318912, "tokens_per_sec": 18453.093555444273, "vram_gb": 5.684933632, "lambda": 0.1264190897345543, "lambda_min": 0.12569008767604828, "lambda_max": 0.12742909789085388, "S_mean": 0.29547119140625, "G_mean": 0.41893654440840083, "G_std": 0.13294484342137972, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.739718437194824, "grad_norm": 0.560026228427887, "tokens_seen": 1482752, "tokens_per_sec": 18473.407708601022, "vram_gb": 5.684933632, "lambda": 0.1264035999774933, "lambda_min": 0.1256154626607895, "lambda_max": 0.12747015058994293, "S_mean": 0.2983805338541667, "G_mean": 0.40707895904779434, "G_std": 0.13473657860110202, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.672647953033447, "grad_norm": 0.6515396237373352, "tokens_seen": 1646592, "tokens_per_sec": 18476.329460388893, "vram_gb": 5.684933632, "lambda": 0.12636210831503072, "lambda_min": 0.12540261447429657, "lambda_max": 0.1274489015340805, "S_mean": 0.2683512369791667, "G_mean": 0.4164770406981309, "G_std": 0.13778129126876593, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.586154937744141, "grad_norm": 0.5118380784988403, "tokens_seen": 1810432, "tokens_per_sec": 18486.684189757896, "vram_gb": 5.684933632, "lambda": 0.12634787956873575, "lambda_min": 0.12527833878993988, "lambda_max": 0.12740960717201233, "S_mean": 0.30389404296875, "G_mean": 0.40114111577471095, "G_std": 0.14484164118766785, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.6082072257995605, "grad_norm": 0.4888658821582794, "tokens_seen": 1974272, "tokens_per_sec": 18485.24804287445, "vram_gb": 5.684933632, "lambda": 0.12632736812035242, "lambda_min": 0.12509234249591827, "lambda_max": 0.1273786872625351, "S_mean": 0.2691141764322917, "G_mean": 0.40214137981335324, "G_std": 0.1534391026943922, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.6109373569488525, "grad_norm": 0.46647804975509644, "tokens_seen": 2138112, "tokens_per_sec": 18484.416587189036, "vram_gb": 5.684933632, "lambda": 0.12627975145975748, "lambda_min": 0.12507280707359314, "lambda_max": 0.12745068967342377, "S_mean": 0.2940266927083333, "G_mean": 0.41223492845892906, "G_std": 0.16227355444182953, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.441583633422852, "grad_norm": 0.5741505026817322, "tokens_seen": 2301952, "tokens_per_sec": 18500.337664847288, "vram_gb": 5.684933632, "lambda": 0.12631403158108392, "lambda_min": 0.12513341009616852, "lambda_max": 0.12759582698345184, "S_mean": 0.3370361328125, "G_mean": 0.43195637812217075, "G_std": 0.17085851604739824, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.333474397659302, "grad_norm": 0.530099630355835, "tokens_seen": 2465792, "tokens_per_sec": 18500.789274170198, "vram_gb": 5.684933632, "lambda": 0.1263508622845014, "lambda_min": 0.12516796588897705, "lambda_max": 0.1279456913471222, "S_mean": 0.3521931966145833, "G_mean": 0.4020000249147415, "G_std": 0.16609600434700647, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.414483547210693, "grad_norm": 0.4535220265388489, "tokens_seen": 2629632, "tokens_per_sec": 18504.618761087822, "vram_gb": 5.684933632, "lambda": 0.1264127374937137, "lambda_min": 0.12511871755123138, "lambda_max": 0.12807156145572662, "S_mean": 0.3412272135416667, "G_mean": 0.41024623562892276, "G_std": 0.1671099541708827, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.478945732116699, "grad_norm": 0.5109475255012512, "tokens_seen": 2793472, "tokens_per_sec": 18511.28493901848, "vram_gb": 5.684933632, "lambda": 0.12639792325596014, "lambda_min": 0.12512533366680145, "lambda_max": 0.12847131490707397, "S_mean": 0.29071044921875, "G_mean": 0.42827024807532627, "G_std": 0.1755801299586892, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.255235433578491, "grad_norm": 0.4536072015762329, "tokens_seen": 2957312, "tokens_per_sec": 18508.78585783076, "vram_gb": 5.684933632, "lambda": 0.1264340846488873, "lambda_min": 0.12511420249938965, "lambda_max": 0.1285872906446457, "S_mean": 0.319000244140625, "G_mean": 0.40848731125394505, "G_std": 0.17717142465213934, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.294102907180786, "grad_norm": 0.42985761165618896, "tokens_seen": 3121152, "tokens_per_sec": 18508.243596357835, "vram_gb": 5.684933632, "lambda": 0.12651421253879866, "lambda_min": 0.12503714859485626, "lambda_max": 0.12880195677280426, "S_mean": 0.3250935872395833, "G_mean": 0.42361511290073395, "G_std": 0.1822340078651905, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.370401620864868, "grad_norm": 0.5143009424209595, "tokens_seen": 3284992, "tokens_per_sec": 18512.426365215313, "vram_gb": 5.684933632, "lambda": 0.1266118629525105, "lambda_min": 0.12511961162090302, "lambda_max": 0.1291266530752182, "S_mean": 0.2855733235677083, "G_mean": 0.45276952038208645, "G_std": 0.19549268142630657, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.278831481933594, "grad_norm": 0.4416904151439667, "tokens_seen": 3448832, "tokens_per_sec": 18512.594441646143, "vram_gb": 5.684933632, "lambda": 0.1267273494352897, "lambda_min": 0.12512733042240143, "lambda_max": 0.12931573390960693, "S_mean": 0.3359375, "G_mean": 0.4435681216418743, "G_std": 0.1946868815769752, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.2219078540802, "grad_norm": 0.49325162172317505, "tokens_seen": 3612672, "tokens_per_sec": 18517.643434678408, "vram_gb": 5.684933632, "lambda": 0.12678298354148865, "lambda_min": 0.12514519691467285, "lambda_max": 0.12960325181484222, "S_mean": 0.3228556315104167, "G_mean": 0.44803983097275096, "G_std": 0.2017272344479958, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.344208240509033, "grad_norm": 0.5530275702476501, "tokens_seen": 3776512, "tokens_per_sec": 18518.67301411064, "vram_gb": 5.684933632, "lambda": 0.12695602948466936, "lambda_min": 0.1251363903284073, "lambda_max": 0.12985168397426605, "S_mean": 0.31573486328125, "G_mean": 0.44404230763514835, "G_std": 0.20274584988753, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.208220958709717, "grad_norm": 0.488131582736969, "tokens_seen": 3940352, "tokens_per_sec": 18521.638851969088, "vram_gb": 5.684933632, "lambda": 0.12705691469212374, "lambda_min": 0.12513865530490875, "lambda_max": 0.13025939464569092, "S_mean": 0.3312174479166667, "G_mean": 0.4475450664758682, "G_std": 0.19931319604317346, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.235521078109741, "grad_norm": 0.48201310634613037, "tokens_seen": 4104192, "tokens_per_sec": 18524.11704969827, "vram_gb": 5.684933632, "lambda": 0.1271813033769528, "lambda_min": 0.12504351139068604, "lambda_max": 0.1305438131093979, "S_mean": 0.3541463216145833, "G_mean": 0.48572882637381554, "G_std": 0.2060528670748075, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.258996248245239, "grad_norm": 0.49790677428245544, "tokens_seen": 4268032, "tokens_per_sec": 18526.5591118626, "vram_gb": 5.684933632, "lambda": 0.12735133493940035, "lambda_min": 0.1251392513513565, "lambda_max": 0.13100233674049377, "S_mean": 0.3273417154947917, "G_mean": 0.5092406285305818, "G_std": 0.2101859382043282, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.161653518676758, "grad_norm": 0.49847474694252014, "tokens_seen": 4431872, "tokens_per_sec": 18528.905150641072, "vram_gb": 5.684933632, "lambda": 0.12745306454598904, "lambda_min": 0.12534666061401367, "lambda_max": 0.13134534657001495, "S_mean": 0.3575846354166667, "G_mean": 0.49537043645977974, "G_std": 0.2109846876313289, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.197483777999878, "grad_norm": 0.5155644416809082, "tokens_seen": 4595712, "tokens_per_sec": 18534.71766302169, "vram_gb": 5.684933632, "lambda": 0.12762730879088244, "lambda_min": 0.12540538609027863, "lambda_max": 0.13170938193798065, "S_mean": 0.3555908203125, "G_mean": 0.5030427513023218, "G_std": 0.21779748176534972, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.176801919937134, "grad_norm": 0.46224212646484375, "tokens_seen": 4759552, "tokens_per_sec": 18537.55784678965, "vram_gb": 5.684933632, "lambda": 0.1278144766887029, "lambda_min": 0.1255912184715271, "lambda_max": 0.13209882378578186, "S_mean": 0.38372802734375, "G_mean": 0.5059734769165516, "G_std": 0.21242521641155085, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.090654611587524, "grad_norm": 0.45104995369911194, "tokens_seen": 4923392, "tokens_per_sec": 18543.009934036723, "vram_gb": 5.684933632, "lambda": 0.12795594334602356, "lambda_min": 0.12570181488990784, "lambda_max": 0.13258589804172516, "S_mean": 0.35302734375, "G_mean": 0.5297425587972006, "G_std": 0.21373161921898523, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 5.997043132781982, "grad_norm": 0.5109930634498596, "tokens_seen": 5087232, "tokens_per_sec": 18544.2751421272, "vram_gb": 5.684933632, "lambda": 0.1280880911896626, "lambda_min": 0.12590965628623962, "lambda_max": 0.13304641842842102, "S_mean": 0.354644775390625, "G_mean": 0.5520224173863729, "G_std": 0.21646007150411606, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.087157487869263, "grad_norm": 0.44303929805755615, "tokens_seen": 5251072, "tokens_per_sec": 18546.200982316434, "vram_gb": 5.684933632, "lambda": 0.12824489859243235, "lambda_min": 0.1257879137992859, "lambda_max": 0.13344497978687286, "S_mean": 0.3891398111979167, "G_mean": 0.5449822805821896, "G_std": 0.21495121779541174, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.88326621055603, "grad_norm": 0.45421329140663147, "tokens_seen": 5414912, "tokens_per_sec": 18549.023392968338, "vram_gb": 5.684933632, "lambda": 0.1284874894966682, "lambda_min": 0.1257873922586441, "lambda_max": 0.13373416662216187, "S_mean": 0.3775431315104167, "G_mean": 0.54122061530749, "G_std": 0.20760517256955305, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.001515626907349, "grad_norm": 0.4739513695240021, "tokens_seen": 5578752, "tokens_per_sec": 18553.038226307173, "vram_gb": 5.684933632, "lambda": 0.12871527733902136, "lambda_min": 0.12550079822540283, "lambda_max": 0.13441219925880432, "S_mean": 0.381072998046875, "G_mean": 0.5474922843277454, "G_std": 0.20688317560901245, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 6.0538649559021, "grad_norm": 0.47732222080230713, "tokens_seen": 5742592, "tokens_per_sec": 18553.63641599157, "vram_gb": 5.684933632, "lambda": 0.12895583423475424, "lambda_min": 0.12533190846443176, "lambda_max": 0.13476522266864777, "S_mean": 0.392486572265625, "G_mean": 0.5579936169087887, "G_std": 0.20571509034683308, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.857337474822998, "grad_norm": 0.4606252908706665, "tokens_seen": 5906432, "tokens_per_sec": 18554.262779041193, "vram_gb": 5.684933632, "lambda": 0.12920383301874003, "lambda_min": 0.12546198070049286, "lambda_max": 0.13520629703998566, "S_mean": 0.4160359700520833, "G_mean": 0.5700867759684721, "G_std": 0.19981457913915315, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.021212100982666, "grad_norm": 0.5048103332519531, "tokens_seen": 6070272, "tokens_per_sec": 18554.24515984854, "vram_gb": 5.684933632, "lambda": 0.1294959963609775, "lambda_min": 0.12563131749629974, "lambda_max": 0.13565096259117126, "S_mean": 0.4109293619791667, "G_mean": 0.5704271011054516, "G_std": 0.20377779814104238, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.911296367645264, "grad_norm": 0.4494790732860565, "tokens_seen": 6234112, "tokens_per_sec": 18555.57736094998, "vram_gb": 5.684933632, "lambda": 0.12978538746635118, "lambda_min": 0.12560850381851196, "lambda_max": 0.13615743815898895, "S_mean": 0.3868408203125, "G_mean": 0.5685371079792579, "G_std": 0.19891680404543877, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.995471239089966, "grad_norm": 0.5042225122451782, "tokens_seen": 6397952, "tokens_per_sec": 18555.566955565853, "vram_gb": 5.684933632, "lambda": 0.13009096619983515, "lambda_min": 0.1255168914794922, "lambda_max": 0.1368076503276825, "S_mean": 0.3843587239583333, "G_mean": 0.579496718943119, "G_std": 0.19944167354454598, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.922982692718506, "grad_norm": 0.48285800218582153, "tokens_seen": 6561792, "tokens_per_sec": 18552.606586173315, "vram_gb": 5.684933632, "lambda": 0.13039981697996458, "lambda_min": 0.12508484721183777, "lambda_max": 0.137301966547966, "S_mean": 0.4429728190104167, "G_mean": 0.5970512684434652, "G_std": 0.18487727921456099, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.83922815322876, "grad_norm": 0.5459604263305664, "tokens_seen": 6725632, "tokens_per_sec": 18554.728019093956, "vram_gb": 5.684933632, "lambda": 0.13074643537402153, "lambda_min": 0.12507548928260803, "lambda_max": 0.13751378655433655, "S_mean": 0.4317830403645833, "G_mean": 0.5568996276706457, "G_std": 0.19394376097867885, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.862159013748169, "grad_norm": 0.5722585916519165, "tokens_seen": 6889472, "tokens_per_sec": 18553.10463398693, "vram_gb": 5.684933632, "lambda": 0.1310335509479046, "lambda_min": 0.12486942857503891, "lambda_max": 0.13814590871334076, "S_mean": 0.43597412109375, "G_mean": 0.5853832804908355, "G_std": 0.18952026528616747, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.830624103546143, "grad_norm": 0.5143104791641235, "tokens_seen": 7053312, "tokens_per_sec": 18553.99876103534, "vram_gb": 5.684933632, "lambda": 0.13133961614221334, "lambda_min": 0.12459781020879745, "lambda_max": 0.1387917846441269, "S_mean": 0.4229329427083333, "G_mean": 0.568771556019783, "G_std": 0.18859815845886865, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.974421262741089, "grad_norm": 0.538038969039917, "tokens_seen": 7217152, "tokens_per_sec": 18556.168421843417, "vram_gb": 5.684933632, "lambda": 0.13164737168699503, "lambda_min": 0.12434092909097672, "lambda_max": 0.1394703984260559, "S_mean": 0.41204833984375, "G_mean": 0.5888839264710745, "G_std": 0.19013671639064947, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.748004674911499, "grad_norm": 0.5141670107841492, "tokens_seen": 7380992, "tokens_per_sec": 18557.70029717384, "vram_gb": 5.684933632, "lambda": 0.1319771527002255, "lambda_min": 0.12420886754989624, "lambda_max": 0.14039665460586548, "S_mean": 0.41046142578125, "G_mean": 0.5817632128794988, "G_std": 0.18879674095660448, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.786068916320801, "grad_norm": 0.5613315105438232, "tokens_seen": 7544832, "tokens_per_sec": 18559.80604887253, "vram_gb": 5.684933632, "lambda": 0.13231594612201056, "lambda_min": 0.12395470589399338, "lambda_max": 0.1411391645669937, "S_mean": 0.4470621744791667, "G_mean": 0.5684681963175535, "G_std": 0.18681212173153958, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.630371570587158, "grad_norm": 0.47859105467796326, "tokens_seen": 7708672, "tokens_per_sec": 18556.09249133302, "vram_gb": 5.684933632, "lambda": 0.1326611020291845, "lambda_min": 0.12362411618232727, "lambda_max": 0.14209076762199402, "S_mean": 0.4251505533854167, "G_mean": 0.5581475893656412, "G_std": 0.18690075259655714, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.792856693267822, "grad_norm": 0.5084601640701294, "tokens_seen": 7872512, "tokens_per_sec": 18556.532634298048, "vram_gb": 5.684933632, "lambda": 0.13294798384110132, "lambda_min": 0.12330695986747742, "lambda_max": 0.14333361387252808, "S_mean": 0.4199422200520833, "G_mean": 0.5835227544109026, "G_std": 0.18878108200927576, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.714043617248535, "grad_norm": 0.5041437745094299, "tokens_seen": 8036352, "tokens_per_sec": 18555.18410149966, "vram_gb": 5.684933632, "lambda": 0.13330694070706764, "lambda_min": 0.1229812428355217, "lambda_max": 0.14448429644107819, "S_mean": 0.3936767578125, "G_mean": 0.5766928369800249, "G_std": 0.1898529321576158, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.892803192138672, "grad_norm": 0.5755292773246765, "tokens_seen": 8200192, "tokens_per_sec": 18549.746484178053, "vram_gb": 5.684933632, "lambda": 0.13373626861721277, "lambda_min": 0.12294690310955048, "lambda_max": 0.14567220211029053, "S_mean": 0.39459228515625, "G_mean": 0.5736776161938906, "G_std": 0.18738140352070332, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.824450206756592, "val_ppl": 338.47499064976955, "lambda": 0.13375560970356068, "lambda_min": 0.1229432001709938, "lambda_max": 0.14573538303375244, "lambda_per_layer": [0.14091213047504425, 0.13194410502910614, 0.13096389174461365, 0.12907898426055908, 0.12579190731048584, 0.1229432001709938, 0.13294349610805511, 0.12435109913349152, 0.12359654903411865, 0.13397249579429626, 0.14573538303375244, 0.13265301287174225, 0.14542661607265472, 0.13177193701267242, 0.13769498467445374, 0.1380087435245514, 0.14515045285224915, 0.13653461635112762, 0.1417997181415558, 0.12904100120067596, 0.1327170729637146, 0.1321791708469391, 0.13573028147220612, 0.12919378280639648], "S_mean": 0.4040120442708333, "G_mean": 0.5683660178134838, "G_std": 0.1854260042309761, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.727666139602661, "grad_norm": 0.4914880394935608, "tokens_seen": 8364032, "tokens_per_sec": 18480.131956242403, "vram_gb": 5.684933632, "lambda": 0.13412510665754476, "lambda_min": 0.12279362231492996, "lambda_max": 0.14710655808448792, "S_mean": 0.40692138671875, "G_mean": 0.5719081349670887, "G_std": 0.18590456712991, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.651137828826904, "grad_norm": 0.5009635090827942, "tokens_seen": 8527872, "tokens_per_sec": 18481.765767173867, "vram_gb": 5.684933632, "lambda": 0.13450418909390768, "lambda_min": 0.12290795892477036, "lambda_max": 0.14845559000968933, "S_mean": 0.4101765950520833, "G_mean": 0.5661529749631882, "G_std": 0.18416888204713663, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.62669038772583, "grad_norm": 0.5566135048866272, "tokens_seen": 8691712, "tokens_per_sec": 18479.9896208061, "vram_gb": 5.684933632, "lambda": 0.13486077108730873, "lambda_min": 0.1228608787059784, "lambda_max": 0.1496606320142746, "S_mean": 0.4092000325520833, "G_mean": 0.5509572364389896, "G_std": 0.18449328374117613, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.678563117980957, "grad_norm": 0.49078235030174255, "tokens_seen": 8855552, "tokens_per_sec": 18482.61558248363, "vram_gb": 5.684933632, "lambda": 0.13524920276055732, "lambda_min": 0.1226850226521492, "lambda_max": 0.15084971487522125, "S_mean": 0.4077962239583333, "G_mean": 0.559166923786203, "G_std": 0.18238388498624167, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.634069919586182, "grad_norm": 0.5495550632476807, "tokens_seen": 9019392, "tokens_per_sec": 18486.595044639067, "vram_gb": 5.684933632, "lambda": 0.13556802769502005, "lambda_min": 0.1224530041217804, "lambda_max": 0.1522122323513031, "S_mean": 0.4017537434895833, "G_mean": 0.5507545315970978, "G_std": 0.1846185422812899, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.7392449378967285, "grad_norm": 0.5011820197105408, "tokens_seen": 9183232, "tokens_per_sec": 18489.400831527648, "vram_gb": 5.684933632, "lambda": 0.13594675871233144, "lambda_min": 0.12230593711137772, "lambda_max": 0.15349939465522766, "S_mean": 0.3993937174479167, "G_mean": 0.5532472642759482, "G_std": 0.18710210903858146, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.6199188232421875, "grad_norm": 0.4961734116077423, "tokens_seen": 9347072, "tokens_per_sec": 18493.035100369365, "vram_gb": 5.684933632, "lambda": 0.13633934780955315, "lambda_min": 0.12201926857233047, "lambda_max": 0.1547488123178482, "S_mean": 0.3873494466145833, "G_mean": 0.5475799143314362, "G_std": 0.18279045230398575, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.687010049819946, "grad_norm": 0.5233210325241089, "tokens_seen": 9510912, "tokens_per_sec": 18495.507395376124, "vram_gb": 5.684933632, "lambda": 0.13672193915893635, "lambda_min": 0.12176747620105743, "lambda_max": 0.15597569942474365, "S_mean": 0.3974405924479167, "G_mean": 0.5607337603966395, "G_std": 0.1817367166901628, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.6099443435668945, "grad_norm": 0.5382334589958191, "tokens_seen": 9674752, "tokens_per_sec": 18496.469951136478, "vram_gb": 5.684933632, "lambda": 0.1370984259992838, "lambda_min": 0.12138533592224121, "lambda_max": 0.15718060731887817, "S_mean": 0.38311767578125, "G_mean": 0.5429668767998616, "G_std": 0.1848029081399242, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.671685457229614, "grad_norm": 0.5651312470436096, "tokens_seen": 9838592, "tokens_per_sec": 18494.220612202895, "vram_gb": 5.684933632, "lambda": 0.13747288659214973, "lambda_min": 0.12121634930372238, "lambda_max": 0.15853886306285858, "S_mean": 0.3972574869791667, "G_mean": 0.5394938569515944, "G_std": 0.18281361538295945, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.48602819442749, "grad_norm": 0.49266907572746277, "tokens_seen": 10002432, "tokens_per_sec": 18494.33053706078, "vram_gb": 5.684933632, "lambda": 0.13780508190393448, "lambda_min": 0.12098805606365204, "lambda_max": 0.15985390543937683, "S_mean": 0.3828837076822917, "G_mean": 0.5513620283454657, "G_std": 0.1778760994784534, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.528331995010376, "grad_norm": 0.50843346118927, "tokens_seen": 10166272, "tokens_per_sec": 18496.20741489379, "vram_gb": 5.684933632, "lambda": 0.1382291903719306, "lambda_min": 0.12101003527641296, "lambda_max": 0.16115708649158478, "S_mean": 0.38909912109375, "G_mean": 0.545398968582352, "G_std": 0.18200791192551455, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.697227954864502, "grad_norm": 0.4997604787349701, "tokens_seen": 10330112, "tokens_per_sec": 18497.315049425455, "vram_gb": 5.684933632, "lambda": 0.1386366734902064, "lambda_min": 0.12074200063943863, "lambda_max": 0.1623886525630951, "S_mean": 0.3820393880208333, "G_mean": 0.547384524717927, "G_std": 0.184224140830338, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.551280498504639, "grad_norm": 0.516150712966919, "tokens_seen": 10493952, "tokens_per_sec": 18497.818872966367, "vram_gb": 5.684933632, "lambda": 0.13910071024050316, "lambda_min": 0.12068673223257065, "lambda_max": 0.16367632150650024, "S_mean": 0.3896077473958333, "G_mean": 0.5477355650315682, "G_std": 0.18614585790783167, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.6446216106414795, "grad_norm": 0.5096935629844666, "tokens_seen": 10657792, "tokens_per_sec": 18497.54759597923, "vram_gb": 5.684933632, "lambda": 0.13955739016334215, "lambda_min": 0.12074626237154007, "lambda_max": 0.16513149440288544, "S_mean": 0.3785196940104167, "G_mean": 0.5341927769283453, "G_std": 0.18371715303510427, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.5351243019104, "grad_norm": 0.5473635792732239, "tokens_seen": 10821632, "tokens_per_sec": 18498.81100130559, "vram_gb": 5.684933632, "lambda": 0.13994202421357235, "lambda_min": 0.12043902277946472, "lambda_max": 0.16631509363651276, "S_mean": 0.3955891927083333, "G_mean": 0.5416294460495313, "G_std": 0.18297482390577594, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.609308481216431, "grad_norm": 0.5175862312316895, "tokens_seen": 10985472, "tokens_per_sec": 18498.989885064093, "vram_gb": 5.684933632, "lambda": 0.1404077528665463, "lambda_min": 0.12012195587158203, "lambda_max": 0.16756504774093628, "S_mean": 0.40679931640625, "G_mean": 0.5324197448790073, "G_std": 0.18235915495703617, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.457603216171265, "grad_norm": 0.5533818006515503, "tokens_seen": 11149312, "tokens_per_sec": 18500.20113535229, "vram_gb": 5.684933632, "lambda": 0.14087652632345757, "lambda_min": 0.12021753191947937, "lambda_max": 0.16852957010269165, "S_mean": 0.3875528971354167, "G_mean": 0.5330395766844352, "G_std": 0.17791840434074402, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.411229133605957, "grad_norm": 0.517105758190155, "tokens_seen": 11313152, "tokens_per_sec": 18499.232972608846, "vram_gb": 5.684933632, "lambda": 0.1412687636911869, "lambda_min": 0.1198248639702797, "lambda_max": 0.16980469226837158, "S_mean": 0.4083251953125, "G_mean": 0.5407715359081825, "G_std": 0.1797588582461079, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.591684341430664, "grad_norm": 0.5401414036750793, "tokens_seen": 11476992, "tokens_per_sec": 18500.893266073523, "vram_gb": 5.684933632, "lambda": 0.14171015564352274, "lambda_min": 0.11989080160856247, "lambda_max": 0.17096804082393646, "S_mean": 0.4126383463541667, "G_mean": 0.5295054685945312, "G_std": 0.17931718720744053, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.460427761077881, "grad_norm": 0.6009930968284607, "tokens_seen": 11640832, "tokens_per_sec": 18503.191607073775, "vram_gb": 5.684933632, "lambda": 0.14217153377830982, "lambda_min": 0.11969751864671707, "lambda_max": 0.17221637070178986, "S_mean": 0.3855387369791667, "G_mean": 0.5367607300480207, "G_std": 0.1769613567739725, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.354045152664185, "grad_norm": 0.5270761847496033, "tokens_seen": 11804672, "tokens_per_sec": 18501.21172425587, "vram_gb": 5.684933632, "lambda": 0.14260988806684813, "lambda_min": 0.11951158940792084, "lambda_max": 0.17347383499145508, "S_mean": 0.38861083984375, "G_mean": 0.5345920709272226, "G_std": 0.17588166209558645, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.526784181594849, "grad_norm": 0.5636498332023621, "tokens_seen": 11968512, "tokens_per_sec": 18500.339139920852, "vram_gb": 5.684933632, "lambda": 0.14297471288591623, "lambda_min": 0.11918537318706512, "lambda_max": 0.17442771792411804, "S_mean": 0.3963826497395833, "G_mean": 0.5240606237202883, "G_std": 0.17961117066442966, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.379661321640015, "grad_norm": 0.5567509531974792, "tokens_seen": 12132352, "tokens_per_sec": 18497.70838570468, "vram_gb": 5.684933632, "lambda": 0.14344882282118002, "lambda_min": 0.11908721178770065, "lambda_max": 0.17562051117420197, "S_mean": 0.4004313151041667, "G_mean": 0.5139474067837, "G_std": 0.17698678265636167, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.497424125671387, "grad_norm": 0.6335088610649109, "tokens_seen": 12296192, "tokens_per_sec": 18492.357168500093, "vram_gb": 5.684933632, "lambda": 0.14391555730253458, "lambda_min": 0.11921963840723038, "lambda_max": 0.17689473927021027, "S_mean": 0.3962605794270833, "G_mean": 0.5151703739538789, "G_std": 0.1778196326146523, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.4816741943359375, "grad_norm": 0.568538248538971, "tokens_seen": 12460032, "tokens_per_sec": 18490.786644899814, "vram_gb": 5.684933632, "lambda": 0.14436287060379982, "lambda_min": 0.11913885176181793, "lambda_max": 0.1780090630054474, "S_mean": 0.3860270182291667, "G_mean": 0.5185290277004242, "G_std": 0.1782848946750164, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.330191135406494, "grad_norm": 0.5509812831878662, "tokens_seen": 12623872, "tokens_per_sec": 18489.47187802266, "vram_gb": 5.684933632, "lambda": 0.14483378641307354, "lambda_min": 0.11871614307165146, "lambda_max": 0.17891982197761536, "S_mean": 0.3993326822916667, "G_mean": 0.5218359949067235, "G_std": 0.17476961947977543, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.405055522918701, "grad_norm": 0.5601121783256531, "tokens_seen": 12787712, "tokens_per_sec": 18487.72065528129, "vram_gb": 5.684933632, "lambda": 0.14527197554707527, "lambda_min": 0.11848816275596619, "lambda_max": 0.17996719479560852, "S_mean": 0.41021728515625, "G_mean": 0.5139392595738173, "G_std": 0.1751702770901223, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.481414794921875, "grad_norm": 0.6137636303901672, "tokens_seen": 12951552, "tokens_per_sec": 18486.944248156586, "vram_gb": 5.684933632, "lambda": 0.1456199719881018, "lambda_min": 0.11815489083528519, "lambda_max": 0.18108679354190826, "S_mean": 0.41943359375, "G_mean": 0.5130662703886628, "G_std": 0.1775730848312378, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.2484517097473145, "grad_norm": 0.5418113470077515, "tokens_seen": 13115392, "tokens_per_sec": 18485.589311282227, "vram_gb": 5.684933632, "lambda": 0.14608451072126627, "lambda_min": 0.11794748157262802, "lambda_max": 0.18221038579940796, "S_mean": 0.4085693359375, "G_mean": 0.5000439692909519, "G_std": 0.17524519376456738, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.453988790512085, "grad_norm": 0.5892783403396606, "tokens_seen": 13279232, "tokens_per_sec": 18482.102733229996, "vram_gb": 5.684933632, "lambda": 0.1465468524644772, "lambda_min": 0.11799652874469757, "lambda_max": 0.18301476538181305, "S_mean": 0.38470458984375, "G_mean": 0.5083229613179961, "G_std": 0.1750634607548515, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.340265512466431, "grad_norm": 0.5696685910224915, "tokens_seen": 13443072, "tokens_per_sec": 18481.46105989775, "vram_gb": 5.684933632, "lambda": 0.1469801211108764, "lambda_min": 0.11798591166734695, "lambda_max": 0.18416883051395416, "S_mean": 0.3992106119791667, "G_mean": 0.4969669912631313, "G_std": 0.17866652024288973, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.354418754577637, "grad_norm": 0.5960283279418945, "tokens_seen": 13606912, "tokens_per_sec": 18479.55994283373, "vram_gb": 5.684933632, "lambda": 0.1474189537887772, "lambda_min": 0.11764615774154663, "lambda_max": 0.18546739220619202, "S_mean": 0.40411376953125, "G_mean": 0.5045342991749445, "G_std": 0.17771363367016116, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.342564105987549, "grad_norm": 0.5402963161468506, "tokens_seen": 13770752, "tokens_per_sec": 18478.43659438004, "vram_gb": 5.684933632, "lambda": 0.147878828458488, "lambda_min": 0.11748509854078293, "lambda_max": 0.18665939569473267, "S_mean": 0.376220703125, "G_mean": 0.5035647389789423, "G_std": 0.17690868691230813, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.520666599273682, "grad_norm": 0.596383810043335, "tokens_seen": 13934592, "tokens_per_sec": 18475.989862386705, "vram_gb": 5.684933632, "lambda": 0.14831875450909138, "lambda_min": 0.11747117340564728, "lambda_max": 0.1879865527153015, "S_mean": 0.4115804036458333, "G_mean": 0.4974391457314293, "G_std": 0.1743831648491323, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.393148899078369, "grad_norm": 0.5712218880653381, "tokens_seen": 14098432, "tokens_per_sec": 18472.966138903474, "vram_gb": 5.684933632, "lambda": 0.1487974381695191, "lambda_min": 0.11715447902679443, "lambda_max": 0.1893395185470581, "S_mean": 0.3893534342447917, "G_mean": 0.5127755999565125, "G_std": 0.17562403421228132, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.302258491516113, "grad_norm": 0.6319090723991394, "tokens_seen": 14262272, "tokens_per_sec": 18471.213062093848, "vram_gb": 5.684933632, "lambda": 0.14924805238842964, "lambda_min": 0.11694256216287613, "lambda_max": 0.19053402543067932, "S_mean": 0.4085693359375, "G_mean": 0.5032419729977846, "G_std": 0.17455663221577802, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.244015693664551, "grad_norm": 0.584231436252594, "tokens_seen": 14426112, "tokens_per_sec": 18469.515151217973, "vram_gb": 5.684933632, "lambda": 0.14969127997756004, "lambda_min": 0.11675123125314713, "lambda_max": 0.19154413044452667, "S_mean": 0.3848876953125, "G_mean": 0.49920669632653397, "G_std": 0.18024352627495924, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.396174430847168, "grad_norm": 0.5558505058288574, "tokens_seen": 14589952, "tokens_per_sec": 18523.535760159106, "vram_gb": 5.684933632, "lambda": 0.15015929068128267, "lambda_min": 0.11638003587722778, "lambda_max": 0.1928337663412094, "S_mean": 0.3832804361979167, "G_mean": 0.5004452212403218, "G_std": 0.17265111782277623, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.303875207901001, "grad_norm": 0.5481938123703003, "tokens_seen": 14753792, "tokens_per_sec": 18520.434022044803, "vram_gb": 5.684933632, "lambda": 0.15061559341847897, "lambda_min": 0.11637649685144424, "lambda_max": 0.1939966231584549, "S_mean": 0.40887451171875, "G_mean": 0.5037053988004724, "G_std": 0.17326601563642421, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.253141641616821, "grad_norm": 0.548874020576477, "tokens_seen": 14917632, "tokens_per_sec": 18516.832687456485, "vram_gb": 5.684933632, "lambda": 0.15110963489860296, "lambda_min": 0.11624506115913391, "lambda_max": 0.1954503059387207, "S_mean": 0.4049275716145833, "G_mean": 0.4981577967604001, "G_std": 0.17449302831664681, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.415846347808838, "grad_norm": 0.5756357312202454, "tokens_seen": 15081472, "tokens_per_sec": 18514.42914160853, "vram_gb": 5.684933632, "lambda": 0.15161037848641476, "lambda_min": 0.11603491753339767, "lambda_max": 0.19665607810020447, "S_mean": 0.4167887369791667, "G_mean": 0.4946722537279129, "G_std": 0.17742512514814734, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.379937648773193, "grad_norm": 0.6094342470169067, "tokens_seen": 15245312, "tokens_per_sec": 18511.989930228272, "vram_gb": 5.684933632, "lambda": 0.15209209950019917, "lambda_min": 0.11571820080280304, "lambda_max": 0.19774693250656128, "S_mean": 0.4215291341145833, "G_mean": 0.5029975604265928, "G_std": 0.17193880421109498, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.269054651260376, "grad_norm": 0.5593279004096985, "tokens_seen": 15409152, "tokens_per_sec": 18508.542887004267, "vram_gb": 5.684933632, "lambda": 0.15254663086185852, "lambda_min": 0.1152762919664383, "lambda_max": 0.19897761940956116, "S_mean": 0.4053141276041667, "G_mean": 0.486190989613533, "G_std": 0.1727474539851149, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.192092180252075, "grad_norm": 0.5354211926460266, "tokens_seen": 15572992, "tokens_per_sec": 18508.087105540617, "vram_gb": 5.684933632, "lambda": 0.15295248478651047, "lambda_min": 0.11491747200489044, "lambda_max": 0.20029708743095398, "S_mean": 0.3886311848958333, "G_mean": 0.49598497586945695, "G_std": 0.1699097533710301, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.290166616439819, "grad_norm": 0.551881730556488, "tokens_seen": 15736832, "tokens_per_sec": 18504.235604051322, "vram_gb": 5.684933632, "lambda": 0.1533584324643016, "lambda_min": 0.11439384520053864, "lambda_max": 0.20146510004997253, "S_mean": 0.40020751953125, "G_mean": 0.47944036809106666, "G_std": 0.17091274680569768, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.407505512237549, "grad_norm": 0.5880167484283447, "tokens_seen": 15900672, "tokens_per_sec": 18500.57577242698, "vram_gb": 5.684933632, "lambda": 0.15379353209088245, "lambda_min": 0.11389501392841339, "lambda_max": 0.20283623039722443, "S_mean": 0.4034220377604167, "G_mean": 0.47853625441590947, "G_std": 0.17289692365253964, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.3451457023620605, "grad_norm": 0.6038150191307068, "tokens_seen": 16064512, "tokens_per_sec": 18499.422256941165, "vram_gb": 5.684933632, "lambda": 0.15428681982060274, "lambda_min": 0.11384426802396774, "lambda_max": 0.20431073009967804, "S_mean": 0.3885701497395833, "G_mean": 0.4896544289464752, "G_std": 0.17253484390676022, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.085910797119141, "grad_norm": 0.5590838193893433, "tokens_seen": 16228352, "tokens_per_sec": 18497.656520983146, "vram_gb": 5.684933632, "lambda": 0.154761364373068, "lambda_min": 0.11328679323196411, "lambda_max": 0.20594604313373566, "S_mean": 0.37896728515625, "G_mean": 0.47815420882155496, "G_std": 0.1726913338061422, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.326976537704468, "grad_norm": 0.5551609396934509, "tokens_seen": 16392192, "tokens_per_sec": 18496.60720610489, "vram_gb": 5.684933632, "lambda": 0.15527928372224173, "lambda_min": 0.11322404444217682, "lambda_max": 0.20716354250907898, "S_mean": 0.3924967447916667, "G_mean": 0.47678756589690846, "G_std": 0.1749884670910736, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.339097285270691, "val_ppl": 208.32456772657778, "lambda": 0.15531065501272678, "lambda_min": 0.11321789026260376, "lambda_max": 0.20723511278629303, "lambda_per_layer": [0.15371182560920715, 0.13892801105976105, 0.13244684040546417, 0.1355598419904709, 0.12345565110445023, 0.11873839050531387, 0.12963464856147766, 0.11321789026260376, 0.1144627034664154, 0.13928891718387604, 0.20301614701747894, 0.17376989126205444, 0.1937839835882187, 0.16176246106624603, 0.18792472779750824, 0.1940060704946518, 0.20723511278629303, 0.1652529537677765, 0.1943381428718567, 0.14137709140777588, 0.15137779712677002, 0.14265747368335724, 0.17112906277179718, 0.14038008451461792], "S_mean": 0.3820088704427083, "G_mean": 0.465896088629961, "G_std": 0.1734843187344571, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.3477935791015625, "grad_norm": 0.5718247890472412, "tokens_seen": 16556032, "tokens_per_sec": 18460.621485470612, "vram_gb": 5.684933632, "lambda": 0.15576473933955035, "lambda_min": 0.11279964447021484, "lambda_max": 0.2085520327091217, "S_mean": 0.3953450520833333, "G_mean": 0.47277219593524933, "G_std": 0.1735306562234958, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.375095367431641, "grad_norm": 0.559147298336029, "tokens_seen": 16719872, "tokens_per_sec": 18459.835391747114, "vram_gb": 5.684933632, "lambda": 0.15621777468671402, "lambda_min": 0.11239967495203018, "lambda_max": 0.2097223997116089, "S_mean": 0.38751220703125, "G_mean": 0.482533598318696, "G_std": 0.16938581814368567, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.276080131530762, "grad_norm": 0.619985044002533, "tokens_seen": 16883712, "tokens_per_sec": 18457.847816427828, "vram_gb": 5.684933632, "lambda": 0.156664385770758, "lambda_min": 0.11182969808578491, "lambda_max": 0.21126064658164978, "S_mean": 0.4086100260416667, "G_mean": 0.478086088784039, "G_std": 0.1687143238571783, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.182413578033447, "grad_norm": 0.6495790481567383, "tokens_seen": 17047552, "tokens_per_sec": 18456.133245021152, "vram_gb": 5.684933632, "lambda": 0.1571388371909658, "lambda_min": 0.11139300465583801, "lambda_max": 0.21255721151828766, "S_mean": 0.4034016927083333, "G_mean": 0.47430708911269903, "G_std": 0.16784575441852212, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.2638585567474365, "grad_norm": 0.5829983353614807, "tokens_seen": 17211392, "tokens_per_sec": 18455.4098933246, "vram_gb": 5.684933632, "lambda": 0.15766878860692182, "lambda_min": 0.11115796864032745, "lambda_max": 0.21396863460540771, "S_mean": 0.391387939453125, "G_mean": 0.4709221289182703, "G_std": 0.16862670107123753, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.314461946487427, "grad_norm": 0.5884328484535217, "tokens_seen": 17375232, "tokens_per_sec": 18453.974181374873, "vram_gb": 5.684933632, "lambda": 0.15816919784992933, "lambda_min": 0.11070400476455688, "lambda_max": 0.21560567617416382, "S_mean": 0.4111328125, "G_mean": 0.47507063873733085, "G_std": 0.170179044517378, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.121938705444336, "grad_norm": 0.5749759674072266, "tokens_seen": 17539072, "tokens_per_sec": 18454.198188484093, "vram_gb": 5.684933632, "lambda": 0.15857153727362552, "lambda_min": 0.11029402166604996, "lambda_max": 0.21692737936973572, "S_mean": 0.4233601888020833, "G_mean": 0.46388923687239486, "G_std": 0.1701923681733509, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.1022186279296875, "grad_norm": 0.5956858396530151, "tokens_seen": 17702912, "tokens_per_sec": 18453.43820435828, "vram_gb": 5.684933632, "lambda": 0.15898399893194437, "lambda_min": 0.1098095253109932, "lambda_max": 0.21818675100803375, "S_mean": 0.403045654296875, "G_mean": 0.4685168449456493, "G_std": 0.16803805615442494, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.141319751739502, "grad_norm": 0.5982304215431213, "tokens_seen": 17866752, "tokens_per_sec": 18453.51633706591, "vram_gb": 5.684933632, "lambda": 0.15945726167410612, "lambda_min": 0.1095309928059578, "lambda_max": 0.2196168750524521, "S_mean": 0.4022623697916667, "G_mean": 0.46697180656095344, "G_std": 0.1673115063458681, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.314951419830322, "grad_norm": 0.6008784174919128, "tokens_seen": 18030592, "tokens_per_sec": 18452.571431280787, "vram_gb": 5.684933632, "lambda": 0.16001113317906857, "lambda_min": 0.1091056615114212, "lambda_max": 0.22105096280574799, "S_mean": 0.423828125, "G_mean": 0.4637915963927905, "G_std": 0.16596792389949164, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.250249624252319, "grad_norm": 0.616727352142334, "tokens_seen": 18194432, "tokens_per_sec": 18452.294940765893, "vram_gb": 5.684933632, "lambda": 0.1605066688110431, "lambda_min": 0.10896102339029312, "lambda_max": 0.2224409580230713, "S_mean": 0.4083048502604167, "G_mean": 0.45119408269723255, "G_std": 0.17077920873028538, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.24396538734436, "grad_norm": 0.6528480052947998, "tokens_seen": 18358272, "tokens_per_sec": 18452.25946599638, "vram_gb": 5.684933632, "lambda": 0.16095415378610292, "lambda_min": 0.1086067259311676, "lambda_max": 0.2237108200788498, "S_mean": 0.4190470377604167, "G_mean": 0.4665068353836735, "G_std": 0.1677902132893602, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.107348680496216, "grad_norm": 0.6316291093826294, "tokens_seen": 18522112, "tokens_per_sec": 18450.295637682706, "vram_gb": 5.684933632, "lambda": 0.16141158497581878, "lambda_min": 0.10810735076665878, "lambda_max": 0.22537903487682343, "S_mean": 0.39813232421875, "G_mean": 0.4652114389464259, "G_std": 0.16863201651722193, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.198465347290039, "grad_norm": 0.7797390222549438, "tokens_seen": 18685952, "tokens_per_sec": 18447.949965380274, "vram_gb": 5.684933632, "lambda": 0.16190082424630722, "lambda_min": 0.10768044739961624, "lambda_max": 0.22690078616142273, "S_mean": 0.4242757161458333, "G_mean": 0.46641289566953975, "G_std": 0.16696066657702127, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.248617887496948, "grad_norm": 0.6104199886322021, "tokens_seen": 18849792, "tokens_per_sec": 18446.41945132491, "vram_gb": 5.684933632, "lambda": 0.16239101563890776, "lambda_min": 0.1074298620223999, "lambda_max": 0.22815124690532684, "S_mean": 0.4096272786458333, "G_mean": 0.46345187289019424, "G_std": 0.16641484930490455, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.164412975311279, "grad_norm": 0.6729648113250732, "tokens_seen": 19013632, "tokens_per_sec": 18446.070835401144, "vram_gb": 5.684933632, "lambda": 0.1628605875497063, "lambda_min": 0.10705021023750305, "lambda_max": 0.22963719069957733, "S_mean": 0.4028116861979167, "G_mean": 0.4583744714036584, "G_std": 0.16797319535786906, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 4.967403888702393, "grad_norm": 0.6118532419204712, "tokens_seen": 19177472, "tokens_per_sec": 18445.404891570503, "vram_gb": 5.684933632, "lambda": 0.16333289972196022, "lambda_min": 0.10679297894239426, "lambda_max": 0.23113368451595306, "S_mean": 0.4132283528645833, "G_mean": 0.4556826410504679, "G_std": 0.16571366352339587, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.200300931930542, "grad_norm": 0.621135413646698, "tokens_seen": 19341312, "tokens_per_sec": 18445.162088817855, "vram_gb": 5.684933632, "lambda": 0.16382271641244492, "lambda_min": 0.10644567012786865, "lambda_max": 0.23263342678546906, "S_mean": 0.3928426106770833, "G_mean": 0.4499886967241764, "G_std": 0.1640518830778698, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.116201162338257, "grad_norm": 0.6615944504737854, "tokens_seen": 19505152, "tokens_per_sec": 18443.044111761687, "vram_gb": 5.684933632, "lambda": 0.16429761486748853, "lambda_min": 0.10632394254207611, "lambda_max": 0.2340618073940277, "S_mean": 0.4328816731770833, "G_mean": 0.4636841270451744, "G_std": 0.16824894522627196, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.064156532287598, "grad_norm": 0.5870950818061829, "tokens_seen": 19668992, "tokens_per_sec": 18440.92202440447, "vram_gb": 5.684933632, "lambda": 0.1647541873777906, "lambda_min": 0.10623303055763245, "lambda_max": 0.23550738394260406, "S_mean": 0.413818359375, "G_mean": 0.4609002306436499, "G_std": 0.16718423614899316, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.077032089233398, "grad_norm": 0.6552935242652893, "tokens_seen": 19832832, "tokens_per_sec": 18439.322730902782, "vram_gb": 5.684933632, "lambda": 0.16524210603286824, "lambda_min": 0.10602451115846634, "lambda_max": 0.23703019320964813, "S_mean": 0.3968709309895833, "G_mean": 0.4543470572680235, "G_std": 0.16851527484444281, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.123900651931763, "grad_norm": 0.6363755464553833, "tokens_seen": 19996672, "tokens_per_sec": 18438.64611757511, "vram_gb": 5.684933632, "lambda": 0.1656868401914835, "lambda_min": 0.10585928708314896, "lambda_max": 0.23836520314216614, "S_mean": 0.404052734375, "G_mean": 0.45704827333490056, "G_std": 0.16377579816617072, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.0653462409973145, "grad_norm": 0.5881608724594116, "tokens_seen": 20160512, "tokens_per_sec": 18438.515056713233, "vram_gb": 5.684933632, "lambda": 0.16611077170819044, "lambda_min": 0.10528169572353363, "lambda_max": 0.2397586703300476, "S_mean": 0.4048055013020833, "G_mean": 0.4449260321756204, "G_std": 0.16391575445110598, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.07970118522644, "grad_norm": 0.5972563624382019, "tokens_seen": 20324352, "tokens_per_sec": 18436.761940685654, "vram_gb": 5.684933632, "lambda": 0.16660322217891613, "lambda_min": 0.10475578904151917, "lambda_max": 0.24066676199436188, "S_mean": 0.3952433268229167, "G_mean": 0.4482532699281971, "G_std": 0.1656199850452443, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.102304458618164, "grad_norm": 0.6416434049606323, "tokens_seen": 20488192, "tokens_per_sec": 18436.865673939967, "vram_gb": 5.684933632, "lambda": 0.167030718177557, "lambda_min": 0.10431529581546783, "lambda_max": 0.24165888130664825, "S_mean": 0.4105021158854167, "G_mean": 0.44357009946058196, "G_std": 0.1663005369094511, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.205454349517822, "grad_norm": 0.602102518081665, "tokens_seen": 20652032, "tokens_per_sec": 18436.334034070558, "vram_gb": 5.684933632, "lambda": 0.16749307047575712, "lambda_min": 0.10403844714164734, "lambda_max": 0.2428217977285385, "S_mean": 0.4012247721354167, "G_mean": 0.43692943174391985, "G_std": 0.1713326204723368, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.040850639343262, "grad_norm": 0.6061774492263794, "tokens_seen": 20815872, "tokens_per_sec": 18435.441300930957, "vram_gb": 5.684933632, "lambda": 0.16796364728361368, "lambda_min": 0.10379993170499802, "lambda_max": 0.24432899057865143, "S_mean": 0.3843180338541667, "G_mean": 0.44534921009714407, "G_std": 0.16653868649154902, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.171906471252441, "grad_norm": 0.7148247957229614, "tokens_seen": 20979712, "tokens_per_sec": 18435.800564012457, "vram_gb": 5.684933632, "lambda": 0.16845977244277796, "lambda_min": 0.10328949242830276, "lambda_max": 0.24573242664337158, "S_mean": 0.3939615885416667, "G_mean": 0.46042316406965256, "G_std": 0.16662009176798165, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.0208494663238525, "grad_norm": 0.6395888924598694, "tokens_seen": 21143552, "tokens_per_sec": 18433.432201920197, "vram_gb": 5.684933632, "lambda": 0.168928362429142, "lambda_min": 0.10279696434736252, "lambda_max": 0.24677447974681854, "S_mean": 0.4021809895833333, "G_mean": 0.44091110738615197, "G_std": 0.16652631550095975, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.085041284561157, "grad_norm": 0.6694987416267395, "tokens_seen": 21307392, "tokens_per_sec": 18432.187218056442, "vram_gb": 5.684933632, "lambda": 0.1694660515834888, "lambda_min": 0.10286812484264374, "lambda_max": 0.24784639477729797, "S_mean": 0.4061075846354167, "G_mean": 0.4511012779548764, "G_std": 0.16796545366135737, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.9870970249176025, "grad_norm": 0.6015433073043823, "tokens_seen": 21471232, "tokens_per_sec": 18431.84609963653, "vram_gb": 5.684933632, "lambda": 0.16993656195700169, "lambda_min": 0.10284414142370224, "lambda_max": 0.24886852502822876, "S_mean": 0.3873697916666667, "G_mean": 0.44406686040262383, "G_std": 0.1661487837942938, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 4.948133945465088, "grad_norm": 0.7105032801628113, "tokens_seen": 21635072, "tokens_per_sec": 18430.42171096664, "vram_gb": 5.684933632, "lambda": 0.17036662995815277, "lambda_min": 0.102753184735775, "lambda_max": 0.25030311942100525, "S_mean": 0.4037679036458333, "G_mean": 0.4365022995819648, "G_std": 0.1677707921868811, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.051587343215942, "grad_norm": 0.6146185994148254, "tokens_seen": 21798912, "tokens_per_sec": 18430.30250730614, "vram_gb": 5.684933632, "lambda": 0.1708889218668143, "lambda_min": 0.10264290869235992, "lambda_max": 0.2517670691013336, "S_mean": 0.4048868815104167, "G_mean": 0.44255409700175125, "G_std": 0.16411769576370716, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.036553382873535, "grad_norm": 0.607150673866272, "tokens_seen": 21962752, "tokens_per_sec": 18430.044634085174, "vram_gb": 5.684933632, "lambda": 0.17134492316593727, "lambda_min": 0.1024797335267067, "lambda_max": 0.25301340222358704, "S_mean": 0.3841756184895833, "G_mean": 0.45262953514854115, "G_std": 0.16463464257928231, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.026325702667236, "grad_norm": 0.6523415446281433, "tokens_seen": 22126592, "tokens_per_sec": 18429.66090371762, "vram_gb": 5.684933632, "lambda": 0.17183905560523272, "lambda_min": 0.10231509804725647, "lambda_max": 0.2542578876018524, "S_mean": 0.3985392252604167, "G_mean": 0.43809363214919966, "G_std": 0.1647857225034386, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.105034589767456, "grad_norm": 0.7402066588401794, "tokens_seen": 22290432, "tokens_per_sec": 18429.554140515025, "vram_gb": 5.684933632, "lambda": 0.17232291772961617, "lambda_min": 0.10218020528554916, "lambda_max": 0.2554352879524231, "S_mean": 0.42364501953125, "G_mean": 0.444405479511867, "G_std": 0.16423840432738265, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.075162887573242, "grad_norm": 0.6418949365615845, "tokens_seen": 22454272, "tokens_per_sec": 18429.131530616298, "vram_gb": 5.684933632, "lambda": 0.17284341311703125, "lambda_min": 0.10168302059173584, "lambda_max": 0.2566573917865753, "S_mean": 0.395660400390625, "G_mean": 0.43679656243572634, "G_std": 0.16786694286080697, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.0743889808654785, "grad_norm": 0.6662697196006775, "tokens_seen": 22618112, "tokens_per_sec": 18429.030040114594, "vram_gb": 5.684933632, "lambda": 0.1733005562176307, "lambda_min": 0.10134933143854141, "lambda_max": 0.25802987813949585, "S_mean": 0.4173177083333333, "G_mean": 0.43310785355667275, "G_std": 0.16394608141854405, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.01816725730896, "grad_norm": 0.6489134430885315, "tokens_seen": 22781952, "tokens_per_sec": 18429.10632487035, "vram_gb": 5.684933632, "lambda": 0.17377315492679676, "lambda_min": 0.10123863816261292, "lambda_max": 0.2592928111553192, "S_mean": 0.3960367838541667, "G_mean": 0.43439330408970517, "G_std": 0.16582675899068514, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.97951078414917, "grad_norm": 0.7038235068321228, "tokens_seen": 22945792, "tokens_per_sec": 18427.52389301149, "vram_gb": 5.684933632, "lambda": 0.1742748220761617, "lambda_min": 0.1011672168970108, "lambda_max": 0.26053205132484436, "S_mean": 0.3939005533854167, "G_mean": 0.4304962479509413, "G_std": 0.16534219806392988, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.109642744064331, "grad_norm": 0.6353573203086853, "tokens_seen": 23109632, "tokens_per_sec": 18426.716685200965, "vram_gb": 5.684933632, "lambda": 0.17479864849398533, "lambda_min": 0.10108477622270584, "lambda_max": 0.2618555724620819, "S_mean": 0.3849080403645833, "G_mean": 0.43422481960927445, "G_std": 0.16571187569449344, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.069643020629883, "grad_norm": 0.62350994348526, "tokens_seen": 23273472, "tokens_per_sec": 18426.47809325915, "vram_gb": 5.684933632, "lambda": 0.17533959603557983, "lambda_min": 0.10104291886091232, "lambda_max": 0.2630457282066345, "S_mean": 0.38897705078125, "G_mean": 0.4356282519487043, "G_std": 0.16423229780048132, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.927711486816406, "grad_norm": 0.6525055170059204, "tokens_seen": 23437312, "tokens_per_sec": 18424.899790070158, "vram_gb": 5.684933632, "lambda": 0.17586671592046818, "lambda_min": 0.10098245739936829, "lambda_max": 0.26474398374557495, "S_mean": 0.3976033528645833, "G_mean": 0.4279988951360186, "G_std": 0.1641893538956841, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 4.975577354431152, "grad_norm": 0.6721015572547913, "tokens_seen": 23601152, "tokens_per_sec": 18423.94598302411, "vram_gb": 5.684933632, "lambda": 0.17634254538764557, "lambda_min": 0.10086706280708313, "lambda_max": 0.2661159038543701, "S_mean": 0.36187744140625, "G_mean": 0.43101070417712134, "G_std": 0.16680174320936203, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.84627628326416, "grad_norm": 0.6339317560195923, "tokens_seen": 23764992, "tokens_per_sec": 18423.51643741874, "vram_gb": 5.684933632, "lambda": 0.1768145483608047, "lambda_min": 0.10080394148826599, "lambda_max": 0.2676182687282562, "S_mean": 0.3831583658854167, "G_mean": 0.43138939049094915, "G_std": 0.16348323311346272, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.963404893875122, "grad_norm": 0.6401458978652954, "tokens_seen": 23928832, "tokens_per_sec": 18422.09474621787, "vram_gb": 5.684933632, "lambda": 0.1772152759755651, "lambda_min": 0.10108331590890884, "lambda_max": 0.2687712609767914, "S_mean": 0.3877970377604167, "G_mean": 0.42836759286001325, "G_std": 0.16245958612610897, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.010130882263184, "grad_norm": 0.6366609334945679, "tokens_seen": 24092672, "tokens_per_sec": 18420.702461546367, "vram_gb": 5.684933632, "lambda": 0.17764333511392275, "lambda_min": 0.10100825130939484, "lambda_max": 0.2699849009513855, "S_mean": 0.38153076171875, "G_mean": 0.42725764277080697, "G_std": 0.16613593247408667, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.849521160125732, "grad_norm": 0.7257722020149231, "tokens_seen": 24256512, "tokens_per_sec": 18419.150965650642, "vram_gb": 5.684933632, "lambda": 0.17811867905159792, "lambda_min": 0.10115128010511398, "lambda_max": 0.27133312821388245, "S_mean": 0.3947041829427083, "G_mean": 0.43923400063067675, "G_std": 0.16150755062699318, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.775094509124756, "grad_norm": 0.8661393523216248, "tokens_seen": 24420352, "tokens_per_sec": 18418.886729775724, "vram_gb": 5.684933632, "lambda": 0.1786124793191751, "lambda_min": 0.1012975350022316, "lambda_max": 0.2725354731082916, "S_mean": 0.393310546875, "G_mean": 0.41878064069896936, "G_std": 0.1655309203391274, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.039130687713623, "grad_norm": 0.6863287687301636, "tokens_seen": 24584192, "tokens_per_sec": 18419.21042846184, "vram_gb": 5.684933632, "lambda": 0.1790725439786911, "lambda_min": 0.10110867023468018, "lambda_max": 0.27356043457984924, "S_mean": 0.37420654296875, "G_mean": 0.42783546323577565, "G_std": 0.16370901698246598, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 4.996057486534118, "val_ppl": 147.82919013346853, "lambda": 0.17910088102022806, "lambda_min": 0.1011166051030159, "lambda_max": 0.27364933490753174, "lambda_per_layer": [0.15979304909706116, 0.14591343700885773, 0.1336611956357956, 0.14126765727996826, 0.12296874821186066, 0.11806882172822952, 0.12671999633312225, 0.1011166051030159, 0.106011301279068, 0.1425892859697342, 0.24896273016929626, 0.21907596290111542, 0.24619370698928833, 0.19629721343517303, 0.2500796318054199, 0.26338890194892883, 0.27364933490753174, 0.20148074626922607, 0.25289386510849, 0.15983135998249054, 0.17323873937129974, 0.1572413146495819, 0.20552614331245422, 0.15245139598846436], "S_mean": 0.3852132161458333, "G_mean": 0.4359427109981577, "G_std": 0.16359886953917643, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.098680734634399, "grad_norm": 0.717848539352417, "tokens_seen": 24748032, "tokens_per_sec": 18394.59328476968, "vram_gb": 5.684933632, "lambda": 0.17960365768522024, "lambda_min": 0.10133430361747742, "lambda_max": 0.27529609203338623, "S_mean": 0.3935445149739583, "G_mean": 0.42852017330005765, "G_std": 0.16534355383676788, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.81731653213501, "grad_norm": 0.6218186020851135, "tokens_seen": 24911872, "tokens_per_sec": 18393.879590359404, "vram_gb": 5.684933632, "lambda": 0.18008054979145527, "lambda_min": 0.1012006476521492, "lambda_max": 0.27710506319999695, "S_mean": 0.403717041015625, "G_mean": 0.4196814966077606, "G_std": 0.16402885325563452, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.9053730964660645, "grad_norm": 0.6755521893501282, "tokens_seen": 25075712, "tokens_per_sec": 18393.09650819771, "vram_gb": 5.684933632, "lambda": 0.18055547835926214, "lambda_min": 0.10108870267868042, "lambda_max": 0.2785934507846832, "S_mean": 0.392303466796875, "G_mean": 0.429230282548815, "G_std": 0.16133573705640933, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.8502397537231445, "grad_norm": 0.6540315747261047, "tokens_seen": 25239552, "tokens_per_sec": 18390.780191598667, "vram_gb": 5.684933632, "lambda": 0.1811045091599226, "lambda_min": 0.10088616609573364, "lambda_max": 0.27968212962150574, "S_mean": 0.38525390625, "G_mean": 0.4260876369662583, "G_std": 0.1659464786450068, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.022327423095703, "grad_norm": 0.800730288028717, "tokens_seen": 25403392, "tokens_per_sec": 18390.42473044081, "vram_gb": 5.684933632, "lambda": 0.18162075243890285, "lambda_min": 0.10072878003120422, "lambda_max": 0.2811240255832672, "S_mean": 0.3817036946614583, "G_mean": 0.4182443098785977, "G_std": 0.16405719278069833, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.749115228652954, "grad_norm": 0.7263131737709045, "tokens_seen": 25567232, "tokens_per_sec": 18384.420295411845, "vram_gb": 5.684933632, "lambda": 0.18212779456128678, "lambda_min": 0.10057920217514038, "lambda_max": 0.282463401556015, "S_mean": 0.3887939453125, "G_mean": 0.4146326758588354, "G_std": 0.16172284244870147, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.899272441864014, "grad_norm": 0.7061173319816589, "tokens_seen": 25731072, "tokens_per_sec": 18384.098381293163, "vram_gb": 5.684933632, "lambda": 0.1825281037017703, "lambda_min": 0.10059704631567001, "lambda_max": 0.28373777866363525, "S_mean": 0.3980509440104167, "G_mean": 0.43241047812625766, "G_std": 0.16222049163964888, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.826907396316528, "grad_norm": 0.7039152383804321, "tokens_seen": 25894912, "tokens_per_sec": 18384.292132908777, "vram_gb": 5.684933632, "lambda": 0.18301562933872143, "lambda_min": 0.1003456637263298, "lambda_max": 0.2852706015110016, "S_mean": 0.3807169596354167, "G_mean": 0.431648048106581, "G_std": 0.16392205430505177, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.869389533996582, "grad_norm": 0.710051417350769, "tokens_seen": 26058752, "tokens_per_sec": 18383.07448839745, "vram_gb": 5.684933632, "lambda": 0.18356924504041672, "lambda_min": 0.1002715528011322, "lambda_max": 0.2865596413612366, "S_mean": 0.39208984375, "G_mean": 0.4205333637073636, "G_std": 0.1654931625816971, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.847876310348511, "grad_norm": 0.6884016990661621, "tokens_seen": 26222592, "tokens_per_sec": 18383.74222056523, "vram_gb": 5.684933632, "lambda": 0.18405138359715542, "lambda_min": 0.10038430243730545, "lambda_max": 0.2880498170852661, "S_mean": 0.3942057291666667, "G_mean": 0.4139159945771098, "G_std": 0.16795548098161817, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.9539079666137695, "grad_norm": 0.788023054599762, "tokens_seen": 26386432, "tokens_per_sec": 18383.402933304413, "vram_gb": 5.684933632, "lambda": 0.18462168363233408, "lambda_min": 0.10053088515996933, "lambda_max": 0.2895452082157135, "S_mean": 0.3854268391927083, "G_mean": 0.4255977653277417, "G_std": 0.16338907143411538, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.927061080932617, "grad_norm": 0.644773542881012, "tokens_seen": 26550272, "tokens_per_sec": 18380.70222089133, "vram_gb": 5.684933632, "lambda": 0.18509670936812958, "lambda_min": 0.10062818974256516, "lambda_max": 0.29066523909568787, "S_mean": 0.3907877604166667, "G_mean": 0.42780584655702114, "G_std": 0.16390967299230397, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.981693506240845, "grad_norm": 0.6703421473503113, "tokens_seen": 26714112, "tokens_per_sec": 18380.340167603455, "vram_gb": 5.684933632, "lambda": 0.1855221272756656, "lambda_min": 0.10058978199958801, "lambda_max": 0.2923053503036499, "S_mean": 0.3935546875, "G_mean": 0.4261376409170528, "G_std": 0.16113439466183385, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.798749208450317, "grad_norm": 0.6229957342147827, "tokens_seen": 26877952, "tokens_per_sec": 18380.414991096666, "vram_gb": 5.684933632, "lambda": 0.18598971391717592, "lambda_min": 0.10054624080657959, "lambda_max": 0.2936655879020691, "S_mean": 0.3699544270833333, "G_mean": 0.4201998362938563, "G_std": 0.16297400967838863, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.065388441085815, "grad_norm": 0.7205125093460083, "tokens_seen": 27041792, "tokens_per_sec": 18380.256952438216, "vram_gb": 5.684933632, "lambda": 0.186452383796374, "lambda_min": 0.10044821351766586, "lambda_max": 0.2950649857521057, "S_mean": 0.3843282063802083, "G_mean": 0.4143383130431175, "G_std": 0.16510337389384708, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.907260894775391, "grad_norm": 0.7224057912826538, "tokens_seen": 27205632, "tokens_per_sec": 18380.537770758005, "vram_gb": 5.684933632, "lambda": 0.1868442486350735, "lambda_min": 0.10025953501462936, "lambda_max": 0.29621487855911255, "S_mean": 0.3841959635416667, "G_mean": 0.4195767054334283, "G_std": 0.1626166911640515, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.859713792800903, "grad_norm": 0.6716839075088501, "tokens_seen": 27369472, "tokens_per_sec": 18380.58302872607, "vram_gb": 5.684933632, "lambda": 0.18725741716722646, "lambda_min": 0.09994899481534958, "lambda_max": 0.2974946200847626, "S_mean": 0.3917236328125, "G_mean": 0.4150834276030461, "G_std": 0.1637334996679177, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.897505760192871, "grad_norm": 0.69284588098526, "tokens_seen": 27533312, "tokens_per_sec": 18380.711491534377, "vram_gb": 5.684933632, "lambda": 0.1877978149180611, "lambda_min": 0.09982669353485107, "lambda_max": 0.29891204833984375, "S_mean": 0.39129638671875, "G_mean": 0.4116564186600347, "G_std": 0.1640859543501089, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.033458948135376, "grad_norm": 0.6909829378128052, "tokens_seen": 27697152, "tokens_per_sec": 18381.314387974442, "vram_gb": 5.684933632, "lambda": 0.18821058298150697, "lambda_min": 0.09956885129213333, "lambda_max": 0.2998456656932831, "S_mean": 0.3899129231770833, "G_mean": 0.4227605794245998, "G_std": 0.16245808762808642, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.973405599594116, "grad_norm": 0.7094146609306335, "tokens_seen": 27860992, "tokens_per_sec": 18380.993862986194, "vram_gb": 5.684933632, "lambda": 0.1887191260854403, "lambda_min": 0.09913355857133865, "lambda_max": 0.3017290532588959, "S_mean": 0.378265380859375, "G_mean": 0.4203516594134271, "G_std": 0.16177163738757372, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.91260027885437, "grad_norm": 0.6837196350097656, "tokens_seen": 28024832, "tokens_per_sec": 18380.803686882387, "vram_gb": 5.684933632, "lambda": 0.18919295879701772, "lambda_min": 0.09910959750413895, "lambda_max": 0.3031040132045746, "S_mean": 0.3859761555989583, "G_mean": 0.4176974833632509, "G_std": 0.16180128992224732, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.701168060302734, "grad_norm": 0.6542478799819946, "tokens_seen": 28188672, "tokens_per_sec": 18380.718307164592, "vram_gb": 5.684933632, "lambda": 0.1896978716055552, "lambda_min": 0.09914547950029373, "lambda_max": 0.3046261668205261, "S_mean": 0.3779703776041667, "G_mean": 0.405692917915682, "G_std": 0.1630079032232364, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.944692611694336, "grad_norm": 0.7380439639091492, "tokens_seen": 28352512, "tokens_per_sec": 18379.587007743758, "vram_gb": 5.684933632, "lambda": 0.19015980946520963, "lambda_min": 0.09918910264968872, "lambda_max": 0.3062094449996948, "S_mean": 0.3774922688802083, "G_mean": 0.4095097441847126, "G_std": 0.16034649219363928, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.938477993011475, "grad_norm": 0.6932872533798218, "tokens_seen": 28516352, "tokens_per_sec": 18379.377322771717, "vram_gb": 5.684933632, "lambda": 0.1906739752739668, "lambda_min": 0.09929008036851883, "lambda_max": 0.30786457657814026, "S_mean": 0.3770345052083333, "G_mean": 0.41404276915515464, "G_std": 0.16131958536182842, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 5.05128026008606, "grad_norm": 0.6972672343254089, "tokens_seen": 28680192, "tokens_per_sec": 18379.44132543694, "vram_gb": 5.684933632, "lambda": 0.19113452328989902, "lambda_min": 0.09947270900011063, "lambda_max": 0.3094756603240967, "S_mean": 0.3810831705729167, "G_mean": 0.4243252792706092, "G_std": 0.16127196156109372, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.81901216506958, "grad_norm": 0.7404661774635315, "tokens_seen": 28844032, "tokens_per_sec": 18379.12875775065, "vram_gb": 5.684933632, "lambda": 0.19158356202145418, "lambda_min": 0.09919849783182144, "lambda_max": 0.31076672673225403, "S_mean": 0.3745930989583333, "G_mean": 0.41158488346263766, "G_std": 0.1624438299331814, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.835433006286621, "grad_norm": 0.7209641337394714, "tokens_seen": 29007872, "tokens_per_sec": 18379.330237879767, "vram_gb": 5.684933632, "lambda": 0.19206356847037873, "lambda_min": 0.09894294291734695, "lambda_max": 0.31205469369888306, "S_mean": 0.3809305826822917, "G_mean": 0.40897127287462354, "G_std": 0.16065229840266207, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.887023448944092, "grad_norm": 0.75126051902771, "tokens_seen": 29171712, "tokens_per_sec": 18378.377045012086, "vram_gb": 5.684933632, "lambda": 0.19251792194942632, "lambda_min": 0.09908480942249298, "lambda_max": 0.31293126940727234, "S_mean": 0.3780721028645833, "G_mean": 0.4083167288141946, "G_std": 0.16283138965566954, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.9019646644592285, "grad_norm": 0.7029838562011719, "tokens_seen": 29335552, "tokens_per_sec": 18377.854751973744, "vram_gb": 5.684933632, "lambda": 0.19298928106824556, "lambda_min": 0.09880290925502777, "lambda_max": 0.314523309469223, "S_mean": 0.3819580078125, "G_mean": 0.4070359130079548, "G_std": 0.1637430542614311, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.882180452346802, "grad_norm": 0.7014172673225403, "tokens_seen": 29499392, "tokens_per_sec": 18377.35148446268, "vram_gb": 5.684933632, "lambda": 0.1934200127919515, "lambda_min": 0.098684161901474, "lambda_max": 0.31633058190345764, "S_mean": 0.3856302897135417, "G_mean": 0.4068047176115215, "G_std": 0.16266248631291091, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.881134510040283, "grad_norm": 0.6671891808509827, "tokens_seen": 29663232, "tokens_per_sec": 18376.45245393413, "vram_gb": 5.684933632, "lambda": 0.19393691793084145, "lambda_min": 0.09866299480199814, "lambda_max": 0.31799575686454773, "S_mean": 0.3732401529947917, "G_mean": 0.4084977060556412, "G_std": 0.1631100857630372, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.791919708251953, "grad_norm": 0.6701276302337646, "tokens_seen": 29827072, "tokens_per_sec": 18376.416449962588, "vram_gb": 5.684933632, "lambda": 0.19448356175174317, "lambda_min": 0.09864422678947449, "lambda_max": 0.31965136528015137, "S_mean": 0.3660176595052083, "G_mean": 0.41052263385305804, "G_std": 0.15947785356547683, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.993736982345581, "grad_norm": 0.7161394953727722, "tokens_seen": 29990912, "tokens_per_sec": 18376.47720168992, "vram_gb": 5.684933632, "lambda": 0.1949152760207653, "lambda_min": 0.09857703000307083, "lambda_max": 0.3207242488861084, "S_mean": 0.3703460693359375, "G_mean": 0.41558714692170423, "G_std": 0.16039845091290772, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.833190202713013, "grad_norm": 0.6931109428405762, "tokens_seen": 30154752, "tokens_per_sec": 18376.00800489382, "vram_gb": 5.684933632, "lambda": 0.19535534580548605, "lambda_min": 0.09842034429311752, "lambda_max": 0.3219769299030304, "S_mean": 0.3823750813802083, "G_mean": 0.40799606319827336, "G_std": 0.16049497132189572, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.985635995864868, "grad_norm": 0.6925478577613831, "tokens_seen": 30318592, "tokens_per_sec": 18376.73649073386, "vram_gb": 5.684933632, "lambda": 0.19586002423117557, "lambda_min": 0.0985562652349472, "lambda_max": 0.32352975010871887, "S_mean": 0.3683675130208333, "G_mean": 0.4145556873021026, "G_std": 0.16215955427226922, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.936825275421143, "grad_norm": 0.7338038086891174, "tokens_seen": 30482432, "tokens_per_sec": 18376.393254707797, "vram_gb": 5.684933632, "lambda": 0.19633463863283396, "lambda_min": 0.09892191737890244, "lambda_max": 0.3246082365512848, "S_mean": 0.38861083984375, "G_mean": 0.4131518246140331, "G_std": 0.16147025620254377, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.881010055541992, "grad_norm": 0.7415544986724854, "tokens_seen": 30646272, "tokens_per_sec": 18376.153495260936, "vram_gb": 5.684933632, "lambda": 0.19679298376043639, "lambda_min": 0.09900882095098495, "lambda_max": 0.32578766345977783, "S_mean": 0.3823445638020833, "G_mean": 0.41025348271553713, "G_std": 0.1602822580995659, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.784968376159668, "grad_norm": 0.6971950531005859, "tokens_seen": 30810112, "tokens_per_sec": 18376.53156150016, "vram_gb": 5.684933632, "lambda": 0.19731919560581446, "lambda_min": 0.09905494004487991, "lambda_max": 0.32713887095451355, "S_mean": 0.3855387369791667, "G_mean": 0.40633049444295466, "G_std": 0.16139482629174987, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.748073577880859, "grad_norm": 0.7294095158576965, "tokens_seen": 30973952, "tokens_per_sec": 18376.419491744848, "vram_gb": 5.684933632, "lambda": 0.19779325587054095, "lambda_min": 0.09924416989088058, "lambda_max": 0.32813760638237, "S_mean": 0.3858439127604167, "G_mean": 0.40265938530986506, "G_std": 0.16087240322182575, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.905496597290039, "grad_norm": 0.7291263937950134, "tokens_seen": 31137792, "tokens_per_sec": 18377.284701780965, "vram_gb": 5.684933632, "lambda": 0.19823697892328104, "lambda_min": 0.09898549318313599, "lambda_max": 0.3294758200645447, "S_mean": 0.3810017903645833, "G_mean": 0.40212862682528794, "G_std": 0.16122474040215215, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 5.004031419754028, "grad_norm": 0.7064239978790283, "tokens_seen": 31301632, "tokens_per_sec": 18377.816522184006, "vram_gb": 5.684933632, "lambda": 0.19875236134976149, "lambda_min": 0.09889458864927292, "lambda_max": 0.331369012594223, "S_mean": 0.3788655598958333, "G_mean": 0.4106232187865923, "G_std": 0.15999325461840877, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.896453857421875, "grad_norm": 0.7210467457771301, "tokens_seen": 31465472, "tokens_per_sec": 18377.713254237333, "vram_gb": 5.684933632, "lambda": 0.19917875186850628, "lambda_min": 0.09884551167488098, "lambda_max": 0.33227571845054626, "S_mean": 0.3972371419270833, "G_mean": 0.4088990224214892, "G_std": 0.1592275632234911, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.848192930221558, "grad_norm": 0.6861921548843384, "tokens_seen": 31629312, "tokens_per_sec": 18377.241146252687, "vram_gb": 5.684933632, "lambda": 0.19970943654576936, "lambda_min": 0.09864574670791626, "lambda_max": 0.33358821272850037, "S_mean": 0.3838704427083333, "G_mean": 0.39758297164614004, "G_std": 0.16226892406120896, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.87947154045105, "grad_norm": 0.7317802906036377, "tokens_seen": 31793152, "tokens_per_sec": 18377.62267278094, "vram_gb": 5.684933632, "lambda": 0.2001916148389379, "lambda_min": 0.09863286465406418, "lambda_max": 0.3352152109146118, "S_mean": 0.39837646484375, "G_mean": 0.4108955553577592, "G_std": 0.15846015827264637, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.883639335632324, "grad_norm": 0.7129117846488953, "tokens_seen": 31956992, "tokens_per_sec": 18377.5544957765, "vram_gb": 5.684933632, "lambda": 0.20069120544940233, "lambda_min": 0.09855438768863678, "lambda_max": 0.3367719054222107, "S_mean": 0.3812052408854167, "G_mean": 0.4026264944113791, "G_std": 0.1639611986465752, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.866237163543701, "grad_norm": 0.6725797057151794, "tokens_seen": 32120832, "tokens_per_sec": 18377.085218196968, "vram_gb": 5.684933632, "lambda": 0.20115812650571266, "lambda_min": 0.09856857359409332, "lambda_max": 0.33854731917381287, "S_mean": 0.3886311848958333, "G_mean": 0.39675399924938876, "G_std": 0.16280402041350803, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.741566181182861, "grad_norm": 0.7566303610801697, "tokens_seen": 32284672, "tokens_per_sec": 18377.352095747327, "vram_gb": 5.684933632, "lambda": 0.2015723322207729, "lambda_min": 0.09851684421300888, "lambda_max": 0.34031370282173157, "S_mean": 0.3702901204427083, "G_mean": 0.3938269172795117, "G_std": 0.16188843214573959, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.734564781188965, "grad_norm": 0.7511621713638306, "tokens_seen": 32448512, "tokens_per_sec": 18377.002660339, "vram_gb": 5.684933632, "lambda": 0.20200621957580248, "lambda_min": 0.09851700812578201, "lambda_max": 0.34184733033180237, "S_mean": 0.3756510416666667, "G_mean": 0.4071318036876619, "G_std": 0.15937524428591132, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.814802169799805, "grad_norm": 0.7336497902870178, "tokens_seen": 32612352, "tokens_per_sec": 18377.080711056813, "vram_gb": 5.684933632, "lambda": 0.20242205758889517, "lambda_min": 0.0983690470457077, "lambda_max": 0.342932790517807, "S_mean": 0.3880411783854167, "G_mean": 0.4006751264290263, "G_std": 0.1599497030644367, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.757092475891113, "grad_norm": 0.7478923201560974, "tokens_seen": 32776192, "tokens_per_sec": 18376.785651779166, "vram_gb": 5.684933632, "lambda": 0.20283319521695375, "lambda_min": 0.09837537258863449, "lambda_max": 0.3440774381160736, "S_mean": 0.3803507486979167, "G_mean": 0.40065090971377987, "G_std": 0.16052369206833342, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.87345073223114, "val_ppl": 130.7713962796398, "lambda": 0.2028553724909822, "lambda_min": 0.09835908561944962, "lambda_max": 0.34415486454963684, "lambda_per_layer": [0.16432423889636993, 0.1479661762714386, 0.1345156729221344, 0.1451212763786316, 0.12383141368627548, 0.12009553611278534, 0.12344764173030853, 0.09835908561944962, 0.10017003864049911, 0.14804314076900482, 0.2878536283969879, 0.2643899917602539, 0.29408055543899536, 0.225801482796669, 0.3090207874774933, 0.33930522203445435, 0.34415486454963684, 0.2397547960281372, 0.30587950348854065, 0.18082869052886963, 0.19150091707706451, 0.17385321855545044, 0.24017080664634705, 0.16606025397777557], "S_mean": 0.38916015625, "G_mean": 0.39961370639503, "G_std": 0.1599031020887196, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.696503400802612, "grad_norm": 0.7219030857086182, "tokens_seen": 32940032, "tokens_per_sec": 18359.232257889747, "vram_gb": 5.684933632, "lambda": 0.20320955539743105, "lambda_min": 0.09828070551156998, "lambda_max": 0.3455701172351837, "S_mean": 0.3751322428385417, "G_mean": 0.3962544760045906, "G_std": 0.1599613424623385, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 5.091588258743286, "grad_norm": 0.7630823850631714, "tokens_seen": 33103872, "tokens_per_sec": 18359.570554821315, "vram_gb": 5.684933632, "lambda": 0.20365866584082445, "lambda_min": 0.0982750803232193, "lambda_max": 0.3469163775444031, "S_mean": 0.378265380859375, "G_mean": 0.3968217431101948, "G_std": 0.16133411702079078, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.733144998550415, "grad_norm": 0.7537781000137329, "tokens_seen": 33267712, "tokens_per_sec": 18359.48031065268, "vram_gb": 5.684933632, "lambda": 0.20415812730789185, "lambda_min": 0.09812133014202118, "lambda_max": 0.3482804000377655, "S_mean": 0.3611246744791667, "G_mean": 0.4043332018579046, "G_std": 0.16032782278489321, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.877220630645752, "grad_norm": 0.7553359270095825, "tokens_seen": 33431552, "tokens_per_sec": 18358.026211017783, "vram_gb": 5.684933632, "lambda": 0.2046098724628488, "lambda_min": 0.09791137278079987, "lambda_max": 0.34951505064964294, "S_mean": 0.3955790201822917, "G_mean": 0.407872352283448, "G_std": 0.1629104270056511, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.805411338806152, "grad_norm": 0.7011367678642273, "tokens_seen": 33595392, "tokens_per_sec": 18358.47902500679, "vram_gb": 5.684933632, "lambda": 0.20507123911132416, "lambda_min": 0.09762157499790192, "lambda_max": 0.35066312551498413, "S_mean": 0.3731791178385417, "G_mean": 0.40093642271434266, "G_std": 0.16255700999560455, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.776690244674683, "grad_norm": 0.736009955406189, "tokens_seen": 33759232, "tokens_per_sec": 18358.807203821034, "vram_gb": 5.684933632, "lambda": 0.20554391325761875, "lambda_min": 0.0975344181060791, "lambda_max": 0.3516292870044708, "S_mean": 0.3763936360677083, "G_mean": 0.40159052874272066, "G_std": 0.1596029367453108, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.797442674636841, "grad_norm": 0.6853618621826172, "tokens_seen": 33923072, "tokens_per_sec": 18358.873085247807, "vram_gb": 5.684933632, "lambda": 0.20597126117597023, "lambda_min": 0.09758703410625458, "lambda_max": 0.35296061635017395, "S_mean": 0.3770548502604167, "G_mean": 0.3990220078267157, "G_std": 0.16006001519660154, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.753027439117432, "grad_norm": 0.8086200952529907, "tokens_seen": 34086912, "tokens_per_sec": 18358.464655721553, "vram_gb": 5.684933632, "lambda": 0.206331388403972, "lambda_min": 0.09757222980260849, "lambda_max": 0.3541526794433594, "S_mean": 0.3767598470052083, "G_mean": 0.4015410087692241, "G_std": 0.16166947374586016, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.718588590621948, "grad_norm": 0.6983075737953186, "tokens_seen": 34250752, "tokens_per_sec": 18358.099437387154, "vram_gb": 5.684933632, "lambda": 0.20676300271103779, "lambda_min": 0.0975838229060173, "lambda_max": 0.3551054894924164, "S_mean": 0.3870442708333333, "G_mean": 0.39857537516703206, "G_std": 0.1606722023182859, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.827619314193726, "grad_norm": 0.7680841088294983, "tokens_seen": 34414592, "tokens_per_sec": 18358.027529516057, "vram_gb": 5.684933632, "lambda": 0.207265869093438, "lambda_min": 0.09760800749063492, "lambda_max": 0.35649099946022034, "S_mean": 0.3935546875, "G_mean": 0.39742020634002984, "G_std": 0.1612584942874188, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.663514614105225, "grad_norm": 0.7298333048820496, "tokens_seen": 34578432, "tokens_per_sec": 18356.651252188887, "vram_gb": 5.684933632, "lambda": 0.20770766430844864, "lambda_min": 0.09773846715688705, "lambda_max": 0.35737887024879456, "S_mean": 0.3821004231770833, "G_mean": 0.3966012925375253, "G_std": 0.1611008265754208, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.833444833755493, "grad_norm": 0.7277626395225525, "tokens_seen": 34742272, "tokens_per_sec": 18357.03559538647, "vram_gb": 5.684933632, "lambda": 0.20812945843984684, "lambda_min": 0.09799428284168243, "lambda_max": 0.3587162494659424, "S_mean": 0.3904012044270833, "G_mean": 0.4100186175201088, "G_std": 0.15985770740856728, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.870382308959961, "grad_norm": 0.7729253768920898, "tokens_seen": 34906112, "tokens_per_sec": 18357.212237046748, "vram_gb": 5.684933632, "lambda": 0.2086132268110911, "lambda_min": 0.09795734286308289, "lambda_max": 0.35994207859039307, "S_mean": 0.3619181315104167, "G_mean": 0.3995835743844509, "G_std": 0.16285533256207904, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.772986888885498, "grad_norm": 0.7304009795188904, "tokens_seen": 35069952, "tokens_per_sec": 18355.99199216392, "vram_gb": 5.684933632, "lambda": 0.20894492821147045, "lambda_min": 0.09779781848192215, "lambda_max": 0.3613351285457611, "S_mean": 0.3846232096354167, "G_mean": 0.3933955443402131, "G_std": 0.16075334115885198, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.708178997039795, "grad_norm": 0.6937177777290344, "tokens_seen": 35233792, "tokens_per_sec": 18355.546811852637, "vram_gb": 5.684933632, "lambda": 0.20945907942950726, "lambda_min": 0.0978938490152359, "lambda_max": 0.3629875183105469, "S_mean": 0.3823649088541667, "G_mean": 0.39098724676296115, "G_std": 0.1622034413351988, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.747265338897705, "grad_norm": 0.7636471390724182, "tokens_seen": 35397632, "tokens_per_sec": 18355.55685920431, "vram_gb": 5.684933632, "lambda": 0.2099128346890211, "lambda_min": 0.09788867086172104, "lambda_max": 0.3644823729991913, "S_mean": 0.38006591796875, "G_mean": 0.3920088961410026, "G_std": 0.15980121633037925, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.859554290771484, "grad_norm": 0.7515493631362915, "tokens_seen": 35561472, "tokens_per_sec": 18355.71303791544, "vram_gb": 5.684933632, "lambda": 0.2103918936724464, "lambda_min": 0.09795436263084412, "lambda_max": 0.36586564779281616, "S_mean": 0.386749267578125, "G_mean": 0.39370348770171404, "G_std": 0.16144771482019374, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.890028715133667, "grad_norm": 0.7372979521751404, "tokens_seen": 35725312, "tokens_per_sec": 18356.01382380486, "vram_gb": 5.684933632, "lambda": 0.21091007503370443, "lambda_min": 0.09813525527715683, "lambda_max": 0.3672970235347748, "S_mean": 0.38134765625, "G_mean": 0.3958664995928605, "G_std": 0.15999380378828695, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.7657105922698975, "grad_norm": 0.7379093170166016, "tokens_seen": 35889152, "tokens_per_sec": 18355.974253660937, "vram_gb": 5.684933632, "lambda": 0.21134600788354874, "lambda_min": 0.09810525923967361, "lambda_max": 0.36897188425064087, "S_mean": 0.3624267578125, "G_mean": 0.3922643482219428, "G_std": 0.16235266719013453, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.685453176498413, "grad_norm": 0.7837237119674683, "tokens_seen": 36052992, "tokens_per_sec": 18355.979665131093, "vram_gb": 5.684933632, "lambda": 0.21184203401207924, "lambda_min": 0.09801023453474045, "lambda_max": 0.37069302797317505, "S_mean": 0.3686726888020833, "G_mean": 0.3896613187001397, "G_std": 0.15966822711440423, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.99739933013916, "grad_norm": 0.7384001612663269, "tokens_seen": 36216832, "tokens_per_sec": 18356.336311366278, "vram_gb": 5.684933632, "lambda": 0.21227172575891018, "lambda_min": 0.09795193374156952, "lambda_max": 0.3720991909503937, "S_mean": 0.3836263020833333, "G_mean": 0.3894556436377267, "G_std": 0.1609553120409449, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.8740856647491455, "grad_norm": 0.7826366424560547, "tokens_seen": 36380672, "tokens_per_sec": 18356.61544996683, "vram_gb": 5.684933632, "lambda": 0.21272388442109028, "lambda_min": 0.09795736521482468, "lambda_max": 0.37392690777778625, "S_mean": 0.3952534993489583, "G_mean": 0.38657534200077254, "G_std": 0.16113461185401926, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.7743165493011475, "grad_norm": 0.7278395891189575, "tokens_seen": 36544512, "tokens_per_sec": 18356.13653605755, "vram_gb": 5.684933632, "lambda": 0.2132046694556872, "lambda_min": 0.09793908149003983, "lambda_max": 0.37540891766548157, "S_mean": 0.3701985677083333, "G_mean": 0.39148894837126136, "G_std": 0.15929776115808636, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.962596893310547, "grad_norm": 0.7620658278465271, "tokens_seen": 36708352, "tokens_per_sec": 18356.50495609489, "vram_gb": 5.684933632, "lambda": 0.21365310779462257, "lambda_min": 0.09790785610675812, "lambda_max": 0.3767627775669098, "S_mean": 0.3863118489583333, "G_mean": 0.392027894500643, "G_std": 0.1605527688904355, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.896784067153931, "grad_norm": 0.7756483554840088, "tokens_seen": 36872192, "tokens_per_sec": 18356.191530269392, "vram_gb": 5.684933632, "lambda": 0.21416930512835583, "lambda_min": 0.0977749451994896, "lambda_max": 0.37839239835739136, "S_mean": 0.3932291666666667, "G_mean": 0.3883055094629526, "G_std": 0.1603662515990436, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.740307807922363, "grad_norm": 0.7834312319755554, "tokens_seen": 37036032, "tokens_per_sec": 18356.565121938256, "vram_gb": 5.684933632, "lambda": 0.21459874076147875, "lambda_min": 0.09776035696268082, "lambda_max": 0.37959519028663635, "S_mean": 0.3953653971354167, "G_mean": 0.3920795447193086, "G_std": 0.1612169426322604, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.7851951122283936, "grad_norm": 0.709290087223053, "tokens_seen": 37199872, "tokens_per_sec": 18355.853281436306, "vram_gb": 5.684933632, "lambda": 0.2150325250501434, "lambda_min": 0.0978274717926979, "lambda_max": 0.381181538105011, "S_mean": 0.3949686686197917, "G_mean": 0.3865039592298369, "G_std": 0.1595305061976736, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.932171821594238, "grad_norm": 0.7514390349388123, "tokens_seen": 37363712, "tokens_per_sec": 18355.158061792827, "vram_gb": 5.684933632, "lambda": 0.21543435441950956, "lambda_min": 0.09778434038162231, "lambda_max": 0.3823964297771454, "S_mean": 0.3918355305989583, "G_mean": 0.392306405507649, "G_std": 0.15982021177963665, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.77733588218689, "grad_norm": 0.7331732511520386, "tokens_seen": 37527552, "tokens_per_sec": 18354.941095700593, "vram_gb": 5.684933632, "lambda": 0.2158889714628458, "lambda_min": 0.09763819724321365, "lambda_max": 0.3836553394794464, "S_mean": 0.36669921875, "G_mean": 0.3827276194933802, "G_std": 0.16143804939929396, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.793029308319092, "grad_norm": 0.7385103702545166, "tokens_seen": 37691392, "tokens_per_sec": 18354.231756260546, "vram_gb": 5.684933632, "lambda": 0.21627711411565542, "lambda_min": 0.09754747897386551, "lambda_max": 0.38486379384994507, "S_mean": 0.375244140625, "G_mean": 0.3918572817929089, "G_std": 0.16014611422239491, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.7366461753845215, "grad_norm": 0.7771276235580444, "tokens_seen": 37855232, "tokens_per_sec": 18354.500612621316, "vram_gb": 5.684933632, "lambda": 0.2167399593939384, "lambda_min": 0.09757324308156967, "lambda_max": 0.3862657845020294, "S_mean": 0.3740132649739583, "G_mean": 0.38438381456459564, "G_std": 0.16196709778159857, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.7423224449157715, "grad_norm": 0.7111656069755554, "tokens_seen": 38019072, "tokens_per_sec": 18354.830874338546, "vram_gb": 5.684933632, "lambda": 0.21717609371989965, "lambda_min": 0.09753679484128952, "lambda_max": 0.3874284625053406, "S_mean": 0.379241943359375, "G_mean": 0.3827361154835671, "G_std": 0.15936236122312644, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.930485010147095, "grad_norm": 0.7666945457458496, "tokens_seen": 38182912, "tokens_per_sec": 18354.77589940709, "vram_gb": 5.684933632, "lambda": 0.21760499477386475, "lambda_min": 0.09735666215419769, "lambda_max": 0.38846728205680847, "S_mean": 0.38848876953125, "G_mean": 0.39626438659615815, "G_std": 0.16061017801985145, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.603254318237305, "grad_norm": 0.7626745700836182, "tokens_seen": 38346752, "tokens_per_sec": 18354.900960490653, "vram_gb": 5.684933632, "lambda": 0.2180883390828967, "lambda_min": 0.09722904115915298, "lambda_max": 0.3894432783126831, "S_mean": 0.3806254069010417, "G_mean": 0.38149609393440187, "G_std": 0.15858500357717276, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.61787223815918, "grad_norm": 0.7402868270874023, "tokens_seen": 38510592, "tokens_per_sec": 18354.644098243687, "vram_gb": 5.684933632, "lambda": 0.21843452968945107, "lambda_min": 0.09719999879598618, "lambda_max": 0.39003175497055054, "S_mean": 0.3663736979166667, "G_mean": 0.38429449902226526, "G_std": 0.160355901850077, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.873325347900391, "grad_norm": 0.769300103187561, "tokens_seen": 38674432, "tokens_per_sec": 18354.213108561125, "vram_gb": 5.684933632, "lambda": 0.21881385706365108, "lambda_min": 0.09709422290325165, "lambda_max": 0.39114710688591003, "S_mean": 0.3780924479166667, "G_mean": 0.38831159425899386, "G_std": 0.15851871226914227, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.635600328445435, "grad_norm": 0.8181292414665222, "tokens_seen": 38838272, "tokens_per_sec": 18353.925157429294, "vram_gb": 5.684933632, "lambda": 0.21927434174964824, "lambda_min": 0.09712813794612885, "lambda_max": 0.39269888401031494, "S_mean": 0.3764241536458333, "G_mean": 0.38864193111658096, "G_std": 0.15893520791238794, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.7866504192352295, "grad_norm": 0.7187062501907349, "tokens_seen": 39002112, "tokens_per_sec": 18353.855001475065, "vram_gb": 5.684933632, "lambda": 0.21966844238340855, "lambda_min": 0.09678701311349869, "lambda_max": 0.3941749632358551, "S_mean": 0.3894805908203125, "G_mean": 0.38316686016817886, "G_std": 0.16001101187430322, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.739722013473511, "grad_norm": 0.7683092951774597, "tokens_seen": 39165952, "tokens_per_sec": 18353.8553303071, "vram_gb": 5.684933632, "lambda": 0.2201324449852109, "lambda_min": 0.09659592807292938, "lambda_max": 0.3955341875553131, "S_mean": 0.3864847819010417, "G_mean": 0.3937210123209904, "G_std": 0.15869813784956932, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.746194362640381, "grad_norm": 0.7231704592704773, "tokens_seen": 39329792, "tokens_per_sec": 18353.76573786556, "vram_gb": 5.684933632, "lambda": 0.2205323874950409, "lambda_min": 0.09630666673183441, "lambda_max": 0.39701926708221436, "S_mean": 0.3818817138671875, "G_mean": 0.3796710352568577, "G_std": 0.1605316665954888, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.614848852157593, "grad_norm": 0.7606956362724304, "tokens_seen": 39493632, "tokens_per_sec": 18353.62824880099, "vram_gb": 5.684933632, "lambda": 0.2209743189935883, "lambda_min": 0.09625287353992462, "lambda_max": 0.39843833446502686, "S_mean": 0.3899434407552083, "G_mean": 0.38393849817415077, "G_std": 0.15845319201859334, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.709752798080444, "grad_norm": 0.7056593894958496, "tokens_seen": 39657472, "tokens_per_sec": 18354.170907844444, "vram_gb": 5.684933632, "lambda": 0.22137274065365395, "lambda_min": 0.09627959132194519, "lambda_max": 0.39950257539749146, "S_mean": 0.369598388671875, "G_mean": 0.38313210910807055, "G_std": 0.1595381359802559, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.7590460777282715, "grad_norm": 0.7276973724365234, "tokens_seen": 39821312, "tokens_per_sec": 18354.274960465893, "vram_gb": 5.684933632, "lambda": 0.22181025240570307, "lambda_min": 0.09622924774885178, "lambda_max": 0.4008546471595764, "S_mean": 0.376068115234375, "G_mean": 0.38630286340291303, "G_std": 0.1589490728608022, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.766224145889282, "grad_norm": 0.7379724979400635, "tokens_seen": 39985152, "tokens_per_sec": 18354.45564445107, "vram_gb": 5.684933632, "lambda": 0.22223842795938253, "lambda_min": 0.0963328555226326, "lambda_max": 0.4020385146141052, "S_mean": 0.3982137044270833, "G_mean": 0.38024405265847844, "G_std": 0.15922744770068675, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.56065559387207, "grad_norm": 0.7920411229133606, "tokens_seen": 40148992, "tokens_per_sec": 18355.02560780252, "vram_gb": 5.684933632, "lambda": 0.22269884124398232, "lambda_min": 0.09652888029813766, "lambda_max": 0.40338456630706787, "S_mean": 0.3834330240885417, "G_mean": 0.38250514888204634, "G_std": 0.16046560664350787, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.782672882080078, "grad_norm": 0.815770149230957, "tokens_seen": 40312832, "tokens_per_sec": 18355.152151821152, "vram_gb": 5.684933632, "lambda": 0.223149956514438, "lambda_min": 0.09647378325462341, "lambda_max": 0.4048198461532593, "S_mean": 0.3884684244791667, "G_mean": 0.3767259134911001, "G_std": 0.158552584820427, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.640488147735596, "grad_norm": 0.7525224685668945, "tokens_seen": 40476672, "tokens_per_sec": 18354.69411345528, "vram_gb": 5.684933632, "lambda": 0.22359248157590628, "lambda_min": 0.09672098606824875, "lambda_max": 0.40616491436958313, "S_mean": 0.3805440266927083, "G_mean": 0.37783804497060675, "G_std": 0.15867825810952732, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.725550174713135, "grad_norm": 0.7507742643356323, "tokens_seen": 40640512, "tokens_per_sec": 18353.90362765023, "vram_gb": 5.684933632, "lambda": 0.22397184185683727, "lambda_min": 0.09668689221143723, "lambda_max": 0.4072750210762024, "S_mean": 0.3837788899739583, "G_mean": 0.3848319853035112, "G_std": 0.1599312882948046, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.680088043212891, "grad_norm": 0.7830149531364441, "tokens_seen": 40804352, "tokens_per_sec": 18353.883964707, "vram_gb": 5.684933632, "lambda": 0.2243904114390413, "lambda_min": 0.0966254323720932, "lambda_max": 0.4084784984588623, "S_mean": 0.3824869791666667, "G_mean": 0.3784740023159732, "G_std": 0.15993462118785828, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.557834625244141, "grad_norm": 0.7968628406524658, "tokens_seen": 40968192, "tokens_per_sec": 18354.066786087344, "vram_gb": 5.684933632, "lambda": 0.22476688865572214, "lambda_min": 0.0965237021446228, "lambda_max": 0.4099399745464325, "S_mean": 0.3796742757161458, "G_mean": 0.3760076643278201, "G_std": 0.15851480168445656, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.753461623191834, "val_ppl": 115.98508708059644, "lambda": 0.22477862176795801, "lambda_min": 0.09651269018650055, "lambda_max": 0.40999072790145874, "lambda_per_layer": [0.16800761222839355, 0.15030811727046967, 0.13543091714382172, 0.14583837985992432, 0.12210466712713242, 0.11957264691591263, 0.12042363733053207, 0.09847884625196457, 0.09651269018650055, 0.15321703255176544, 0.31578564643859863, 0.29859575629234314, 0.34022530913352966, 0.2539556324481964, 0.365649938583374, 0.4083320200443268, 0.40999072790145874, 0.27985578775405884, 0.35871514678001404, 0.19523736834526062, 0.21393901109695435, 0.1930132359266281, 0.27453458309173584, 0.176962211728096], "S_mean": 0.383026123046875, "G_mean": 0.37226028468770284, "G_std": 0.15923852733491609, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.739062547683716, "grad_norm": 0.7241436243057251, "tokens_seen": 41132032, "tokens_per_sec": 18340.15867416873, "vram_gb": 5.684933632, "lambda": 0.2251187264919281, "lambda_min": 0.09642722457647324, "lambda_max": 0.41077059507369995, "S_mean": 0.3669331868489583, "G_mean": 0.3876501211586098, "G_std": 0.15832154522649944, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.611332416534424, "grad_norm": 0.7250497341156006, "tokens_seen": 41295872, "tokens_per_sec": 18339.963762810065, "vram_gb": 5.684933632, "lambda": 0.22546842818458876, "lambda_min": 0.09637759625911713, "lambda_max": 0.4118337333202362, "S_mean": 0.3893025716145833, "G_mean": 0.377045548055321, "G_std": 0.1601221455493942, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.648852586746216, "grad_norm": 0.8431228399276733, "tokens_seen": 41459712, "tokens_per_sec": 18339.810207787596, "vram_gb": 5.684933632, "lambda": 0.22586799009392658, "lambda_min": 0.09612400829792023, "lambda_max": 0.4124666154384613, "S_mean": 0.390106201171875, "G_mean": 0.3862920430644105, "G_std": 0.1573022746403391, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.715535640716553, "grad_norm": 0.7806175947189331, "tokens_seen": 41623552, "tokens_per_sec": 18339.59005514439, "vram_gb": 5.684933632, "lambda": 0.22629977824787298, "lambda_min": 0.09581471234560013, "lambda_max": 0.41357338428497314, "S_mean": 0.3863423665364583, "G_mean": 0.3779819467260192, "G_std": 0.15805072349030524, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.614898681640625, "grad_norm": 0.7325793504714966, "tokens_seen": 41787392, "tokens_per_sec": 18339.530194471172, "vram_gb": 5.684933632, "lambda": 0.22678093736370405, "lambda_min": 0.09567401558160782, "lambda_max": 0.41495460271835327, "S_mean": 0.3878682454427083, "G_mean": 0.38296296711390215, "G_std": 0.15886602356719473, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.724207878112793, "grad_norm": 0.7244077324867249, "tokens_seen": 41951232, "tokens_per_sec": 18339.1484063176, "vram_gb": 5.684933632, "lambda": 0.22719127995272478, "lambda_min": 0.0955873429775238, "lambda_max": 0.4164377450942993, "S_mean": 0.368499755859375, "G_mean": 0.37534866326798993, "G_std": 0.1600916520304357, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.653923988342285, "grad_norm": 0.7577067017555237, "tokens_seen": 42115072, "tokens_per_sec": 18339.45293579419, "vram_gb": 5.684933632, "lambda": 0.22765192917237678, "lambda_min": 0.09563030302524567, "lambda_max": 0.41801950335502625, "S_mean": 0.373016357421875, "G_mean": 0.376819768998151, "G_std": 0.160482297030588, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.78248405456543, "grad_norm": 0.7377466559410095, "tokens_seen": 42278912, "tokens_per_sec": 18339.602610480735, "vram_gb": 5.684933632, "lambda": 0.2279865968351563, "lambda_min": 0.09571211785078049, "lambda_max": 0.41887903213500977, "S_mean": 0.3814798990885417, "G_mean": 0.3827038930418591, "G_std": 0.16110420180484653, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.49979043006897, "grad_norm": 0.762711226940155, "tokens_seen": 42442752, "tokens_per_sec": 18340.211860050047, "vram_gb": 5.684933632, "lambda": 0.22841981643189987, "lambda_min": 0.09588182717561722, "lambda_max": 0.4197772145271301, "S_mean": 0.3767445882161458, "G_mean": 0.3739876184457292, "G_std": 0.15847236814443022, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.523251533508301, "grad_norm": 0.929036021232605, "tokens_seen": 42606592, "tokens_per_sec": 18340.04008163339, "vram_gb": 5.684933632, "lambda": 0.2288213592643539, "lambda_min": 0.09595559537410736, "lambda_max": 0.42103856801986694, "S_mean": 0.3886311848958333, "G_mean": 0.3792594246721516, "G_std": 0.1596173703437671, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.724256992340088, "grad_norm": 0.7923519015312195, "tokens_seen": 42770432, "tokens_per_sec": 18339.91033326519, "vram_gb": 5.684933632, "lambda": 0.22922184442480406, "lambda_min": 0.09602437913417816, "lambda_max": 0.4221543073654175, "S_mean": 0.3741861979166667, "G_mean": 0.38024860286774737, "G_std": 0.15871842298656702, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.5415496826171875, "grad_norm": 0.7508048415184021, "tokens_seen": 42934272, "tokens_per_sec": 18339.811125199656, "vram_gb": 5.684933632, "lambda": 0.2296016135563453, "lambda_min": 0.09607740491628647, "lambda_max": 0.4234744608402252, "S_mean": 0.380828857421875, "G_mean": 0.38339143246412277, "G_std": 0.1590172244468704, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.699728727340698, "grad_norm": 0.8115954995155334, "tokens_seen": 43098112, "tokens_per_sec": 18338.669407955902, "vram_gb": 5.684933632, "lambda": 0.22997133309642473, "lambda_min": 0.09579341858625412, "lambda_max": 0.4247959852218628, "S_mean": 0.39056396484375, "G_mean": 0.38412390757973, "G_std": 0.15834211224379638, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.53961706161499, "grad_norm": 0.7815262675285339, "tokens_seen": 43261952, "tokens_per_sec": 18337.937866506923, "vram_gb": 5.684933632, "lambda": 0.23045416766156754, "lambda_min": 0.09557567536830902, "lambda_max": 0.4264112114906311, "S_mean": 0.37506103515625, "G_mean": 0.37054836718986434, "G_std": 0.1582855285378173, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.738724708557129, "grad_norm": 0.7685094475746155, "tokens_seen": 43425792, "tokens_per_sec": 18338.282193696316, "vram_gb": 5.684933632, "lambda": 0.23085753619670868, "lambda_min": 0.09543846547603607, "lambda_max": 0.42786702513694763, "S_mean": 0.3758900960286458, "G_mean": 0.3778844013189276, "G_std": 0.15835314880435666, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.57245397567749, "grad_norm": 0.7217435836791992, "tokens_seen": 43589632, "tokens_per_sec": 18337.827828332916, "vram_gb": 5.684933632, "lambda": 0.23125094237426916, "lambda_min": 0.09545902162790298, "lambda_max": 0.42983582615852356, "S_mean": 0.374542236328125, "G_mean": 0.37145498480337363, "G_std": 0.1585417384825026, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.558765649795532, "grad_norm": 0.729750394821167, "tokens_seen": 43753472, "tokens_per_sec": 18337.678044447643, "vram_gb": 5.684933632, "lambda": 0.23167960718274117, "lambda_min": 0.09541754424571991, "lambda_max": 0.43119269609451294, "S_mean": 0.3681488037109375, "G_mean": 0.3761174362152815, "G_std": 0.15767798592181256, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.594025135040283, "grad_norm": 0.7754662036895752, "tokens_seen": 43917312, "tokens_per_sec": 18337.46683124687, "vram_gb": 5.684933632, "lambda": 0.23204128226886192, "lambda_min": 0.09532595425844193, "lambda_max": 0.43188321590423584, "S_mean": 0.38067626953125, "G_mean": 0.37326604236538213, "G_std": 0.15964414489765963, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.723252773284912, "grad_norm": 0.8039615750312805, "tokens_seen": 44081152, "tokens_per_sec": 18337.750573641468, "vram_gb": 5.684933632, "lambda": 0.23242796305567026, "lambda_min": 0.09519600123167038, "lambda_max": 0.433165967464447, "S_mean": 0.3818206787109375, "G_mean": 0.38310300139710307, "G_std": 0.16053058517475924, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.655408143997192, "grad_norm": 0.7727210521697998, "tokens_seen": 44244992, "tokens_per_sec": 18338.322404431517, "vram_gb": 5.684933632, "lambda": 0.2328367323304216, "lambda_min": 0.09498397260904312, "lambda_max": 0.43431729078292847, "S_mean": 0.3752797444661458, "G_mean": 0.37658443907275796, "G_std": 0.15845293322733292, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.792031764984131, "grad_norm": 0.7614063620567322, "tokens_seen": 44408832, "tokens_per_sec": 18338.58330257829, "vram_gb": 5.684933632, "lambda": 0.23328816114614406, "lambda_min": 0.09487365931272507, "lambda_max": 0.4358718693256378, "S_mean": 0.3803609212239583, "G_mean": 0.3740104866058876, "G_std": 0.15852465907422206, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.523010015487671, "grad_norm": 0.817383348941803, "tokens_seen": 44572672, "tokens_per_sec": 18338.633725800828, "vram_gb": 5.684933632, "lambda": 0.2337073109423121, "lambda_min": 0.09470474720001221, "lambda_max": 0.4368143081665039, "S_mean": 0.3745320638020833, "G_mean": 0.3782471634913236, "G_std": 0.15975384414196014, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.574182987213135, "grad_norm": 0.7511439919471741, "tokens_seen": 44736512, "tokens_per_sec": 18338.931642532654, "vram_gb": 5.684933632, "lambda": 0.23415192868560553, "lambda_min": 0.09466918557882309, "lambda_max": 0.43835532665252686, "S_mean": 0.3908284505208333, "G_mean": 0.3781446445112427, "G_std": 0.15659897793860486, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.54306173324585, "grad_norm": 0.7546665668487549, "tokens_seen": 44900352, "tokens_per_sec": 18338.57395351538, "vram_gb": 5.684933632, "lambda": 0.2345329439267516, "lambda_min": 0.09462752193212509, "lambda_max": 0.4397296905517578, "S_mean": 0.37933349609375, "G_mean": 0.38066133623942733, "G_std": 0.15834497974719852, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.778616428375244, "grad_norm": 0.8313029408454895, "tokens_seen": 45064192, "tokens_per_sec": 18338.19558426986, "vram_gb": 5.684933632, "lambda": 0.2349592304478089, "lambda_min": 0.0946241170167923, "lambda_max": 0.441111296415329, "S_mean": 0.3698527018229167, "G_mean": 0.3821127105038613, "G_std": 0.15736546320840716, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.774779319763184, "grad_norm": 0.7825353145599365, "tokens_seen": 45228032, "tokens_per_sec": 18337.601960817374, "vram_gb": 5.684933632, "lambda": 0.23543389576176801, "lambda_min": 0.09459239989519119, "lambda_max": 0.44261088967323303, "S_mean": 0.3668619791666667, "G_mean": 0.37501552964871127, "G_std": 0.1575143896819403, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.666112661361694, "grad_norm": 0.7712629437446594, "tokens_seen": 45391872, "tokens_per_sec": 18337.27104001216, "vram_gb": 5.684933632, "lambda": 0.23586657115568718, "lambda_min": 0.09450797736644745, "lambda_max": 0.44368988275527954, "S_mean": 0.3773295084635417, "G_mean": 0.3711177813820541, "G_std": 0.1574566496613746, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.6788365840911865, "grad_norm": 0.8377493619918823, "tokens_seen": 45555712, "tokens_per_sec": 18337.4280164664, "vram_gb": 5.684933632, "lambda": 0.23617792191604772, "lambda_min": 0.09439928084611893, "lambda_max": 0.4452861249446869, "S_mean": 0.3813374837239583, "G_mean": 0.37177359134269256, "G_std": 0.15896523262684545, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.512436866760254, "grad_norm": 0.7436960935592651, "tokens_seen": 45719552, "tokens_per_sec": 18337.024417290588, "vram_gb": 5.684933632, "lambda": 0.23648266214877367, "lambda_min": 0.09424923360347748, "lambda_max": 0.4464370012283325, "S_mean": 0.37359619140625, "G_mean": 0.36999211196477216, "G_std": 0.15732708181409785, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.717585325241089, "grad_norm": 0.8639277815818787, "tokens_seen": 45883392, "tokens_per_sec": 18337.059369846807, "vram_gb": 5.684933632, "lambda": 0.236950953801473, "lambda_min": 0.09428654611110687, "lambda_max": 0.4477827548980713, "S_mean": 0.37548828125, "G_mean": 0.3828983975884815, "G_std": 0.15808864349188903, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.577305316925049, "grad_norm": 0.7900910973548889, "tokens_seen": 46047232, "tokens_per_sec": 18336.53617317603, "vram_gb": 5.684933632, "lambda": 0.23739150445908308, "lambda_min": 0.09421492367982864, "lambda_max": 0.44895562529563904, "S_mean": 0.3730367024739583, "G_mean": 0.37806196689295274, "G_std": 0.15920947787041465, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.58471155166626, "grad_norm": 0.7691118121147156, "tokens_seen": 46211072, "tokens_per_sec": 18336.021307537474, "vram_gb": 5.684933632, "lambda": 0.23773668085535368, "lambda_min": 0.09421037137508392, "lambda_max": 0.4500494599342346, "S_mean": 0.3696492513020833, "G_mean": 0.37807010137476027, "G_std": 0.15734877802121142, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.639315366744995, "grad_norm": 0.7814778089523315, "tokens_seen": 46374912, "tokens_per_sec": 18336.17639784584, "vram_gb": 5.684933632, "lambda": 0.23809135208527246, "lambda_min": 0.09414026141166687, "lambda_max": 0.45135122537612915, "S_mean": 0.3680419921875, "G_mean": 0.3763364808789144, "G_std": 0.1566957779771959, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.739697456359863, "grad_norm": 0.8009423017501831, "tokens_seen": 46538752, "tokens_per_sec": 18336.006605312486, "vram_gb": 5.684933632, "lambda": 0.23844033479690552, "lambda_min": 0.09402991831302643, "lambda_max": 0.4528133273124695, "S_mean": 0.3916015625, "G_mean": 0.3738628790403406, "G_std": 0.15739371026090035, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.605375051498413, "grad_norm": 0.7929614782333374, "tokens_seen": 46702592, "tokens_per_sec": 18335.610870991302, "vram_gb": 5.684933632, "lambda": 0.23875730050106844, "lambda_min": 0.0940077155828476, "lambda_max": 0.45409584045410156, "S_mean": 0.3757120768229167, "G_mean": 0.3788930573500693, "G_std": 0.15755901485681534, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.602857351303101, "grad_norm": 0.7185047268867493, "tokens_seen": 46866432, "tokens_per_sec": 18335.756653575274, "vram_gb": 5.684933632, "lambda": 0.23920176457613707, "lambda_min": 0.09425407648086548, "lambda_max": 0.45567190647125244, "S_mean": 0.3646647135416667, "G_mean": 0.36759191115076345, "G_std": 0.15888842106020698, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.624661207199097, "grad_norm": 0.7895858287811279, "tokens_seen": 47030272, "tokens_per_sec": 18335.611148775966, "vram_gb": 5.684933632, "lambda": 0.2395819534237186, "lambda_min": 0.09433465451002121, "lambda_max": 0.45677289366722107, "S_mean": 0.3772074381510417, "G_mean": 0.3680620277300477, "G_std": 0.157014131002749, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.542094707489014, "grad_norm": 0.7404147386550903, "tokens_seen": 47194112, "tokens_per_sec": 18335.743103476478, "vram_gb": 5.684933632, "lambda": 0.2398928158606092, "lambda_min": 0.09416070580482483, "lambda_max": 0.4579826593399048, "S_mean": 0.3705902099609375, "G_mean": 0.3635546457177649, "G_std": 0.15687538233275214, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.655616760253906, "grad_norm": 0.7946863770484924, "tokens_seen": 47357952, "tokens_per_sec": 18335.99273875011, "vram_gb": 5.684933632, "lambda": 0.24029029409090677, "lambda_min": 0.09409205615520477, "lambda_max": 0.4586772918701172, "S_mean": 0.3832906087239583, "G_mean": 0.3740282092864315, "G_std": 0.15667633436775455, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.634594440460205, "grad_norm": 0.7775639295578003, "tokens_seen": 47521792, "tokens_per_sec": 18335.91251535445, "vram_gb": 5.684933632, "lambda": 0.2407055295382937, "lambda_min": 0.0941230058670044, "lambda_max": 0.4600546658039093, "S_mean": 0.3668111165364583, "G_mean": 0.3686633388667057, "G_std": 0.15667774757215133, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.469614267349243, "grad_norm": 0.8541117906570435, "tokens_seen": 47685632, "tokens_per_sec": 18336.070969865057, "vram_gb": 5.684933632, "lambda": 0.2410897510126233, "lambda_min": 0.09421174973249435, "lambda_max": 0.46095073223114014, "S_mean": 0.3755594889322917, "G_mean": 0.3790580452575038, "G_std": 0.1586640204380577, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.528120756149292, "grad_norm": 0.8308905959129333, "tokens_seen": 47849472, "tokens_per_sec": 18335.91194052894, "vram_gb": 5.684933632, "lambda": 0.24152207809189954, "lambda_min": 0.09416134655475616, "lambda_max": 0.46231508255004883, "S_mean": 0.4032389322916667, "G_mean": 0.3736343678707878, "G_std": 0.15682879032101482, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.475728750228882, "grad_norm": 0.7390794157981873, "tokens_seen": 48013312, "tokens_per_sec": 18336.112637971604, "vram_gb": 5.684933632, "lambda": 0.24188381309310594, "lambda_min": 0.09404940158128738, "lambda_max": 0.46383702754974365, "S_mean": 0.3750712076822917, "G_mean": 0.36535973629603785, "G_std": 0.15723048790823668, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.479450464248657, "grad_norm": 0.7714105844497681, "tokens_seen": 48177152, "tokens_per_sec": 18336.206869443427, "vram_gb": 5.684933632, "lambda": 0.24223223390678564, "lambda_min": 0.0940147265791893, "lambda_max": 0.46522057056427, "S_mean": 0.389892578125, "G_mean": 0.3701937525377919, "G_std": 0.1583991746107737, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.526724815368652, "grad_norm": 0.7782655358314514, "tokens_seen": 48340992, "tokens_per_sec": 18336.12156540735, "vram_gb": 5.684933632, "lambda": 0.24261655813703933, "lambda_min": 0.09381407499313354, "lambda_max": 0.4665427803993225, "S_mean": 0.3856099446614583, "G_mean": 0.37502494268119335, "G_std": 0.1593291936442256, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.533942699432373, "grad_norm": 0.7538461685180664, "tokens_seen": 48504832, "tokens_per_sec": 18336.226977900726, "vram_gb": 5.684933632, "lambda": 0.24299323682983717, "lambda_min": 0.0937190055847168, "lambda_max": 0.46750545501708984, "S_mean": 0.3744608561197917, "G_mean": 0.37229222897440195, "G_std": 0.15632489083024362, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.677088737487793, "grad_norm": 0.7848824858665466, "tokens_seen": 48668672, "tokens_per_sec": 18336.81965474946, "vram_gb": 5.684933632, "lambda": 0.24334117801239094, "lambda_min": 0.09368109703063965, "lambda_max": 0.46846163272857666, "S_mean": 0.3772074381510417, "G_mean": 0.3706366525342067, "G_std": 0.1565968714033564, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.610686302185059, "grad_norm": 0.7662794589996338, "tokens_seen": 48832512, "tokens_per_sec": 18336.709869388735, "vram_gb": 5.684933632, "lambda": 0.2436943125600616, "lambda_min": 0.09370418637990952, "lambda_max": 0.4695708751678467, "S_mean": 0.369384765625, "G_mean": 0.36438395276976127, "G_std": 0.1566044130595401, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.443368434906006, "grad_norm": 0.745613694190979, "tokens_seen": 48996352, "tokens_per_sec": 18336.991929902917, "vram_gb": 5.684933632, "lambda": 0.24405634496361017, "lambda_min": 0.09365205466747284, "lambda_max": 0.47075575590133667, "S_mean": 0.381072998046875, "G_mean": 0.3691699397750199, "G_std": 0.1552593915645654, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.6710429191589355, "grad_norm": 0.7958568334579468, "tokens_seen": 49160192, "tokens_per_sec": 18336.97245778586, "vram_gb": 5.684933632, "lambda": 0.2443741594130794, "lambda_min": 0.09373684227466583, "lambda_max": 0.4719805121421814, "S_mean": 0.3780314127604167, "G_mean": 0.37089095381088555, "G_std": 0.15769116107063988, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.71041259765625, "val_ppl": 111.09798922048151, "lambda": 0.24438632931560278, "lambda_min": 0.09373912215232849, "lambda_max": 0.47205767035484314, "lambda_per_layer": [0.1706209033727646, 0.15205031633377075, 0.13616718351840973, 0.14599624276161194, 0.12330140173435211, 0.11949952691793442, 0.11769932508468628, 0.09960688650608063, 0.09373912215232849, 0.15874211490154266, 0.3383456766605377, 0.3236042559146881, 0.3777337670326233, 0.2776631712913513, 0.41329294443130493, 0.47205767035484314, 0.4655124545097351, 0.31457069516181946, 0.40777188539505005, 0.21600331366062164, 0.23424583673477173, 0.2118281126022339, 0.306810587644577, 0.1884085088968277], "S_mean": 0.387298583984375, "G_mean": 0.3777211553727587, "G_std": 0.1574702758419638, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.762540817260742, "grad_norm": 0.7804802656173706, "tokens_seen": 49324032, "tokens_per_sec": 18325.262299856953, "vram_gb": 5.684933632, "lambda": 0.2447616591428717, "lambda_min": 0.09382037818431854, "lambda_max": 0.4734938144683838, "S_mean": 0.3837992350260417, "G_mean": 0.36853502426917356, "G_std": 0.15724075612767288, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.543668508529663, "grad_norm": 0.7834106683731079, "tokens_seen": 49487872, "tokens_per_sec": 18324.34167504661, "vram_gb": 5.684933632, "lambda": 0.2451704159999887, "lambda_min": 0.09373447299003601, "lambda_max": 0.4752342104911804, "S_mean": 0.3557942708333333, "G_mean": 0.3647017839054267, "G_std": 0.15714364518256238, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.48848032951355, "grad_norm": 0.8290496468544006, "tokens_seen": 49651712, "tokens_per_sec": 18324.24072693939, "vram_gb": 5.684933632, "lambda": 0.2455772248407205, "lambda_min": 0.09366009384393692, "lambda_max": 0.4766501784324646, "S_mean": 0.3708089192708333, "G_mean": 0.3752304071870943, "G_std": 0.1564697118398423, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.508368492126465, "grad_norm": 0.7745351791381836, "tokens_seen": 49815552, "tokens_per_sec": 18323.975662800975, "vram_gb": 5.684933632, "lambda": 0.24604046189536652, "lambda_min": 0.09373282641172409, "lambda_max": 0.4780433475971222, "S_mean": 0.364776611328125, "G_mean": 0.3645928835806747, "G_std": 0.15800831148711345, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.638512372970581, "grad_norm": 0.7694043517112732, "tokens_seen": 49979392, "tokens_per_sec": 18323.81543207671, "vram_gb": 5.684933632, "lambda": 0.24637730636944374, "lambda_min": 0.09367170184850693, "lambda_max": 0.47924354672431946, "S_mean": 0.36541748046875, "G_mean": 0.3722895449803521, "G_std": 0.15955170849338174, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.676563262939453, "grad_norm": 0.8074434399604797, "tokens_seen": 50143232, "tokens_per_sec": 18323.84719197397, "vram_gb": 5.684933632, "lambda": 0.24672406073659658, "lambda_min": 0.09360644966363907, "lambda_max": 0.48085400462150574, "S_mean": 0.36754608154296875, "G_mean": 0.374088972962151, "G_std": 0.1582768209045753, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.638275384902954, "grad_norm": 0.8415700793266296, "tokens_seen": 50307072, "tokens_per_sec": 18323.891987728504, "vram_gb": 5.684933632, "lambda": 0.24704495444893837, "lambda_min": 0.09358421713113785, "lambda_max": 0.4822787046432495, "S_mean": 0.373260498046875, "G_mean": 0.3757207386661321, "G_std": 0.15514754061587155, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.5123937129974365, "grad_norm": 0.8000009655952454, "tokens_seen": 50470912, "tokens_per_sec": 18324.50593934422, "vram_gb": 5.684933632, "lambda": 0.24736965789149204, "lambda_min": 0.0935681164264679, "lambda_max": 0.48334595561027527, "S_mean": 0.3739980061848958, "G_mean": 0.37236811057664454, "G_std": 0.15662812197115272, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.594329833984375, "grad_norm": 0.8122153878211975, "tokens_seen": 50634752, "tokens_per_sec": 18324.24360690878, "vram_gb": 5.684933632, "lambda": 0.24769688645998636, "lambda_min": 0.0937170684337616, "lambda_max": 0.4842680096626282, "S_mean": 0.3615010579427083, "G_mean": 0.36542558840786415, "G_std": 0.15656646725255996, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.556471824645996, "grad_norm": 0.9812518358230591, "tokens_seen": 50798592, "tokens_per_sec": 18324.304081613427, "vram_gb": 5.684933632, "lambda": 0.24799491185694933, "lambda_min": 0.09382671862840652, "lambda_max": 0.4847395420074463, "S_mean": 0.3808441162109375, "G_mean": 0.37386235866385203, "G_std": 0.15720891929231584, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.575837850570679, "grad_norm": 0.8587139248847961, "tokens_seen": 50962432, "tokens_per_sec": 18324.53783220757, "vram_gb": 5.684933632, "lambda": 0.24830244295299053, "lambda_min": 0.09376519173383713, "lambda_max": 0.4858865439891815, "S_mean": 0.3813374837239583, "G_mean": 0.3705524099059403, "G_std": 0.15733935385166356, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.672672510147095, "grad_norm": 0.8218334913253784, "tokens_seen": 51126272, "tokens_per_sec": 18324.21065788293, "vram_gb": 5.684933632, "lambda": 0.24865479643146196, "lambda_min": 0.09371811151504517, "lambda_max": 0.48703694343566895, "S_mean": 0.3863169352213542, "G_mean": 0.3735965151960651, "G_std": 0.1570377138365681, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.54192852973938, "grad_norm": 0.7825934290885925, "tokens_seen": 51290112, "tokens_per_sec": 18324.349624707826, "vram_gb": 5.684933632, "lambda": 0.2489823174352447, "lambda_min": 0.09366089850664139, "lambda_max": 0.4882575273513794, "S_mean": 0.3834737141927083, "G_mean": 0.3607375754509121, "G_std": 0.1568485073124369, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.539745092391968, "grad_norm": 0.8034579157829285, "tokens_seen": 51453952, "tokens_per_sec": 18324.122062585124, "vram_gb": 5.684933632, "lambda": 0.24932513013482094, "lambda_min": 0.09362120181322098, "lambda_max": 0.48955920338630676, "S_mean": 0.380950927734375, "G_mean": 0.36319390738693375, "G_std": 0.1564410081676518, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.654960632324219, "grad_norm": 0.7891222238540649, "tokens_seen": 51617792, "tokens_per_sec": 18324.20232615137, "vram_gb": 5.684933632, "lambda": 0.24966280286510786, "lambda_min": 0.09372098743915558, "lambda_max": 0.4909554719924927, "S_mean": 0.3824055989583333, "G_mean": 0.37563608470372856, "G_std": 0.15546239194615433, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.494154691696167, "grad_norm": 0.8036468029022217, "tokens_seen": 51781632, "tokens_per_sec": 18324.100683795514, "vram_gb": 5.684933632, "lambda": 0.24997488564501205, "lambda_min": 0.09375574439764023, "lambda_max": 0.4926117956638336, "S_mean": 0.3729604085286458, "G_mean": 0.3644452599498133, "G_std": 0.15571149178625396, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.62382960319519, "grad_norm": 0.8452504277229309, "tokens_seen": 51945472, "tokens_per_sec": 18323.911591365548, "vram_gb": 5.684933632, "lambda": 0.2503116171186169, "lambda_min": 0.09370021522045135, "lambda_max": 0.4936631917953491, "S_mean": 0.3982391357421875, "G_mean": 0.3648172076791525, "G_std": 0.15502051322255284, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.745845556259155, "grad_norm": 0.8366919159889221, "tokens_seen": 52109312, "tokens_per_sec": 18323.95789403809, "vram_gb": 5.684933632, "lambda": 0.25062658451497555, "lambda_min": 0.09350971132516861, "lambda_max": 0.4945426285266876, "S_mean": 0.3715159098307292, "G_mean": 0.3705599857494235, "G_std": 0.15554156061261892, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.302160024642944, "grad_norm": 0.7733508348464966, "tokens_seen": 52273152, "tokens_per_sec": 18323.965392392805, "vram_gb": 5.684933632, "lambda": 0.25100943353027105, "lambda_min": 0.09348674863576889, "lambda_max": 0.49558329582214355, "S_mean": 0.3741658528645833, "G_mean": 0.36804948141798377, "G_std": 0.15664155420381576, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.5178773403167725, "grad_norm": 0.789945125579834, "tokens_seen": 52436992, "tokens_per_sec": 18324.197361472787, "vram_gb": 5.684933632, "lambda": 0.25131553318351507, "lambda_min": 0.09340783953666687, "lambda_max": 0.49651020765304565, "S_mean": 0.3740590413411458, "G_mean": 0.35871486730563146, "G_std": 0.15526937413960695, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.4468958377838135, "grad_norm": 0.7994849681854248, "tokens_seen": 52600832, "tokens_per_sec": 18324.563771451718, "vram_gb": 5.684933632, "lambda": 0.2515677126745383, "lambda_min": 0.09326024353504181, "lambda_max": 0.49760180711746216, "S_mean": 0.372955322265625, "G_mean": 0.3666742652033766, "G_std": 0.15771981483946243, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.642230749130249, "grad_norm": 0.7756033539772034, "tokens_seen": 52764672, "tokens_per_sec": 18324.7044501271, "vram_gb": 5.684933632, "lambda": 0.25197612183789414, "lambda_min": 0.0932043120265007, "lambda_max": 0.49899783730506897, "S_mean": 0.366180419921875, "G_mean": 0.3664530764023463, "G_std": 0.15455757345383367, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.501636505126953, "grad_norm": 0.7981887459754944, "tokens_seen": 52928512, "tokens_per_sec": 18324.584807813462, "vram_gb": 5.684933632, "lambda": 0.2522820355370641, "lambda_min": 0.09327548742294312, "lambda_max": 0.4999340772628784, "S_mean": 0.3788604736328125, "G_mean": 0.3585635678221782, "G_std": 0.15420947829261422, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.520312547683716, "grad_norm": 0.8594127297401428, "tokens_seen": 53092352, "tokens_per_sec": 18325.2903096508, "vram_gb": 5.684933632, "lambda": 0.2526124917591612, "lambda_min": 0.09333417564630508, "lambda_max": 0.5004730224609375, "S_mean": 0.376251220703125, "G_mean": 0.3723067866327862, "G_std": 0.15500243365143737, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.486122131347656, "grad_norm": 0.7806779742240906, "tokens_seen": 53256192, "tokens_per_sec": 18325.105840268734, "vram_gb": 5.684933632, "lambda": 0.25299744835744303, "lambda_min": 0.0933128222823143, "lambda_max": 0.5011733174324036, "S_mean": 0.3722686767578125, "G_mean": 0.3608428945299238, "G_std": 0.15464201600601277, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.599834442138672, "grad_norm": 0.8114641904830933, "tokens_seen": 53420032, "tokens_per_sec": 18325.438243355657, "vram_gb": 5.684933632, "lambda": 0.2533393809571862, "lambda_min": 0.0932958573102951, "lambda_max": 0.5020647048950195, "S_mean": 0.3784077962239583, "G_mean": 0.36904907459393144, "G_std": 0.1561867033985133, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.519338130950928, "grad_norm": 0.7708404064178467, "tokens_seen": 53583872, "tokens_per_sec": 18325.77065936377, "vram_gb": 5.684933632, "lambda": 0.25364147406071424, "lambda_min": 0.09313808381557465, "lambda_max": 0.5028515458106995, "S_mean": 0.378173828125, "G_mean": 0.35961776599287987, "G_std": 0.1561923762007306, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.656770944595337, "grad_norm": 0.8142878413200378, "tokens_seen": 53747712, "tokens_per_sec": 18325.976893386527, "vram_gb": 5.684933632, "lambda": 0.2539324639365077, "lambda_min": 0.09303019195795059, "lambda_max": 0.5036261677742004, "S_mean": 0.3628184000651042, "G_mean": 0.36233421647921205, "G_std": 0.15502233717901012, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.593535900115967, "grad_norm": 0.8196409344673157, "tokens_seen": 53911552, "tokens_per_sec": 18326.27656574447, "vram_gb": 5.684933632, "lambda": 0.2541594185555975, "lambda_min": 0.09294069558382034, "lambda_max": 0.5047242641448975, "S_mean": 0.3786417643229167, "G_mean": 0.3654542335619529, "G_std": 0.15644317015539855, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.498193740844727, "grad_norm": 0.8088771104812622, "tokens_seen": 54075392, "tokens_per_sec": 18326.951762124045, "vram_gb": 5.684933632, "lambda": 0.2544409865513444, "lambda_min": 0.09281613677740097, "lambda_max": 0.505781888961792, "S_mean": 0.3766988118489583, "G_mean": 0.361358394070218, "G_std": 0.15636610930475095, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.49854040145874, "grad_norm": 0.8947829008102417, "tokens_seen": 54239232, "tokens_per_sec": 18327.10286200779, "vram_gb": 5.684933632, "lambda": 0.25474490380535525, "lambda_min": 0.092701256275177, "lambda_max": 0.5070215463638306, "S_mean": 0.3904978434244792, "G_mean": 0.3651293655857444, "G_std": 0.156227476351584, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.405088424682617, "grad_norm": 0.8194224834442139, "tokens_seen": 54403072, "tokens_per_sec": 18327.416671351784, "vram_gb": 5.684933632, "lambda": 0.25505856797099113, "lambda_min": 0.09264972805976868, "lambda_max": 0.5081087350845337, "S_mean": 0.3902587890625, "G_mean": 0.36830374016426504, "G_std": 0.1574726770243918, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.444033861160278, "grad_norm": 0.7787858843803406, "tokens_seen": 54566912, "tokens_per_sec": 18327.783010589912, "vram_gb": 5.684933632, "lambda": 0.2554361637060841, "lambda_min": 0.09260386973619461, "lambda_max": 0.509424090385437, "S_mean": 0.363800048828125, "G_mean": 0.36484195897355676, "G_std": 0.15556894526040801, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.466922760009766, "grad_norm": 0.8502992391586304, "tokens_seen": 54730752, "tokens_per_sec": 18327.86186184092, "vram_gb": 5.684933632, "lambda": 0.255720604211092, "lambda_min": 0.09259649366140366, "lambda_max": 0.5102810263633728, "S_mean": 0.3847757975260417, "G_mean": 0.36581984798734385, "G_std": 0.1579311693785712, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.773017644882202, "grad_norm": 0.8874080181121826, "tokens_seen": 54894592, "tokens_per_sec": 18328.344685156466, "vram_gb": 5.684933632, "lambda": 0.2560019101947546, "lambda_min": 0.09253986924886703, "lambda_max": 0.5113767385482788, "S_mean": 0.3808695475260417, "G_mean": 0.3671158990667512, "G_std": 0.1556091937236488, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.4172704219818115, "grad_norm": 0.7895466685295105, "tokens_seen": 55058432, "tokens_per_sec": 18327.986071262112, "vram_gb": 5.684933632, "lambda": 0.2563060764223337, "lambda_min": 0.09259067475795746, "lambda_max": 0.51227867603302, "S_mean": 0.375213623046875, "G_mean": 0.36336848194090027, "G_std": 0.15708740478536734, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.478964805603027, "grad_norm": 0.8336508870124817, "tokens_seen": 55222272, "tokens_per_sec": 18327.970574709958, "vram_gb": 5.684933632, "lambda": 0.25663555692881346, "lambda_min": 0.09261494129896164, "lambda_max": 0.5131139159202576, "S_mean": 0.3834431966145833, "G_mean": 0.3700487726988892, "G_std": 0.1546237295260653, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.5539655685424805, "grad_norm": 0.8258456587791443, "tokens_seen": 55386112, "tokens_per_sec": 18328.27913086522, "vram_gb": 5.684933632, "lambda": 0.25692305869112414, "lambda_min": 0.09261013567447662, "lambda_max": 0.5139862298965454, "S_mean": 0.384124755859375, "G_mean": 0.36684143954577547, "G_std": 0.15626509817472348, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.5289599895477295, "grad_norm": 0.7742685675621033, "tokens_seen": 55549952, "tokens_per_sec": 18327.964561423647, "vram_gb": 5.684933632, "lambda": 0.2572859919940432, "lambda_min": 0.09258544445037842, "lambda_max": 0.5152280330657959, "S_mean": 0.3774617513020833, "G_mean": 0.3671871934396525, "G_std": 0.1550816746118168, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.660651922225952, "grad_norm": 0.7990791201591492, "tokens_seen": 55713792, "tokens_per_sec": 18328.07347212526, "vram_gb": 5.684933632, "lambda": 0.2576060987388094, "lambda_min": 0.09266293793916702, "lambda_max": 0.5162712335586548, "S_mean": 0.3700358072916667, "G_mean": 0.3643089454465856, "G_std": 0.157039203409416, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.567905902862549, "grad_norm": 0.8460344672203064, "tokens_seen": 55877632, "tokens_per_sec": 18328.482250042573, "vram_gb": 5.684933632, "lambda": 0.25792034653325874, "lambda_min": 0.09269239753484726, "lambda_max": 0.5174644589424133, "S_mean": 0.3717803955078125, "G_mean": 0.3542603384703398, "G_std": 0.15723384245454022, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.55572772026062, "grad_norm": 0.8415594100952148, "tokens_seen": 56041472, "tokens_per_sec": 18328.462064249765, "vram_gb": 5.684933632, "lambda": 0.2582073525215189, "lambda_min": 0.09257954359054565, "lambda_max": 0.5183126926422119, "S_mean": 0.372833251953125, "G_mean": 0.36102844029664993, "G_std": 0.1571662121762832, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.550168514251709, "grad_norm": 0.8251811265945435, "tokens_seen": 56205312, "tokens_per_sec": 18328.4703322867, "vram_gb": 5.684933632, "lambda": 0.25846464342127246, "lambda_min": 0.09256748110055923, "lambda_max": 0.5192344188690186, "S_mean": 0.3827972412109375, "G_mean": 0.367682463179032, "G_std": 0.1540438944939524, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.546874284744263, "grad_norm": 0.8239948749542236, "tokens_seen": 56369152, "tokens_per_sec": 18328.61781470042, "vram_gb": 5.684933632, "lambda": 0.2587856752797961, "lambda_min": 0.09246877580881119, "lambda_max": 0.520731508731842, "S_mean": 0.3778584798177083, "G_mean": 0.35594909607122344, "G_std": 0.1562268527535101, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.542713642120361, "grad_norm": 0.8647258877754211, "tokens_seen": 56532992, "tokens_per_sec": 18328.362192254514, "vram_gb": 5.684933632, "lambda": 0.2591222968573372, "lambda_min": 0.09250786900520325, "lambda_max": 0.5216574668884277, "S_mean": 0.388916015625, "G_mean": 0.35906137991696596, "G_std": 0.1552751186924676, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.4827797412872314, "grad_norm": 0.8830364346504211, "tokens_seen": 56696832, "tokens_per_sec": 18328.661438122912, "vram_gb": 5.684933632, "lambda": 0.2594361972684662, "lambda_min": 0.09258320927619934, "lambda_max": 0.5224869251251221, "S_mean": 0.3754018147786458, "G_mean": 0.3619955932566275, "G_std": 0.15758449475591382, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.372863292694092, "grad_norm": 0.8026102781295776, "tokens_seen": 56860672, "tokens_per_sec": 18328.47587484841, "vram_gb": 5.684933632, "lambda": 0.25967891917874414, "lambda_min": 0.09250340610742569, "lambda_max": 0.5234297513961792, "S_mean": 0.3791300455729167, "G_mean": 0.3594913366250694, "G_std": 0.15769171994179487, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.422245025634766, "grad_norm": 0.8041717410087585, "tokens_seen": 57024512, "tokens_per_sec": 18328.39903172293, "vram_gb": 5.684933632, "lambda": 0.2599240705991785, "lambda_min": 0.0924212634563446, "lambda_max": 0.5241400003433228, "S_mean": 0.3852284749348958, "G_mean": 0.36103813668402535, "G_std": 0.15653998618169376, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.611138105392456, "grad_norm": 0.8380231857299805, "tokens_seen": 57188352, "tokens_per_sec": 18328.5096898062, "vram_gb": 5.684933632, "lambda": 0.260183507266144, "lambda_min": 0.09239207208156586, "lambda_max": 0.5248955488204956, "S_mean": 0.3747812906901042, "G_mean": 0.36581425050583977, "G_std": 0.15479589102324098, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.400866985321045, "grad_norm": 0.7983009219169617, "tokens_seen": 57352192, "tokens_per_sec": 18328.97404248453, "vram_gb": 5.684933632, "lambda": 0.26047360183050233, "lambda_min": 0.09237696975469589, "lambda_max": 0.526003360748291, "S_mean": 0.3818613688151042, "G_mean": 0.3618803847736369, "G_std": 0.15472426001603404, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.603547430038452, "val_ppl": 99.83785600068748, "lambda": 0.26048777376612026, "lambda_min": 0.09238262474536896, "lambda_max": 0.5260547399520874, "lambda_per_layer": [0.17296427488327026, 0.15459901094436646, 0.13707534968852997, 0.14746743440628052, 0.12236664444208145, 0.11923999339342117, 0.11660511791706085, 0.10146641731262207, 0.09238262474536896, 0.1635117381811142, 0.3540101647377014, 0.344686895608902, 0.40621668100357056, 0.29394349455833435, 0.45280343294143677, 0.5260547399520874, 0.5131683349609375, 0.3417070508003235, 0.44958314299583435, 0.2303842455148697, 0.24886013567447662, 0.22593656182289124, 0.3381825089454651, 0.19849057495594025], "S_mean": 0.3849639892578125, "G_mean": 0.36641578484947485, "G_std": 0.15590935367314765, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.480022430419922, "grad_norm": 0.7869158983230591, "tokens_seen": 57516032, "tokens_per_sec": 18319.291446674557, "vram_gb": 5.684933632, "lambda": 0.2607491910457611, "lambda_min": 0.09230950474739075, "lambda_max": 0.5266657471656799, "S_mean": 0.3663075764973958, "G_mean": 0.36203518657324213, "G_std": 0.15360955665043244, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.466287612915039, "grad_norm": 0.8198327422142029, "tokens_seen": 57679872, "tokens_per_sec": 18319.81462896611, "vram_gb": 5.684933632, "lambda": 0.26100581884384155, "lambda_min": 0.09225313365459442, "lambda_max": 0.5273758172988892, "S_mean": 0.3725128173828125, "G_mean": 0.3612016891905417, "G_std": 0.15577515626015762, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.526907920837402, "grad_norm": 0.855184018611908, "tokens_seen": 57843712, "tokens_per_sec": 18320.00545448683, "vram_gb": 5.684933632, "lambda": 0.2611930112664898, "lambda_min": 0.09227145463228226, "lambda_max": 0.5277004241943359, "S_mean": 0.3797556559244792, "G_mean": 0.3575824243016541, "G_std": 0.15550692783047756, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.47664737701416, "grad_norm": 0.8929058313369751, "tokens_seen": 58007552, "tokens_per_sec": 18320.474269610004, "vram_gb": 5.684933632, "lambda": 0.26153569140781957, "lambda_min": 0.09239119291305542, "lambda_max": 0.528799295425415, "S_mean": 0.3875020345052083, "G_mean": 0.36780355296408135, "G_std": 0.155649893063431, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.477123260498047, "grad_norm": 0.8601927757263184, "tokens_seen": 58171392, "tokens_per_sec": 18320.21824273964, "vram_gb": 5.684933632, "lambda": 0.2617917073269685, "lambda_min": 0.09239110350608826, "lambda_max": 0.5296104550361633, "S_mean": 0.3802286783854167, "G_mean": 0.36062288431761164, "G_std": 0.15574775360679874, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.6049933433532715, "grad_norm": 0.8622409105300903, "tokens_seen": 58335232, "tokens_per_sec": 18320.10211034552, "vram_gb": 5.684933632, "lambda": 0.26204233927031356, "lambda_min": 0.0923854410648346, "lambda_max": 0.5302602052688599, "S_mean": 0.382659912109375, "G_mean": 0.3581040167870621, "G_std": 0.15352728640815863, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.500436782836914, "grad_norm": 0.8608163595199585, "tokens_seen": 58499072, "tokens_per_sec": 18317.63962118193, "vram_gb": 5.684933632, "lambda": 0.26232007797807455, "lambda_min": 0.09247730672359467, "lambda_max": 0.5314634442329407, "S_mean": 0.3864949544270833, "G_mean": 0.3623841015311579, "G_std": 0.15499271297206482, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.6677093505859375, "grad_norm": 0.8403571844100952, "tokens_seen": 58662912, "tokens_per_sec": 18317.24931150604, "vram_gb": 5.684933632, "lambda": 0.2625846279164155, "lambda_min": 0.0925690233707428, "lambda_max": 0.5324300527572632, "S_mean": 0.3954060872395833, "G_mean": 0.36276452980625135, "G_std": 0.15355333185289055, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.484957456588745, "grad_norm": 0.8263989686965942, "tokens_seen": 58826752, "tokens_per_sec": 18317.24382097776, "vram_gb": 5.684933632, "lambda": 0.2628077395881216, "lambda_min": 0.09261618554592133, "lambda_max": 0.5334300398826599, "S_mean": 0.3901112874348958, "G_mean": 0.36237675611240167, "G_std": 0.15498651024730256, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.409796237945557, "grad_norm": 0.8273744583129883, "tokens_seen": 58990592, "tokens_per_sec": 18317.732198000376, "vram_gb": 5.684933632, "lambda": 0.26308702925841015, "lambda_min": 0.09261523932218552, "lambda_max": 0.5342508554458618, "S_mean": 0.3811594645182292, "G_mean": 0.35581129835918546, "G_std": 0.15483260484567532, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.4060704708099365, "grad_norm": 0.8112235069274902, "tokens_seen": 59154432, "tokens_per_sec": 18317.466427405758, "vram_gb": 5.684933632, "lambda": 0.26343684053669375, "lambda_min": 0.09256181120872498, "lambda_max": 0.5352741479873657, "S_mean": 0.3751627604166667, "G_mean": 0.3613181870896369, "G_std": 0.1538844662718475, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.562521457672119, "grad_norm": 0.8278475403785706, "tokens_seen": 59318272, "tokens_per_sec": 18317.746816854888, "vram_gb": 5.684933632, "lambda": 0.26374314818531275, "lambda_min": 0.09265100955963135, "lambda_max": 0.5360382199287415, "S_mean": 0.3783620198567708, "G_mean": 0.3562360542515914, "G_std": 0.15440344550491622, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.392167806625366, "grad_norm": 0.8979431390762329, "tokens_seen": 59482112, "tokens_per_sec": 18318.04039891017, "vram_gb": 5.684933632, "lambda": 0.26396917179226875, "lambda_min": 0.09271283447742462, "lambda_max": 0.5368741750717163, "S_mean": 0.3754933675130208, "G_mean": 0.3630204804552098, "G_std": 0.15498824281773219, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.36637020111084, "grad_norm": 0.8347077369689941, "tokens_seen": 59645952, "tokens_per_sec": 18317.69673659188, "vram_gb": 5.684933632, "lambda": 0.2642477198193471, "lambda_min": 0.09264200925827026, "lambda_max": 0.537990391254425, "S_mean": 0.3874766031901042, "G_mean": 0.359071810496971, "G_std": 0.1533277592776964, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.620541572570801, "grad_norm": 0.8414024710655212, "tokens_seen": 59809792, "tokens_per_sec": 18317.10752128041, "vram_gb": 5.684933632, "lambda": 0.26445930792639655, "lambda_min": 0.09258011728525162, "lambda_max": 0.5390115976333618, "S_mean": 0.3786112467447917, "G_mean": 0.35940942090625566, "G_std": 0.1551252871674175, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.620790481567383, "grad_norm": 0.8890026807785034, "tokens_seen": 59973632, "tokens_per_sec": 18316.60431011623, "vram_gb": 5.684933632, "lambda": 0.2647029170766473, "lambda_min": 0.092559814453125, "lambda_max": 0.5399703979492188, "S_mean": 0.3705393473307292, "G_mean": 0.3659455721111347, "G_std": 0.15552003790313998, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.428167819976807, "grad_norm": 0.9491268992424011, "tokens_seen": 60137472, "tokens_per_sec": 18316.564584652988, "vram_gb": 5.684933632, "lambda": 0.2650334605326255, "lambda_min": 0.09266535937786102, "lambda_max": 0.5409834980964661, "S_mean": 0.3646341959635417, "G_mean": 0.3622627378596614, "G_std": 0.1557047611956174, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.403503656387329, "grad_norm": 0.8308011293411255, "tokens_seen": 60301312, "tokens_per_sec": 18316.719315871927, "vram_gb": 5.684933632, "lambda": 0.2653195010498166, "lambda_min": 0.09273556619882584, "lambda_max": 0.5414289236068726, "S_mean": 0.3899943033854167, "G_mean": 0.3551031503205498, "G_std": 0.1525909424526617, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.448542356491089, "grad_norm": 0.8214101195335388, "tokens_seen": 60465152, "tokens_per_sec": 18316.71023924117, "vram_gb": 5.684933632, "lambda": 0.2656047359729807, "lambda_min": 0.092764712870121, "lambda_max": 0.5422253608703613, "S_mean": 0.3784128824869792, "G_mean": 0.35867582975576323, "G_std": 0.15390797471627593, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.575074672698975, "grad_norm": 0.8471735119819641, "tokens_seen": 60628992, "tokens_per_sec": 18316.891973313697, "vram_gb": 5.684933632, "lambda": 0.26588967877129716, "lambda_min": 0.09283207356929779, "lambda_max": 0.5428378582000732, "S_mean": 0.3675689697265625, "G_mean": 0.36235731467604637, "G_std": 0.15516244686053446, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.431132078170776, "grad_norm": 0.8140209913253784, "tokens_seen": 60792832, "tokens_per_sec": 18317.100145444972, "vram_gb": 5.684933632, "lambda": 0.2660984347263972, "lambda_min": 0.09285572171211243, "lambda_max": 0.5437064170837402, "S_mean": 0.38641357421875, "G_mean": 0.3553557097135733, "G_std": 0.15427735552657396, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.5047101974487305, "grad_norm": 0.8798801302909851, "tokens_seen": 60956672, "tokens_per_sec": 18316.916119709615, "vram_gb": 5.684933632, "lambda": 0.2663535649577777, "lambda_min": 0.09287993609905243, "lambda_max": 0.5444660186767578, "S_mean": 0.383819580078125, "G_mean": 0.36501626390963793, "G_std": 0.15480865363497287, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.5825114250183105, "grad_norm": 0.8071960806846619, "tokens_seen": 61120512, "tokens_per_sec": 18316.98715884089, "vram_gb": 5.684933632, "lambda": 0.2665923833847046, "lambda_min": 0.09291654825210571, "lambda_max": 0.5449588298797607, "S_mean": 0.381317138671875, "G_mean": 0.3595247194947054, "G_std": 0.15351386251859367, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.50519061088562, "grad_norm": 0.8590418696403503, "tokens_seen": 61284352, "tokens_per_sec": 18317.268766559333, "vram_gb": 5.684933632, "lambda": 0.2668519727885723, "lambda_min": 0.09293091297149658, "lambda_max": 0.5460038185119629, "S_mean": 0.3730723063151042, "G_mean": 0.365518368082121, "G_std": 0.15453546315742037, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.576849699020386, "grad_norm": 0.8759456872940063, "tokens_seen": 61448192, "tokens_per_sec": 18317.50646912444, "vram_gb": 5.684933632, "lambda": 0.2671386795118451, "lambda_min": 0.0929919108748436, "lambda_max": 0.5466700792312622, "S_mean": 0.3699289957682292, "G_mean": 0.3575469844508916, "G_std": 0.15508127802362046, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.393507242202759, "grad_norm": 0.8426020741462708, "tokens_seen": 61612032, "tokens_per_sec": 18317.83031133096, "vram_gb": 5.684933632, "lambda": 0.2674341605355342, "lambda_min": 0.09287871420383453, "lambda_max": 0.5473259687423706, "S_mean": 0.38214111328125, "G_mean": 0.3529654696273307, "G_std": 0.15403700418149432, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.489802360534668, "grad_norm": 0.8308683633804321, "tokens_seen": 61775872, "tokens_per_sec": 18317.835471151677, "vram_gb": 5.684933632, "lambda": 0.26766815936813754, "lambda_min": 0.09274382144212723, "lambda_max": 0.5483146905899048, "S_mean": 0.38104248046875, "G_mean": 0.35181507871796686, "G_std": 0.15529752317039916, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.5311338901519775, "grad_norm": 0.8393840789794922, "tokens_seen": 61939712, "tokens_per_sec": 18318.26205413677, "vram_gb": 5.684933632, "lambda": 0.26784639277805883, "lambda_min": 0.09266412258148193, "lambda_max": 0.5490279793739319, "S_mean": 0.3944295247395833, "G_mean": 0.3544789324514568, "G_std": 0.15477262906885395, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.429268836975098, "grad_norm": 0.9091805815696716, "tokens_seen": 62103552, "tokens_per_sec": 18318.52429470167, "vram_gb": 5.684933632, "lambda": 0.268088989580671, "lambda_min": 0.09261710196733475, "lambda_max": 0.5498184561729431, "S_mean": 0.396575927734375, "G_mean": 0.35859119535113376, "G_std": 0.1536213367168481, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.379465818405151, "grad_norm": 0.846755862236023, "tokens_seen": 62267392, "tokens_per_sec": 18318.849890261215, "vram_gb": 5.684933632, "lambda": 0.26832651533186436, "lambda_min": 0.09263855218887329, "lambda_max": 0.5504955649375916, "S_mean": 0.373016357421875, "G_mean": 0.35889100974115234, "G_std": 0.1549540909860904, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.289026260375977, "grad_norm": 0.8933413624763489, "tokens_seen": 62431232, "tokens_per_sec": 18318.843913679142, "vram_gb": 5.684933632, "lambda": 0.2684954299281041, "lambda_min": 0.09269571304321289, "lambda_max": 0.5506057739257812, "S_mean": 0.3859456380208333, "G_mean": 0.36144600676683086, "G_std": 0.15512475611952445, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.502156496047974, "grad_norm": 0.8376107811927795, "tokens_seen": 62595072, "tokens_per_sec": 18319.209083358215, "vram_gb": 5.684933632, "lambda": 0.26877264585345984, "lambda_min": 0.09278002381324768, "lambda_max": 0.5513750314712524, "S_mean": 0.3902638753255208, "G_mean": 0.35688538864875835, "G_std": 0.15481401673362902, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.410747051239014, "grad_norm": 0.888069748878479, "tokens_seen": 62758912, "tokens_per_sec": 18319.451490665637, "vram_gb": 5.684933632, "lambda": 0.26903496589511633, "lambda_min": 0.09281358122825623, "lambda_max": 0.5521039962768555, "S_mean": 0.3846638997395833, "G_mean": 0.3646016671943168, "G_std": 0.15491511187671372, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.520700931549072, "grad_norm": 0.8756272196769714, "tokens_seen": 62922752, "tokens_per_sec": 18319.36417851219, "vram_gb": 5.684933632, "lambda": 0.2692575032512347, "lambda_min": 0.09280740469694138, "lambda_max": 0.5528494119644165, "S_mean": 0.3804423014322917, "G_mean": 0.3663042043335736, "G_std": 0.15418372893085083, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.4419567584991455, "grad_norm": 0.874674916267395, "tokens_seen": 63086592, "tokens_per_sec": 18319.711708772014, "vram_gb": 5.684933632, "lambda": 0.2694818538924058, "lambda_min": 0.09282596409320831, "lambda_max": 0.5538234114646912, "S_mean": 0.3807576497395833, "G_mean": 0.3576083799513678, "G_std": 0.15477013510341445, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.3247504234313965, "grad_norm": 0.92699134349823, "tokens_seen": 63250432, "tokens_per_sec": 18319.962372393242, "vram_gb": 5.684933632, "lambda": 0.26970157648126286, "lambda_min": 0.09281284362077713, "lambda_max": 0.5546479225158691, "S_mean": 0.3882039388020833, "G_mean": 0.3640928976237774, "G_std": 0.15295213790765652, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.316501617431641, "grad_norm": 0.822303831577301, "tokens_seen": 63414272, "tokens_per_sec": 18320.391852029796, "vram_gb": 5.684933632, "lambda": 0.26994194462895393, "lambda_min": 0.09283193945884705, "lambda_max": 0.5554987788200378, "S_mean": 0.3854268391927083, "G_mean": 0.3571810941211879, "G_std": 0.15381961079159132, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.418036460876465, "grad_norm": 0.8476288914680481, "tokens_seen": 63578112, "tokens_per_sec": 18320.38858849554, "vram_gb": 5.684933632, "lambda": 0.27017108506212634, "lambda_min": 0.09279411286115646, "lambda_max": 0.5563149452209473, "S_mean": 0.388458251953125, "G_mean": 0.3607874579417209, "G_std": 0.15396131796296686, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.436534881591797, "grad_norm": 0.7991369962692261, "tokens_seen": 63741952, "tokens_per_sec": 18320.33670119163, "vram_gb": 5.684933632, "lambda": 0.27032163242499035, "lambda_min": 0.09279122203588486, "lambda_max": 0.5569008588790894, "S_mean": 0.3677266438802083, "G_mean": 0.35984506939227384, "G_std": 0.15491168030227223, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.437005519866943, "grad_norm": 0.9398518204689026, "tokens_seen": 63905792, "tokens_per_sec": 18320.41165826656, "vram_gb": 5.684933632, "lambda": 0.27051810671885806, "lambda_min": 0.09276925027370453, "lambda_max": 0.5574358701705933, "S_mean": 0.3917236328125, "G_mean": 0.3564123718533665, "G_std": 0.15480234776623547, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.525414705276489, "grad_norm": 0.8560163378715515, "tokens_seen": 64069632, "tokens_per_sec": 18320.268523524366, "vram_gb": 5.684933632, "lambda": 0.27074479032307863, "lambda_min": 0.09274551272392273, "lambda_max": 0.55808424949646, "S_mean": 0.39971923828125, "G_mean": 0.3639300264573346, "G_std": 0.15480328265887996, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.460307598114014, "grad_norm": 0.8643019795417786, "tokens_seen": 64233472, "tokens_per_sec": 18320.71262799788, "vram_gb": 5.684933632, "lambda": 0.2710071870436271, "lambda_min": 0.0927957221865654, "lambda_max": 0.5590125322341919, "S_mean": 0.3866678873697917, "G_mean": 0.35547690900663537, "G_std": 0.15411037586939832, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.230233669281006, "grad_norm": 0.8296508193016052, "tokens_seen": 64397312, "tokens_per_sec": 18320.856449139, "vram_gb": 5.684933632, "lambda": 0.27123791351914406, "lambda_min": 0.09277225285768509, "lambda_max": 0.5600362420082092, "S_mean": 0.3941548665364583, "G_mean": 0.353484148470064, "G_std": 0.15367551629121104, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.5280232429504395, "grad_norm": 0.8835127949714661, "tokens_seen": 64561152, "tokens_per_sec": 18321.22515372376, "vram_gb": 5.684933632, "lambda": 0.271484666193525, "lambda_min": 0.09274758398532867, "lambda_max": 0.5609267354011536, "S_mean": 0.3909200032552083, "G_mean": 0.35691421704056364, "G_std": 0.15400374067636827, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.522432804107666, "grad_norm": 0.9200502634048462, "tokens_seen": 64724992, "tokens_per_sec": 18321.55950009455, "vram_gb": 5.684933632, "lambda": 0.2716900072991848, "lambda_min": 0.0927158072590828, "lambda_max": 0.5617771148681641, "S_mean": 0.3859405517578125, "G_mean": 0.35698518824453157, "G_std": 0.15398931600308666, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.301319599151611, "grad_norm": 0.8198204040527344, "tokens_seen": 64888832, "tokens_per_sec": 18321.66629808999, "vram_gb": 5.684933632, "lambda": 0.2719091434652607, "lambda_min": 0.09265664964914322, "lambda_max": 0.5624663829803467, "S_mean": 0.3986867268880208, "G_mean": 0.3541745458108683, "G_std": 0.1537488076913481, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.482980012893677, "grad_norm": 0.8557148575782776, "tokens_seen": 65052672, "tokens_per_sec": 18322.008791718017, "vram_gb": 5.684933632, "lambda": 0.2721147056048115, "lambda_min": 0.09264510124921799, "lambda_max": 0.5629743337631226, "S_mean": 0.3806355794270833, "G_mean": 0.356257867688934, "G_std": 0.15373502799775451, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.523675441741943, "grad_norm": 0.847021758556366, "tokens_seen": 65216512, "tokens_per_sec": 18322.09972902391, "vram_gb": 5.684933632, "lambda": 0.2723151324316859, "lambda_min": 0.0926356390118599, "lambda_max": 0.5636572241783142, "S_mean": 0.3842976888020833, "G_mean": 0.35259134446581203, "G_std": 0.15413720079232007, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.3992414474487305, "grad_norm": 0.8715120553970337, "tokens_seen": 65380352, "tokens_per_sec": 18322.34762973792, "vram_gb": 5.684933632, "lambda": 0.2724986433361967, "lambda_min": 0.0926167219877243, "lambda_max": 0.5642507672309875, "S_mean": 0.3974609375, "G_mean": 0.3558991802080224, "G_std": 0.15584264035957554, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.461987257003784, "grad_norm": 0.8334783911705017, "tokens_seen": 65544192, "tokens_per_sec": 18322.76392861144, "vram_gb": 5.684933632, "lambda": 0.2727404066051046, "lambda_min": 0.09261342883110046, "lambda_max": 0.5650339126586914, "S_mean": 0.38812255859375, "G_mean": 0.35677089379169047, "G_std": 0.15296769192597517, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.487992906570435, "val_ppl": 88.94275019965237, "lambda": 0.2727547464892268, "lambda_min": 0.09262008965015411, "lambda_max": 0.5650748610496521, "lambda_per_layer": [0.17488573491573334, 0.1552041918039322, 0.13829238712787628, 0.14759981632232666, 0.12326497584581375, 0.12024299055337906, 0.11521586775779724, 0.10180539637804031, 0.09262008965015411, 0.16634702682495117, 0.3661434054374695, 0.35836872458457947, 0.428580105304718, 0.30610528588294983, 0.48382097482681274, 0.5650748610496521, 0.5489732027053833, 0.36262649297714233, 0.4799865484237671, 0.24125832319259644, 0.26280659437179565, 0.23767100274562836, 0.3624168932437897, 0.20680302381515503], "S_mean": 0.3953959147135417, "G_mean": 0.35996700505105156, "G_std": 0.15396464604418725, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.515766382217407, "grad_norm": 0.8843803405761719, "tokens_seen": 65708032, "tokens_per_sec": 18314.50894902121, "vram_gb": 5.684933632, "lambda": 0.273012169636786, "lambda_min": 0.09266949445009232, "lambda_max": 0.5656787157058716, "S_mean": 0.39288330078125, "G_mean": 0.36113277124240994, "G_std": 0.15379893814679235, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.529287576675415, "grad_norm": 0.8736494183540344, "tokens_seen": 65871872, "tokens_per_sec": 18314.607191359235, "vram_gb": 5.684933632, "lambda": 0.2732170249025027, "lambda_min": 0.09259923547506332, "lambda_max": 0.566214919090271, "S_mean": 0.3975931803385417, "G_mean": 0.36325217977476615, "G_std": 0.15345756461222967, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.484212160110474, "grad_norm": 0.8811337947845459, "tokens_seen": 66035712, "tokens_per_sec": 18314.92009837343, "vram_gb": 5.684933632, "lambda": 0.27344741610189277, "lambda_min": 0.09260150790214539, "lambda_max": 0.5670565366744995, "S_mean": 0.3918965657552083, "G_mean": 0.36031649021121365, "G_std": 0.1535372177604586, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.40395712852478, "grad_norm": 0.8728591203689575, "tokens_seen": 66199552, "tokens_per_sec": 18315.473461498932, "vram_gb": 5.684933632, "lambda": 0.27367102075368166, "lambda_min": 0.09270033240318298, "lambda_max": 0.5675456523895264, "S_mean": 0.4044189453125, "G_mean": 0.3604877424271156, "G_std": 0.15200084540992975, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.270718097686768, "grad_norm": 0.8508017659187317, "tokens_seen": 66363392, "tokens_per_sec": 18315.688077901756, "vram_gb": 5.684933632, "lambda": 0.2738702005396287, "lambda_min": 0.09269572794437408, "lambda_max": 0.5680968761444092, "S_mean": 0.3897450764973958, "G_mean": 0.349901888674746, "G_std": 0.1543076722494637, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.71124267578125, "grad_norm": 0.9128912687301636, "tokens_seen": 66527232, "tokens_per_sec": 18316.112427900596, "vram_gb": 5.684933632, "lambda": 0.2740819612517953, "lambda_min": 0.09255371987819672, "lambda_max": 0.568689227104187, "S_mean": 0.3873494466145833, "G_mean": 0.356723407516256, "G_std": 0.15334074276809892, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.409017086029053, "grad_norm": 0.925187885761261, "tokens_seen": 66691072, "tokens_per_sec": 18316.348762677553, "vram_gb": 5.684933632, "lambda": 0.2743065633500616, "lambda_min": 0.09249063581228256, "lambda_max": 0.5698387622833252, "S_mean": 0.3750813802083333, "G_mean": 0.3510019814129919, "G_std": 0.15372044880253574, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.3738014698028564, "grad_norm": 0.8611316680908203, "tokens_seen": 66854912, "tokens_per_sec": 18316.53735657211, "vram_gb": 5.684933632, "lambda": 0.27453477028757334, "lambda_min": 0.09247591346502304, "lambda_max": 0.570388913154602, "S_mean": 0.3839772542317708, "G_mean": 0.355182617281874, "G_std": 0.15360712985663363, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.494545221328735, "grad_norm": 0.8688015341758728, "tokens_seen": 67018752, "tokens_per_sec": 18316.58066552665, "vram_gb": 5.684933632, "lambda": 0.2747163601840536, "lambda_min": 0.09253374487161636, "lambda_max": 0.5708707571029663, "S_mean": 0.3904876708984375, "G_mean": 0.3580277485307306, "G_std": 0.15434972553824386, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.442841529846191, "grad_norm": 0.8930180668830872, "tokens_seen": 67182592, "tokens_per_sec": 18316.356328471244, "vram_gb": 5.684933632, "lambda": 0.27490822163720924, "lambda_min": 0.09259562939405441, "lambda_max": 0.5714730024337769, "S_mean": 0.3841654459635417, "G_mean": 0.35885201146205264, "G_std": 0.15256905412146202, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.447427272796631, "grad_norm": 0.8777070641517639, "tokens_seen": 67346432, "tokens_per_sec": 18316.44345721835, "vram_gb": 5.684933632, "lambda": 0.2751280531908075, "lambda_min": 0.09261195361614227, "lambda_max": 0.572509229183197, "S_mean": 0.3910013834635417, "G_mean": 0.35241165997770924, "G_std": 0.15227127622347325, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.5847063064575195, "grad_norm": 0.9476744532585144, "tokens_seen": 67510272, "tokens_per_sec": 18316.65542158923, "vram_gb": 5.684933632, "lambda": 0.2753311734025677, "lambda_min": 0.09258057922124863, "lambda_max": 0.5731105208396912, "S_mean": 0.3861134847005208, "G_mean": 0.356693366697679, "G_std": 0.15508790391807756, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.426464080810547, "grad_norm": 0.874173104763031, "tokens_seen": 67674112, "tokens_per_sec": 18315.146722969796, "vram_gb": 5.684933632, "lambda": 0.27550910351177055, "lambda_min": 0.09256215393543243, "lambda_max": 0.5738093852996826, "S_mean": 0.3926849365234375, "G_mean": 0.35543801775202155, "G_std": 0.15164189770196876, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.3671345710754395, "grad_norm": 0.8530479669570923, "tokens_seen": 67837952, "tokens_per_sec": 18313.119167381294, "vram_gb": 5.684933632, "lambda": 0.2757022924100359, "lambda_min": 0.09251794219017029, "lambda_max": 0.5743544101715088, "S_mean": 0.3773295084635417, "G_mean": 0.3557431463462611, "G_std": 0.15271732370213917, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.414748430252075, "grad_norm": 0.8632885813713074, "tokens_seen": 68001792, "tokens_per_sec": 18311.5840875447, "vram_gb": 5.684933632, "lambda": 0.2758780065923929, "lambda_min": 0.09251892566680908, "lambda_max": 0.5748251676559448, "S_mean": 0.3861897786458333, "G_mean": 0.3571136014846464, "G_std": 0.15284084998226413, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.367074489593506, "grad_norm": 0.8527672290802002, "tokens_seen": 68165632, "tokens_per_sec": 18308.81033083685, "vram_gb": 5.684933632, "lambda": 0.27606100620081025, "lambda_min": 0.09252670407295227, "lambda_max": 0.5757650136947632, "S_mean": 0.3826700846354167, "G_mean": 0.35937180331287283, "G_std": 0.15315689557852843, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.463095188140869, "grad_norm": 0.8710979223251343, "tokens_seen": 68329472, "tokens_per_sec": 18306.132317360847, "vram_gb": 5.684933632, "lambda": 0.2761922487989068, "lambda_min": 0.0925096794962883, "lambda_max": 0.5761409401893616, "S_mean": 0.398895263671875, "G_mean": 0.35712250353147584, "G_std": 0.15393247734755278, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.310042142868042, "grad_norm": 0.8403646349906921, "tokens_seen": 68493312, "tokens_per_sec": 18301.893557003084, "vram_gb": 5.684933632, "lambda": 0.2763822330161929, "lambda_min": 0.09252512454986572, "lambda_max": 0.5767630338668823, "S_mean": 0.378662109375, "G_mean": 0.3528022455672423, "G_std": 0.154688106267713, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.4089906215667725, "grad_norm": 0.9289424419403076, "tokens_seen": 68657152, "tokens_per_sec": 18299.45809285632, "vram_gb": 5.684933632, "lambda": 0.27663516501585644, "lambda_min": 0.09258879721164703, "lambda_max": 0.5774741768836975, "S_mean": 0.40716552734375, "G_mean": 0.3559374023073663, "G_std": 0.1546115766201789, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.474010705947876, "grad_norm": 0.9106419682502747, "tokens_seen": 68820992, "tokens_per_sec": 18296.432018276515, "vram_gb": 5.684933632, "lambda": 0.276860348880291, "lambda_min": 0.09260650724172592, "lambda_max": 0.5782979130744934, "S_mean": 0.3842824300130208, "G_mean": 0.3563720517947028, "G_std": 0.1558270708968242, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.4768054485321045, "grad_norm": 0.9411298036575317, "tokens_seen": 68984832, "tokens_per_sec": 18293.48039649553, "vram_gb": 5.684933632, "lambda": 0.2770481885721286, "lambda_min": 0.0926816314458847, "lambda_max": 0.5786656141281128, "S_mean": 0.3857625325520833, "G_mean": 0.35559853445738554, "G_std": 0.15315895504318178, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.408663988113403, "grad_norm": 0.9040068984031677, "tokens_seen": 69148672, "tokens_per_sec": 18292.71433019724, "vram_gb": 5.684933632, "lambda": 0.27720764372497797, "lambda_min": 0.09266688674688339, "lambda_max": 0.579197108745575, "S_mean": 0.4061991373697917, "G_mean": 0.3593213196533422, "G_std": 0.15342086483724415, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.481833219528198, "grad_norm": 0.87090003490448, "tokens_seen": 69312512, "tokens_per_sec": 18292.033006078073, "vram_gb": 5.684933632, "lambda": 0.2773886499926448, "lambda_min": 0.09261789917945862, "lambda_max": 0.5797327160835266, "S_mean": 0.4079386393229167, "G_mean": 0.3614325935486704, "G_std": 0.15479784664542726, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.439408302307129, "grad_norm": 0.8836545348167419, "tokens_seen": 69476352, "tokens_per_sec": 18290.99906373736, "vram_gb": 5.684933632, "lambda": 0.2775308834388852, "lambda_min": 0.09259139001369476, "lambda_max": 0.5802685022354126, "S_mean": 0.388763427734375, "G_mean": 0.35719790984876454, "G_std": 0.1537446832129111, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.415359973907471, "grad_norm": 0.924168050289154, "tokens_seen": 69640192, "tokens_per_sec": 18290.794153464823, "vram_gb": 5.684933632, "lambda": 0.2777457196886341, "lambda_min": 0.0925803855061531, "lambda_max": 0.5807296633720398, "S_mean": 0.3833567301432292, "G_mean": 0.36430941817040247, "G_std": 0.1537196000572294, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.41207218170166, "grad_norm": 0.9450178146362305, "tokens_seen": 69804032, "tokens_per_sec": 18291.367751845188, "vram_gb": 5.684933632, "lambda": 0.27790119281659525, "lambda_min": 0.0925983414053917, "lambda_max": 0.5812606811523438, "S_mean": 0.4006449381510417, "G_mean": 0.3538981336168945, "G_std": 0.15405129711143672, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.4045703411102295, "grad_norm": 0.8689968585968018, "tokens_seen": 69967872, "tokens_per_sec": 18291.394540679292, "vram_gb": 5.684933632, "lambda": 0.2780565644303958, "lambda_min": 0.09259252995252609, "lambda_max": 0.5818749070167542, "S_mean": 0.3891398111979167, "G_mean": 0.3579304240799199, "G_std": 0.15222462736225376, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.376838207244873, "grad_norm": 0.9185594320297241, "tokens_seen": 70131712, "tokens_per_sec": 18291.655295116587, "vram_gb": 5.684933632, "lambda": 0.27820424269884825, "lambda_min": 0.09261941909790039, "lambda_max": 0.5825592279434204, "S_mean": 0.3773295084635417, "G_mean": 0.3534997010913988, "G_std": 0.1535102316023161, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.4818501472473145, "grad_norm": 0.9378319382667542, "tokens_seen": 70295552, "tokens_per_sec": 18292.20087399706, "vram_gb": 5.684933632, "lambda": 0.27834729260454577, "lambda_min": 0.09262225776910782, "lambda_max": 0.5827524065971375, "S_mean": 0.3740234375, "G_mean": 0.3533222808813055, "G_std": 0.1530360817365969, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.32229208946228, "grad_norm": 0.898036777973175, "tokens_seen": 70459392, "tokens_per_sec": 18292.412998177013, "vram_gb": 5.684933632, "lambda": 0.2785165250922243, "lambda_min": 0.0926201343536377, "lambda_max": 0.5833158493041992, "S_mean": 0.3891855875651042, "G_mean": 0.3584717948300143, "G_std": 0.15402663704783967, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.418740749359131, "grad_norm": 0.8880966901779175, "tokens_seen": 70623232, "tokens_per_sec": 18292.62824930639, "vram_gb": 5.684933632, "lambda": 0.27866464977463085, "lambda_min": 0.09266941249370575, "lambda_max": 0.5836362838745117, "S_mean": 0.3923746744791667, "G_mean": 0.35663530440069735, "G_std": 0.15465017206345996, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.4237260818481445, "grad_norm": 0.8866780400276184, "tokens_seen": 70787072, "tokens_per_sec": 18293.141129502415, "vram_gb": 5.684933632, "lambda": 0.2788654025644064, "lambda_min": 0.09271222352981567, "lambda_max": 0.584385335445404, "S_mean": 0.3861287434895833, "G_mean": 0.34952424427804846, "G_std": 0.15276294124002257, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.435948133468628, "grad_norm": 0.8832370638847351, "tokens_seen": 70950912, "tokens_per_sec": 18293.293598442997, "vram_gb": 5.684933632, "lambda": 0.2790245621775587, "lambda_min": 0.0926990658044815, "lambda_max": 0.5848637819290161, "S_mean": 0.3874409993489583, "G_mean": 0.35005378754188615, "G_std": 0.15310432113862285, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.4694037437438965, "grad_norm": 0.9094716310501099, "tokens_seen": 71114752, "tokens_per_sec": 18293.57167900536, "vram_gb": 5.684933632, "lambda": 0.27916327863931656, "lambda_min": 0.09267900884151459, "lambda_max": 0.5851301550865173, "S_mean": 0.3795725504557292, "G_mean": 0.35912969793813926, "G_std": 0.15296936147691062, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.417012691497803, "grad_norm": 0.8455795049667358, "tokens_seen": 71278592, "tokens_per_sec": 18293.930867703588, "vram_gb": 5.684933632, "lambda": 0.2793611378098528, "lambda_min": 0.09272333979606628, "lambda_max": 0.5857863426208496, "S_mean": 0.3864695231119792, "G_mean": 0.3561600255779922, "G_std": 0.15366989246103913, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.456503868103027, "grad_norm": 0.8822804689407349, "tokens_seen": 71442432, "tokens_per_sec": 18294.144750320193, "vram_gb": 5.684933632, "lambda": 0.2795328112939994, "lambda_min": 0.09267129004001617, "lambda_max": 0.5861748456954956, "S_mean": 0.3889312744140625, "G_mean": 0.3502627472237994, "G_std": 0.1527574494636307, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.346948862075806, "grad_norm": 0.941135585308075, "tokens_seen": 71606272, "tokens_per_sec": 18294.610026030292, "vram_gb": 5.684933632, "lambda": 0.2797471219673753, "lambda_min": 0.09261121600866318, "lambda_max": 0.5870485305786133, "S_mean": 0.38983154296875, "G_mean": 0.3606275020477672, "G_std": 0.1537431531275312, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.421114444732666, "grad_norm": 0.8634566068649292, "tokens_seen": 71770112, "tokens_per_sec": 18294.464408367134, "vram_gb": 5.684933632, "lambda": 0.27992266820122796, "lambda_min": 0.0926005020737648, "lambda_max": 0.5876520872116089, "S_mean": 0.390350341796875, "G_mean": 0.3572605062897007, "G_std": 0.1528165867396941, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.279085874557495, "grad_norm": 0.86607426404953, "tokens_seen": 71933952, "tokens_per_sec": 18295.490559309186, "vram_gb": 5.684933632, "lambda": 0.2800466225792964, "lambda_min": 0.09258400648832321, "lambda_max": 0.5878465175628662, "S_mean": 0.3884735107421875, "G_mean": 0.357064885708193, "G_std": 0.15250435941076526, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.459425449371338, "grad_norm": 0.9164977073669434, "tokens_seen": 72097792, "tokens_per_sec": 18296.53506835238, "vram_gb": 5.684933632, "lambda": 0.28015340336908895, "lambda_min": 0.0926140621304512, "lambda_max": 0.5882574915885925, "S_mean": 0.3949076334635417, "G_mean": 0.35757391519534093, "G_std": 0.15277134471883377, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.501662254333496, "grad_norm": 0.9108008146286011, "tokens_seen": 72261632, "tokens_per_sec": 18296.847799437048, "vram_gb": 5.684933632, "lambda": 0.28032159867386025, "lambda_min": 0.09264559298753738, "lambda_max": 0.5889700651168823, "S_mean": 0.3901011149088542, "G_mean": 0.35366191947832704, "G_std": 0.15484876422366747, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.411789417266846, "grad_norm": 0.9128734469413757, "tokens_seen": 72425472, "tokens_per_sec": 18296.808577633394, "vram_gb": 5.684933632, "lambda": 0.2804915237550934, "lambda_min": 0.09262704849243164, "lambda_max": 0.5895780920982361, "S_mean": 0.3987833658854167, "G_mean": 0.35293584464428324, "G_std": 0.15325874354069433, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.350538969039917, "grad_norm": 0.889193058013916, "tokens_seen": 72589312, "tokens_per_sec": 18296.82993170757, "vram_gb": 5.684933632, "lambda": 0.28066204208880663, "lambda_min": 0.09260325878858566, "lambda_max": 0.5900541543960571, "S_mean": 0.391387939453125, "G_mean": 0.3551287646405399, "G_std": 0.15294591384008527, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.56316065788269, "grad_norm": 0.9119324088096619, "tokens_seen": 72753152, "tokens_per_sec": 18296.843007582793, "vram_gb": 5.684933632, "lambda": 0.28078710070500773, "lambda_min": 0.09261243790388107, "lambda_max": 0.5902270674705505, "S_mean": 0.393096923828125, "G_mean": 0.3576138530236979, "G_std": 0.15400170057546347, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.3984339237213135, "grad_norm": 0.8379104137420654, "tokens_seen": 72916992, "tokens_per_sec": 18297.225415752135, "vram_gb": 5.684933632, "lambda": 0.2809343704332908, "lambda_min": 0.09254143387079239, "lambda_max": 0.5906435251235962, "S_mean": 0.3891448974609375, "G_mean": 0.3556620665670683, "G_std": 0.1532964485231787, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.40939998626709, "grad_norm": 0.8950060606002808, "tokens_seen": 73080832, "tokens_per_sec": 18297.16757121686, "vram_gb": 5.684933632, "lambda": 0.2811157278095682, "lambda_min": 0.09255631268024445, "lambda_max": 0.5910281538963318, "S_mean": 0.3935953776041667, "G_mean": 0.35252772682967287, "G_std": 0.15383710716074953, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.283891677856445, "grad_norm": 0.9132924675941467, "tokens_seen": 73244672, "tokens_per_sec": 18297.339409473352, "vram_gb": 5.684933632, "lambda": 0.28128890817364055, "lambda_min": 0.0925552174448967, "lambda_max": 0.5912057161331177, "S_mean": 0.392913818359375, "G_mean": 0.3453702407423407, "G_std": 0.154086984305953, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.4481141567230225, "grad_norm": 0.9069195985794067, "tokens_seen": 73408512, "tokens_per_sec": 18297.244337433243, "vram_gb": 5.684933632, "lambda": 0.2814262059206764, "lambda_min": 0.09254442900419235, "lambda_max": 0.591754674911499, "S_mean": 0.3948872884114583, "G_mean": 0.35315877889903885, "G_std": 0.15300028010581931, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.4406578540802, "grad_norm": 0.9569308757781982, "tokens_seen": 73572352, "tokens_per_sec": 18297.326879551958, "vram_gb": 5.684933632, "lambda": 0.2815959909930825, "lambda_min": 0.09262694418430328, "lambda_max": 0.5921714305877686, "S_mean": 0.3931070963541667, "G_mean": 0.3529163314184795, "G_std": 0.15254531536872187, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.52816104888916, "grad_norm": 0.8695442080497742, "tokens_seen": 73736192, "tokens_per_sec": 18297.387699537114, "vram_gb": 5.684933632, "lambda": 0.28172016764680546, "lambda_min": 0.09263095259666443, "lambda_max": 0.5925078988075256, "S_mean": 0.403656005859375, "G_mean": 0.3579088693950325, "G_std": 0.15390684797118107, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.462886905670166, "val_ppl": 86.73755112695724, "lambda": 0.2817256869748235, "lambda_min": 0.09262578189373016, "lambda_max": 0.5925370454788208, "lambda_per_layer": [0.17600758373737335, 0.15597374737262726, 0.13936248421669006, 0.1476914882659912, 0.12316252291202545, 0.12038423120975494, 0.11464573442935944, 0.10233204811811447, 0.09262578189373016, 0.1690189242362976, 0.373361736536026, 0.36658379435539246, 0.4429101049900055, 0.31540367007255554, 0.506958544254303, 0.5925370454788208, 0.5755859613418579, 0.3762671947479248, 0.5064725875854492, 0.24944625794887543, 0.27300456166267395, 0.247803196310997, 0.38039493560791016, 0.2134823501110077], "S_mean": 0.3932139078776042, "G_mean": 0.35569133058500785, "G_std": 0.15416160264673331, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.343264579772949, "grad_norm": 0.9083709120750427, "tokens_seen": 73900032, "tokens_per_sec": 18289.309398916095, "vram_gb": 5.684933632, "lambda": 0.2818328182523449, "lambda_min": 0.09254756569862366, "lambda_max": 0.5929043889045715, "S_mean": 0.3937276204427083, "G_mean": 0.35178415616974235, "G_std": 0.1528354520754268, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.429128885269165, "grad_norm": 0.9002455472946167, "tokens_seen": 74063872, "tokens_per_sec": 18288.924763423773, "vram_gb": 5.684933632, "lambda": 0.28196887268374365, "lambda_min": 0.09253861010074615, "lambda_max": 0.5933128595352173, "S_mean": 0.40069580078125, "G_mean": 0.35286782938055694, "G_std": 0.15386559433924654, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.2426841259002686, "grad_norm": 0.8895514011383057, "tokens_seen": 74227712, "tokens_per_sec": 18289.21215722729, "vram_gb": 5.684933632, "lambda": 0.2821289875234167, "lambda_min": 0.09255310148000717, "lambda_max": 0.5937197208404541, "S_mean": 0.3914998372395833, "G_mean": 0.3456715321323524, "G_std": 0.1532463908273106, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.387920379638672, "grad_norm": 0.9171867966651917, "tokens_seen": 74391552, "tokens_per_sec": 18289.483922033927, "vram_gb": 5.684933632, "lambda": 0.2822478556384643, "lambda_min": 0.09250731766223907, "lambda_max": 0.5939006209373474, "S_mean": 0.3978068033854167, "G_mean": 0.35451657449205715, "G_std": 0.15254157952343425, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.500537395477295, "grad_norm": 0.8612919449806213, "tokens_seen": 74555392, "tokens_per_sec": 18289.49090404945, "vram_gb": 5.684933632, "lambda": 0.28241043258458376, "lambda_min": 0.09253395348787308, "lambda_max": 0.594214677810669, "S_mean": 0.392059326171875, "G_mean": 0.3522710995748639, "G_std": 0.15483952837530524, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.4696877002716064, "grad_norm": 0.9075753688812256, "tokens_seen": 74719232, "tokens_per_sec": 18289.98949399576, "vram_gb": 5.684933632, "lambda": 0.28252696214864653, "lambda_min": 0.0925663635134697, "lambda_max": 0.5947107672691345, "S_mean": 0.4001719156901042, "G_mean": 0.35066289082169533, "G_std": 0.15220270197217664, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.245713472366333, "grad_norm": 0.8763635158538818, "tokens_seen": 74883072, "tokens_per_sec": 18290.078608250824, "vram_gb": 5.684933632, "lambda": 0.28268453913430375, "lambda_min": 0.09262671321630478, "lambda_max": 0.5952436327934265, "S_mean": 0.3886159261067708, "G_mean": 0.34992530872114, "G_std": 0.15288299915846437, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.5323262214660645, "grad_norm": 1.0575977563858032, "tokens_seen": 75046912, "tokens_per_sec": 18290.386618852423, "vram_gb": 5.684933632, "lambda": 0.2828342750047644, "lambda_min": 0.09265875071287155, "lambda_max": 0.5960479378700256, "S_mean": 0.3978169759114583, "G_mean": 0.3583304548325638, "G_std": 0.15276654650612423, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.335851192474365, "grad_norm": 0.921244740486145, "tokens_seen": 75210752, "tokens_per_sec": 18290.67476093241, "vram_gb": 5.684933632, "lambda": 0.2829824403549234, "lambda_min": 0.09266422688961029, "lambda_max": 0.5965883135795593, "S_mean": 0.39404296875, "G_mean": 0.35510068930064637, "G_std": 0.15197107836138457, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.432732582092285, "grad_norm": 0.8949102759361267, "tokens_seen": 75374592, "tokens_per_sec": 18291.068948546537, "vram_gb": 5.684933632, "lambda": 0.28314910798023146, "lambda_min": 0.092694953083992, "lambda_max": 0.5967788100242615, "S_mean": 0.391021728515625, "G_mean": 0.3491475930592666, "G_std": 0.15270393865648657, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.375286340713501, "grad_norm": 0.9943711757659912, "tokens_seen": 75538432, "tokens_per_sec": 18290.936112160503, "vram_gb": 5.684933632, "lambda": 0.2833139232049386, "lambda_min": 0.09269220381975174, "lambda_max": 0.5972110629081726, "S_mean": 0.3806355794270833, "G_mean": 0.3494332314003259, "G_std": 0.15240866388194263, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.395967483520508, "grad_norm": 0.9788076281547546, "tokens_seen": 75702272, "tokens_per_sec": 18290.9548631805, "vram_gb": 5.684933632, "lambda": 0.28345864980171126, "lambda_min": 0.09270083904266357, "lambda_max": 0.5975567102432251, "S_mean": 0.3902994791666667, "G_mean": 0.35085476787450415, "G_std": 0.15344058629125357, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.426356554031372, "grad_norm": 0.90457683801651, "tokens_seen": 75866112, "tokens_per_sec": 18290.788017533552, "vram_gb": 5.684933632, "lambda": 0.28358458417157334, "lambda_min": 0.09265217185020447, "lambda_max": 0.5980236530303955, "S_mean": 0.3876291910807292, "G_mean": 0.35199599697565037, "G_std": 0.1518523161454747, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.410011529922485, "grad_norm": 0.9407506585121155, "tokens_seen": 76029952, "tokens_per_sec": 18290.90940764322, "vram_gb": 5.684933632, "lambda": 0.2837104744588335, "lambda_min": 0.09268797934055328, "lambda_max": 0.5984314680099487, "S_mean": 0.4138590494791667, "G_mean": 0.3574292487464845, "G_std": 0.15206404212707034, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.293400287628174, "grad_norm": 0.9709521532058716, "tokens_seen": 76193792, "tokens_per_sec": 18290.28207356061, "vram_gb": 5.684933632, "lambda": 0.28383142842600745, "lambda_min": 0.09274496138095856, "lambda_max": 0.5986327528953552, "S_mean": 0.3938395182291667, "G_mean": 0.3506297883577645, "G_std": 0.15179314832979193, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.532415866851807, "grad_norm": 0.8992860913276672, "tokens_seen": 76357632, "tokens_per_sec": 18289.908716259193, "vram_gb": 5.684933632, "lambda": 0.2839677256221573, "lambda_min": 0.09282412379980087, "lambda_max": 0.59909987449646, "S_mean": 0.3886311848958333, "G_mean": 0.35241075480977696, "G_std": 0.15251737045279393, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.526261329650879, "grad_norm": 0.9739509224891663, "tokens_seen": 76521472, "tokens_per_sec": 18290.25890180567, "vram_gb": 5.684933632, "lambda": 0.28410009760409594, "lambda_min": 0.09280648082494736, "lambda_max": 0.5997313261032104, "S_mean": 0.3901265462239583, "G_mean": 0.35862439568154514, "G_std": 0.15168090048246086, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.357471466064453, "grad_norm": 0.9741159081459045, "tokens_seen": 76685312, "tokens_per_sec": 18290.45256092445, "vram_gb": 5.684933632, "lambda": 0.2842389636983474, "lambda_min": 0.09278769046068192, "lambda_max": 0.6001995205879211, "S_mean": 0.39471435546875, "G_mean": 0.3513757574837655, "G_std": 0.151869590433004, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.419425010681152, "grad_norm": 0.8890888094902039, "tokens_seen": 76849152, "tokens_per_sec": 18290.342732645768, "vram_gb": 5.684933632, "lambda": 0.2843957481284936, "lambda_min": 0.09269043803215027, "lambda_max": 0.600481390953064, "S_mean": 0.3949228922526042, "G_mean": 0.35033719257141155, "G_std": 0.15050246745037535, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.387607097625732, "grad_norm": 0.9557620286941528, "tokens_seen": 77012992, "tokens_per_sec": 18290.35965319034, "vram_gb": 5.684933632, "lambda": 0.28453630798806745, "lambda_min": 0.09270680695772171, "lambda_max": 0.6010342836380005, "S_mean": 0.3863525390625, "G_mean": 0.3539602153468877, "G_std": 0.15389553158699223, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.381150007247925, "grad_norm": 0.9343851208686829, "tokens_seen": 77176832, "tokens_per_sec": 18289.935217207163, "vram_gb": 5.684933632, "lambda": 0.28466417888800305, "lambda_min": 0.09274627268314362, "lambda_max": 0.6016250848770142, "S_mean": 0.3939666748046875, "G_mean": 0.35042750649154186, "G_std": 0.15159897269525877, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.2422261238098145, "grad_norm": 0.8838435411453247, "tokens_seen": 77340672, "tokens_per_sec": 18290.060349791638, "vram_gb": 5.684933632, "lambda": 0.2847707948336999, "lambda_min": 0.09271084517240524, "lambda_max": 0.6019365787506104, "S_mean": 0.411865234375, "G_mean": 0.35526701738126576, "G_std": 0.15350837376900017, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.483277320861816, "grad_norm": 0.9077817797660828, "tokens_seen": 77504512, "tokens_per_sec": 18289.9838769916, "vram_gb": 5.684933632, "lambda": 0.28490042717506486, "lambda_min": 0.09265704452991486, "lambda_max": 0.6023927330970764, "S_mean": 0.3895975748697917, "G_mean": 0.35027493233792484, "G_std": 0.15255864907521755, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.5536017417907715, "grad_norm": 0.9680621027946472, "tokens_seen": 77668352, "tokens_per_sec": 18290.170286591347, "vram_gb": 5.684933632, "lambda": 0.28500588176151115, "lambda_min": 0.09267885237932205, "lambda_max": 0.6028047204017639, "S_mean": 0.4001973470052083, "G_mean": 0.3535595720944305, "G_std": 0.15115512752284607, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.285616636276245, "grad_norm": 0.8780069351196289, "tokens_seen": 77832192, "tokens_per_sec": 18290.115478262855, "vram_gb": 5.684933632, "lambda": 0.2851064608742793, "lambda_min": 0.09264951199293137, "lambda_max": 0.6031078696250916, "S_mean": 0.4019266764322917, "G_mean": 0.3493980606241773, "G_std": 0.15356360274987915, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.401906728744507, "grad_norm": 1.0000102519989014, "tokens_seen": 77996032, "tokens_per_sec": 18289.574691331236, "vram_gb": 5.684933632, "lambda": 0.28521721189220745, "lambda_min": 0.09263641387224197, "lambda_max": 0.6033067107200623, "S_mean": 0.4002583821614583, "G_mean": 0.36052481713704765, "G_std": 0.15088108011210957, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.353095054626465, "grad_norm": 0.9202643036842346, "tokens_seen": 78159872, "tokens_per_sec": 18289.567805351053, "vram_gb": 5.684933632, "lambda": 0.2853229319055875, "lambda_min": 0.09263639897108078, "lambda_max": 0.6034201383590698, "S_mean": 0.3977813720703125, "G_mean": 0.35144722725575167, "G_std": 0.1525846616908287, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.290523529052734, "grad_norm": 0.8831409811973572, "tokens_seen": 78323712, "tokens_per_sec": 18289.874227606306, "vram_gb": 5.684933632, "lambda": 0.2854468484098713, "lambda_min": 0.09261391311883926, "lambda_max": 0.6036627292633057, "S_mean": 0.3979390462239583, "G_mean": 0.35453080967999995, "G_std": 0.15264206558155516, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.389373779296875, "grad_norm": 0.9381522536277771, "tokens_seen": 78487552, "tokens_per_sec": 18290.13472742843, "vram_gb": 5.684933632, "lambda": 0.28555870801210403, "lambda_min": 0.09261302649974823, "lambda_max": 0.6039992570877075, "S_mean": 0.3955790201822917, "G_mean": 0.3502388385434945, "G_std": 0.15310548905593654, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.595818519592285, "grad_norm": 0.9363499283790588, "tokens_seen": 78651392, "tokens_per_sec": 18290.55258419695, "vram_gb": 5.684933632, "lambda": 0.28568361699581146, "lambda_min": 0.09261453151702881, "lambda_max": 0.6045666337013245, "S_mean": 0.40057373046875, "G_mean": 0.35032409251046676, "G_std": 0.1536802981669704, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.406392335891724, "grad_norm": 0.9658107757568359, "tokens_seen": 78815232, "tokens_per_sec": 18290.534323310825, "vram_gb": 5.684933632, "lambda": 0.28582616802304983, "lambda_min": 0.09265536814928055, "lambda_max": 0.605117678642273, "S_mean": 0.4017791748046875, "G_mean": 0.3507808418944478, "G_std": 0.15096134742877135, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.240245819091797, "grad_norm": 0.9512710571289062, "tokens_seen": 78979072, "tokens_per_sec": 18290.91226135648, "vram_gb": 5.684933632, "lambda": 0.28598050959408283, "lambda_min": 0.09263882786035538, "lambda_max": 0.6055597066879272, "S_mean": 0.4034627278645833, "G_mean": 0.35422945573615533, "G_std": 0.15035667619667947, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.4774489402771, "grad_norm": 0.9523465037345886, "tokens_seen": 79142912, "tokens_per_sec": 18291.13901172526, "vram_gb": 5.684933632, "lambda": 0.2860807413235307, "lambda_min": 0.09265045821666718, "lambda_max": 0.6057602167129517, "S_mean": 0.3924458821614583, "G_mean": 0.3541374326062699, "G_std": 0.1517334480692322, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.3923962116241455, "grad_norm": 1.006157636642456, "tokens_seen": 79306752, "tokens_per_sec": 18291.504266271553, "vram_gb": 5.684933632, "lambda": 0.28621187899261713, "lambda_min": 0.09262575954198837, "lambda_max": 0.6061992645263672, "S_mean": 0.391357421875, "G_mean": 0.35010652026782435, "G_std": 0.151413090294227, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.253800868988037, "grad_norm": 0.9284365773200989, "tokens_seen": 79470592, "tokens_per_sec": 18291.986228180966, "vram_gb": 5.684933632, "lambda": 0.2863213500628869, "lambda_min": 0.09257897734642029, "lambda_max": 0.6066283583641052, "S_mean": 0.3945515950520833, "G_mean": 0.3442042422636102, "G_std": 0.15142352634575218, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.332883358001709, "grad_norm": 0.9267266392707825, "tokens_seen": 79634432, "tokens_per_sec": 18292.301638873883, "vram_gb": 5.684933632, "lambda": 0.2864219282443325, "lambda_min": 0.09254741668701172, "lambda_max": 0.6070076823234558, "S_mean": 0.3857421875, "G_mean": 0.35216054460033774, "G_std": 0.15219700798237076, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.318997859954834, "grad_norm": 0.9530343413352966, "tokens_seen": 79798272, "tokens_per_sec": 18292.58527838482, "vram_gb": 5.684933632, "lambda": 0.28650954955567914, "lambda_min": 0.09251856803894043, "lambda_max": 0.6073665618896484, "S_mean": 0.3889211018880208, "G_mean": 0.34929375528978807, "G_std": 0.1513613131052504, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.390061140060425, "grad_norm": 0.9305859804153442, "tokens_seen": 79962112, "tokens_per_sec": 18292.749211047925, "vram_gb": 5.684933632, "lambda": 0.28662734292447567, "lambda_min": 0.09253226965665817, "lambda_max": 0.6075683236122131, "S_mean": 0.3844960530598958, "G_mean": 0.34823179555435974, "G_std": 0.1519437034924825, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.331621408462524, "grad_norm": 0.9248923659324646, "tokens_seen": 80125952, "tokens_per_sec": 18292.998320711395, "vram_gb": 5.684933632, "lambda": 0.2867491543292999, "lambda_min": 0.09251520782709122, "lambda_max": 0.6078282594680786, "S_mean": 0.393768310546875, "G_mean": 0.351815311005339, "G_std": 0.15193212358281016, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.331635475158691, "grad_norm": 0.930621862411499, "tokens_seen": 80289792, "tokens_per_sec": 18293.139317605146, "vram_gb": 5.684933632, "lambda": 0.2868690388277173, "lambda_min": 0.09250868856906891, "lambda_max": 0.6081444025039673, "S_mean": 0.3875223795572917, "G_mean": 0.3507756139151752, "G_std": 0.15209216209283719, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.2923665046691895, "grad_norm": 0.9320075511932373, "tokens_seen": 80453632, "tokens_per_sec": 18293.25718931994, "vram_gb": 5.684933632, "lambda": 0.2869994780048728, "lambda_min": 0.09249500185251236, "lambda_max": 0.608698844909668, "S_mean": 0.4014689127604167, "G_mean": 0.35083743215849, "G_std": 0.15099345011791834, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.3264172077178955, "grad_norm": 0.9246284365653992, "tokens_seen": 80617472, "tokens_per_sec": 18293.60956624683, "vram_gb": 5.684933632, "lambda": 0.2870847520728906, "lambda_min": 0.09252404421567917, "lambda_max": 0.6088879108428955, "S_mean": 0.3960164388020833, "G_mean": 0.3460657140240073, "G_std": 0.15088550214810917, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.319313049316406, "grad_norm": 0.9783167243003845, "tokens_seen": 80781312, "tokens_per_sec": 18293.60123631923, "vram_gb": 5.684933632, "lambda": 0.2871894808486104, "lambda_min": 0.09253901243209839, "lambda_max": 0.6090027093887329, "S_mean": 0.3897959391276042, "G_mean": 0.3540173589717597, "G_std": 0.15291080763563514, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.2287938594818115, "grad_norm": 0.9094874262809753, "tokens_seen": 80945152, "tokens_per_sec": 18293.85023165897, "vram_gb": 5.684933632, "lambda": 0.28727624099701643, "lambda_min": 0.092491015791893, "lambda_max": 0.6092544198036194, "S_mean": 0.4145456949869792, "G_mean": 0.35040352881575626, "G_std": 0.1513187144494926, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.337738990783691, "grad_norm": 0.9577129483222961, "tokens_seen": 81108992, "tokens_per_sec": 18293.91918753692, "vram_gb": 5.684933632, "lambda": 0.287381986156106, "lambda_min": 0.0925457775592804, "lambda_max": 0.6094653606414795, "S_mean": 0.3948262532552083, "G_mean": 0.3488711613851289, "G_std": 0.15089834469836205, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.330282211303711, "grad_norm": 0.8945531845092773, "tokens_seen": 81272832, "tokens_per_sec": 18294.183318563028, "vram_gb": 5.684933632, "lambda": 0.2874921889354785, "lambda_min": 0.09254346042871475, "lambda_max": 0.6097546219825745, "S_mean": 0.3908335367838542, "G_mean": 0.34826913064656156, "G_std": 0.1520003314362839, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.300934314727783, "grad_norm": 1.006401538848877, "tokens_seen": 81436672, "tokens_per_sec": 18294.418257109686, "vram_gb": 5.684933632, "lambda": 0.2875727778300643, "lambda_min": 0.09259246289730072, "lambda_max": 0.6100114583969116, "S_mean": 0.3996938069661458, "G_mean": 0.3496007982175797, "G_std": 0.15347355623574307, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.478017807006836, "grad_norm": 0.981929361820221, "tokens_seen": 81600512, "tokens_per_sec": 18293.85117801923, "vram_gb": 5.684933632, "lambda": 0.2876891726627946, "lambda_min": 0.09259189665317535, "lambda_max": 0.6104807257652283, "S_mean": 0.4026590983072917, "G_mean": 0.3487881791467468, "G_std": 0.15147066547069699, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.237415313720703, "grad_norm": 0.9544782042503357, "tokens_seen": 81764352, "tokens_per_sec": 18293.831799720472, "vram_gb": 5.684933632, "lambda": 0.28780122691144544, "lambda_min": 0.09261158853769302, "lambda_max": 0.6108718514442444, "S_mean": 0.3894195556640625, "G_mean": 0.3509773470771809, "G_std": 0.15222489653388038, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.386967420578003, "grad_norm": 1.026269555091858, "tokens_seen": 81928192, "tokens_per_sec": 18294.131242130898, "vram_gb": 5.684933632, "lambda": 0.2879038928076625, "lambda_min": 0.0926743671298027, "lambda_max": 0.6112435460090637, "S_mean": 0.3854624430338542, "G_mean": 0.3539778118332227, "G_std": 0.15256051191439232, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.4536034107208256, "val_ppl": 85.93604963301473, "lambda": 0.2879080940037966, "lambda_min": 0.09267839044332504, "lambda_max": 0.6112597584724426, "lambda_per_layer": [0.17689859867095947, 0.15692421793937683, 0.14017675817012787, 0.14765140414237976, 0.12278556078672409, 0.12034495174884796, 0.11414262652397156, 0.10261514782905579, 0.09267839044332504, 0.17009709775447845, 0.37803658843040466, 0.3727608025074005, 0.4513325095176697, 0.3213946521282196, 0.5231440663337708, 0.6112597584724426, 0.5938779711723328, 0.3861500918865204, 0.5239206552505493, 0.2569129168987274, 0.28076061606407166, 0.254179984331131, 0.3931465446949005, 0.21860234439373016], "S_mean": 0.385284423828125, "G_mean": 0.35151939707187313, "G_std": 0.15186167159117758, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.253297805786133, "grad_norm": 0.9503504633903503, "tokens_seen": 82092032, "tokens_per_sec": 18287.297530649917, "vram_gb": 5.684933632, "lambda": 0.2880248427391052, "lambda_min": 0.09270847588777542, "lambda_max": 0.6116799712181091, "S_mean": 0.4016215006510417, "G_mean": 0.348458306863904, "G_std": 0.15204496561394384, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.354042053222656, "grad_norm": 0.9373801946640015, "tokens_seen": 82255872, "tokens_per_sec": 18287.410419888263, "vram_gb": 5.684933632, "lambda": 0.28811350588997203, "lambda_min": 0.0926855206489563, "lambda_max": 0.6119304895401001, "S_mean": 0.3991597493489583, "G_mean": 0.35458426635401946, "G_std": 0.1518398500435675, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.249661445617676, "grad_norm": 0.9785416722297668, "tokens_seen": 82419712, "tokens_per_sec": 18287.50797782134, "vram_gb": 5.684933632, "lambda": 0.28822563712795574, "lambda_min": 0.09269336611032486, "lambda_max": 0.6121680736541748, "S_mean": 0.3824920654296875, "G_mean": 0.3482105729635805, "G_std": 0.15239757920304933, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.190526485443115, "grad_norm": 0.8985538482666016, "tokens_seen": 82583552, "tokens_per_sec": 18287.791519667706, "vram_gb": 5.684933632, "lambda": 0.28830761307229597, "lambda_min": 0.09272509813308716, "lambda_max": 0.6122215390205383, "S_mean": 0.3844451904296875, "G_mean": 0.34428616147488356, "G_std": 0.15097621865182495, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.483482122421265, "grad_norm": 1.0413103103637695, "tokens_seen": 82747392, "tokens_per_sec": 18287.876705695144, "vram_gb": 5.684933632, "lambda": 0.28839656213919324, "lambda_min": 0.09277628362178802, "lambda_max": 0.6125181317329407, "S_mean": 0.3861033121744792, "G_mean": 0.3511688280850649, "G_std": 0.1520777929496641, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.387130260467529, "grad_norm": 0.9516112804412842, "tokens_seen": 82911232, "tokens_per_sec": 18288.061733070484, "vram_gb": 5.684933632, "lambda": 0.2884990470483899, "lambda_min": 0.09278657287359238, "lambda_max": 0.612903892993927, "S_mean": 0.378875732421875, "G_mean": 0.34762357563401264, "G_std": 0.15232274580436447, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.276258945465088, "grad_norm": 0.9280924797058105, "tokens_seen": 83075072, "tokens_per_sec": 18288.28132796317, "vram_gb": 5.684933632, "lambda": 0.288606157215933, "lambda_min": 0.09277354180812836, "lambda_max": 0.6131715178489685, "S_mean": 0.3900655110677083, "G_mean": 0.34878535677368444, "G_std": 0.15232972249699137, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.412110090255737, "grad_norm": 0.9435945749282837, "tokens_seen": 83238912, "tokens_per_sec": 18288.38905877608, "vram_gb": 5.684933632, "lambda": 0.28870289990057546, "lambda_min": 0.092747263610363, "lambda_max": 0.6135055422782898, "S_mean": 0.3895212809244792, "G_mean": 0.3464270392432809, "G_std": 0.15148647467140108, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.503855466842651, "grad_norm": 0.9622884392738342, "tokens_seen": 83402752, "tokens_per_sec": 18288.508985609125, "vram_gb": 5.684933632, "lambda": 0.2887761077533166, "lambda_min": 0.09271744638681412, "lambda_max": 0.6138114333152771, "S_mean": 0.3812306722005208, "G_mean": 0.35000671764525276, "G_std": 0.1522306431628143, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.484653949737549, "grad_norm": 0.9665144681930542, "tokens_seen": 83566592, "tokens_per_sec": 18288.974526496753, "vram_gb": 5.684933632, "lambda": 0.2888695476576686, "lambda_min": 0.09270909428596497, "lambda_max": 0.6141111850738525, "S_mean": 0.391693115234375, "G_mean": 0.34927926554034155, "G_std": 0.15262039215303957, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.388233661651611, "grad_norm": 0.962575376033783, "tokens_seen": 83730432, "tokens_per_sec": 18289.282038310375, "vram_gb": 5.684933632, "lambda": 0.28896275473137695, "lambda_min": 0.09271805733442307, "lambda_max": 0.6144339442253113, "S_mean": 0.3919219970703125, "G_mean": 0.3504551756195724, "G_std": 0.15100016025826335, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.260573863983154, "grad_norm": 0.9744824767112732, "tokens_seen": 83894272, "tokens_per_sec": 18289.24652539898, "vram_gb": 5.684933632, "lambda": 0.2890955038989584, "lambda_min": 0.09273609519004822, "lambda_max": 0.6150547862052917, "S_mean": 0.385986328125, "G_mean": 0.3490292942151427, "G_std": 0.15088921564165503, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.394684314727783, "grad_norm": 0.9855300188064575, "tokens_seen": 84058112, "tokens_per_sec": 18289.45330391273, "vram_gb": 5.684933632, "lambda": 0.2892116916676362, "lambda_min": 0.0927659124135971, "lambda_max": 0.6154608130455017, "S_mean": 0.3962809244791667, "G_mean": 0.34765375514204305, "G_std": 0.15101108703917512, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.3979387283325195, "grad_norm": 1.0369853973388672, "tokens_seen": 84221952, "tokens_per_sec": 18289.854555697737, "vram_gb": 5.684933632, "lambda": 0.2893339504177372, "lambda_min": 0.09280688315629959, "lambda_max": 0.6158871650695801, "S_mean": 0.3864390055338542, "G_mean": 0.3496775613166392, "G_std": 0.1510680332624664, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.297255277633667, "grad_norm": 0.9368658065795898, "tokens_seen": 84385792, "tokens_per_sec": 18289.847343083857, "vram_gb": 5.684933632, "lambda": 0.28941441761950654, "lambda_min": 0.09284672141075134, "lambda_max": 0.6161969304084778, "S_mean": 0.3923136393229167, "G_mean": 0.3531998172402382, "G_std": 0.15153645099295923, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.420623540878296, "grad_norm": 0.9478351473808289, "tokens_seen": 84549632, "tokens_per_sec": 18290.001990854045, "vram_gb": 5.684933632, "lambda": 0.2895001145079732, "lambda_min": 0.09282443672418594, "lambda_max": 0.6165686845779419, "S_mean": 0.3819376627604167, "G_mean": 0.35077680779310566, "G_std": 0.15154915543583533, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.264033555984497, "grad_norm": 0.9346973299980164, "tokens_seen": 84713472, "tokens_per_sec": 18290.093220413073, "vram_gb": 5.684933632, "lambda": 0.28957391685495776, "lambda_min": 0.09277660399675369, "lambda_max": 0.6167977452278137, "S_mean": 0.3927001953125, "G_mean": 0.34559814166277647, "G_std": 0.1508256211721649, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.295927286148071, "grad_norm": 0.9168721437454224, "tokens_seen": 84877312, "tokens_per_sec": 18290.23376612612, "vram_gb": 5.684933632, "lambda": 0.2896354825546344, "lambda_min": 0.09277752041816711, "lambda_max": 0.616671621799469, "S_mean": 0.3828023274739583, "G_mean": 0.3475181677689155, "G_std": 0.15075343428179622, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.429450273513794, "grad_norm": 1.01190185546875, "tokens_seen": 85041152, "tokens_per_sec": 18290.570127474253, "vram_gb": 5.684933632, "lambda": 0.2896922367314498, "lambda_min": 0.0927877351641655, "lambda_max": 0.617002010345459, "S_mean": 0.3808085123697917, "G_mean": 0.354249512621512, "G_std": 0.1518342731287703, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.385357618331909, "grad_norm": 0.9644330143928528, "tokens_seen": 85204992, "tokens_per_sec": 18290.62188519707, "vram_gb": 5.684933632, "lambda": 0.28975357456753653, "lambda_min": 0.09278017282485962, "lambda_max": 0.6169899106025696, "S_mean": 0.4001007080078125, "G_mean": 0.3485590770530204, "G_std": 0.15216485841665417, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.2386157512664795, "grad_norm": 0.9453426599502563, "tokens_seen": 85368832, "tokens_per_sec": 18290.965304831083, "vram_gb": 5.684933632, "lambda": 0.28985659312456846, "lambda_min": 0.09278275072574615, "lambda_max": 0.6170695424079895, "S_mean": 0.3909708658854167, "G_mean": 0.34802380825082463, "G_std": 0.15077085505860546, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.235895156860352, "grad_norm": 0.9336043000221252, "tokens_seen": 85532672, "tokens_per_sec": 18290.524435661024, "vram_gb": 5.684933632, "lambda": 0.28993251143644255, "lambda_min": 0.0927601084113121, "lambda_max": 0.6173263192176819, "S_mean": 0.3877003987630208, "G_mean": 0.3480362536696096, "G_std": 0.15042090835049748, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.441034317016602, "grad_norm": 1.0139710903167725, "tokens_seen": 85696512, "tokens_per_sec": 18290.697513014802, "vram_gb": 5.684933632, "lambda": 0.29002055681000155, "lambda_min": 0.09277988225221634, "lambda_max": 0.6177753806114197, "S_mean": 0.3864898681640625, "G_mean": 0.35174041070664924, "G_std": 0.15111586645555994, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.5702431201934814, "grad_norm": 0.9841073155403137, "tokens_seen": 85860352, "tokens_per_sec": 18291.064006230543, "vram_gb": 5.684933632, "lambda": 0.2901010476052761, "lambda_min": 0.09277518838644028, "lambda_max": 0.6181241273880005, "S_mean": 0.3916015625, "G_mean": 0.3486231987675031, "G_std": 0.15092595953804752, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.397331237792969, "grad_norm": 0.9870181083679199, "tokens_seen": 86024192, "tokens_per_sec": 18291.029334760256, "vram_gb": 5.684933632, "lambda": 0.2901807675758998, "lambda_min": 0.09275016188621521, "lambda_max": 0.6184167861938477, "S_mean": 0.3884175618489583, "G_mean": 0.34877660490262014, "G_std": 0.15128757963733128, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.302988529205322, "grad_norm": 0.9867922067642212, "tokens_seen": 86188032, "tokens_per_sec": 18291.400975916284, "vram_gb": 5.684933632, "lambda": 0.2902714600786567, "lambda_min": 0.09271971136331558, "lambda_max": 0.6187528371810913, "S_mean": 0.3805592854817708, "G_mean": 0.3472804863316317, "G_std": 0.15095734491478652, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.259705305099487, "grad_norm": 0.9439285397529602, "tokens_seen": 86351872, "tokens_per_sec": 18291.819622813353, "vram_gb": 5.684933632, "lambda": 0.2903604970003168, "lambda_min": 0.09271219372749329, "lambda_max": 0.6190522909164429, "S_mean": 0.4029744466145833, "G_mean": 0.35422316162536543, "G_std": 0.15015901275910437, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.229067802429199, "grad_norm": 0.9428063035011292, "tokens_seen": 86515712, "tokens_per_sec": 18291.841996876014, "vram_gb": 5.684933632, "lambda": 0.290447188851734, "lambda_min": 0.09273432195186615, "lambda_max": 0.6192021369934082, "S_mean": 0.3943328857421875, "G_mean": 0.345743351848796, "G_std": 0.15030247489145646, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.435324668884277, "grad_norm": 0.974852979183197, "tokens_seen": 86679552, "tokens_per_sec": 18292.07517674089, "vram_gb": 5.684933632, "lambda": 0.29052525293082, "lambda_min": 0.09273351728916168, "lambda_max": 0.6193854808807373, "S_mean": 0.391204833984375, "G_mean": 0.35062486802538234, "G_std": 0.14988700672984123, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.232860088348389, "grad_norm": 0.9383203387260437, "tokens_seen": 86843392, "tokens_per_sec": 18292.438566507954, "vram_gb": 5.684933632, "lambda": 0.29059709639598924, "lambda_min": 0.09273075312376022, "lambda_max": 0.6196130514144897, "S_mean": 0.4030100504557292, "G_mean": 0.35067679717515904, "G_std": 0.1499474591886004, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.392475843429565, "grad_norm": 0.9515361785888672, "tokens_seen": 87007232, "tokens_per_sec": 18292.654142650463, "vram_gb": 5.684933632, "lambda": 0.29067428316920996, "lambda_min": 0.09268292039632797, "lambda_max": 0.6199138164520264, "S_mean": 0.3951924641927083, "G_mean": 0.34875851597947377, "G_std": 0.15118807884088406, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.233252048492432, "grad_norm": 0.923592746257782, "tokens_seen": 87171072, "tokens_per_sec": 18293.033219299254, "vram_gb": 5.684933632, "lambda": 0.2907598266998927, "lambda_min": 0.09264342486858368, "lambda_max": 0.6202577352523804, "S_mean": 0.4235636393229167, "G_mean": 0.35635501115272444, "G_std": 0.15131298025759557, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.32257080078125, "grad_norm": 0.9576848149299622, "tokens_seen": 87334912, "tokens_per_sec": 18293.038563388858, "vram_gb": 5.684933632, "lambda": 0.2908205473795533, "lambda_min": 0.09261924773454666, "lambda_max": 0.6204869747161865, "S_mean": 0.4024861653645833, "G_mean": 0.3437254980672151, "G_std": 0.14978889628158262, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.378699779510498, "grad_norm": 1.0219860076904297, "tokens_seen": 87498752, "tokens_per_sec": 18293.059280864432, "vram_gb": 5.684933632, "lambda": 0.29090423447390396, "lambda_min": 0.09259893745183945, "lambda_max": 0.6207413077354431, "S_mean": 0.3782246907552083, "G_mean": 0.3481454163168867, "G_std": 0.15112452674657106, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.383992671966553, "grad_norm": 1.1484297513961792, "tokens_seen": 87662592, "tokens_per_sec": 18293.442222071033, "vram_gb": 5.684933632, "lambda": 0.2910111201927066, "lambda_min": 0.09261007606983185, "lambda_max": 0.62102210521698, "S_mean": 0.3929189046223958, "G_mean": 0.344527550196896, "G_std": 0.15227651231301328, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.307342529296875, "grad_norm": 0.9189075231552124, "tokens_seen": 87826432, "tokens_per_sec": 18293.866931467357, "vram_gb": 5.684933632, "lambda": 0.29109403925637406, "lambda_min": 0.09262067824602127, "lambda_max": 0.6212480664253235, "S_mean": 0.4043223063151042, "G_mean": 0.34858141707566875, "G_std": 0.15139389934483916, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.241979598999023, "grad_norm": 0.9527115821838379, "tokens_seen": 87990272, "tokens_per_sec": 18294.22578051401, "vram_gb": 5.684933632, "lambda": 0.29116902127861977, "lambda_min": 0.09268652647733688, "lambda_max": 0.6213793158531189, "S_mean": 0.3964335123697917, "G_mean": 0.3501182604425897, "G_std": 0.15121822567501417, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.356188058853149, "grad_norm": 0.9698405861854553, "tokens_seen": 88154112, "tokens_per_sec": 18294.59135777345, "vram_gb": 5.684933632, "lambda": 0.29123306771119434, "lambda_min": 0.09269220381975174, "lambda_max": 0.6217787861824036, "S_mean": 0.3878682454427083, "G_mean": 0.3490511835552752, "G_std": 0.15123568649869412, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.332229137420654, "grad_norm": 0.9641404747962952, "tokens_seen": 88317952, "tokens_per_sec": 18294.842731931152, "vram_gb": 5.684933632, "lambda": 0.2913126712664962, "lambda_min": 0.09267308562994003, "lambda_max": 0.6220096349716187, "S_mean": 0.4045867919921875, "G_mean": 0.3471968839876354, "G_std": 0.15135352526946613, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.37186598777771, "grad_norm": 0.9919405579566956, "tokens_seen": 88481792, "tokens_per_sec": 18295.30420688648, "vram_gb": 5.684933632, "lambda": 0.29138096049427986, "lambda_min": 0.09265531599521637, "lambda_max": 0.6221480369567871, "S_mean": 0.3923746744791667, "G_mean": 0.34559186338447034, "G_std": 0.1525937101105228, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.31039834022522, "grad_norm": 0.9866604208946228, "tokens_seen": 88645632, "tokens_per_sec": 18295.39654565131, "vram_gb": 5.684933632, "lambda": 0.2914893363292019, "lambda_min": 0.09266447275876999, "lambda_max": 0.6223985552787781, "S_mean": 0.3831888834635417, "G_mean": 0.3520235445660849, "G_std": 0.15159999404568225, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.467325448989868, "grad_norm": 0.9726029634475708, "tokens_seen": 88809472, "tokens_per_sec": 18295.634724579675, "vram_gb": 5.684933632, "lambda": 0.29156228620558977, "lambda_min": 0.09266844391822815, "lambda_max": 0.6225489974021912, "S_mean": 0.4029795328776042, "G_mean": 0.3486957602823774, "G_std": 0.15056710046095154, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.39924955368042, "grad_norm": 0.9874943494796753, "tokens_seen": 88973312, "tokens_per_sec": 18295.840014560006, "vram_gb": 5.684933632, "lambda": 0.29162190947681665, "lambda_min": 0.09267464280128479, "lambda_max": 0.6227236986160278, "S_mean": 0.3884226481119792, "G_mean": 0.3492409452640762, "G_std": 0.15085435080497214, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.412136793136597, "grad_norm": 0.9928034543991089, "tokens_seen": 89137152, "tokens_per_sec": 18295.961185543234, "vram_gb": 5.684933632, "lambda": 0.2916844558591644, "lambda_min": 0.09270020574331284, "lambda_max": 0.6229745745658875, "S_mean": 0.3999735514322917, "G_mean": 0.34933063744877774, "G_std": 0.15200623885417977, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.264726638793945, "grad_norm": 0.9636192321777344, "tokens_seen": 89300992, "tokens_per_sec": 18296.27264541344, "vram_gb": 5.684933632, "lambda": 0.2917492634927233, "lambda_min": 0.09270664304494858, "lambda_max": 0.6231563687324524, "S_mean": 0.39312744140625, "G_mean": 0.34308602986857295, "G_std": 0.1500921972716848, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.162081480026245, "grad_norm": 0.9474366903305054, "tokens_seen": 89464832, "tokens_per_sec": 18296.327430017827, "vram_gb": 5.684933632, "lambda": 0.2918381628890832, "lambda_min": 0.09272993355989456, "lambda_max": 0.6233847141265869, "S_mean": 0.3976949055989583, "G_mean": 0.3437670817753921, "G_std": 0.1509570931084454, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.433305025100708, "grad_norm": 0.9749447703361511, "tokens_seen": 89628672, "tokens_per_sec": 18296.145096051838, "vram_gb": 5.684933632, "lambda": 0.2919292940447728, "lambda_min": 0.09275845438241959, "lambda_max": 0.623612642288208, "S_mean": 0.3949686686197917, "G_mean": 0.35436840712403256, "G_std": 0.149591198656708, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.346801996231079, "grad_norm": 0.9828968048095703, "tokens_seen": 89792512, "tokens_per_sec": 18296.17820379057, "vram_gb": 5.684933632, "lambda": 0.29202997280905646, "lambda_min": 0.09276014566421509, "lambda_max": 0.6237978339195251, "S_mean": 0.3867848714192708, "G_mean": 0.3468780176093181, "G_std": 0.15195620133696744, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.3203959465026855, "grad_norm": 1.008508563041687, "tokens_seen": 89956352, "tokens_per_sec": 18296.181184480665, "vram_gb": 5.684933632, "lambda": 0.2921091364696622, "lambda_min": 0.09281138330698013, "lambda_max": 0.6241332292556763, "S_mean": 0.3899688720703125, "G_mean": 0.3481501841451973, "G_std": 0.15024121462677917, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.326669931411743, "grad_norm": 0.9916279911994934, "tokens_seen": 90120192, "tokens_per_sec": 18296.247338776957, "vram_gb": 5.684933632, "lambda": 0.29220844618976116, "lambda_min": 0.09284160286188126, "lambda_max": 0.6244406700134277, "S_mean": 0.3882191975911458, "G_mean": 0.3521840472240001, "G_std": 0.14965260191820562, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.3724324941635135, "val_ppl": 79.23613891375149, "lambda": 0.2922128752494852, "lambda_min": 0.09284373372793198, "lambda_max": 0.6244505643844604, "lambda_per_layer": [0.17782586812973022, 0.1577310860157013, 0.1408786177635193, 0.14774176478385925, 0.12292543798685074, 0.12066355347633362, 0.11410331726074219, 0.1024189367890358, 0.09284373372793198, 0.171176940202713, 0.38078656792640686, 0.37626343965530396, 0.4570145606994629, 0.32509657740592957, 0.5334392189979553, 0.6244505643844604, 0.6071454882621765, 0.39371198415756226, 0.5371505618095398, 0.261089563369751, 0.2854559123516083, 0.25877511501312256, 0.4027375280857086, 0.22168266773223877], "S_mean": 0.392181396484375, "G_mean": 0.351289534708485, "G_std": 0.1505053159004698, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.284345626831055, "grad_norm": 1.0140501260757446, "tokens_seen": 90284032, "tokens_per_sec": 18289.922348544624, "vram_gb": 5.684933632, "lambda": 0.2922888156026602, "lambda_min": 0.09288040548563004, "lambda_max": 0.6245660185813904, "S_mean": 0.3880106608072917, "G_mean": 0.35106598570321995, "G_std": 0.15047365081651756, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.273317337036133, "grad_norm": 0.9426714777946472, "tokens_seen": 90447872, "tokens_per_sec": 18289.9965483338, "vram_gb": 5.684933632, "lambda": 0.29234910539040965, "lambda_min": 0.09285904467105865, "lambda_max": 0.6246558427810669, "S_mean": 0.3919881184895833, "G_mean": 0.34253992000594735, "G_std": 0.15164051488197097, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.411778450012207, "grad_norm": 0.9350086450576782, "tokens_seen": 90611712, "tokens_per_sec": 18290.25266640442, "vram_gb": 5.684933632, "lambda": 0.29243073302010697, "lambda_min": 0.0928385853767395, "lambda_max": 0.6248332262039185, "S_mean": 0.4069620768229167, "G_mean": 0.34784327959641814, "G_std": 0.15160011585491398, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.121892690658569, "grad_norm": 0.9948318600654602, "tokens_seen": 90775552, "tokens_per_sec": 18290.62393139304, "vram_gb": 5.684933632, "lambda": 0.2925067792336146, "lambda_min": 0.09283874928951263, "lambda_max": 0.6250826120376587, "S_mean": 0.3939921061197917, "G_mean": 0.34792202594690025, "G_std": 0.1515560382977128, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.370027303695679, "grad_norm": 1.0313183069229126, "tokens_seen": 90939392, "tokens_per_sec": 18290.831768745298, "vram_gb": 5.684933632, "lambda": 0.2925861269856493, "lambda_min": 0.09285805374383926, "lambda_max": 0.6253776550292969, "S_mean": 0.3936411539713542, "G_mean": 0.3476062035188079, "G_std": 0.15151190621933588, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.311955213546753, "grad_norm": 0.9257754683494568, "tokens_seen": 91103232, "tokens_per_sec": 18291.057179859406, "vram_gb": 5.684933632, "lambda": 0.29266684347142774, "lambda_min": 0.09285217523574829, "lambda_max": 0.6256355047225952, "S_mean": 0.397003173828125, "G_mean": 0.34260160150006413, "G_std": 0.15166569924137244, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.330958604812622, "grad_norm": 0.9532983899116516, "tokens_seen": 91267072, "tokens_per_sec": 18289.952509794726, "vram_gb": 5.684933632, "lambda": 0.2927453387528658, "lambda_min": 0.09286732226610184, "lambda_max": 0.6258604526519775, "S_mean": 0.4023488362630208, "G_mean": 0.3406032353329162, "G_std": 0.15149507745324323, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.35309362411499, "grad_norm": 0.9667759537696838, "tokens_seen": 91430912, "tokens_per_sec": 18290.090307822797, "vram_gb": 5.684933632, "lambda": 0.29279179343332845, "lambda_min": 0.09287665784358978, "lambda_max": 0.6259471774101257, "S_mean": 0.3935292561848958, "G_mean": 0.3500772487993042, "G_std": 0.1514761925985416, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.305786848068237, "grad_norm": 0.965487539768219, "tokens_seen": 91594752, "tokens_per_sec": 18290.270979818823, "vram_gb": 5.684933632, "lambda": 0.2928537093102932, "lambda_min": 0.092866912484169, "lambda_max": 0.6260407567024231, "S_mean": 0.4041544596354167, "G_mean": 0.3445511457199852, "G_std": 0.1501024383275459, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.214428663253784, "grad_norm": 0.9728545546531677, "tokens_seen": 91758592, "tokens_per_sec": 18290.595603254646, "vram_gb": 5.684933632, "lambda": 0.2929401332512498, "lambda_min": 0.0928933322429657, "lambda_max": 0.6262540221214294, "S_mean": 0.3900655110677083, "G_mean": 0.34925990148137015, "G_std": 0.15061762769861767, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.276432991027832, "grad_norm": 0.9610673785209656, "tokens_seen": 91922432, "tokens_per_sec": 18290.636825086887, "vram_gb": 5.684933632, "lambda": 0.2930014804005623, "lambda_min": 0.09290257096290588, "lambda_max": 0.6264901161193848, "S_mean": 0.3977559407552083, "G_mean": 0.34661903271141153, "G_std": 0.1506285936726878, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.32619309425354, "grad_norm": 1.0046343803405762, "tokens_seen": 92086272, "tokens_per_sec": 18291.151546656285, "vram_gb": 5.684933632, "lambda": 0.2930523691078027, "lambda_min": 0.09288141131401062, "lambda_max": 0.6268100142478943, "S_mean": 0.4046071370442708, "G_mean": 0.34840430443485576, "G_std": 0.15276199004923305, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.355987787246704, "grad_norm": 1.0600258111953735, "tokens_seen": 92250112, "tokens_per_sec": 18291.45574298614, "vram_gb": 5.684933632, "lambda": 0.2931395623212059, "lambda_min": 0.0928758755326271, "lambda_max": 0.6271191239356995, "S_mean": 0.3953348795572917, "G_mean": 0.3550705541856587, "G_std": 0.14981030176083246, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.2732532024383545, "grad_norm": 0.9831656813621521, "tokens_seen": 92413952, "tokens_per_sec": 18291.261539757914, "vram_gb": 5.684933632, "lambda": 0.2931939549744129, "lambda_min": 0.09287037700414658, "lambda_max": 0.6273710131645203, "S_mean": 0.3900146484375, "G_mean": 0.34917017646754783, "G_std": 0.15047796772948155, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.26854395866394, "grad_norm": 1.0034016370773315, "tokens_seen": 92577792, "tokens_per_sec": 18291.603990099244, "vram_gb": 5.684933632, "lambda": 0.293256051838398, "lambda_min": 0.09287190437316895, "lambda_max": 0.6275747418403625, "S_mean": 0.3943583170572917, "G_mean": 0.3460798390830557, "G_std": 0.15166742731040964, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.309455394744873, "grad_norm": 0.969134509563446, "tokens_seen": 92741632, "tokens_per_sec": 18291.7194586897, "vram_gb": 5.684933632, "lambda": 0.2933314563706517, "lambda_min": 0.09287255257368088, "lambda_max": 0.6278421878814697, "S_mean": 0.3884989420572917, "G_mean": 0.3468385186667244, "G_std": 0.1507614904160922, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.360682964324951, "grad_norm": 0.9709041714668274, "tokens_seen": 92905472, "tokens_per_sec": 18291.3858727567, "vram_gb": 5.684933632, "lambda": 0.29338633300115663, "lambda_min": 0.0928923487663269, "lambda_max": 0.6280672550201416, "S_mean": 0.3949178059895833, "G_mean": 0.3440479977677266, "G_std": 0.15124210385450473, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.465503931045532, "grad_norm": 1.0388432741165161, "tokens_seen": 93069312, "tokens_per_sec": 18291.470400430764, "vram_gb": 5.684933632, "lambda": 0.29345695953816175, "lambda_min": 0.09293897449970245, "lambda_max": 0.6282331943511963, "S_mean": 0.3976491292317708, "G_mean": 0.34573800178865594, "G_std": 0.1496254320954904, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.29705548286438, "grad_norm": 0.9782949686050415, "tokens_seen": 93233152, "tokens_per_sec": 18291.518946487733, "vram_gb": 5.684933632, "lambda": 0.29350832911829156, "lambda_min": 0.09295755624771118, "lambda_max": 0.6283864378929138, "S_mean": 0.3965911865234375, "G_mean": 0.34761730387496453, "G_std": 0.1492969540025418, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.3955299854278564, "grad_norm": 1.010913610458374, "tokens_seen": 93396992, "tokens_per_sec": 18291.674461490366, "vram_gb": 5.684933632, "lambda": 0.2935714243600766, "lambda_min": 0.09296000748872757, "lambda_max": 0.6284182667732239, "S_mean": 0.4054972330729167, "G_mean": 0.35067992261610925, "G_std": 0.15059011666259417, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.296755790710449, "grad_norm": 1.0269739627838135, "tokens_seen": 93560832, "tokens_per_sec": 18291.87100109077, "vram_gb": 5.684933632, "lambda": 0.2936336289470394, "lambda_min": 0.09298500418663025, "lambda_max": 0.628327488899231, "S_mean": 0.3992716471354167, "G_mean": 0.34616675072660047, "G_std": 0.1503379330582296, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.329143762588501, "grad_norm": 0.9668381214141846, "tokens_seen": 93724672, "tokens_per_sec": 18292.140741379735, "vram_gb": 5.684933632, "lambda": 0.29370351228863, "lambda_min": 0.09300629049539566, "lambda_max": 0.6284797191619873, "S_mean": 0.3997701009114583, "G_mean": 0.34957460857306916, "G_std": 0.1502012360530595, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.352807521820068, "grad_norm": 0.9777989983558655, "tokens_seen": 93888512, "tokens_per_sec": 18292.51400670605, "vram_gb": 5.684933632, "lambda": 0.2937691028540333, "lambda_min": 0.09300719946622849, "lambda_max": 0.6287463903427124, "S_mean": 0.4005533854166667, "G_mean": 0.3479409112284581, "G_std": 0.15053071229097745, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.351879358291626, "grad_norm": 0.9418855309486389, "tokens_seen": 94052352, "tokens_per_sec": 18292.310606976404, "vram_gb": 5.684933632, "lambda": 0.29381995151440304, "lambda_min": 0.09299469739198685, "lambda_max": 0.6288085579872131, "S_mean": 0.3930257161458333, "G_mean": 0.34382960743581253, "G_std": 0.1505099825638657, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.258826017379761, "grad_norm": 0.9763834476470947, "tokens_seen": 94216192, "tokens_per_sec": 18292.437472763377, "vram_gb": 5.684933632, "lambda": 0.2938999254256487, "lambda_min": 0.09297774732112885, "lambda_max": 0.6290642619132996, "S_mean": 0.3991190592447917, "G_mean": 0.35056097619235516, "G_std": 0.15024119786297283, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.15775203704834, "grad_norm": 0.9817291498184204, "tokens_seen": 94380032, "tokens_per_sec": 18292.357747833372, "vram_gb": 5.684933632, "lambda": 0.2939649348457654, "lambda_min": 0.09297917783260345, "lambda_max": 0.6292673349380493, "S_mean": 0.3965911865234375, "G_mean": 0.3493179399520159, "G_std": 0.15009273181203753, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.4020373821258545, "grad_norm": 0.9700790643692017, "tokens_seen": 94543872, "tokens_per_sec": 18292.24547172268, "vram_gb": 5.684933632, "lambda": 0.2940227873623371, "lambda_min": 0.09297838807106018, "lambda_max": 0.6294648051261902, "S_mean": 0.3947041829427083, "G_mean": 0.3503534502815455, "G_std": 0.15035086617960283, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.318603038787842, "grad_norm": 0.9874248504638672, "tokens_seen": 94707712, "tokens_per_sec": 18292.395203297485, "vram_gb": 5.684933632, "lambda": 0.29409127434094745, "lambda_min": 0.09297525137662888, "lambda_max": 0.6297470927238464, "S_mean": 0.3931630452473958, "G_mean": 0.3488332185273369, "G_std": 0.14993999747093767, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.180169582366943, "grad_norm": 0.9481203556060791, "tokens_seen": 94871552, "tokens_per_sec": 18292.59936068242, "vram_gb": 5.684933632, "lambda": 0.2941502497221033, "lambda_min": 0.09299041330814362, "lambda_max": 0.6298454999923706, "S_mean": 0.3946431477864583, "G_mean": 0.3452120421764751, "G_std": 0.14984800484186658, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.333926200866699, "grad_norm": 0.9721952080726624, "tokens_seen": 95035392, "tokens_per_sec": 18292.6135686505, "vram_gb": 5.684933632, "lambda": 0.29423069426168996, "lambda_min": 0.09297733008861542, "lambda_max": 0.6300617456436157, "S_mean": 0.3976287841796875, "G_mean": 0.344015776955833, "G_std": 0.15145314034695426, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.2808754444122314, "grad_norm": 0.957699179649353, "tokens_seen": 95199232, "tokens_per_sec": 18292.732242232647, "vram_gb": 5.684933632, "lambda": 0.2942902386809389, "lambda_min": 0.09296447783708572, "lambda_max": 0.6303128600120544, "S_mean": 0.4030354817708333, "G_mean": 0.3498778151503454, "G_std": 0.14988278601473817, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.333872556686401, "grad_norm": 0.971627414226532, "tokens_seen": 95363072, "tokens_per_sec": 18292.769026705962, "vram_gb": 5.684933632, "lambda": 0.2943502801160018, "lambda_min": 0.09296167641878128, "lambda_max": 0.6305729746818542, "S_mean": 0.3981424967447917, "G_mean": 0.34863538093244034, "G_std": 0.1523532500335326, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.418755292892456, "grad_norm": 1.0397671461105347, "tokens_seen": 95526912, "tokens_per_sec": 18292.92209024224, "vram_gb": 5.684933632, "lambda": 0.29441843337068957, "lambda_min": 0.09292219579219818, "lambda_max": 0.6308472752571106, "S_mean": 0.392486572265625, "G_mean": 0.3469986109994352, "G_std": 0.1510793377722924, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.2024149894714355, "grad_norm": 1.0203921794891357, "tokens_seen": 95690752, "tokens_per_sec": 18293.047368948002, "vram_gb": 5.684933632, "lambda": 0.2944743921980262, "lambda_min": 0.0929209515452385, "lambda_max": 0.6310478448867798, "S_mean": 0.410888671875, "G_mean": 0.3484587719819198, "G_std": 0.1490046518156305, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.352796316146851, "grad_norm": 1.023280382156372, "tokens_seen": 95854592, "tokens_per_sec": 18293.223905922456, "vram_gb": 5.684933632, "lambda": 0.2945383774737517, "lambda_min": 0.09293277561664581, "lambda_max": 0.6313241124153137, "S_mean": 0.3944142659505208, "G_mean": 0.34353929773593944, "G_std": 0.1498010451129327, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.271266222000122, "grad_norm": 1.015249490737915, "tokens_seen": 96018432, "tokens_per_sec": 18293.21235374889, "vram_gb": 5.684933632, "lambda": 0.2945883894960086, "lambda_min": 0.0929296687245369, "lambda_max": 0.6313708424568176, "S_mean": 0.3904622395833333, "G_mean": 0.3450699915799002, "G_std": 0.15163893155598393, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.312066555023193, "grad_norm": 1.0225818157196045, "tokens_seen": 96182272, "tokens_per_sec": 18293.313163832016, "vram_gb": 5.684933632, "lambda": 0.294662047487994, "lambda_min": 0.0929415225982666, "lambda_max": 0.6315548419952393, "S_mean": 0.3911539713541667, "G_mean": 0.3498500226996839, "G_std": 0.15113551320973784, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.310425281524658, "grad_norm": 0.961117684841156, "tokens_seen": 96346112, "tokens_per_sec": 18293.770976333053, "vram_gb": 5.684933632, "lambda": 0.2947271605953574, "lambda_min": 0.09294616430997849, "lambda_max": 0.6317216753959656, "S_mean": 0.3934529622395833, "G_mean": 0.34226030328621465, "G_std": 0.15080120581357428, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.339244842529297, "grad_norm": 0.9601591229438782, "tokens_seen": 96509952, "tokens_per_sec": 18294.13186744892, "vram_gb": 5.684933632, "lambda": 0.29479620171089965, "lambda_min": 0.0929679274559021, "lambda_max": 0.6319711208343506, "S_mean": 0.398956298828125, "G_mean": 0.34503818380956847, "G_std": 0.1498808447504416, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.289920806884766, "grad_norm": 0.9988320469856262, "tokens_seen": 96673792, "tokens_per_sec": 18294.369617600365, "vram_gb": 5.684933632, "lambda": 0.2948615749677022, "lambda_min": 0.09296837449073792, "lambda_max": 0.6321313381195068, "S_mean": 0.392120361328125, "G_mean": 0.3434130629369368, "G_std": 0.15112126735039055, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.34259033203125, "grad_norm": 1.0423237085342407, "tokens_seen": 96837632, "tokens_per_sec": 18294.714272169153, "vram_gb": 5.684933632, "lambda": 0.29493217697987956, "lambda_min": 0.09295878559350967, "lambda_max": 0.632323145866394, "S_mean": 0.400634765625, "G_mean": 0.3436102520984908, "G_std": 0.15011257852893323, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.238344192504883, "grad_norm": 0.9926890730857849, "tokens_seen": 97001472, "tokens_per_sec": 18295.09410240262, "vram_gb": 5.684933632, "lambda": 0.29499644103149575, "lambda_min": 0.09292710572481155, "lambda_max": 0.6325970888137817, "S_mean": 0.4057057698567708, "G_mean": 0.34280447103083134, "G_std": 0.15036905622885874, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.247864723205566, "grad_norm": 0.9681630730628967, "tokens_seen": 97165312, "tokens_per_sec": 18295.35642817871, "vram_gb": 5.684933632, "lambda": 0.2950252477700512, "lambda_min": 0.09289269149303436, "lambda_max": 0.6327704191207886, "S_mean": 0.39996337890625, "G_mean": 0.3454628235194832, "G_std": 0.1494477385034164, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.452517747879028, "grad_norm": 1.1080548763275146, "tokens_seen": 97329152, "tokens_per_sec": 18295.44832739146, "vram_gb": 5.684933632, "lambda": 0.2950887664531668, "lambda_min": 0.09292899072170258, "lambda_max": 0.6330624222755432, "S_mean": 0.392333984375, "G_mean": 0.3487356298137456, "G_std": 0.15048943432824066, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.263861417770386, "grad_norm": 1.090919017791748, "tokens_seen": 97492992, "tokens_per_sec": 18295.596566492877, "vram_gb": 5.684933632, "lambda": 0.29515782619516057, "lambda_min": 0.09291836619377136, "lambda_max": 0.633324921131134, "S_mean": 0.3926188151041667, "G_mean": 0.34090893948450685, "G_std": 0.1507148543605581, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.2608044147491455, "grad_norm": 1.0110666751861572, "tokens_seen": 97656832, "tokens_per_sec": 18295.51174734395, "vram_gb": 5.684933632, "lambda": 0.2952043395489454, "lambda_min": 0.09293550997972488, "lambda_max": 0.633354127407074, "S_mean": 0.4001261393229167, "G_mean": 0.34289556586494047, "G_std": 0.15004337128872672, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.400369644165039, "grad_norm": 1.2527533769607544, "tokens_seen": 97820672, "tokens_per_sec": 18295.75079278016, "vram_gb": 5.684933632, "lambda": 0.29525130180021125, "lambda_min": 0.09292245656251907, "lambda_max": 0.6335160136222839, "S_mean": 0.4051920572916667, "G_mean": 0.34044637670740485, "G_std": 0.14872040421081087, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.241017580032349, "grad_norm": 1.0209674835205078, "tokens_seen": 97984512, "tokens_per_sec": 18295.995422602067, "vram_gb": 5.684933632, "lambda": 0.29530744316677254, "lambda_min": 0.092919260263443, "lambda_max": 0.6337149143218994, "S_mean": 0.3872426350911458, "G_mean": 0.34899194926644367, "G_std": 0.15104760838827738, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.285902500152588, "grad_norm": 0.9659044146537781, "tokens_seen": 98148352, "tokens_per_sec": 18296.09847400087, "vram_gb": 5.684933632, "lambda": 0.29537375799069804, "lambda_min": 0.0929143875837326, "lambda_max": 0.6338273882865906, "S_mean": 0.401031494140625, "G_mean": 0.3447236562302957, "G_std": 0.15081971573332945, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.352875447273254, "val_ppl": 77.70156877103959, "lambda": 0.2954302333916227, "lambda_min": 0.09290912002325058, "lambda_max": 0.6339113712310791, "lambda_per_layer": [0.17844241857528687, 0.15804831683635712, 0.14136068522930145, 0.148036926984787, 0.12272127717733383, 0.12057282775640488, 0.11383507400751114, 0.10231970995664597, 0.09290912002325058, 0.1721991002559662, 0.38244181871414185, 0.3786458671092987, 0.46153759956359863, 0.32758787274360657, 0.5410665273666382, 0.6339113712310791, 0.6170794367790222, 0.39891403913497925, 0.5463242530822754, 0.26453304290771484, 0.2896743416786194, 0.2632298767566681, 0.410540372133255, 0.22439372539520264], "S_mean": 0.4008941650390625, "G_mean": 0.3457400150752316, "G_std": 0.15042622915158668, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5374.616672039032, "nan_failures": 0} diff --git a/full_s4/config.json b/full_s4/config.json new file mode 100644 index 0000000000000000000000000000000000000000..6ea9d14c1d6949d4a18a6d2291f467d730879b43 --- /dev/null +++ b/full_s4/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 4, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s4", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 4001349, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/full_s4/final.pt b/full_s4/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..0160e97f2b93a7f70d96999987ff0af96fdff6a3 --- /dev/null +++ b/full_s4/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:324684aa157a690a58646c82d7c760e6278b709168dc4be9372d2c0778f0e999 +size 256003111 diff --git a/full_s4/metrics.jsonl b/full_s4/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..baad6f7338e57809f1089f9e273f1bec292cf341 --- /dev/null +++ b/full_s4/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 4, "config": {"variant": "full", "seed": 4, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s4", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 4001349, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.901908874511719, "grad_norm": 11.078120231628418, "tokens_seen": 8192, "tokens_per_sec": 6616.664834296153, "vram_gb": 5.172759552, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.17313385009765625, "G_mean": 0.49705679963032406, "G_std": 0.08551615104079247, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.786567687988281, "grad_norm": 1.9155511856079102, "tokens_seen": 172032, "tokens_per_sec": 15502.142103750546, "vram_gb": 5.684933632, "lambda": 0.12692486184338728, "lambda_min": 0.12688536942005157, "lambda_max": 0.12695829570293427, "S_mean": 0.4159596761067708, "G_mean": 0.4553276238342126, "G_std": 0.06331148805717628, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.037954330444336, "grad_norm": 1.7289838790893555, "tokens_seen": 335872, "tokens_per_sec": 15970.290665085662, "vram_gb": 5.684933632, "lambda": 0.12685753032565117, "lambda_min": 0.12680833041667938, "lambda_max": 0.1269741803407669, "S_mean": 0.4843190511067708, "G_mean": 0.4317767421404521, "G_std": 0.062058891635388136, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.1768217086792, "grad_norm": 1.4561188220977783, "tokens_seen": 499712, "tokens_per_sec": 16107.539327753264, "vram_gb": 5.684933632, "lambda": 0.12675989419221878, "lambda_min": 0.1266596019268036, "lambda_max": 0.12700580060482025, "S_mean": 0.499237060546875, "G_mean": 0.43582314004500705, "G_std": 0.07308286211142938, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.533573150634766, "grad_norm": 0.8069532513618469, "tokens_seen": 663552, "tokens_per_sec": 16108.011848237986, "vram_gb": 5.684933632, "lambda": 0.12662202616532645, "lambda_min": 0.126404270529747, "lambda_max": 0.12699225544929504, "S_mean": 0.3835957845052083, "G_mean": 0.4294958549241225, "G_std": 0.08964463090524077, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 6.994081020355225, "grad_norm": 0.7314291596412659, "tokens_seen": 827392, "tokens_per_sec": 16018.160942967821, "vram_gb": 5.684933632, "lambda": 0.12649238420029482, "lambda_min": 0.12616148591041565, "lambda_max": 0.12713254988193512, "S_mean": 0.2610575358072917, "G_mean": 0.4353996366262436, "G_std": 0.10620803494627278, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.177103281021118, "grad_norm": 0.5767114758491516, "tokens_seen": 991232, "tokens_per_sec": 16003.363468457628, "vram_gb": 5.684933632, "lambda": 0.126394705226024, "lambda_min": 0.12589097023010254, "lambda_max": 0.12733639776706696, "S_mean": 0.2886962890625, "G_mean": 0.4306877888739109, "G_std": 0.11588117340579629, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.810484886169434, "grad_norm": 0.5891932249069214, "tokens_seen": 1155072, "tokens_per_sec": 16134.187006893831, "vram_gb": 5.684933632, "lambda": 0.1263536624610424, "lambda_min": 0.12551794946193695, "lambda_max": 0.12740348279476166, "S_mean": 0.2857462565104167, "G_mean": 0.40672072644035023, "G_std": 0.11532471685980757, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.726803779602051, "grad_norm": 0.6107571721076965, "tokens_seen": 1318912, "tokens_per_sec": 16337.435897789688, "vram_gb": 5.684933632, "lambda": 0.12635492533445358, "lambda_min": 0.12533743679523468, "lambda_max": 0.12756559252738953, "S_mean": 0.2576700846354167, "G_mean": 0.4088233100871245, "G_std": 0.1281888710024456, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.743535280227661, "grad_norm": 0.598254919052124, "tokens_seen": 1482752, "tokens_per_sec": 16530.2611545984, "vram_gb": 5.684933632, "lambda": 0.12638081051409245, "lambda_min": 0.1252683848142624, "lambda_max": 0.12759651243686676, "S_mean": 0.2719523111979167, "G_mean": 0.38508129368225735, "G_std": 0.1333815773638586, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.691915273666382, "grad_norm": 0.5073958039283752, "tokens_seen": 1646592, "tokens_per_sec": 16692.130326207254, "vram_gb": 5.684933632, "lambda": 0.12631004924575487, "lambda_min": 0.12501424551010132, "lambda_max": 0.12759841978549957, "S_mean": 0.247344970703125, "G_mean": 0.400378526498874, "G_std": 0.14135082671418786, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.7360899448394775, "grad_norm": 0.5595394968986511, "tokens_seen": 1810432, "tokens_per_sec": 16832.501422062778, "vram_gb": 5.684933632, "lambda": 0.12634136465688547, "lambda_min": 0.12481248378753662, "lambda_max": 0.12771739065647125, "S_mean": 0.24434407552083334, "G_mean": 0.3930956597129504, "G_std": 0.15446821227669716, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.520927667617798, "grad_norm": 0.537175714969635, "tokens_seen": 1974272, "tokens_per_sec": 16958.68271478925, "vram_gb": 5.684933632, "lambda": 0.12635038855175176, "lambda_min": 0.12468205392360687, "lambda_max": 0.12785452604293823, "S_mean": 0.2772216796875, "G_mean": 0.3756640963256359, "G_std": 0.1529799895361066, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.617964744567871, "grad_norm": 0.4712919592857361, "tokens_seen": 2138112, "tokens_per_sec": 17048.404171108974, "vram_gb": 5.684933632, "lambda": 0.12634874321520329, "lambda_min": 0.12464487552642822, "lambda_max": 0.1279430389404297, "S_mean": 0.23854573567708334, "G_mean": 0.39126606409748393, "G_std": 0.16262721053014198, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.669438123703003, "grad_norm": 0.5098272562026978, "tokens_seen": 2301952, "tokens_per_sec": 17132.840056298628, "vram_gb": 5.684933632, "lambda": 0.12637338073303303, "lambda_min": 0.12448006868362427, "lambda_max": 0.12816274166107178, "S_mean": 0.2852783203125, "G_mean": 0.40059853717684746, "G_std": 0.17114591225981712, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.340326547622681, "grad_norm": 0.4711325168609619, "tokens_seen": 2465792, "tokens_per_sec": 17209.488657909213, "vram_gb": 5.684933632, "lambda": 0.12637322520216307, "lambda_min": 0.12441889196634293, "lambda_max": 0.1283169984817505, "S_mean": 0.27740478515625, "G_mean": 0.37721312915285427, "G_std": 0.17197068314999342, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.394336700439453, "grad_norm": 0.6527551412582397, "tokens_seen": 2629632, "tokens_per_sec": 17265.306364292697, "vram_gb": 5.684933632, "lambda": 0.12644734481970468, "lambda_min": 0.12446720153093338, "lambda_max": 0.1285964697599411, "S_mean": 0.2874348958333333, "G_mean": 0.383746216694514, "G_std": 0.17421626020222902, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.433804035186768, "grad_norm": 0.45131760835647583, "tokens_seen": 2793472, "tokens_per_sec": 17322.74284903307, "vram_gb": 5.684933632, "lambda": 0.12653786626954874, "lambda_min": 0.12443292886018753, "lambda_max": 0.12893174588680267, "S_mean": 0.2698771158854167, "G_mean": 0.40952223042647046, "G_std": 0.18679112568497658, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.465924263000488, "grad_norm": 0.47156840562820435, "tokens_seen": 2957312, "tokens_per_sec": 17364.01698915853, "vram_gb": 5.684933632, "lambda": 0.12659325419614711, "lambda_min": 0.12444287538528442, "lambda_max": 0.12916383147239685, "S_mean": 0.2667439778645833, "G_mean": 0.40441223109761876, "G_std": 0.18949187578012547, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.396562099456787, "grad_norm": 0.5106698274612427, "tokens_seen": 3121152, "tokens_per_sec": 17409.696784601147, "vram_gb": 5.684933632, "lambda": 0.12670292984694242, "lambda_min": 0.12453321367502213, "lambda_max": 0.12962636351585388, "S_mean": 0.31915283203125, "G_mean": 0.4013722079495589, "G_std": 0.1926906512429317, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.286241054534912, "grad_norm": 0.4735393524169922, "tokens_seen": 3284992, "tokens_per_sec": 17443.860943189065, "vram_gb": 5.684933632, "lambda": 0.1268038945272565, "lambda_min": 0.1246466413140297, "lambda_max": 0.12995770573616028, "S_mean": 0.31982421875, "G_mean": 0.4099446510275205, "G_std": 0.1989651449645559, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.159085273742676, "grad_norm": 0.47873571515083313, "tokens_seen": 3448832, "tokens_per_sec": 17478.776945373775, "vram_gb": 5.684933632, "lambda": 0.1268520106871923, "lambda_min": 0.12455998361110687, "lambda_max": 0.1303674727678299, "S_mean": 0.3333943684895833, "G_mean": 0.39448800310492516, "G_std": 0.19843264545003572, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.262938022613525, "grad_norm": 0.5126109719276428, "tokens_seen": 3612672, "tokens_per_sec": 17515.436307299613, "vram_gb": 5.684933632, "lambda": 0.1269506150856614, "lambda_min": 0.12454991787672043, "lambda_max": 0.13096202909946442, "S_mean": 0.3011271158854167, "G_mean": 0.4188994790116946, "G_std": 0.20710614106307426, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.286677837371826, "grad_norm": 0.4795948266983032, "tokens_seen": 3776512, "tokens_per_sec": 17533.49039948426, "vram_gb": 5.684933632, "lambda": 0.12700073514133692, "lambda_min": 0.1244175061583519, "lambda_max": 0.13125300407409668, "S_mean": 0.301849365234375, "G_mean": 0.42387475073337555, "G_std": 0.207086352010568, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.2813920974731445, "grad_norm": 0.5165824294090271, "tokens_seen": 3940352, "tokens_per_sec": 17565.640167696067, "vram_gb": 5.684933632, "lambda": 0.12710468067477146, "lambda_min": 0.12455398589372635, "lambda_max": 0.13162070512771606, "S_mean": 0.32354736328125, "G_mean": 0.43032630036274594, "G_std": 0.21195003048827252, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.203155040740967, "grad_norm": 0.4861752688884735, "tokens_seen": 4104192, "tokens_per_sec": 17593.380613187386, "vram_gb": 5.684933632, "lambda": 0.12728733321030936, "lambda_min": 0.12479010969400406, "lambda_max": 0.13208410143852234, "S_mean": 0.3756306966145833, "G_mean": 0.4179424283405145, "G_std": 0.206623001024127, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.259288311004639, "grad_norm": 0.4720209240913391, "tokens_seen": 4268032, "tokens_per_sec": 17624.328827966365, "vram_gb": 5.684933632, "lambda": 0.1274381559342146, "lambda_min": 0.124880850315094, "lambda_max": 0.13273750245571136, "S_mean": 0.3590901692708333, "G_mean": 0.4382787458598614, "G_std": 0.2154399665693442, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.076217174530029, "grad_norm": 0.4973568320274353, "tokens_seen": 4431872, "tokens_per_sec": 17647.468446922434, "vram_gb": 5.684933632, "lambda": 0.12755477366348109, "lambda_min": 0.12475013732910156, "lambda_max": 0.13315793871879578, "S_mean": 0.3816731770833333, "G_mean": 0.4541743149360021, "G_std": 0.21566800183306137, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.2182817459106445, "grad_norm": 0.5455684065818787, "tokens_seen": 4595712, "tokens_per_sec": 17668.91576512112, "vram_gb": 5.684933632, "lambda": 0.12771027100582918, "lambda_min": 0.12475495040416718, "lambda_max": 0.13354863226413727, "S_mean": 0.37200927734375, "G_mean": 0.48742624123891193, "G_std": 0.21768820813546577, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.062690734863281, "grad_norm": 0.45980125665664673, "tokens_seen": 4759552, "tokens_per_sec": 17687.809240876923, "vram_gb": 5.684933632, "lambda": 0.12789143652965626, "lambda_min": 0.12476421147584915, "lambda_max": 0.13378028571605682, "S_mean": 0.3623860677083333, "G_mean": 0.4936484396457672, "G_std": 0.22097176034003496, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.0132951736450195, "grad_norm": 0.5192211270332336, "tokens_seen": 4923392, "tokens_per_sec": 17703.687328058055, "vram_gb": 5.684933632, "lambda": 0.12807843772073588, "lambda_min": 0.12485496699810028, "lambda_max": 0.13428901135921478, "S_mean": 0.3548075358072917, "G_mean": 0.49055100108186406, "G_std": 0.21941259937981764, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.129741907119751, "grad_norm": 0.4762781858444214, "tokens_seen": 5087232, "tokens_per_sec": 17728.123640266014, "vram_gb": 5.684933632, "lambda": 0.12824134156107903, "lambda_min": 0.12459437549114227, "lambda_max": 0.13462989032268524, "S_mean": 0.3702392578125, "G_mean": 0.49828387548526126, "G_std": 0.22356261064608893, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.230600357055664, "grad_norm": 0.5174452662467957, "tokens_seen": 5251072, "tokens_per_sec": 17748.28655223447, "vram_gb": 5.684933632, "lambda": 0.1284517695506414, "lambda_min": 0.1247766762971878, "lambda_max": 0.13506579399108887, "S_mean": 0.3940226236979167, "G_mean": 0.5151649701098601, "G_std": 0.21172791036466757, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.053154230117798, "grad_norm": 0.4961080551147461, "tokens_seen": 5414912, "tokens_per_sec": 17766.235091666513, "vram_gb": 5.684933632, "lambda": 0.12867949033776918, "lambda_min": 0.12485918402671814, "lambda_max": 0.13557696342468262, "S_mean": 0.3749491373697917, "G_mean": 0.5473651749392351, "G_std": 0.21519022497038046, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.065998315811157, "grad_norm": 0.5209652781486511, "tokens_seen": 5578752, "tokens_per_sec": 17784.288142228546, "vram_gb": 5.684933632, "lambda": 0.1289023533463478, "lambda_min": 0.12504778802394867, "lambda_max": 0.1357070803642273, "S_mean": 0.3972371419270833, "G_mean": 0.5441190103689829, "G_std": 0.2133390928308169, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.967520236968994, "grad_norm": 0.4969852566719055, "tokens_seen": 5742592, "tokens_per_sec": 17789.065436458586, "vram_gb": 5.684933632, "lambda": 0.1291481200605631, "lambda_min": 0.12514911592006683, "lambda_max": 0.13590164482593536, "S_mean": 0.4157307942708333, "G_mean": 0.5307757084568342, "G_std": 0.21231743724395832, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.935401916503906, "grad_norm": 0.5087997913360596, "tokens_seen": 5906432, "tokens_per_sec": 17798.50409679277, "vram_gb": 5.684933632, "lambda": 0.1293968198200067, "lambda_min": 0.1250627040863037, "lambda_max": 0.13629809021949768, "S_mean": 0.3787943522135417, "G_mean": 0.5471401537458102, "G_std": 0.21093643456697464, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.034818410873413, "grad_norm": 0.4823954999446869, "tokens_seen": 6070272, "tokens_per_sec": 17813.104059029552, "vram_gb": 5.684933632, "lambda": 0.1297471054519216, "lambda_min": 0.12486136704683304, "lambda_max": 0.13679903745651245, "S_mean": 0.3913981119791667, "G_mean": 0.5491782960792383, "G_std": 0.2006792069102327, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.002120494842529, "grad_norm": 0.51756352186203, "tokens_seen": 6234112, "tokens_per_sec": 17825.3694742782, "vram_gb": 5.684933632, "lambda": 0.1301203661908706, "lambda_min": 0.1248553991317749, "lambda_max": 0.13733267784118652, "S_mean": 0.3698933919270833, "G_mean": 0.5811063498258591, "G_std": 0.2014544028788805, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.128162860870361, "grad_norm": 0.5190848112106323, "tokens_seen": 6397952, "tokens_per_sec": 17835.964142994337, "vram_gb": 5.684933632, "lambda": 0.1304818003748854, "lambda_min": 0.12478034943342209, "lambda_max": 0.1378890424966812, "S_mean": 0.3798929850260417, "G_mean": 0.5913497967024645, "G_std": 0.20086126370976368, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.850389719009399, "grad_norm": 0.4661721885204315, "tokens_seen": 6561792, "tokens_per_sec": 17841.952338531108, "vram_gb": 5.684933632, "lambda": 0.13081089903910956, "lambda_min": 0.12446007132530212, "lambda_max": 0.13920065760612488, "S_mean": 0.3893636067708333, "G_mean": 0.5748367843528589, "G_std": 0.19700581301003695, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.90027928352356, "grad_norm": 0.4737221598625183, "tokens_seen": 6725632, "tokens_per_sec": 17846.014259746808, "vram_gb": 5.684933632, "lambda": 0.13111076193551222, "lambda_min": 0.12396207451820374, "lambda_max": 0.14048072695732117, "S_mean": 0.3616434733072917, "G_mean": 0.5593334771692753, "G_std": 0.2014954974874854, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.850017070770264, "grad_norm": 0.5344657897949219, "tokens_seen": 6889472, "tokens_per_sec": 17858.847665078778, "vram_gb": 5.684933632, "lambda": 0.13142910258223614, "lambda_min": 0.12357138842344284, "lambda_max": 0.14186936616897583, "S_mean": 0.3442281087239583, "G_mean": 0.5577694786091646, "G_std": 0.2007428597037991, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.935296535491943, "grad_norm": 0.5666730999946594, "tokens_seen": 7053312, "tokens_per_sec": 17871.150687907768, "vram_gb": 5.684933632, "lambda": 0.1318246591836214, "lambda_min": 0.12318015098571777, "lambda_max": 0.14301836490631104, "S_mean": 0.3935139973958333, "G_mean": 0.5480199865996838, "G_std": 0.1966401214400927, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.767166376113892, "grad_norm": 0.46878156065940857, "tokens_seen": 7217152, "tokens_per_sec": 17883.486421491918, "vram_gb": 5.684933632, "lambda": 0.1322418594111999, "lambda_min": 0.12265609204769135, "lambda_max": 0.1443464308977127, "S_mean": 0.36541748046875, "G_mean": 0.5581615579624971, "G_std": 0.19639057448754707, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.773815870285034, "grad_norm": 0.5303638577461243, "tokens_seen": 7380992, "tokens_per_sec": 17894.451296658222, "vram_gb": 5.684933632, "lambda": 0.13264422677457333, "lambda_min": 0.12215641140937805, "lambda_max": 0.1456979513168335, "S_mean": 0.355712890625, "G_mean": 0.5846103429794312, "G_std": 0.19911482091993093, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.839080333709717, "grad_norm": 0.49565616250038147, "tokens_seen": 7544832, "tokens_per_sec": 17908.150680966733, "vram_gb": 5.684933632, "lambda": 0.13303990351657072, "lambda_min": 0.12157709896564484, "lambda_max": 0.14698834717273712, "S_mean": 0.3561604817708333, "G_mean": 0.5821945940454801, "G_std": 0.19681487046182156, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.767991065979004, "grad_norm": 0.5569437146186829, "tokens_seen": 7708672, "tokens_per_sec": 17914.749187059835, "vram_gb": 5.684933632, "lambda": 0.133400014291207, "lambda_min": 0.12107665836811066, "lambda_max": 0.1482752412557602, "S_mean": 0.3717854817708333, "G_mean": 0.5585003029555082, "G_std": 0.20040419573585191, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.802011966705322, "grad_norm": 0.5176509022712708, "tokens_seen": 7872512, "tokens_per_sec": 17925.1979348676, "vram_gb": 5.684933632, "lambda": 0.13383774862935147, "lambda_min": 0.12056050449609756, "lambda_max": 0.14953495562076569, "S_mean": 0.349365234375, "G_mean": 0.5636651776731014, "G_std": 0.19351087603718042, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.7962260246276855, "grad_norm": 0.5433199405670166, "tokens_seen": 8036352, "tokens_per_sec": 17935.495351131496, "vram_gb": 5.684933632, "lambda": 0.13432009921719631, "lambda_min": 0.12001685053110123, "lambda_max": 0.15071988105773926, "S_mean": 0.35662841796875, "G_mean": 0.5589304659515619, "G_std": 0.19587160150210062, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.717461109161377, "grad_norm": 0.5804958343505859, "tokens_seen": 8200192, "tokens_per_sec": 17944.86969960597, "vram_gb": 5.684933632, "lambda": 0.1346960850059986, "lambda_min": 0.11941196024417877, "lambda_max": 0.15185850858688354, "S_mean": 0.32794189453125, "G_mean": 0.5498163489003977, "G_std": 0.19117959992339215, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.776382684707642, "val_ppl": 322.59016698459436, "lambda": 0.13471540343016386, "lambda_min": 0.11936924606561661, "lambda_max": 0.15191997587680817, "lambda_per_layer": [0.140584334731102, 0.13155010342597961, 0.13162308931350708, 0.1308876872062683, 0.1275871992111206, 0.11936924606561661, 0.12690405547618866, 0.13046111166477203, 0.13930228352546692, 0.15191997587680817, 0.13761845231056213, 0.1323430836200714, 0.13191555440425873, 0.14258399605751038, 0.1379074901342392, 0.1481291502714157, 0.14740391075611115, 0.13379068672657013, 0.13085225224494934, 0.13824467360973358, 0.13479503989219666, 0.12744373083114624, 0.1285652220249176, 0.1313873529434204], "S_mean": 0.3604532877604167, "G_mean": 0.5607797342042128, "G_std": 0.19462808749328056, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.7660298347473145, "grad_norm": 0.5116082429885864, "tokens_seen": 8364032, "tokens_per_sec": 17894.559167612922, "vram_gb": 5.684933632, "lambda": 0.13511213505019745, "lambda_min": 0.11855726689100266, "lambda_max": 0.15328119695186615, "S_mean": 0.3475138346354167, "G_mean": 0.5540812549491724, "G_std": 0.19853175214181343, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.7096312046051025, "grad_norm": 0.5062384009361267, "tokens_seen": 8527872, "tokens_per_sec": 17903.518022955035, "vram_gb": 5.684933632, "lambda": 0.13551606175800165, "lambda_min": 0.1176198273897171, "lambda_max": 0.15469537675380707, "S_mean": 0.3575642903645833, "G_mean": 0.5549819283187389, "G_std": 0.1921793498719732, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.611855506896973, "grad_norm": 0.5375374555587769, "tokens_seen": 8691712, "tokens_per_sec": 17912.57111278514, "vram_gb": 5.684933632, "lambda": 0.13597818495084843, "lambda_min": 0.1168421134352684, "lambda_max": 0.1561058759689331, "S_mean": 0.3495076497395833, "G_mean": 0.5292880274355412, "G_std": 0.20006655932714543, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.660183906555176, "grad_norm": 0.5151681900024414, "tokens_seen": 8855552, "tokens_per_sec": 17918.06730993569, "vram_gb": 5.684933632, "lambda": 0.13638119989385208, "lambda_min": 0.1160527691245079, "lambda_max": 0.15755099058151245, "S_mean": 0.35784912109375, "G_mean": 0.5479242789248625, "G_std": 0.19626558820406595, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.8141539096832275, "grad_norm": 0.5202354192733765, "tokens_seen": 9019392, "tokens_per_sec": 17925.429560290617, "vram_gb": 5.684933632, "lambda": 0.1368334861472249, "lambda_min": 0.11569663137197495, "lambda_max": 0.15890221297740936, "S_mean": 0.3362019856770833, "G_mean": 0.5517748631536961, "G_std": 0.19772627608229718, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.761934280395508, "grad_norm": 0.49205920100212097, "tokens_seen": 9183232, "tokens_per_sec": 17930.574621856576, "vram_gb": 5.684933632, "lambda": 0.137236377534767, "lambda_min": 0.1147778257727623, "lambda_max": 0.16016730666160583, "S_mean": 0.3309529622395833, "G_mean": 0.554235452786088, "G_std": 0.1941429659103354, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.572589874267578, "grad_norm": 0.5308946371078491, "tokens_seen": 9347072, "tokens_per_sec": 17937.01190712814, "vram_gb": 5.684933632, "lambda": 0.1376870290065805, "lambda_min": 0.11401060968637466, "lambda_max": 0.16147364675998688, "S_mean": 0.3328450520833333, "G_mean": 0.5445777500669161, "G_std": 0.1957367416471243, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.613469362258911, "grad_norm": 0.5121581554412842, "tokens_seen": 9510912, "tokens_per_sec": 17940.530388687734, "vram_gb": 5.684933632, "lambda": 0.1381812443335851, "lambda_min": 0.11343128979206085, "lambda_max": 0.16279390454292297, "S_mean": 0.3464152018229167, "G_mean": 0.5404178481549025, "G_std": 0.19455218532433113, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.697718620300293, "grad_norm": 0.5283424854278564, "tokens_seen": 9674752, "tokens_per_sec": 17947.05734009457, "vram_gb": 5.684933632, "lambda": 0.13866149572034678, "lambda_min": 0.11291943490505219, "lambda_max": 0.16418923437595367, "S_mean": 0.3240763346354167, "G_mean": 0.528130225216349, "G_std": 0.1956396714473764, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.6702141761779785, "grad_norm": 0.5193036794662476, "tokens_seen": 9838592, "tokens_per_sec": 17950.839154566525, "vram_gb": 5.684933632, "lambda": 0.13909149138877788, "lambda_min": 0.11229213327169418, "lambda_max": 0.16558727622032166, "S_mean": 0.3086649576822917, "G_mean": 0.5411007522294918, "G_std": 0.19560385464380184, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.880238056182861, "grad_norm": 0.5169352293014526, "tokens_seen": 10002432, "tokens_per_sec": 17956.923556431353, "vram_gb": 5.684933632, "lambda": 0.13950619287788868, "lambda_min": 0.11157779395580292, "lambda_max": 0.16707192361354828, "S_mean": 0.33355712890625, "G_mean": 0.5300985214610895, "G_std": 0.19292791870733103, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.535277605056763, "grad_norm": 0.5109989643096924, "tokens_seen": 10166272, "tokens_per_sec": 17963.637057080214, "vram_gb": 5.684933632, "lambda": 0.13996233884245157, "lambda_min": 0.11076473444700241, "lambda_max": 0.1686728596687317, "S_mean": 0.3293050130208333, "G_mean": 0.5317762636890014, "G_std": 0.19803755482037863, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.55954647064209, "grad_norm": 0.5586907863616943, "tokens_seen": 10330112, "tokens_per_sec": 17967.358681079517, "vram_gb": 5.684933632, "lambda": 0.14037496565530697, "lambda_min": 0.11025089770555496, "lambda_max": 0.17015832662582397, "S_mean": 0.3271077473958333, "G_mean": 0.5282444568971792, "G_std": 0.19590365265806517, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.440014123916626, "grad_norm": 0.5305277109146118, "tokens_seen": 10493952, "tokens_per_sec": 17976.368586118326, "vram_gb": 5.684933632, "lambda": 0.1407824136937658, "lambda_min": 0.10956790298223495, "lambda_max": 0.1716192364692688, "S_mean": 0.3140869140625, "G_mean": 0.5381634713461002, "G_std": 0.19063116889446974, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.754671573638916, "grad_norm": 0.5816652774810791, "tokens_seen": 10657792, "tokens_per_sec": 17983.023966354776, "vram_gb": 5.684933632, "lambda": 0.14121810408929983, "lambda_min": 0.10892939567565918, "lambda_max": 0.17290498316287994, "S_mean": 0.3303019205729167, "G_mean": 0.5288880628844103, "G_std": 0.1933183427900076, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.518969535827637, "grad_norm": 0.5544753074645996, "tokens_seen": 10821632, "tokens_per_sec": 17988.746387947805, "vram_gb": 5.684933632, "lambda": 0.1417385016878446, "lambda_min": 0.10849544405937195, "lambda_max": 0.1741577684879303, "S_mean": 0.3548990885416667, "G_mean": 0.536684749647975, "G_std": 0.19229061591128507, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.491321086883545, "grad_norm": 0.5263372659683228, "tokens_seen": 10985472, "tokens_per_sec": 17995.740599394052, "vram_gb": 5.684933632, "lambda": 0.14229163092871508, "lambda_min": 0.10798238962888718, "lambda_max": 0.17559561133384705, "S_mean": 0.3401082356770833, "G_mean": 0.5238935432086388, "G_std": 0.19096339804430804, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.592747211456299, "grad_norm": 0.5433462858200073, "tokens_seen": 11149312, "tokens_per_sec": 18002.577733825136, "vram_gb": 5.684933632, "lambda": 0.1427511948471268, "lambda_min": 0.10731740295886993, "lambda_max": 0.17706207931041718, "S_mean": 0.34002685546875, "G_mean": 0.5289315891762575, "G_std": 0.19541600812226534, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.548146486282349, "grad_norm": 0.5820518136024475, "tokens_seen": 11313152, "tokens_per_sec": 18010.72633705323, "vram_gb": 5.684933632, "lambda": 0.14321863558143377, "lambda_min": 0.10668758302927017, "lambda_max": 0.17865563929080963, "S_mean": 0.3072306315104167, "G_mean": 0.5182925295084715, "G_std": 0.1963453038285176, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.430954456329346, "grad_norm": 0.5842150449752808, "tokens_seen": 11476992, "tokens_per_sec": 18018.13504074818, "vram_gb": 5.684933632, "lambda": 0.14365649906297526, "lambda_min": 0.10646630078554153, "lambda_max": 0.18006309866905212, "S_mean": 0.3700358072916667, "G_mean": 0.5173999114582936, "G_std": 0.19699849529812732, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.541476726531982, "grad_norm": 0.5505357980728149, "tokens_seen": 11640832, "tokens_per_sec": 18025.468967216228, "vram_gb": 5.684933632, "lambda": 0.1440195751686891, "lambda_min": 0.10588878393173218, "lambda_max": 0.18149487674236298, "S_mean": 0.32696533203125, "G_mean": 0.5070090169707934, "G_std": 0.19538159326960644, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.490448474884033, "grad_norm": 0.6261901259422302, "tokens_seen": 11804672, "tokens_per_sec": 18032.810082229294, "vram_gb": 5.684933632, "lambda": 0.14447169036914906, "lambda_min": 0.10535836219787598, "lambda_max": 0.18311139941215515, "S_mean": 0.30633544921875, "G_mean": 0.5108577590435743, "G_std": 0.19606760734071335, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.406242847442627, "grad_norm": 0.5375993251800537, "tokens_seen": 11968512, "tokens_per_sec": 18039.470233486212, "vram_gb": 5.684933632, "lambda": 0.14492431903878847, "lambda_min": 0.10492339730262756, "lambda_max": 0.184409037232399, "S_mean": 0.3295491536458333, "G_mean": 0.500774477298061, "G_std": 0.19667847671856484, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.448437929153442, "grad_norm": 0.5575626492500305, "tokens_seen": 12132352, "tokens_per_sec": 18046.55664487774, "vram_gb": 5.684933632, "lambda": 0.1454170017192761, "lambda_min": 0.10449796169996262, "lambda_max": 0.18595659732818604, "S_mean": 0.3240152994791667, "G_mean": 0.4957709365213911, "G_std": 0.19655054854229093, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.482463598251343, "grad_norm": 0.5608839988708496, "tokens_seen": 12296192, "tokens_per_sec": 18053.79418974591, "vram_gb": 5.684933632, "lambda": 0.14591137195626894, "lambda_min": 0.10394413769245148, "lambda_max": 0.18749302625656128, "S_mean": 0.32568359375, "G_mean": 0.5027264775708318, "G_std": 0.19451019478340945, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.336889982223511, "grad_norm": 0.5765790343284607, "tokens_seen": 12460032, "tokens_per_sec": 18060.702286503772, "vram_gb": 5.684933632, "lambda": 0.14632880128920078, "lambda_min": 0.10376759618520737, "lambda_max": 0.1885804384946823, "S_mean": 0.3255818684895833, "G_mean": 0.4997792715827624, "G_std": 0.193785493572553, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.4203269481658936, "grad_norm": 0.5784818530082703, "tokens_seen": 12623872, "tokens_per_sec": 18068.103464168555, "vram_gb": 5.684933632, "lambda": 0.1468188784395655, "lambda_min": 0.10344336926937103, "lambda_max": 0.18989747762680054, "S_mean": 0.3104451497395833, "G_mean": 0.4947134479880333, "G_std": 0.193304522583882, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.480249881744385, "grad_norm": 0.5588568449020386, "tokens_seen": 12787712, "tokens_per_sec": 18076.215732580193, "vram_gb": 5.684933632, "lambda": 0.14729215142627558, "lambda_min": 0.10317175835371017, "lambda_max": 0.19132640957832336, "S_mean": 0.3287556966145833, "G_mean": 0.5041752470036348, "G_std": 0.19100659294053912, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.55444860458374, "grad_norm": 0.557573676109314, "tokens_seen": 12951552, "tokens_per_sec": 18081.27868112927, "vram_gb": 5.684933632, "lambda": 0.14775694658358893, "lambda_min": 0.10291968286037445, "lambda_max": 0.19279751181602478, "S_mean": 0.3042399088541667, "G_mean": 0.48459414951503277, "G_std": 0.19362159554536143, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.261246681213379, "grad_norm": 0.6137188076972961, "tokens_seen": 13115392, "tokens_per_sec": 18086.13200681295, "vram_gb": 5.684933632, "lambda": 0.14829887946446738, "lambda_min": 0.1026804968714714, "lambda_max": 0.1943478286266327, "S_mean": 0.302490234375, "G_mean": 0.4895890870441993, "G_std": 0.19304401086022457, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.387177467346191, "grad_norm": 0.5983961224555969, "tokens_seen": 13279232, "tokens_per_sec": 18091.654814051082, "vram_gb": 5.684933632, "lambda": 0.14880152978003025, "lambda_min": 0.1023407056927681, "lambda_max": 0.195809006690979, "S_mean": 0.3032430013020833, "G_mean": 0.49519906224062044, "G_std": 0.18899012062077722, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.417119026184082, "grad_norm": 0.6103706359863281, "tokens_seen": 13443072, "tokens_per_sec": 18097.77539346406, "vram_gb": 5.684933632, "lambda": 0.14926041848957539, "lambda_min": 0.10186585038900375, "lambda_max": 0.19688434898853302, "S_mean": 0.3443196614583333, "G_mean": 0.48630159720778465, "G_std": 0.19133556711797914, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.356652021408081, "grad_norm": 0.5435746312141418, "tokens_seen": 13606912, "tokens_per_sec": 18103.730876078334, "vram_gb": 5.684933632, "lambda": 0.14977348254372677, "lambda_min": 0.10127109289169312, "lambda_max": 0.19822506606578827, "S_mean": 0.3214111328125, "G_mean": 0.48405859308938187, "G_std": 0.19050064186255136, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.2417731285095215, "grad_norm": 0.5380285382270813, "tokens_seen": 13770752, "tokens_per_sec": 18108.108701787274, "vram_gb": 5.684933632, "lambda": 0.15024863028277954, "lambda_min": 0.10071961581707001, "lambda_max": 0.19963207840919495, "S_mean": 0.3238118489583333, "G_mean": 0.48323536819467944, "G_std": 0.19017754355445504, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.298517942428589, "grad_norm": 0.5721272230148315, "tokens_seen": 13934592, "tokens_per_sec": 18114.106264544163, "vram_gb": 5.684933632, "lambda": 0.1507082600146532, "lambda_min": 0.10025032609701157, "lambda_max": 0.20118097960948944, "S_mean": 0.342041015625, "G_mean": 0.47795741943021613, "G_std": 0.19252090947702527, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.411051034927368, "grad_norm": 0.6150281429290771, "tokens_seen": 14098432, "tokens_per_sec": 18119.34424365276, "vram_gb": 5.684933632, "lambda": 0.151184001006186, "lambda_min": 0.09961959719657898, "lambda_max": 0.20259255170822144, "S_mean": 0.3494059244791667, "G_mean": 0.4829820667703946, "G_std": 0.1922030975110829, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.462866306304932, "grad_norm": 0.5976377725601196, "tokens_seen": 14262272, "tokens_per_sec": 18123.041517391557, "vram_gb": 5.684933632, "lambda": 0.15166255397101244, "lambda_min": 0.0992424488067627, "lambda_max": 0.20364393293857574, "S_mean": 0.3196614583333333, "G_mean": 0.4837846638013919, "G_std": 0.19183440087363124, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.2999351024627686, "grad_norm": 0.6238687038421631, "tokens_seen": 14426112, "tokens_per_sec": 18127.360364774544, "vram_gb": 5.684933632, "lambda": 0.15222225400308767, "lambda_min": 0.0989033505320549, "lambda_max": 0.205338254570961, "S_mean": 0.3236083984375, "G_mean": 0.4825120819732547, "G_std": 0.19157388604556522, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.4570770263671875, "grad_norm": 0.5841648578643799, "tokens_seen": 14589952, "tokens_per_sec": 18131.999570438205, "vram_gb": 5.684933632, "lambda": 0.1527447026843826, "lambda_min": 0.09839332103729248, "lambda_max": 0.20685303211212158, "S_mean": 0.3155924479166667, "G_mean": 0.4672636315226555, "G_std": 0.1946095055900514, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.207701683044434, "grad_norm": 0.6241361498832703, "tokens_seen": 14753792, "tokens_per_sec": 18135.690336771757, "vram_gb": 5.684933632, "lambda": 0.1532875563328465, "lambda_min": 0.09799381345510483, "lambda_max": 0.20835453271865845, "S_mean": 0.3344523111979167, "G_mean": 0.4682622719556093, "G_std": 0.1911842504826685, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.2890944480896, "grad_norm": 0.5639853477478027, "tokens_seen": 14917632, "tokens_per_sec": 18139.18263366083, "vram_gb": 5.684933632, "lambda": 0.1537786809106668, "lambda_min": 0.09727925807237625, "lambda_max": 0.20947590470314026, "S_mean": 0.3145345052083333, "G_mean": 0.4795135458310445, "G_std": 0.1911729018514355, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.3388848304748535, "grad_norm": 0.606713056564331, "tokens_seen": 15081472, "tokens_per_sec": 18143.502961329563, "vram_gb": 5.684933632, "lambda": 0.15427333333839974, "lambda_min": 0.09679292887449265, "lambda_max": 0.21078145503997803, "S_mean": 0.3159993489583333, "G_mean": 0.46935582409302395, "G_std": 0.19184399753188094, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.536444425582886, "grad_norm": 0.5906962752342224, "tokens_seen": 15245312, "tokens_per_sec": 18148.168513371413, "vram_gb": 5.684933632, "lambda": 0.15482969333728155, "lambda_min": 0.09639807790517807, "lambda_max": 0.21227772533893585, "S_mean": 0.3176066080729167, "G_mean": 0.4691696744412184, "G_std": 0.19291254381338754, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.398151159286499, "grad_norm": 0.584104597568512, "tokens_seen": 15409152, "tokens_per_sec": 18153.587319598788, "vram_gb": 5.684933632, "lambda": 0.15534744380662838, "lambda_min": 0.09611710160970688, "lambda_max": 0.21366499364376068, "S_mean": 0.3238932291666667, "G_mean": 0.47755700380851823, "G_std": 0.19048991970097026, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.299365282058716, "grad_norm": 0.6915989518165588, "tokens_seen": 15572992, "tokens_per_sec": 18157.82319273611, "vram_gb": 5.684933632, "lambda": 0.15589065880825123, "lambda_min": 0.09599851071834564, "lambda_max": 0.21538564562797546, "S_mean": 0.30316162109375, "G_mean": 0.469314473370711, "G_std": 0.18985350088526806, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.32050085067749, "grad_norm": 0.5973790884017944, "tokens_seen": 15736832, "tokens_per_sec": 18158.218469430776, "vram_gb": 5.684933632, "lambda": 0.1564192163447539, "lambda_min": 0.0959283709526062, "lambda_max": 0.21704934537410736, "S_mean": 0.3238525390625, "G_mean": 0.45790130520860356, "G_std": 0.19195640133693814, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.316622734069824, "grad_norm": 0.5596320033073425, "tokens_seen": 15900672, "tokens_per_sec": 18153.251683138842, "vram_gb": 5.684933632, "lambda": 0.15701258275657892, "lambda_min": 0.09591328352689743, "lambda_max": 0.21834078431129456, "S_mean": 0.3273518880208333, "G_mean": 0.4565594106291731, "G_std": 0.19003993971273303, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.429080009460449, "grad_norm": 0.5760782957077026, "tokens_seen": 16064512, "tokens_per_sec": 18156.968910803284, "vram_gb": 5.684933632, "lambda": 0.157519092472891, "lambda_min": 0.09601359069347382, "lambda_max": 0.21968096494674683, "S_mean": 0.3167724609375, "G_mean": 0.46012505516409874, "G_std": 0.19021068746224046, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.203201055526733, "grad_norm": 0.5542515516281128, "tokens_seen": 16228352, "tokens_per_sec": 18159.37653061937, "vram_gb": 5.684933632, "lambda": 0.15799559094011784, "lambda_min": 0.09584613889455795, "lambda_max": 0.22122634947299957, "S_mean": 0.312744140625, "G_mean": 0.45588354859501123, "G_std": 0.18785253229240576, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.272127866744995, "grad_norm": 0.6240807175636292, "tokens_seen": 16392192, "tokens_per_sec": 18164.0609447769, "vram_gb": 5.684933632, "lambda": 0.15849375197043022, "lambda_min": 0.09556696563959122, "lambda_max": 0.22285954654216766, "S_mean": 0.3298136393229167, "G_mean": 0.45250152113536996, "G_std": 0.1891722121896843, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.303819108009338, "val_ppl": 201.10338076691752, "lambda": 0.1585189988836646, "lambda_min": 0.0955720916390419, "lambda_max": 0.22293050587177277, "lambda_per_layer": [0.15163466334342957, 0.13431623578071594, 0.1335357278585434, 0.1330529749393463, 0.11877848207950592, 0.0955720916390419, 0.12583130598068237, 0.14177431166172028, 0.16559350490570068, 0.22293050587177277, 0.18759332597255707, 0.1624341458082199, 0.16025866568088531, 0.2093695104122162, 0.17731457948684692, 0.19939030706882477, 0.19425280392169952, 0.1645204871892929, 0.1550453007221222, 0.17824405431747437, 0.16139660775661469, 0.13746920228004456, 0.14728212356567383, 0.14686505496501923], "S_mean": 0.3069966634114583, "G_mean": 0.4576698609938224, "G_std": 0.18808379024267197, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.199825763702393, "grad_norm": 0.558273196220398, "tokens_seen": 16556032, "tokens_per_sec": 18133.349090483873, "vram_gb": 5.684933632, "lambda": 0.1590616898611188, "lambda_min": 0.09562083333730698, "lambda_max": 0.2244933545589447, "S_mean": 0.3106282552083333, "G_mean": 0.4552545587842663, "G_std": 0.18539267281691232, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.387546062469482, "grad_norm": 0.6258214712142944, "tokens_seen": 16719872, "tokens_per_sec": 18138.059621508262, "vram_gb": 5.684933632, "lambda": 0.15955673903226852, "lambda_min": 0.09556011855602264, "lambda_max": 0.2260676473379135, "S_mean": 0.32122802734375, "G_mean": 0.4415124723066886, "G_std": 0.1874851679118971, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.297123432159424, "grad_norm": 0.5802142024040222, "tokens_seen": 16883712, "tokens_per_sec": 18141.34094653339, "vram_gb": 5.684933632, "lambda": 0.16005013200143972, "lambda_min": 0.09527832269668579, "lambda_max": 0.22728559374809265, "S_mean": 0.3241780598958333, "G_mean": 0.45267399804045755, "G_std": 0.1872725049033761, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.204250812530518, "grad_norm": 0.6224308609962463, "tokens_seen": 17047552, "tokens_per_sec": 18146.253540688784, "vram_gb": 5.684933632, "lambda": 0.16062999547769627, "lambda_min": 0.0953163132071495, "lambda_max": 0.22850404679775238, "S_mean": 0.3112386067708333, "G_mean": 0.4425801398853461, "G_std": 0.18490396284808716, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.131780385971069, "grad_norm": 0.778224766254425, "tokens_seen": 17211392, "tokens_per_sec": 18150.967048254344, "vram_gb": 5.684933632, "lambda": 0.16112989280372858, "lambda_min": 0.09529486298561096, "lambda_max": 0.22970722615718842, "S_mean": 0.3104654947916667, "G_mean": 0.45081614376977086, "G_std": 0.18383795442059636, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.239620208740234, "grad_norm": 0.6282277703285217, "tokens_seen": 17375232, "tokens_per_sec": 18154.61385930519, "vram_gb": 5.684933632, "lambda": 0.16166726561884084, "lambda_min": 0.0951622873544693, "lambda_max": 0.23100198805332184, "S_mean": 0.3293863932291667, "G_mean": 0.434432848667105, "G_std": 0.1851799264550209, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.187174081802368, "grad_norm": 0.6019474864006042, "tokens_seen": 17539072, "tokens_per_sec": 18157.75031984109, "vram_gb": 5.684933632, "lambda": 0.1622170489281416, "lambda_min": 0.09494610130786896, "lambda_max": 0.232337087392807, "S_mean": 0.32293701171875, "G_mean": 0.43607635827114183, "G_std": 0.18374242835367718, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.159860610961914, "grad_norm": 0.5742467045783997, "tokens_seen": 17702912, "tokens_per_sec": 18161.592832317056, "vram_gb": 5.684933632, "lambda": 0.16275733895599842, "lambda_min": 0.09489466995000839, "lambda_max": 0.23371239006519318, "S_mean": 0.3297322591145833, "G_mean": 0.45487274198482436, "G_std": 0.18874145112931728, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.2968974113464355, "grad_norm": 0.6202488541603088, "tokens_seen": 17866752, "tokens_per_sec": 18164.60098262563, "vram_gb": 5.684933632, "lambda": 0.1632342105731368, "lambda_min": 0.09493356198072433, "lambda_max": 0.2350669801235199, "S_mean": 0.3306070963541667, "G_mean": 0.44051054616769153, "G_std": 0.18556019896641374, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.192620277404785, "grad_norm": 0.7108427286148071, "tokens_seen": 18030592, "tokens_per_sec": 18166.801452038108, "vram_gb": 5.684933632, "lambda": 0.1638197060674429, "lambda_min": 0.09473351389169693, "lambda_max": 0.23656950891017914, "S_mean": 0.2908121744791667, "G_mean": 0.45547968801110983, "G_std": 0.187321403529495, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.1364946365356445, "grad_norm": 0.641004204750061, "tokens_seen": 18194432, "tokens_per_sec": 18170.434261085982, "vram_gb": 5.684933632, "lambda": 0.16435561360170445, "lambda_min": 0.09439200162887573, "lambda_max": 0.23792552947998047, "S_mean": 0.3160909016927083, "G_mean": 0.4370485738230248, "G_std": 0.1861642268486321, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.073402166366577, "grad_norm": 0.6491034626960754, "tokens_seen": 18358272, "tokens_per_sec": 18174.015309527964, "vram_gb": 5.684933632, "lambda": 0.1648531397804618, "lambda_min": 0.09404254704713821, "lambda_max": 0.23922376334667206, "S_mean": 0.3243611653645833, "G_mean": 0.44154948415234685, "G_std": 0.1826013127962748, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.245572805404663, "grad_norm": 0.6386318206787109, "tokens_seen": 18522112, "tokens_per_sec": 18176.398227799215, "vram_gb": 5.684933632, "lambda": 0.16537275413672128, "lambda_min": 0.09384442120790482, "lambda_max": 0.2407040297985077, "S_mean": 0.32415771484375, "G_mean": 0.4294685274362564, "G_std": 0.18637365382164717, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.121899843215942, "grad_norm": 0.6348481774330139, "tokens_seen": 18685952, "tokens_per_sec": 18179.151368298735, "vram_gb": 5.684933632, "lambda": 0.16597962907205024, "lambda_min": 0.09372762590646744, "lambda_max": 0.24227209389209747, "S_mean": 0.3124491373697917, "G_mean": 0.43983693902070325, "G_std": 0.18371356589098772, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.118544578552246, "grad_norm": 0.6664955019950867, "tokens_seen": 18849792, "tokens_per_sec": 18181.567870066487, "vram_gb": 5.684933632, "lambda": 0.16647502624740204, "lambda_min": 0.09344638884067535, "lambda_max": 0.24366891384124756, "S_mean": 0.2993977864583333, "G_mean": 0.4488981816296776, "G_std": 0.18343934835866094, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 4.9937217235565186, "grad_norm": 0.62654709815979, "tokens_seen": 19013632, "tokens_per_sec": 18184.61017798422, "vram_gb": 5.684933632, "lambda": 0.16704025057454905, "lambda_min": 0.09318311512470245, "lambda_max": 0.24508193135261536, "S_mean": 0.3119710286458333, "G_mean": 0.431168085274597, "G_std": 0.18241934633503357, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.172971248626709, "grad_norm": 0.6458927989006042, "tokens_seen": 19177472, "tokens_per_sec": 18186.960573420674, "vram_gb": 5.684933632, "lambda": 0.1675521352638801, "lambda_min": 0.09290789812803268, "lambda_max": 0.24661040306091309, "S_mean": 0.3093668619791667, "G_mean": 0.432249682645003, "G_std": 0.1845628285470108, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 4.906905174255371, "grad_norm": 0.8623917698860168, "tokens_seen": 19341312, "tokens_per_sec": 18188.795030589703, "vram_gb": 5.684933632, "lambda": 0.1681030976275603, "lambda_min": 0.09256590902805328, "lambda_max": 0.24808625876903534, "S_mean": 0.30303955078125, "G_mean": 0.4428117916298409, "G_std": 0.18656126076045135, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.080040693283081, "grad_norm": 0.6448522806167603, "tokens_seen": 19505152, "tokens_per_sec": 18192.14776304661, "vram_gb": 5.684933632, "lambda": 0.16861153487116098, "lambda_min": 0.09230341017246246, "lambda_max": 0.2494966983795166, "S_mean": 0.3069864908854167, "G_mean": 0.43597085820510983, "G_std": 0.18302834763502082, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.073209047317505, "grad_norm": 0.6956644654273987, "tokens_seen": 19668992, "tokens_per_sec": 18195.089582630404, "vram_gb": 5.684933632, "lambda": 0.16914716052512327, "lambda_min": 0.09228965640068054, "lambda_max": 0.2508683204650879, "S_mean": 0.2842508951822917, "G_mean": 0.43587293770785135, "G_std": 0.18509867756317058, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.011825084686279, "grad_norm": 0.6434435844421387, "tokens_seen": 19832832, "tokens_per_sec": 18197.0647134006, "vram_gb": 5.684933632, "lambda": 0.16977064808209738, "lambda_min": 0.09220708906650543, "lambda_max": 0.2524888515472412, "S_mean": 0.2993570963541667, "G_mean": 0.42831080096463364, "G_std": 0.18612878381585082, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.062101602554321, "grad_norm": 0.6583250761032104, "tokens_seen": 19996672, "tokens_per_sec": 18199.68386542325, "vram_gb": 5.684933632, "lambda": 0.17031096884359917, "lambda_min": 0.09201917052268982, "lambda_max": 0.253763884305954, "S_mean": 0.2820332845052083, "G_mean": 0.4288631229040523, "G_std": 0.18272099799166122, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.044578313827515, "grad_norm": 0.6974340081214905, "tokens_seen": 20160512, "tokens_per_sec": 18201.762692510136, "vram_gb": 5.684933632, "lambda": 0.1709296703338623, "lambda_min": 0.09198006242513657, "lambda_max": 0.2553781569004059, "S_mean": 0.290283203125, "G_mean": 0.42226751257355016, "G_std": 0.18239997831794122, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.1667139530181885, "grad_norm": 0.8430538773536682, "tokens_seen": 20324352, "tokens_per_sec": 18204.41396127984, "vram_gb": 5.684933632, "lambda": 0.17144799449791512, "lambda_min": 0.09221246093511581, "lambda_max": 0.2568461298942566, "S_mean": 0.31121826171875, "G_mean": 0.42335283011198044, "G_std": 0.17826849649039408, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.097948312759399, "grad_norm": 0.6556472778320312, "tokens_seen": 20488192, "tokens_per_sec": 18206.325795341883, "vram_gb": 5.684933632, "lambda": 0.17200684019674858, "lambda_min": 0.09213072061538696, "lambda_max": 0.25828295946121216, "S_mean": 0.2920328776041667, "G_mean": 0.41905749527116615, "G_std": 0.18151724462707838, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 4.981981515884399, "grad_norm": 0.6528576612472534, "tokens_seen": 20652032, "tokens_per_sec": 18208.5046727988, "vram_gb": 5.684933632, "lambda": 0.17253927420824766, "lambda_min": 0.09207963198423386, "lambda_max": 0.2597573399543762, "S_mean": 0.2910563151041667, "G_mean": 0.422932642356803, "G_std": 0.1835515208852788, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.017252206802368, "grad_norm": 0.672002911567688, "tokens_seen": 20815872, "tokens_per_sec": 18211.750318440267, "vram_gb": 5.684933632, "lambda": 0.17308621325840554, "lambda_min": 0.09189962595701218, "lambda_max": 0.2613862454891205, "S_mean": 0.30474853515625, "G_mean": 0.42393536074087024, "G_std": 0.1836422512618204, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.169118881225586, "grad_norm": 0.6313623189926147, "tokens_seen": 20979712, "tokens_per_sec": 18215.029736445314, "vram_gb": 5.684933632, "lambda": 0.17360110425700745, "lambda_min": 0.09158807247877121, "lambda_max": 0.2628994882106781, "S_mean": 0.2843729654947917, "G_mean": 0.42018990172073245, "G_std": 0.18499241777074835, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.019164562225342, "grad_norm": 0.6805318593978882, "tokens_seen": 21143552, "tokens_per_sec": 18217.734787722875, "vram_gb": 5.684933632, "lambda": 0.17409977378944555, "lambda_min": 0.09120435267686844, "lambda_max": 0.26450061798095703, "S_mean": 0.3046468098958333, "G_mean": 0.414114067175736, "G_std": 0.1835920449035863, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.004042148590088, "grad_norm": 0.653674840927124, "tokens_seen": 21307392, "tokens_per_sec": 18220.51156937183, "vram_gb": 5.684933632, "lambda": 0.17465302037696043, "lambda_min": 0.09109377861022949, "lambda_max": 0.26578229665756226, "S_mean": 0.3063557942708333, "G_mean": 0.4249946547982593, "G_std": 0.1814652238972485, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.053502082824707, "grad_norm": 0.6303574442863464, "tokens_seen": 21471232, "tokens_per_sec": 18223.578849899637, "vram_gb": 5.684933632, "lambda": 0.17518282247086367, "lambda_min": 0.09114272892475128, "lambda_max": 0.2675345540046692, "S_mean": 0.3059285481770833, "G_mean": 0.41459306205312413, "G_std": 0.1815651076224943, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.0279762744903564, "grad_norm": 0.7050998210906982, "tokens_seen": 21635072, "tokens_per_sec": 18226.104730687654, "vram_gb": 5.684933632, "lambda": 0.17561589553952217, "lambda_min": 0.09097033739089966, "lambda_max": 0.2688919007778168, "S_mean": 0.31182861328125, "G_mean": 0.40045891515910625, "G_std": 0.18013697466813028, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.173388957977295, "grad_norm": 0.6078730225563049, "tokens_seen": 21798912, "tokens_per_sec": 18228.74639102561, "vram_gb": 5.684933632, "lambda": 0.17612195728967586, "lambda_min": 0.0908246636390686, "lambda_max": 0.270303338766098, "S_mean": 0.3190104166666667, "G_mean": 0.42116983653977513, "G_std": 0.17910972354002297, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 4.901092290878296, "grad_norm": 0.6596304774284363, "tokens_seen": 21962752, "tokens_per_sec": 18230.46982903353, "vram_gb": 5.684933632, "lambda": 0.1767290337011218, "lambda_min": 0.09056088328361511, "lambda_max": 0.2720855474472046, "S_mean": 0.32305908203125, "G_mean": 0.41566596971824765, "G_std": 0.17843931866809726, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.129070281982422, "grad_norm": 0.6074743866920471, "tokens_seen": 22126592, "tokens_per_sec": 18233.26209471876, "vram_gb": 5.684933632, "lambda": 0.17726234843333563, "lambda_min": 0.09055820107460022, "lambda_max": 0.2733597159385681, "S_mean": 0.3132527669270833, "G_mean": 0.4082351458879809, "G_std": 0.18039915431290865, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.112071752548218, "grad_norm": 0.615066409111023, "tokens_seen": 22290432, "tokens_per_sec": 18235.945852153804, "vram_gb": 5.684933632, "lambda": 0.17775640667726597, "lambda_min": 0.0904833972454071, "lambda_max": 0.2748464047908783, "S_mean": 0.3280436197916667, "G_mean": 0.4193593015273412, "G_std": 0.18269333483961722, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.0768537521362305, "grad_norm": 0.6570385694503784, "tokens_seen": 22454272, "tokens_per_sec": 18237.9909524389, "vram_gb": 5.684933632, "lambda": 0.1782360104843974, "lambda_min": 0.09045448154211044, "lambda_max": 0.2760810852050781, "S_mean": 0.30108642578125, "G_mean": 0.4178630290552974, "G_std": 0.18048303186272582, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.01089334487915, "grad_norm": 0.6762102842330933, "tokens_seen": 22618112, "tokens_per_sec": 18237.638970652224, "vram_gb": 5.684933632, "lambda": 0.17878720288475355, "lambda_min": 0.09076790511608124, "lambda_max": 0.27747946977615356, "S_mean": 0.294921875, "G_mean": 0.40693980253612, "G_std": 0.18233742184626558, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.061131000518799, "grad_norm": 0.7615111470222473, "tokens_seen": 22781952, "tokens_per_sec": 18239.465745038706, "vram_gb": 5.684933632, "lambda": 0.17930879164487123, "lambda_min": 0.09094950556755066, "lambda_max": 0.2789343595504761, "S_mean": 0.3088480631510417, "G_mean": 0.41738420833523077, "G_std": 0.18165710483056804, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.9859747886657715, "grad_norm": 0.657135546207428, "tokens_seen": 22945792, "tokens_per_sec": 18240.583893410258, "vram_gb": 5.684933632, "lambda": 0.17976458619038263, "lambda_min": 0.09099782258272171, "lambda_max": 0.2803116738796234, "S_mean": 0.3058675130208333, "G_mean": 0.40641565434634686, "G_std": 0.18276287064266702, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.91358757019043, "grad_norm": 0.6722871661186218, "tokens_seen": 23109632, "tokens_per_sec": 18243.16220229107, "vram_gb": 5.684933632, "lambda": 0.18034270188460746, "lambda_min": 0.09101639688014984, "lambda_max": 0.2814684808254242, "S_mean": 0.2999064127604167, "G_mean": 0.4083407069556415, "G_std": 0.18036490733114383, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 4.905674934387207, "grad_norm": 0.6401274800300598, "tokens_seen": 23273472, "tokens_per_sec": 18245.555267815533, "vram_gb": 5.684933632, "lambda": 0.18086891124645868, "lambda_min": 0.09093216806650162, "lambda_max": 0.28296250104904175, "S_mean": 0.3153889973958333, "G_mean": 0.4038863863485555, "G_std": 0.1804882053596278, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.014226675033569, "grad_norm": 0.6094387769699097, "tokens_seen": 23437312, "tokens_per_sec": 18247.878117942724, "vram_gb": 5.684933632, "lambda": 0.18138409405946732, "lambda_min": 0.09094202518463135, "lambda_max": 0.28437966108322144, "S_mean": 0.298126220703125, "G_mean": 0.40856135729700327, "G_std": 0.18054011557251215, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 4.995115280151367, "grad_norm": 0.6504446268081665, "tokens_seen": 23601152, "tokens_per_sec": 18250.17066613435, "vram_gb": 5.684933632, "lambda": 0.18189018654326597, "lambda_min": 0.09113200008869171, "lambda_max": 0.285454660654068, "S_mean": 0.2974853515625, "G_mean": 0.4082016162574291, "G_std": 0.18041851557791233, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.998856067657471, "grad_norm": 0.6940333247184753, "tokens_seen": 23764992, "tokens_per_sec": 18251.745778161974, "vram_gb": 5.684933632, "lambda": 0.18243986709664264, "lambda_min": 0.09130154550075531, "lambda_max": 0.28684133291244507, "S_mean": 0.3203125, "G_mean": 0.4087769103546937, "G_std": 0.1807685064462324, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.0316078662872314, "grad_norm": 0.6354218125343323, "tokens_seen": 23928832, "tokens_per_sec": 18254.193946637828, "vram_gb": 5.684933632, "lambda": 0.18294594685236612, "lambda_min": 0.0914330929517746, "lambda_max": 0.2879490554332733, "S_mean": 0.2965901692708333, "G_mean": 0.40686397192378837, "G_std": 0.17978697083890438, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 4.971907377243042, "grad_norm": 0.6744305491447449, "tokens_seen": 24092672, "tokens_per_sec": 18256.250025448586, "vram_gb": 5.684933632, "lambda": 0.18348023730019727, "lambda_min": 0.09144801646471024, "lambda_max": 0.28941482305526733, "S_mean": 0.2983601888020833, "G_mean": 0.3998545231297612, "G_std": 0.17918149024869004, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.977857351303101, "grad_norm": 0.6661168932914734, "tokens_seen": 24256512, "tokens_per_sec": 18258.09862712205, "vram_gb": 5.684933632, "lambda": 0.1839823660751184, "lambda_min": 0.09140177071094513, "lambda_max": 0.2906617522239685, "S_mean": 0.3022257486979167, "G_mean": 0.40471926362564165, "G_std": 0.1797318367753178, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.971401214599609, "grad_norm": 0.732746958732605, "tokens_seen": 24420352, "tokens_per_sec": 18260.539026108007, "vram_gb": 5.684933632, "lambda": 0.18449570704251528, "lambda_min": 0.09154004603624344, "lambda_max": 0.2921118438243866, "S_mean": 0.32135009765625, "G_mean": 0.4136940537331005, "G_std": 0.18098175222985446, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.954671859741211, "grad_norm": 0.7387804388999939, "tokens_seen": 24584192, "tokens_per_sec": 18262.42677139515, "vram_gb": 5.684933632, "lambda": 0.18499127123504877, "lambda_min": 0.09156782925128937, "lambda_max": 0.2932596802711487, "S_mean": 0.2989908854166667, "G_mean": 0.4013473060913384, "G_std": 0.18010669814733168, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 4.9694363355636595, "val_ppl": 143.94572741227176, "lambda": 0.18501864932477474, "lambda_min": 0.09156768769025803, "lambda_max": 0.29331713914871216, "lambda_per_layer": [0.1578034907579422, 0.13653402030467987, 0.13559499382972717, 0.1292022168636322, 0.11927295476198196, 0.09156768769025803, 0.12408813834190369, 0.14876285195350647, 0.18075069785118103, 0.29331713914871216, 0.23623664677143097, 0.20245838165283203, 0.19873185455799103, 0.28057530522346497, 0.22460737824440002, 0.25656193494796753, 0.2505255937576294, 0.20244915783405304, 0.18262559175491333, 0.2180100828409195, 0.19127118587493896, 0.14859604835510254, 0.17096562683582306, 0.15993860363960266], "S_mean": 0.3176066080729167, "G_mean": 0.40879801847040653, "G_std": 0.1799176890247812, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.861040115356445, "grad_norm": 0.7117879390716553, "tokens_seen": 24748032, "tokens_per_sec": 18242.57228262343, "vram_gb": 5.684933632, "lambda": 0.18554292681316534, "lambda_min": 0.09161266684532166, "lambda_max": 0.2944045066833496, "S_mean": 0.2867024739583333, "G_mean": 0.4113841637348135, "G_std": 0.18005852540954947, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.84910249710083, "grad_norm": 0.7935280799865723, "tokens_seen": 24911872, "tokens_per_sec": 18243.449130123212, "vram_gb": 5.684933632, "lambda": 0.18602137981603542, "lambda_min": 0.09169016033411026, "lambda_max": 0.2954813241958618, "S_mean": 0.29736328125, "G_mean": 0.40322845925887424, "G_std": 0.17907167101899782, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.120336055755615, "grad_norm": 0.7571646571159363, "tokens_seen": 25075712, "tokens_per_sec": 18245.23258604413, "vram_gb": 5.684933632, "lambda": 0.18650012680639824, "lambda_min": 0.09155314415693283, "lambda_max": 0.29654526710510254, "S_mean": 0.3101399739583333, "G_mean": 0.4081446124861638, "G_std": 0.17946603358723223, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.984074831008911, "grad_norm": 0.6714295744895935, "tokens_seen": 25239552, "tokens_per_sec": 18247.959407379527, "vram_gb": 5.684933632, "lambda": 0.1870460631325841, "lambda_min": 0.09149077534675598, "lambda_max": 0.2979375422000885, "S_mean": 0.3232218424479167, "G_mean": 0.40182723058387637, "G_std": 0.1798251906875521, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.028688907623291, "grad_norm": 0.7565917372703552, "tokens_seen": 25403392, "tokens_per_sec": 18249.39939651959, "vram_gb": 5.684933632, "lambda": 0.18771144033720097, "lambda_min": 0.09130308777093887, "lambda_max": 0.29961875081062317, "S_mean": 0.30029296875, "G_mean": 0.40480426761011284, "G_std": 0.18024306596877673, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.880733489990234, "grad_norm": 0.6652807593345642, "tokens_seen": 25567232, "tokens_per_sec": 18251.02822947073, "vram_gb": 5.684933632, "lambda": 0.188196227264901, "lambda_min": 0.09099733829498291, "lambda_max": 0.30113786458969116, "S_mean": 0.2889607747395833, "G_mean": 0.40665279763440293, "G_std": 0.1794161506307622, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.880537986755371, "grad_norm": 0.7173463702201843, "tokens_seen": 25731072, "tokens_per_sec": 18253.243024603726, "vram_gb": 5.684933632, "lambda": 0.1886526901895801, "lambda_min": 0.09082169085741043, "lambda_max": 0.30227410793304443, "S_mean": 0.3150431315104167, "G_mean": 0.40034463442862034, "G_std": 0.18007082282565534, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.99385666847229, "grad_norm": 0.6817997097969055, "tokens_seen": 25894912, "tokens_per_sec": 18255.469721878904, "vram_gb": 5.684933632, "lambda": 0.18926715602477392, "lambda_min": 0.09090879559516907, "lambda_max": 0.30330967903137207, "S_mean": 0.3109334309895833, "G_mean": 0.4086717843698959, "G_std": 0.18004012806341052, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.873083591461182, "grad_norm": 0.685822069644928, "tokens_seen": 26058752, "tokens_per_sec": 18257.560021808004, "vram_gb": 5.684933632, "lambda": 0.18983030350257954, "lambda_min": 0.09098535031080246, "lambda_max": 0.3046022951602936, "S_mean": 0.3029581705729167, "G_mean": 0.394584809585164, "G_std": 0.1786127968225628, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.91709566116333, "grad_norm": 0.6849649548530579, "tokens_seen": 26222592, "tokens_per_sec": 18259.3087069901, "vram_gb": 5.684933632, "lambda": 0.19045191258192062, "lambda_min": 0.09108539670705795, "lambda_max": 0.30595529079437256, "S_mean": 0.3217264811197917, "G_mean": 0.39557831765462953, "G_std": 0.17851017027472457, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.060052394866943, "grad_norm": 0.7039570808410645, "tokens_seen": 26386432, "tokens_per_sec": 18261.315005051358, "vram_gb": 5.684933632, "lambda": 0.19100599270313978, "lambda_min": 0.09132984280586243, "lambda_max": 0.30702847242355347, "S_mean": 0.3111368815104167, "G_mean": 0.4029040946625173, "G_std": 0.17842422301570574, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.9346513748168945, "grad_norm": 0.6395095586776733, "tokens_seen": 26550272, "tokens_per_sec": 18263.146429689175, "vram_gb": 5.684933632, "lambda": 0.19167656047890583, "lambda_min": 0.09161356836557388, "lambda_max": 0.30840420722961426, "S_mean": 0.3058878580729167, "G_mean": 0.39778519809866947, "G_std": 0.17989750936006507, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.940542221069336, "grad_norm": 0.7060293555259705, "tokens_seen": 26714112, "tokens_per_sec": 18265.087051531533, "vram_gb": 5.684933632, "lambda": 0.19222778206070265, "lambda_min": 0.09180547297000885, "lambda_max": 0.3100941479206085, "S_mean": 0.3024088541666667, "G_mean": 0.39729892648756504, "G_std": 0.1781865085940808, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.8379926681518555, "grad_norm": 0.7186973094940186, "tokens_seen": 26877952, "tokens_per_sec": 18267.156198970013, "vram_gb": 5.684933632, "lambda": 0.192804009343187, "lambda_min": 0.09209088236093521, "lambda_max": 0.3114731013774872, "S_mean": 0.29229736328125, "G_mean": 0.4013814479112625, "G_std": 0.179600411405166, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.938971757888794, "grad_norm": 0.6952468156814575, "tokens_seen": 27041792, "tokens_per_sec": 18268.49860680279, "vram_gb": 5.684933632, "lambda": 0.19334296075006327, "lambda_min": 0.09200672060251236, "lambda_max": 0.31272566318511963, "S_mean": 0.29998779296875, "G_mean": 0.39750910953929025, "G_std": 0.17611812963150442, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.804870367050171, "grad_norm": 0.6798903942108154, "tokens_seen": 27205632, "tokens_per_sec": 18270.725955291087, "vram_gb": 5.684933632, "lambda": 0.19394026417285204, "lambda_min": 0.09191394597291946, "lambda_max": 0.3141447901725769, "S_mean": 0.2917073567708333, "G_mean": 0.39434102084487677, "G_std": 0.18000758749743304, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.9296815395355225, "grad_norm": 0.7819356322288513, "tokens_seen": 27369472, "tokens_per_sec": 18272.1924543036, "vram_gb": 5.684933632, "lambda": 0.19449275576819977, "lambda_min": 0.09216232597827911, "lambda_max": 0.3155289888381958, "S_mean": 0.2855936686197917, "G_mean": 0.3952434839059909, "G_std": 0.1805493861126403, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.882831811904907, "grad_norm": 0.7007386088371277, "tokens_seen": 27533312, "tokens_per_sec": 18273.711004746827, "vram_gb": 5.684933632, "lambda": 0.19503929683317742, "lambda_min": 0.09249545633792877, "lambda_max": 0.3168238699436188, "S_mean": 0.3099365234375, "G_mean": 0.4020528110365073, "G_std": 0.17643949443784854, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.797125339508057, "grad_norm": 0.6852650046348572, "tokens_seen": 27697152, "tokens_per_sec": 18275.524628404124, "vram_gb": 5.684933632, "lambda": 0.19556872298320135, "lambda_min": 0.09271715581417084, "lambda_max": 0.3182205855846405, "S_mean": 0.2996012369791667, "G_mean": 0.39762444406126934, "G_std": 0.17783055415687463, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.810554027557373, "grad_norm": 0.742425262928009, "tokens_seen": 27860992, "tokens_per_sec": 18276.8113681898, "vram_gb": 5.684933632, "lambda": 0.19608369811127582, "lambda_min": 0.09248320758342743, "lambda_max": 0.31953540444374084, "S_mean": 0.2999471028645833, "G_mean": 0.39049244082222384, "G_std": 0.17852369517398378, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.824530124664307, "grad_norm": 0.7621369361877441, "tokens_seen": 28024832, "tokens_per_sec": 18278.58800257272, "vram_gb": 5.684933632, "lambda": 0.196524474148949, "lambda_min": 0.09224408119916916, "lambda_max": 0.3206537663936615, "S_mean": 0.2950948079427083, "G_mean": 0.3920802442977826, "G_std": 0.1773611606719593, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.764597415924072, "grad_norm": 0.7626374959945679, "tokens_seen": 28188672, "tokens_per_sec": 18280.4449048182, "vram_gb": 5.684933632, "lambda": 0.19707401748746634, "lambda_min": 0.09239406883716583, "lambda_max": 0.3215290307998657, "S_mean": 0.308837890625, "G_mean": 0.3907691747881472, "G_std": 0.17726888748196265, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.765315294265747, "grad_norm": 0.7155002355575562, "tokens_seen": 28352512, "tokens_per_sec": 18281.23441918475, "vram_gb": 5.684933632, "lambda": 0.19763365108519793, "lambda_min": 0.09224458783864975, "lambda_max": 0.3227371275424957, "S_mean": 0.2978719075520833, "G_mean": 0.4012765483930707, "G_std": 0.17571521441762647, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 5.004003524780273, "grad_norm": 0.691945493221283, "tokens_seen": 28516352, "tokens_per_sec": 18283.041033123307, "vram_gb": 5.684933632, "lambda": 0.19814186884711185, "lambda_min": 0.09217827767133713, "lambda_max": 0.32398539781570435, "S_mean": 0.2987060546875, "G_mean": 0.3972883661432813, "G_std": 0.17603313395132622, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.925710201263428, "grad_norm": 0.6903700232505798, "tokens_seen": 28680192, "tokens_per_sec": 18283.949075974313, "vram_gb": 5.684933632, "lambda": 0.19863510535409054, "lambda_min": 0.09216108918190002, "lambda_max": 0.3250170946121216, "S_mean": 0.3006591796875, "G_mean": 0.39292741008102894, "G_std": 0.1779157662919412, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.747330188751221, "grad_norm": 0.7280983328819275, "tokens_seen": 28844032, "tokens_per_sec": 18285.392708492156, "vram_gb": 5.684933632, "lambda": 0.19910092682888111, "lambda_min": 0.09228678047657013, "lambda_max": 0.32607612013816833, "S_mean": 0.3163248697916667, "G_mean": 0.38777242104212445, "G_std": 0.18132933353384337, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 5.046241283416748, "grad_norm": 0.7428289651870728, "tokens_seen": 29007872, "tokens_per_sec": 18286.954098211565, "vram_gb": 5.684933632, "lambda": 0.1996177090331912, "lambda_min": 0.09223195910453796, "lambda_max": 0.32726845145225525, "S_mean": 0.3044230143229167, "G_mean": 0.3928560757388671, "G_std": 0.179959719767794, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.897424697875977, "grad_norm": 0.6771010756492615, "tokens_seen": 29171712, "tokens_per_sec": 18288.865726064072, "vram_gb": 5.684933632, "lambda": 0.2001770973826448, "lambda_min": 0.09212891012430191, "lambda_max": 0.32841044664382935, "S_mean": 0.30548095703125, "G_mean": 0.39261298812925816, "G_std": 0.17607263343719146, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 5.119072675704956, "grad_norm": 0.7170913219451904, "tokens_seen": 29335552, "tokens_per_sec": 18290.29666887053, "vram_gb": 5.684933632, "lambda": 0.20078064315021038, "lambda_min": 0.09205139428377151, "lambda_max": 0.3296525180339813, "S_mean": 0.3142496744791667, "G_mean": 0.39639657037332654, "G_std": 0.1773112320030729, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.879709482192993, "grad_norm": 0.6985851526260376, "tokens_seen": 29499392, "tokens_per_sec": 18291.69949231311, "vram_gb": 5.684933632, "lambda": 0.20130669015149275, "lambda_min": 0.09208005666732788, "lambda_max": 0.3307918608188629, "S_mean": 0.2969462076822917, "G_mean": 0.3940795712793867, "G_std": 0.17686036058391133, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.7344276905059814, "grad_norm": 0.7014895081520081, "tokens_seen": 29663232, "tokens_per_sec": 18293.445853244357, "vram_gb": 5.684933632, "lambda": 0.20188738591969013, "lambda_min": 0.09230038523674011, "lambda_max": 0.3318723142147064, "S_mean": 0.2982177734375, "G_mean": 0.38934176430727047, "G_std": 0.17668399098329246, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.8665008544921875, "grad_norm": 0.7351472973823547, "tokens_seen": 29827072, "tokens_per_sec": 18295.411475277426, "vram_gb": 5.684933632, "lambda": 0.20240317502369484, "lambda_min": 0.09234312921762466, "lambda_max": 0.3330087661743164, "S_mean": 0.3032430013020833, "G_mean": 0.38745234347879887, "G_std": 0.17368364225452146, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.791673898696899, "grad_norm": 0.9774790406227112, "tokens_seen": 29990912, "tokens_per_sec": 18297.41645286428, "vram_gb": 5.684933632, "lambda": 0.20301236057033142, "lambda_min": 0.09228994697332382, "lambda_max": 0.3344666063785553, "S_mean": 0.3260701497395833, "G_mean": 0.3943103381122152, "G_std": 0.17930291391288242, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.907187461853027, "grad_norm": 0.7257916927337646, "tokens_seen": 30154752, "tokens_per_sec": 18299.200551930506, "vram_gb": 5.684933632, "lambda": 0.20357152509192625, "lambda_min": 0.09233938157558441, "lambda_max": 0.335700124502182, "S_mean": 0.296142578125, "G_mean": 0.38405395167258877, "G_std": 0.17761516586566964, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.871164321899414, "grad_norm": 0.7619187831878662, "tokens_seen": 30318592, "tokens_per_sec": 18300.88193090428, "vram_gb": 5.684933632, "lambda": 0.20402023692925772, "lambda_min": 0.09237753599882126, "lambda_max": 0.33645737171173096, "S_mean": 0.29827880859375, "G_mean": 0.39341605245135725, "G_std": 0.1757767036712418, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.793386936187744, "grad_norm": 0.7445179224014282, "tokens_seen": 30482432, "tokens_per_sec": 18301.946020756102, "vram_gb": 5.684933632, "lambda": 0.2044753565763434, "lambda_min": 0.09233617782592773, "lambda_max": 0.33769214153289795, "S_mean": 0.3068033854166667, "G_mean": 0.39481373100231093, "G_std": 0.17634173257586858, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.61638879776001, "grad_norm": 0.7206400036811829, "tokens_seen": 30646272, "tokens_per_sec": 18303.480362019745, "vram_gb": 5.684933632, "lambda": 0.20487616521616778, "lambda_min": 0.09244043380022049, "lambda_max": 0.3384494483470917, "S_mean": 0.3104044596354167, "G_mean": 0.3878398030065, "G_std": 0.17700418752307692, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.747677326202393, "grad_norm": 0.7691967487335205, "tokens_seen": 30810112, "tokens_per_sec": 18304.791874533406, "vram_gb": 5.684933632, "lambda": 0.20537519361823797, "lambda_min": 0.09247766435146332, "lambda_max": 0.339432954788208, "S_mean": 0.2988484700520833, "G_mean": 0.38938867424925167, "G_std": 0.1761956577344487, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.758114337921143, "grad_norm": 0.7707050442695618, "tokens_seen": 30973952, "tokens_per_sec": 18305.341109498448, "vram_gb": 5.684933632, "lambda": 0.20594936050474644, "lambda_min": 0.0925191193819046, "lambda_max": 0.3403659462928772, "S_mean": 0.2880452473958333, "G_mean": 0.39192002212318283, "G_std": 0.17760495305992663, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.824378490447998, "grad_norm": 0.7563660740852356, "tokens_seen": 31137792, "tokens_per_sec": 18307.078484389265, "vram_gb": 5.684933632, "lambda": 0.20652295059214035, "lambda_min": 0.09272480010986328, "lambda_max": 0.3418124318122864, "S_mean": 0.28973388671875, "G_mean": 0.3849789120722562, "G_std": 0.1748325324151665, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.796894550323486, "grad_norm": 0.7597107887268066, "tokens_seen": 31301632, "tokens_per_sec": 18308.695555504164, "vram_gb": 5.684933632, "lambda": 0.207056429858009, "lambda_min": 0.09286727011203766, "lambda_max": 0.34305334091186523, "S_mean": 0.2947794596354167, "G_mean": 0.38775424457465607, "G_std": 0.17634832750385007, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.597880840301514, "grad_norm": 0.693475604057312, "tokens_seen": 31465472, "tokens_per_sec": 18309.972105487373, "vram_gb": 5.684933632, "lambda": 0.20758097246289253, "lambda_min": 0.09278160333633423, "lambda_max": 0.34374889731407166, "S_mean": 0.291015625, "G_mean": 0.3837149573955685, "G_std": 0.1769140837714076, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.8910441398620605, "grad_norm": 0.7627325654029846, "tokens_seen": 31629312, "tokens_per_sec": 18311.652604401723, "vram_gb": 5.684933632, "lambda": 0.2081136191263795, "lambda_min": 0.09274276345968246, "lambda_max": 0.34477025270462036, "S_mean": 0.313720703125, "G_mean": 0.38992374188577134, "G_std": 0.1742388477238516, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.814699649810791, "grad_norm": 0.6863903403282166, "tokens_seen": 31793152, "tokens_per_sec": 18312.757524146524, "vram_gb": 5.684933632, "lambda": 0.20866297433773676, "lambda_min": 0.09259883314371109, "lambda_max": 0.3459770381450653, "S_mean": 0.2805074055989583, "G_mean": 0.38782511341075104, "G_std": 0.17635771702043712, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.852787256240845, "grad_norm": 0.7211596965789795, "tokens_seen": 31956992, "tokens_per_sec": 18314.031460599705, "vram_gb": 5.684933632, "lambda": 0.2091416676218311, "lambda_min": 0.09266519546508789, "lambda_max": 0.3466874063014984, "S_mean": 0.2961629231770833, "G_mean": 0.3915074047787736, "G_std": 0.1762245079347243, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.84792160987854, "grad_norm": 0.7154310345649719, "tokens_seen": 32120832, "tokens_per_sec": 18315.38235415336, "vram_gb": 5.684933632, "lambda": 0.20965079497545958, "lambda_min": 0.09271245449781418, "lambda_max": 0.3474475145339966, "S_mean": 0.2999267578125, "G_mean": 0.3919199933297932, "G_std": 0.1761113617879649, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.755326986312866, "grad_norm": 0.7672743201255798, "tokens_seen": 32284672, "tokens_per_sec": 18316.457262273718, "vram_gb": 5.684933632, "lambda": 0.21021234430372715, "lambda_min": 0.09276285022497177, "lambda_max": 0.34852248430252075, "S_mean": 0.3004150390625, "G_mean": 0.38708692028497654, "G_std": 0.17601543866718808, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.931282997131348, "grad_norm": 0.9254279732704163, "tokens_seen": 32448512, "tokens_per_sec": 18318.219468797, "vram_gb": 5.684933632, "lambda": 0.2106619564195474, "lambda_min": 0.09294866770505905, "lambda_max": 0.34934136271476746, "S_mean": 0.3197835286458333, "G_mean": 0.3804688314752032, "G_std": 0.17670552052247027, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.770971059799194, "grad_norm": 0.7523133754730225, "tokens_seen": 32612352, "tokens_per_sec": 18319.117863169195, "vram_gb": 5.684933632, "lambda": 0.21116971597075462, "lambda_min": 0.09308286011219025, "lambda_max": 0.3501623272895813, "S_mean": 0.3075968424479167, "G_mean": 0.3778089371044189, "G_std": 0.17634014350672564, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.710554361343384, "grad_norm": 0.677518904209137, "tokens_seen": 32776192, "tokens_per_sec": 18319.607211977156, "vram_gb": 5.684933632, "lambda": 0.21163225763787827, "lambda_min": 0.09318749606609344, "lambda_max": 0.35077446699142456, "S_mean": 0.2981974283854167, "G_mean": 0.3721036963785688, "G_std": 0.17487957964961728, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.882496976852417, "val_ppl": 131.95975329995977, "lambda": 0.21165564811478058, "lambda_min": 0.0931912511587143, "lambda_max": 0.3507741689682007, "lambda_per_layer": [0.16284865140914917, 0.13784871995449066, 0.13874013721942902, 0.1292075216770172, 0.12212973833084106, 0.0931912511587143, 0.12270531803369522, 0.15358325839042664, 0.19321076571941376, 0.3507741689682007, 0.28707408905029297, 0.239138662815094, 0.23443932831287384, 0.3482271730899811, 0.27434760332107544, 0.3173823356628418, 0.30987775325775146, 0.24648071825504303, 0.2123783379793167, 0.25518736243247986, 0.22656100988388062, 0.15889795124530792, 0.19410836696624756, 0.17139533162117004], "S_mean": 0.3034464518229167, "G_mean": 0.38209380007659394, "G_std": 0.17592539141575494, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.654444217681885, "grad_norm": 0.763248860836029, "tokens_seen": 32940032, "tokens_per_sec": 18303.599679434934, "vram_gb": 5.684933632, "lambda": 0.21211693404863277, "lambda_min": 0.09331408888101578, "lambda_max": 0.35148903727531433, "S_mean": 0.3086954752604167, "G_mean": 0.38415986206382513, "G_std": 0.17676027577059963, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.778432846069336, "grad_norm": 0.7760003805160522, "tokens_seen": 33103872, "tokens_per_sec": 18305.412614967045, "vram_gb": 5.684933632, "lambda": 0.21262797713279724, "lambda_min": 0.09329342842102051, "lambda_max": 0.35222113132476807, "S_mean": 0.2974650065104167, "G_mean": 0.37267336063086987, "G_std": 0.17687432110930482, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.765820503234863, "grad_norm": 0.7514824271202087, "tokens_seen": 33267712, "tokens_per_sec": 18305.637505826406, "vram_gb": 5.684933632, "lambda": 0.2131420054162542, "lambda_min": 0.09310377389192581, "lambda_max": 0.35328811407089233, "S_mean": 0.3222859700520833, "G_mean": 0.3757492514171948, "G_std": 0.17648412566632032, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.782573223114014, "grad_norm": 0.7710630297660828, "tokens_seen": 33431552, "tokens_per_sec": 18306.016858582112, "vram_gb": 5.684933632, "lambda": 0.2136875381693244, "lambda_min": 0.09297933429479599, "lambda_max": 0.35420694947242737, "S_mean": 0.308349609375, "G_mean": 0.38047011600186426, "G_std": 0.17521455558016896, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.830446004867554, "grad_norm": 0.7224828600883484, "tokens_seen": 33595392, "tokens_per_sec": 18307.392607305876, "vram_gb": 5.684933632, "lambda": 0.21415663696825504, "lambda_min": 0.09299760311841965, "lambda_max": 0.35505616664886475, "S_mean": 0.30438232421875, "G_mean": 0.3848730355966836, "G_std": 0.1758339370135218, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.791447162628174, "grad_norm": 0.7425313591957092, "tokens_seen": 33759232, "tokens_per_sec": 18309.120055235417, "vram_gb": 5.684933632, "lambda": 0.2147201191013058, "lambda_min": 0.09287754446268082, "lambda_max": 0.35651907324790955, "S_mean": 0.3066507975260417, "G_mean": 0.3816737122057627, "G_std": 0.17466750624589622, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.810526371002197, "grad_norm": 0.80149906873703, "tokens_seen": 33923072, "tokens_per_sec": 18310.271417049906, "vram_gb": 5.684933632, "lambda": 0.21524732622007528, "lambda_min": 0.09290741384029388, "lambda_max": 0.35776498913764954, "S_mean": 0.3079833984375, "G_mean": 0.37305716052651405, "G_std": 0.17664833979991576, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.889995098114014, "grad_norm": 0.8385878801345825, "tokens_seen": 34086912, "tokens_per_sec": 18312.775294087518, "vram_gb": 5.684933632, "lambda": 0.21570021690179905, "lambda_min": 0.09306002408266068, "lambda_max": 0.35884150862693787, "S_mean": 0.31634521484375, "G_mean": 0.3762646787023793, "G_std": 0.17351391181970635, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.794987678527832, "grad_norm": 0.7124825716018677, "tokens_seen": 34250752, "tokens_per_sec": 18314.22551567594, "vram_gb": 5.684933632, "lambda": 0.21615707750121751, "lambda_min": 0.09304223954677582, "lambda_max": 0.3600066602230072, "S_mean": 0.286773681640625, "G_mean": 0.3863213734390835, "G_std": 0.17479519825428724, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.802309513092041, "grad_norm": 0.7690416574478149, "tokens_seen": 34414592, "tokens_per_sec": 18315.99118656374, "vram_gb": 5.684933632, "lambda": 0.21662112108121315, "lambda_min": 0.09328446537256241, "lambda_max": 0.3609502613544464, "S_mean": 0.2962443033854167, "G_mean": 0.3795070277216534, "G_std": 0.17617396785256764, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.742512226104736, "grad_norm": 0.7081183195114136, "tokens_seen": 34578432, "tokens_per_sec": 18317.490874028375, "vram_gb": 5.684933632, "lambda": 0.21706727550675473, "lambda_min": 0.09338411688804626, "lambda_max": 0.36224091053009033, "S_mean": 0.292144775390625, "G_mean": 0.3678625469716887, "G_std": 0.17570019816048443, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.653459072113037, "grad_norm": 0.7404568791389465, "tokens_seen": 34742272, "tokens_per_sec": 18318.76860372408, "vram_gb": 5.684933632, "lambda": 0.21753226779401302, "lambda_min": 0.0932677760720253, "lambda_max": 0.3635645806789398, "S_mean": 0.2939656575520833, "G_mean": 0.37695939869930345, "G_std": 0.1733266324736178, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.73853063583374, "grad_norm": 0.7664743661880493, "tokens_seen": 34906112, "tokens_per_sec": 18320.286253778475, "vram_gb": 5.684933632, "lambda": 0.21805682281653085, "lambda_min": 0.0932331532239914, "lambda_max": 0.3652076721191406, "S_mean": 0.3064982096354167, "G_mean": 0.3748184246942401, "G_std": 0.1755623194233825, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.8637611865997314, "grad_norm": 0.8649498820304871, "tokens_seen": 35069952, "tokens_per_sec": 18321.508545582336, "vram_gb": 5.684933632, "lambda": 0.21858093328773975, "lambda_min": 0.0929323062300682, "lambda_max": 0.3667638301849365, "S_mean": 0.2958170572916667, "G_mean": 0.3801777984481305, "G_std": 0.1752824221427242, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.881865978240967, "grad_norm": 0.7358690500259399, "tokens_seen": 35233792, "tokens_per_sec": 18322.7238198219, "vram_gb": 5.684933632, "lambda": 0.21913841273635626, "lambda_min": 0.09253410249948502, "lambda_max": 0.36814773082733154, "S_mean": 0.2942911783854167, "G_mean": 0.37636969742986065, "G_std": 0.17341872118413448, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.693397521972656, "grad_norm": 0.8841035962104797, "tokens_seen": 35397632, "tokens_per_sec": 18323.785878149618, "vram_gb": 5.684933632, "lambda": 0.21963754141082367, "lambda_min": 0.0925804078578949, "lambda_max": 0.36935651302337646, "S_mean": 0.3031209309895833, "G_mean": 0.37204661468664807, "G_std": 0.17289102845825255, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.926431655883789, "grad_norm": 0.7208322882652283, "tokens_seen": 35561472, "tokens_per_sec": 18324.75575594681, "vram_gb": 5.684933632, "lambda": 0.22017129758993784, "lambda_min": 0.09260760992765427, "lambda_max": 0.37092357873916626, "S_mean": 0.294586181640625, "G_mean": 0.37679766638514894, "G_std": 0.17441362181367973, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.72038197517395, "grad_norm": 0.8263587951660156, "tokens_seen": 35725312, "tokens_per_sec": 18326.071748875496, "vram_gb": 5.684933632, "lambda": 0.2207070253789425, "lambda_min": 0.09260110557079315, "lambda_max": 0.3724939227104187, "S_mean": 0.30755615234375, "G_mean": 0.3793068192123125, "G_std": 0.17461336365280053, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.4376091957092285, "grad_norm": 0.7587844729423523, "tokens_seen": 35889152, "tokens_per_sec": 18327.370979171214, "vram_gb": 5.684933632, "lambda": 0.22126330342143774, "lambda_min": 0.09240095317363739, "lambda_max": 0.37394094467163086, "S_mean": 0.2858378092447917, "G_mean": 0.3744656940301259, "G_std": 0.17341265097881356, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.9046471118927, "grad_norm": 0.7715455889701843, "tokens_seen": 36052992, "tokens_per_sec": 18328.671010168528, "vram_gb": 5.684933632, "lambda": 0.22184408673395714, "lambda_min": 0.09239291399717331, "lambda_max": 0.3754502832889557, "S_mean": 0.3025105794270833, "G_mean": 0.3763333691749722, "G_std": 0.17186871551287672, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.716322422027588, "grad_norm": 0.6934325098991394, "tokens_seen": 36216832, "tokens_per_sec": 18329.755319578428, "vram_gb": 5.684933632, "lambda": 0.22239872658004364, "lambda_min": 0.09262656420469284, "lambda_max": 0.376920223236084, "S_mean": 0.2955220540364583, "G_mean": 0.3712809159575651, "G_std": 0.17408742165813842, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.909254789352417, "grad_norm": 0.7967830300331116, "tokens_seen": 36380672, "tokens_per_sec": 18330.626045581626, "vram_gb": 5.684933632, "lambda": 0.2228445258612434, "lambda_min": 0.0927460640668869, "lambda_max": 0.3779698312282562, "S_mean": 0.2909647623697917, "G_mean": 0.38024752323205274, "G_std": 0.1727160319375495, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.614481210708618, "grad_norm": 0.8067598342895508, "tokens_seen": 36544512, "tokens_per_sec": 18331.12822638473, "vram_gb": 5.684933632, "lambda": 0.22330477833747864, "lambda_min": 0.09282892942428589, "lambda_max": 0.3788101077079773, "S_mean": 0.2921193440755208, "G_mean": 0.3779793406526248, "G_std": 0.17453162868817648, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.740115165710449, "grad_norm": 0.7524868845939636, "tokens_seen": 36708352, "tokens_per_sec": 18332.03991791571, "vram_gb": 5.684933632, "lambda": 0.22387578307340542, "lambda_min": 0.09307325631380081, "lambda_max": 0.38047298789024353, "S_mean": 0.2898152669270833, "G_mean": 0.3764781868085265, "G_std": 0.17552698566578329, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.826943635940552, "grad_norm": 0.7098895907402039, "tokens_seen": 36872192, "tokens_per_sec": 18332.927973474983, "vram_gb": 5.684933632, "lambda": 0.2243939656764269, "lambda_min": 0.09323372691869736, "lambda_max": 0.38143691420555115, "S_mean": 0.28533935546875, "G_mean": 0.368300042891254, "G_std": 0.17485206962252656, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.50516414642334, "grad_norm": 0.7898439168930054, "tokens_seen": 37036032, "tokens_per_sec": 18334.01460957715, "vram_gb": 5.684933632, "lambda": 0.22492891177535057, "lambda_min": 0.09337818622589111, "lambda_max": 0.3831692337989807, "S_mean": 0.30596923828125, "G_mean": 0.3792603921610862, "G_std": 0.1752440056297928, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.647199392318726, "grad_norm": 0.6881875395774841, "tokens_seen": 37199872, "tokens_per_sec": 18335.11443072318, "vram_gb": 5.684933632, "lambda": 0.22541071816037098, "lambda_min": 0.0935373455286026, "lambda_max": 0.38503289222717285, "S_mean": 0.28582763671875, "G_mean": 0.37404131004586816, "G_std": 0.1739391329077383, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.642747640609741, "grad_norm": 0.7527104616165161, "tokens_seen": 37363712, "tokens_per_sec": 18334.24075507812, "vram_gb": 5.684933632, "lambda": 0.2258819549654921, "lambda_min": 0.09356187283992767, "lambda_max": 0.3862290680408478, "S_mean": 0.2855631510416667, "G_mean": 0.3724429303159316, "G_std": 0.1724396915330241, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.693514108657837, "grad_norm": 0.7644874453544617, "tokens_seen": 37527552, "tokens_per_sec": 18335.264456879813, "vram_gb": 5.684933632, "lambda": 0.2263390471537908, "lambda_min": 0.09375700354576111, "lambda_max": 0.38732126355171204, "S_mean": 0.2973124186197917, "G_mean": 0.370774118307357, "G_std": 0.17452543679003915, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.579543590545654, "grad_norm": 0.8468172550201416, "tokens_seen": 37691392, "tokens_per_sec": 18336.12226648092, "vram_gb": 5.684933632, "lambda": 0.22683018011351427, "lambda_min": 0.0936831384897232, "lambda_max": 0.3887081444263458, "S_mean": 0.292144775390625, "G_mean": 0.3693339053230981, "G_std": 0.17298402140537897, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.684196472167969, "grad_norm": 0.7909974455833435, "tokens_seen": 37855232, "tokens_per_sec": 18336.842014382415, "vram_gb": 5.684933632, "lambda": 0.22729485481977463, "lambda_min": 0.09361395239830017, "lambda_max": 0.3899502158164978, "S_mean": 0.2913411458333333, "G_mean": 0.3688609595410526, "G_std": 0.17410021655571958, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.74045991897583, "grad_norm": 0.8201084136962891, "tokens_seen": 38019072, "tokens_per_sec": 18337.792206960206, "vram_gb": 5.684933632, "lambda": 0.22790196320662895, "lambda_min": 0.09367533028125763, "lambda_max": 0.3916754722595215, "S_mean": 0.2983194986979167, "G_mean": 0.3723896581990023, "G_std": 0.17354817075344423, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.658891201019287, "grad_norm": 0.8145051002502441, "tokens_seen": 38182912, "tokens_per_sec": 18338.6567136766, "vram_gb": 5.684933632, "lambda": 0.22828997081766525, "lambda_min": 0.09398053586483002, "lambda_max": 0.39255350828170776, "S_mean": 0.2941487630208333, "G_mean": 0.36876893835142255, "G_std": 0.17344354869176945, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.621283769607544, "grad_norm": 0.7806306481361389, "tokens_seen": 38346752, "tokens_per_sec": 18339.633016259548, "vram_gb": 5.684933632, "lambda": 0.22888455602029958, "lambda_min": 0.09406591951847076, "lambda_max": 0.39405927062034607, "S_mean": 0.3331095377604167, "G_mean": 0.3721026332738499, "G_std": 0.17323479242622852, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.659347057342529, "grad_norm": 0.7721532583236694, "tokens_seen": 38510592, "tokens_per_sec": 18340.896958302597, "vram_gb": 5.684933632, "lambda": 0.2294286210089922, "lambda_min": 0.0939425528049469, "lambda_max": 0.395548552274704, "S_mean": 0.2896321614583333, "G_mean": 0.36960707247878116, "G_std": 0.17506522336043417, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.665808439254761, "grad_norm": 0.8335934281349182, "tokens_seen": 38674432, "tokens_per_sec": 18342.16651588335, "vram_gb": 5.684933632, "lambda": 0.22990209857622781, "lambda_min": 0.09395799785852432, "lambda_max": 0.3971382677555084, "S_mean": 0.289886474609375, "G_mean": 0.37146521809821326, "G_std": 0.17437701346352696, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.6990861892700195, "grad_norm": 0.7844802737236023, "tokens_seen": 38838272, "tokens_per_sec": 18343.61855957631, "vram_gb": 5.684933632, "lambda": 0.23036170502503714, "lambda_min": 0.09400350600481033, "lambda_max": 0.39883580803871155, "S_mean": 0.3018798828125, "G_mean": 0.3650000742636621, "G_std": 0.17416710670416555, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.759946346282959, "grad_norm": 0.7725723385810852, "tokens_seen": 39002112, "tokens_per_sec": 18344.510098149945, "vram_gb": 5.684933632, "lambda": 0.23082886760433516, "lambda_min": 0.0941966250538826, "lambda_max": 0.40002089738845825, "S_mean": 0.3049825032552083, "G_mean": 0.36394425372903544, "G_std": 0.17340527785321078, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.6523566246032715, "grad_norm": 0.8535985350608826, "tokens_seen": 39165952, "tokens_per_sec": 18345.49895622843, "vram_gb": 5.684933632, "lambda": 0.23133567906916142, "lambda_min": 0.0944066271185875, "lambda_max": 0.40101584792137146, "S_mean": 0.314483642578125, "G_mean": 0.3761297492310405, "G_std": 0.17303685859466592, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.826184034347534, "grad_norm": 0.752882719039917, "tokens_seen": 39329792, "tokens_per_sec": 18346.819077651904, "vram_gb": 5.684933632, "lambda": 0.2318769305323561, "lambda_min": 0.09456466138362885, "lambda_max": 0.4021492302417755, "S_mean": 0.2837931315104167, "G_mean": 0.37248026533052325, "G_std": 0.17572360128785172, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.631640672683716, "grad_norm": 0.7983090281486511, "tokens_seen": 39493632, "tokens_per_sec": 18347.759762197165, "vram_gb": 5.684933632, "lambda": 0.23229312368979058, "lambda_min": 0.09455988556146622, "lambda_max": 0.40391501784324646, "S_mean": 0.2969563802083333, "G_mean": 0.3666700889977316, "G_std": 0.17344494590846202, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.545327425003052, "grad_norm": 0.7679954171180725, "tokens_seen": 39657472, "tokens_per_sec": 18348.91293405252, "vram_gb": 5.684933632, "lambda": 0.23275776424755654, "lambda_min": 0.09457871317863464, "lambda_max": 0.40556830167770386, "S_mean": 0.2893168131510417, "G_mean": 0.36984084694025415, "G_std": 0.17197686387225986, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.624759912490845, "grad_norm": 0.7711767554283142, "tokens_seen": 39821312, "tokens_per_sec": 18350.088831150104, "vram_gb": 5.684933632, "lambda": 0.23325791272024313, "lambda_min": 0.09465260058641434, "lambda_max": 0.40687355399131775, "S_mean": 0.2990824381510417, "G_mean": 0.36500615378220874, "G_std": 0.17291064925181368, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.61198616027832, "grad_norm": 0.7505030035972595, "tokens_seen": 39985152, "tokens_per_sec": 18351.21185292783, "vram_gb": 5.684933632, "lambda": 0.23370311129838228, "lambda_min": 0.09454835206270218, "lambda_max": 0.4078127145767212, "S_mean": 0.298919677734375, "G_mean": 0.3674835685330133, "G_std": 0.174742634796227, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.68882417678833, "grad_norm": 0.7873495221138, "tokens_seen": 40148992, "tokens_per_sec": 18351.67672337818, "vram_gb": 5.684933632, "lambda": 0.23415747315933308, "lambda_min": 0.0945388525724411, "lambda_max": 0.4093916416168213, "S_mean": 0.3002726236979167, "G_mean": 0.3680456392467022, "G_std": 0.17348229015866914, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.6678626537323, "grad_norm": 0.827453076839447, "tokens_seen": 40312832, "tokens_per_sec": 18352.519448864612, "vram_gb": 5.684933632, "lambda": 0.23462593959023556, "lambda_min": 0.09453397244215012, "lambda_max": 0.4104582667350769, "S_mean": 0.2945149739583333, "G_mean": 0.37244097671161097, "G_std": 0.17365150627059242, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.620879650115967, "grad_norm": 0.7357204556465149, "tokens_seen": 40476672, "tokens_per_sec": 18353.459289437425, "vram_gb": 5.684933632, "lambda": 0.2350979701926311, "lambda_min": 0.09461595863103867, "lambda_max": 0.4117431342601776, "S_mean": 0.31011962890625, "G_mean": 0.369708958780393, "G_std": 0.1725697269042333, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.6880598068237305, "grad_norm": 0.7516381144523621, "tokens_seen": 40640512, "tokens_per_sec": 18353.968568823522, "vram_gb": 5.684933632, "lambda": 0.2356088658173879, "lambda_min": 0.09481722116470337, "lambda_max": 0.4128461480140686, "S_mean": 0.2954203287760417, "G_mean": 0.36593357799574733, "G_std": 0.1742232653001944, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.628538370132446, "grad_norm": 0.7756951451301575, "tokens_seen": 40804352, "tokens_per_sec": 18354.228989197563, "vram_gb": 5.684933632, "lambda": 0.23611569063117108, "lambda_min": 0.09493368864059448, "lambda_max": 0.4138014614582062, "S_mean": 0.2929484049479167, "G_mean": 0.37179499957710505, "G_std": 0.17237817014877996, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.793579816818237, "grad_norm": 0.8143602013587952, "tokens_seen": 40968192, "tokens_per_sec": 18354.738620998094, "vram_gb": 5.684933632, "lambda": 0.2365231616422534, "lambda_min": 0.09522886574268341, "lambda_max": 0.41486790776252747, "S_mean": 0.295135498046875, "G_mean": 0.3711463672419389, "G_std": 0.173138183929647, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.751354026794433, "val_ppl": 115.74089474874374, "lambda": 0.23654645960777998, "lambda_min": 0.09524653851985931, "lambda_max": 0.4149286448955536, "lambda_per_layer": [0.166298046708107, 0.13967441022396088, 0.1422603875398636, 0.12977077066898346, 0.11929643154144287, 0.09524653851985931, 0.12124723941087723, 0.15900562703609467, 0.20443490147590637, 0.3982017934322357, 0.3320275843143463, 0.27406445145606995, 0.2669595777988434, 0.4149286448955536, 0.31867533922195435, 0.37216639518737793, 0.3665224611759186, 0.2885182797908783, 0.23773691058158875, 0.2968685030937195, 0.2621390223503113, 0.17074516415596008, 0.21847666800022125, 0.1818498820066452], "S_mean": 0.2972920735677083, "G_mean": 0.3737017248446743, "G_std": 0.17338554134281972, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.740778207778931, "grad_norm": 0.7943148612976074, "tokens_seen": 41132032, "tokens_per_sec": 18341.415325347414, "vram_gb": 5.684933632, "lambda": 0.23692470074941716, "lambda_min": 0.0952397882938385, "lambda_max": 0.4160352945327759, "S_mean": 0.2870076497395833, "G_mean": 0.36803771601989865, "G_std": 0.1749807842231045, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.802516460418701, "grad_norm": 0.8816511631011963, "tokens_seen": 41295872, "tokens_per_sec": 18341.928228833407, "vram_gb": 5.684933632, "lambda": 0.23732075281441212, "lambda_min": 0.09521815925836563, "lambda_max": 0.4175220727920532, "S_mean": 0.2953694661458333, "G_mean": 0.36967655814563233, "G_std": 0.17277909621285895, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.637090444564819, "grad_norm": 0.8316570520401001, "tokens_seen": 41459712, "tokens_per_sec": 18341.606250661545, "vram_gb": 5.684933632, "lambda": 0.23775403232624134, "lambda_min": 0.09538309276103973, "lambda_max": 0.4184693694114685, "S_mean": 0.319427490234375, "G_mean": 0.3784525617181013, "G_std": 0.17445041813577214, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.736264705657959, "grad_norm": 0.8009878993034363, "tokens_seen": 41623552, "tokens_per_sec": 18342.214308372175, "vram_gb": 5.684933632, "lambda": 0.23817113259186348, "lambda_min": 0.09564530849456787, "lambda_max": 0.41948115825653076, "S_mean": 0.2980550130208333, "G_mean": 0.3594268919744839, "G_std": 0.1702343572396785, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.6795570850372314, "grad_norm": 0.8131768107414246, "tokens_seen": 41787392, "tokens_per_sec": 18343.093752134802, "vram_gb": 5.684933632, "lambda": 0.2386739533394575, "lambda_min": 0.09567923843860626, "lambda_max": 0.4209589660167694, "S_mean": 0.2994791666666667, "G_mean": 0.3685165634378791, "G_std": 0.173960650106892, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.70674991607666, "grad_norm": 0.7514963150024414, "tokens_seen": 41951232, "tokens_per_sec": 18343.698893920064, "vram_gb": 5.684933632, "lambda": 0.2392581362898151, "lambda_min": 0.09542540460824966, "lambda_max": 0.4220051169395447, "S_mean": 0.298614501953125, "G_mean": 0.3697026485266785, "G_std": 0.17167742736637592, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.632500886917114, "grad_norm": 0.7634615302085876, "tokens_seen": 42115072, "tokens_per_sec": 18344.63934519853, "vram_gb": 5.684933632, "lambda": 0.23977564089000225, "lambda_min": 0.09536871314048767, "lambda_max": 0.4229992628097534, "S_mean": 0.2886454264322917, "G_mean": 0.3653328693471849, "G_std": 0.17510863595331708, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.728556394577026, "grad_norm": 0.8561191558837891, "tokens_seen": 42278912, "tokens_per_sec": 18345.195161001484, "vram_gb": 5.684933632, "lambda": 0.2402218865851561, "lambda_min": 0.09553378820419312, "lambda_max": 0.42454877495765686, "S_mean": 0.294525146484375, "G_mean": 0.3707390239772697, "G_std": 0.17332150119667253, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.54503870010376, "grad_norm": 0.7751049399375916, "tokens_seen": 42442752, "tokens_per_sec": 18345.98155561266, "vram_gb": 5.684933632, "lambda": 0.24065693467855453, "lambda_min": 0.09541444480419159, "lambda_max": 0.42579877376556396, "S_mean": 0.2995402018229167, "G_mean": 0.36386202306797105, "G_std": 0.17036034539341927, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.5143797397613525, "grad_norm": 0.7983046174049377, "tokens_seen": 42606592, "tokens_per_sec": 18346.21766103309, "vram_gb": 5.684933632, "lambda": 0.24107904639095068, "lambda_min": 0.09536217153072357, "lambda_max": 0.4267883598804474, "S_mean": 0.3025309244791667, "G_mean": 0.35834514047019184, "G_std": 0.17392763177243373, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.6839330196380615, "grad_norm": 0.7637577056884766, "tokens_seen": 42770432, "tokens_per_sec": 18347.144784617456, "vram_gb": 5.684933632, "lambda": 0.24153699912130833, "lambda_min": 0.09515630453824997, "lambda_max": 0.4280770421028137, "S_mean": 0.290496826171875, "G_mean": 0.36557072944318253, "G_std": 0.17245389171876013, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.745244741439819, "grad_norm": 0.8891811966896057, "tokens_seen": 42934272, "tokens_per_sec": 18348.55269721217, "vram_gb": 5.684933632, "lambda": 0.24198819821079573, "lambda_min": 0.09502056986093521, "lambda_max": 0.429448664188385, "S_mean": 0.3060099283854167, "G_mean": 0.3731331959522019, "G_std": 0.1729315995083501, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.617391586303711, "grad_norm": 0.7862179279327393, "tokens_seen": 43098112, "tokens_per_sec": 18349.42231256759, "vram_gb": 5.684933632, "lambda": 0.24252067878842354, "lambda_min": 0.09518991410732269, "lambda_max": 0.4311928153038025, "S_mean": 0.2911783854166667, "G_mean": 0.3642084232221047, "G_std": 0.1731772793767353, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.595859050750732, "grad_norm": 0.8553651571273804, "tokens_seen": 43261952, "tokens_per_sec": 18350.41894490853, "vram_gb": 5.684933632, "lambda": 0.24299585850288472, "lambda_min": 0.09541676193475723, "lambda_max": 0.43271604180336, "S_mean": 0.2889302571614583, "G_mean": 0.358302519734328, "G_std": 0.16988038993440568, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.615639686584473, "grad_norm": 0.7850041389465332, "tokens_seen": 43425792, "tokens_per_sec": 18351.2197534512, "vram_gb": 5.684933632, "lambda": 0.24344167144348225, "lambda_min": 0.09546012431383133, "lambda_max": 0.4339657425880432, "S_mean": 0.303253173828125, "G_mean": 0.36473490910915035, "G_std": 0.1735751877228419, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.628255128860474, "grad_norm": 0.7880725264549255, "tokens_seen": 43589632, "tokens_per_sec": 18352.272498804963, "vram_gb": 5.684933632, "lambda": 0.24390633000681797, "lambda_min": 0.09547745436429977, "lambda_max": 0.4346320629119873, "S_mean": 0.30230712890625, "G_mean": 0.3726271258977552, "G_std": 0.1724106960464269, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.713900327682495, "grad_norm": 0.8540233969688416, "tokens_seen": 43753472, "tokens_per_sec": 18353.466035699777, "vram_gb": 5.684933632, "lambda": 0.24426641842971245, "lambda_min": 0.09541989117860794, "lambda_max": 0.43600255250930786, "S_mean": 0.3041585286458333, "G_mean": 0.36805650948857266, "G_std": 0.17131264127480486, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.643073081970215, "grad_norm": 0.8515545725822449, "tokens_seen": 43917312, "tokens_per_sec": 18354.697617682898, "vram_gb": 5.684933632, "lambda": 0.24474171486993632, "lambda_min": 0.09534537047147751, "lambda_max": 0.4372078478336334, "S_mean": 0.2922465006510417, "G_mean": 0.3668080548911045, "G_std": 0.17267780199957392, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.595464706420898, "grad_norm": 0.8044107556343079, "tokens_seen": 44081152, "tokens_per_sec": 18355.33276555723, "vram_gb": 5.684933632, "lambda": 0.24516871416320404, "lambda_min": 0.09544458240270615, "lambda_max": 0.438025563955307, "S_mean": 0.3030598958333333, "G_mean": 0.3570839643167953, "G_std": 0.17078858148306608, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.698471546173096, "grad_norm": 0.7878754734992981, "tokens_seen": 44244992, "tokens_per_sec": 18356.110030686617, "vram_gb": 5.684933632, "lambda": 0.24562156479805708, "lambda_min": 0.0955362394452095, "lambda_max": 0.4387945532798767, "S_mean": 0.2994893391927083, "G_mean": 0.3670561285689473, "G_std": 0.17348597384989262, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.586088418960571, "grad_norm": 0.8726659417152405, "tokens_seen": 44408832, "tokens_per_sec": 18357.06506014081, "vram_gb": 5.684933632, "lambda": 0.2460547067845861, "lambda_min": 0.0953756645321846, "lambda_max": 0.43992358446121216, "S_mean": 0.305694580078125, "G_mean": 0.3717958456836641, "G_std": 0.17288240619624654, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.74377179145813, "grad_norm": 0.7842851877212524, "tokens_seen": 44572672, "tokens_per_sec": 18357.836091464957, "vram_gb": 5.684933632, "lambda": 0.2465185613060991, "lambda_min": 0.09538431465625763, "lambda_max": 0.4414728283882141, "S_mean": 0.3129781087239583, "G_mean": 0.3657076076293985, "G_std": 0.17305507964920253, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.561523199081421, "grad_norm": 0.794880211353302, "tokens_seen": 44736512, "tokens_per_sec": 18358.77536739991, "vram_gb": 5.684933632, "lambda": 0.24690743070095778, "lambda_min": 0.09558834880590439, "lambda_max": 0.4425770342350006, "S_mean": 0.2905680338541667, "G_mean": 0.36334628658369184, "G_std": 0.17005327662142614, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.6771838665008545, "grad_norm": 0.8790001273155212, "tokens_seen": 44900352, "tokens_per_sec": 18359.728746395445, "vram_gb": 5.684933632, "lambda": 0.24736797933777174, "lambda_min": 0.09572669863700867, "lambda_max": 0.4439508318901062, "S_mean": 0.3180745442708333, "G_mean": 0.3658526196765403, "G_std": 0.17098473897203803, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.715211391448975, "grad_norm": 0.8457003831863403, "tokens_seen": 45064192, "tokens_per_sec": 18360.01418406743, "vram_gb": 5.684933632, "lambda": 0.24782517738640308, "lambda_min": 0.09575283527374268, "lambda_max": 0.44558727741241455, "S_mean": 0.3152364095052083, "G_mean": 0.35793535714037716, "G_std": 0.17178647957431772, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.815738916397095, "grad_norm": 0.820027232170105, "tokens_seen": 45228032, "tokens_per_sec": 18361.043186902654, "vram_gb": 5.684933632, "lambda": 0.24830062159647545, "lambda_min": 0.09577678143978119, "lambda_max": 0.44696831703186035, "S_mean": 0.286041259765625, "G_mean": 0.3657913162993888, "G_std": 0.17245381554433456, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.720362901687622, "grad_norm": 0.7745139002799988, "tokens_seen": 45391872, "tokens_per_sec": 18361.449423785696, "vram_gb": 5.684933632, "lambda": 0.24877672332028547, "lambda_min": 0.09570091217756271, "lambda_max": 0.448136568069458, "S_mean": 0.2922770182291667, "G_mean": 0.3674296786387761, "G_std": 0.1720795133151114, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.507068634033203, "grad_norm": 0.8066146373748779, "tokens_seen": 45555712, "tokens_per_sec": 18362.040268392364, "vram_gb": 5.684933632, "lambda": 0.24915645737200975, "lambda_min": 0.09578288346529007, "lambda_max": 0.44935905933380127, "S_mean": 0.3080851236979167, "G_mean": 0.36000380308056873, "G_std": 0.1714816086459905, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.542153358459473, "grad_norm": 0.8033159375190735, "tokens_seen": 45719552, "tokens_per_sec": 18362.93806473119, "vram_gb": 5.684933632, "lambda": 0.24966059966633716, "lambda_min": 0.09581033140420914, "lambda_max": 0.45039570331573486, "S_mean": 0.3044942220052083, "G_mean": 0.36728547769598663, "G_std": 0.17129114596173167, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.6742119789123535, "grad_norm": 0.8691620826721191, "tokens_seen": 45883392, "tokens_per_sec": 18363.300108387317, "vram_gb": 5.684933632, "lambda": 0.2501507631192605, "lambda_min": 0.09590820223093033, "lambda_max": 0.45170870423316956, "S_mean": 0.2881368001302083, "G_mean": 0.3631510976701975, "G_std": 0.1723190783523023, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.4130144119262695, "grad_norm": 0.771591067314148, "tokens_seen": 46047232, "tokens_per_sec": 18364.20683706393, "vram_gb": 5.684933632, "lambda": 0.25062733454008895, "lambda_min": 0.09595988690853119, "lambda_max": 0.45312604308128357, "S_mean": 0.291595458984375, "G_mean": 0.3629524919670075, "G_std": 0.17084747118254504, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.6791417598724365, "grad_norm": 0.8401879668235779, "tokens_seen": 46211072, "tokens_per_sec": 18364.409432454177, "vram_gb": 5.684933632, "lambda": 0.25102281974007684, "lambda_min": 0.09609422832727432, "lambda_max": 0.4543769359588623, "S_mean": 0.30035400390625, "G_mean": 0.35773366658637923, "G_std": 0.17006552553114793, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.710679292678833, "grad_norm": 0.8173484206199646, "tokens_seen": 46374912, "tokens_per_sec": 18364.99240856833, "vram_gb": 5.684933632, "lambda": 0.25146817105511826, "lambda_min": 0.09615708887577057, "lambda_max": 0.4556434154510498, "S_mean": 0.2973836263020833, "G_mean": 0.3644516132771969, "G_std": 0.17265355912968516, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.5012617111206055, "grad_norm": 0.8183082342147827, "tokens_seen": 46538752, "tokens_per_sec": 18365.78993576504, "vram_gb": 5.684933632, "lambda": 0.2518983759606878, "lambda_min": 0.0961885005235672, "lambda_max": 0.4567233920097351, "S_mean": 0.3095296223958333, "G_mean": 0.3574371750000864, "G_std": 0.17161078844219446, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.475147724151611, "grad_norm": 0.7798191905021667, "tokens_seen": 46702592, "tokens_per_sec": 18366.320874147383, "vram_gb": 5.684933632, "lambda": 0.252339162863791, "lambda_min": 0.09615293890237808, "lambda_max": 0.4578307271003723, "S_mean": 0.2890828450520833, "G_mean": 0.3581862701879193, "G_std": 0.17064290846853206, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.620668411254883, "grad_norm": 0.7912061810493469, "tokens_seen": 46866432, "tokens_per_sec": 18366.800021138202, "vram_gb": 5.684933632, "lambda": 0.25282227247953415, "lambda_min": 0.09610816836357117, "lambda_max": 0.45946162939071655, "S_mean": 0.29986572265625, "G_mean": 0.35826594072083634, "G_std": 0.17165038245730102, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.661371946334839, "grad_norm": 0.8579557538032532, "tokens_seen": 47030272, "tokens_per_sec": 18367.363676399124, "vram_gb": 5.684933632, "lambda": 0.2532856147736311, "lambda_min": 0.09597570449113846, "lambda_max": 0.4605156481266022, "S_mean": 0.315673828125, "G_mean": 0.36944881174713373, "G_std": 0.16986994872180125, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.554885149002075, "grad_norm": 0.8239679336547852, "tokens_seen": 47194112, "tokens_per_sec": 18367.927837735286, "vram_gb": 5.684933632, "lambda": 0.2536843453223507, "lambda_min": 0.09578151255846024, "lambda_max": 0.4618993103504181, "S_mean": 0.28759765625, "G_mean": 0.3682170473815252, "G_std": 0.17081073586208126, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.523930072784424, "grad_norm": 0.8168082237243652, "tokens_seen": 47357952, "tokens_per_sec": 18369.017522376493, "vram_gb": 5.684933632, "lambda": 0.2540863190467159, "lambda_min": 0.09575054794549942, "lambda_max": 0.4630641043186188, "S_mean": 0.3027445475260417, "G_mean": 0.35625974445914227, "G_std": 0.17064483184367418, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.486393213272095, "grad_norm": 0.8063188791275024, "tokens_seen": 47521792, "tokens_per_sec": 18369.295307938213, "vram_gb": 5.684933632, "lambda": 0.2545256847515702, "lambda_min": 0.09577260911464691, "lambda_max": 0.46457594633102417, "S_mean": 0.3171590169270833, "G_mean": 0.3682216592909147, "G_std": 0.16874093702062964, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.539267539978027, "grad_norm": 0.939861536026001, "tokens_seen": 47685632, "tokens_per_sec": 18369.31441078628, "vram_gb": 5.684933632, "lambda": 0.25500068968782824, "lambda_min": 0.09577066451311111, "lambda_max": 0.46574580669403076, "S_mean": 0.2916971842447917, "G_mean": 0.36250115232542157, "G_std": 0.1717093183348576, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.728235960006714, "grad_norm": 0.8127169013023376, "tokens_seen": 47849472, "tokens_per_sec": 18370.209268307044, "vram_gb": 5.684933632, "lambda": 0.25546469787756604, "lambda_min": 0.09595773369073868, "lambda_max": 0.46705061197280884, "S_mean": 0.2953287760416667, "G_mean": 0.35975503098840517, "G_std": 0.17147134190114835, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.545278787612915, "grad_norm": 0.7928160429000854, "tokens_seen": 48013312, "tokens_per_sec": 18371.083176002427, "vram_gb": 5.684933632, "lambda": 0.2559572284420331, "lambda_min": 0.09601417928934097, "lambda_max": 0.46785813570022583, "S_mean": 0.2885233561197917, "G_mean": 0.3585992289396624, "G_std": 0.1715753930620849, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.663610935211182, "grad_norm": 0.8572279214859009, "tokens_seen": 48177152, "tokens_per_sec": 18371.987644289555, "vram_gb": 5.684933632, "lambda": 0.2564028777802984, "lambda_min": 0.0960320308804512, "lambda_max": 0.4687894284725189, "S_mean": 0.3027750651041667, "G_mean": 0.36334170781386393, "G_std": 0.16779711081956825, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.482780456542969, "grad_norm": 0.8299276828765869, "tokens_seen": 48340992, "tokens_per_sec": 18372.87334689044, "vram_gb": 5.684933632, "lambda": 0.2567604922999938, "lambda_min": 0.0958724245429039, "lambda_max": 0.4694904088973999, "S_mean": 0.2916666666666667, "G_mean": 0.3558735668193549, "G_std": 0.1703094057738781, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.43078088760376, "grad_norm": 0.8623465299606323, "tokens_seen": 48504832, "tokens_per_sec": 18373.388416908052, "vram_gb": 5.684933632, "lambda": 0.25714112104227144, "lambda_min": 0.09583507478237152, "lambda_max": 0.47077205777168274, "S_mean": 0.2847900390625, "G_mean": 0.358245681039989, "G_std": 0.1711004126506547, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.619410753250122, "grad_norm": 0.830647885799408, "tokens_seen": 48668672, "tokens_per_sec": 18373.892712839322, "vram_gb": 5.684933632, "lambda": 0.25753561966121197, "lambda_min": 0.09583117067813873, "lambda_max": 0.47171229124069214, "S_mean": 0.2848103841145833, "G_mean": 0.3626527128120263, "G_std": 0.17364863585680723, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.518354654312134, "grad_norm": 0.8505178689956665, "tokens_seen": 48832512, "tokens_per_sec": 18374.75934741302, "vram_gb": 5.684933632, "lambda": 0.2579338162516554, "lambda_min": 0.09582982957363129, "lambda_max": 0.47298663854599, "S_mean": 0.2955423990885417, "G_mean": 0.36487312777899206, "G_std": 0.1711149246742328, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.6433515548706055, "grad_norm": 0.840333104133606, "tokens_seen": 48996352, "tokens_per_sec": 18374.340640030925, "vram_gb": 5.684933632, "lambda": 0.2584414106483261, "lambda_min": 0.09577202051877975, "lambda_max": 0.4742015600204468, "S_mean": 0.2904866536458333, "G_mean": 0.3603122190882762, "G_std": 0.17076579333903888, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.6822593212127686, "grad_norm": 0.8432538509368896, "tokens_seen": 49160192, "tokens_per_sec": 18370.722285782525, "vram_gb": 5.684933632, "lambda": 0.25883820187300444, "lambda_min": 0.09581489861011505, "lambda_max": 0.4750109910964966, "S_mean": 0.28985595703125, "G_mean": 0.35807150617862743, "G_std": 0.17097215920997164, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.643370652198792, "val_ppl": 103.89394842656749, "lambda": 0.2588627024864157, "lambda_min": 0.09581530839204788, "lambda_max": 0.4750651717185974, "lambda_per_layer": [0.16941577196121216, 0.14117933809757233, 0.1457095891237259, 0.1290091872215271, 0.11949381232261658, 0.09581530839204788, 0.12094190716743469, 0.1636698842048645, 0.21438346803188324, 0.4352802336215973, 0.3675641715526581, 0.3053717315196991, 0.291287362575531, 0.4750651717185974, 0.3585360646247864, 0.42553380131721497, 0.4199862480163574, 0.3281351625919342, 0.2589150369167328, 0.33617162704467773, 0.29242220520973206, 0.1833430528640747, 0.24401898682117462, 0.19145573675632477], "S_mean": 0.3112691243489583, "G_mean": 0.35370160173624754, "G_std": 0.17156253242865205, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.6617090702056885, "grad_norm": 0.8297393918037415, "tokens_seen": 49324032, "tokens_per_sec": 18358.986291878064, "vram_gb": 5.684933632, "lambda": 0.2592731745292743, "lambda_min": 0.09578309953212738, "lambda_max": 0.4762701690196991, "S_mean": 0.2889251708984375, "G_mean": 0.35848265420645475, "G_std": 0.17057556294215223, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.471333980560303, "grad_norm": 0.8558259606361389, "tokens_seen": 49487872, "tokens_per_sec": 18358.500878339724, "vram_gb": 5.684933632, "lambda": 0.2596320478866498, "lambda_min": 0.09573446214199066, "lambda_max": 0.47733500599861145, "S_mean": 0.3020121256510417, "G_mean": 0.3596945912577212, "G_std": 0.17154517428328594, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.5453290939331055, "grad_norm": 0.7953281402587891, "tokens_seen": 49651712, "tokens_per_sec": 18358.216506444354, "vram_gb": 5.684933632, "lambda": 0.2600602243716518, "lambda_min": 0.09562640637159348, "lambda_max": 0.4783494472503662, "S_mean": 0.2991943359375, "G_mean": 0.3565436414598177, "G_std": 0.16978794305274883, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.732675790786743, "grad_norm": 0.779789924621582, "tokens_seen": 49815552, "tokens_per_sec": 18359.015763552554, "vram_gb": 5.684933632, "lambda": 0.26047793837885064, "lambda_min": 0.09540869295597076, "lambda_max": 0.4792613387107849, "S_mean": 0.3167012532552083, "G_mean": 0.356764108253022, "G_std": 0.1686527559068054, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.390136003494263, "grad_norm": 0.8126724362373352, "tokens_seen": 49979392, "tokens_per_sec": 18355.970345027636, "vram_gb": 5.684933632, "lambda": 0.2608625286569198, "lambda_min": 0.09532873332500458, "lambda_max": 0.48054516315460205, "S_mean": 0.29876708984375, "G_mean": 0.3546972152156134, "G_std": 0.1686471375481536, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.6539225578308105, "grad_norm": 0.8156017661094666, "tokens_seen": 50143232, "tokens_per_sec": 18356.104544669863, "vram_gb": 5.684933632, "lambda": 0.26122099968294304, "lambda_min": 0.09535017609596252, "lambda_max": 0.4819808006286621, "S_mean": 0.3069051106770833, "G_mean": 0.3615856012329459, "G_std": 0.16986405081115663, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.4892497062683105, "grad_norm": 0.798835813999176, "tokens_seen": 50307072, "tokens_per_sec": 18356.31349056971, "vram_gb": 5.684933632, "lambda": 0.2615624973550439, "lambda_min": 0.09539175033569336, "lambda_max": 0.4830566942691803, "S_mean": 0.2928670247395833, "G_mean": 0.35908133854779106, "G_std": 0.17136443802155554, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.562302112579346, "grad_norm": 0.8603989481925964, "tokens_seen": 50470912, "tokens_per_sec": 18353.55948171712, "vram_gb": 5.684933632, "lambda": 0.2620219448581338, "lambda_min": 0.09549394994974136, "lambda_max": 0.48411473631858826, "S_mean": 0.2998250325520833, "G_mean": 0.36358975906235474, "G_std": 0.16813443523521224, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.474977493286133, "grad_norm": 0.8107099533081055, "tokens_seen": 50634752, "tokens_per_sec": 18351.170580779766, "vram_gb": 5.684933632, "lambda": 0.2624577156578501, "lambda_min": 0.09548582881689072, "lambda_max": 0.4855002760887146, "S_mean": 0.2945454915364583, "G_mean": 0.3516097213917722, "G_std": 0.1699120351113379, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.610996246337891, "grad_norm": 0.8573105931282043, "tokens_seen": 50798592, "tokens_per_sec": 18348.20130092587, "vram_gb": 5.684933632, "lambda": 0.26288323880483705, "lambda_min": 0.09530788660049438, "lambda_max": 0.4867139458656311, "S_mean": 0.2927449544270833, "G_mean": 0.3624000658746809, "G_std": 0.17131966965583464, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.672152042388916, "grad_norm": 0.769991397857666, "tokens_seen": 50962432, "tokens_per_sec": 18345.317093956557, "vram_gb": 5.684933632, "lambda": 0.26325015164911747, "lambda_min": 0.09526586532592773, "lambda_max": 0.4874356985092163, "S_mean": 0.29364013671875, "G_mean": 0.35825196661365527, "G_std": 0.17251891433261335, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.602458477020264, "grad_norm": 0.796809196472168, "tokens_seen": 51126272, "tokens_per_sec": 18342.836063941177, "vram_gb": 5.684933632, "lambda": 0.2636394447957476, "lambda_min": 0.09540782868862152, "lambda_max": 0.48853063583374023, "S_mean": 0.3023783365885417, "G_mean": 0.3582523966518541, "G_std": 0.17165487500218055, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.472127676010132, "grad_norm": 0.7713797688484192, "tokens_seen": 51290112, "tokens_per_sec": 18340.44285851935, "vram_gb": 5.684933632, "lambda": 0.26397693529725075, "lambda_min": 0.09556394070386887, "lambda_max": 0.4894629716873169, "S_mean": 0.3097127278645833, "G_mean": 0.3529267829532425, "G_std": 0.17141674676289162, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.460530996322632, "grad_norm": 0.8780010938644409, "tokens_seen": 51453952, "tokens_per_sec": 18340.07536108695, "vram_gb": 5.684933632, "lambda": 0.2643559059749047, "lambda_min": 0.09547465294599533, "lambda_max": 0.49065178632736206, "S_mean": 0.3001708984375, "G_mean": 0.35116600967012346, "G_std": 0.17091993677119413, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.58116340637207, "grad_norm": 1.0352119207382202, "tokens_seen": 51617792, "tokens_per_sec": 18340.59126004965, "vram_gb": 5.684933632, "lambda": 0.2647238438948989, "lambda_min": 0.09538279473781586, "lambda_max": 0.49150583148002625, "S_mean": 0.3071797688802083, "G_mean": 0.360798515068988, "G_std": 0.169387319823727, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.344851970672607, "grad_norm": 0.7812787890434265, "tokens_seen": 51781632, "tokens_per_sec": 18340.08916038077, "vram_gb": 5.684933632, "lambda": 0.26511879296352464, "lambda_min": 0.09537608921527863, "lambda_max": 0.49260708689689636, "S_mean": 0.3070475260416667, "G_mean": 0.3554890203134467, "G_std": 0.17024426348507404, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.588832139968872, "grad_norm": 0.7858982086181641, "tokens_seen": 51945472, "tokens_per_sec": 18338.92784564856, "vram_gb": 5.684933632, "lambda": 0.26546994006882113, "lambda_min": 0.09541483968496323, "lambda_max": 0.49354469776153564, "S_mean": 0.306427001953125, "G_mean": 0.35363700628901523, "G_std": 0.16929990185114244, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.437335252761841, "grad_norm": 0.8202621340751648, "tokens_seen": 52109312, "tokens_per_sec": 18336.525346526767, "vram_gb": 5.684933632, "lambda": 0.26587214848647517, "lambda_min": 0.09548693895339966, "lambda_max": 0.4943743348121643, "S_mean": 0.2947794596354167, "G_mean": 0.3576130832079798, "G_std": 0.16933067717278996, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.594384670257568, "grad_norm": 0.8033254742622375, "tokens_seen": 52273152, "tokens_per_sec": 18335.17267707296, "vram_gb": 5.684933632, "lambda": 0.26626183092594147, "lambda_min": 0.09547632187604904, "lambda_max": 0.4953778386116028, "S_mean": 0.2955830891927083, "G_mean": 0.3553391710544626, "G_std": 0.16946567913206914, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.560847520828247, "grad_norm": 0.8502037525177002, "tokens_seen": 52436992, "tokens_per_sec": 18335.199421243018, "vram_gb": 5.684933632, "lambda": 0.2666119408483307, "lambda_min": 0.09543471783399582, "lambda_max": 0.4963386356830597, "S_mean": 0.324462890625, "G_mean": 0.3579733884738137, "G_std": 0.17043883904504278, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.612593173980713, "grad_norm": 0.8440104722976685, "tokens_seen": 52600832, "tokens_per_sec": 18334.684679592785, "vram_gb": 5.684933632, "lambda": 0.2670399636651079, "lambda_min": 0.09553247690200806, "lambda_max": 0.49710068106651306, "S_mean": 0.29473876953125, "G_mean": 0.35876511114959914, "G_std": 0.1700375898896406, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.648001432418823, "grad_norm": 0.8397901654243469, "tokens_seen": 52764672, "tokens_per_sec": 18333.779809633626, "vram_gb": 5.684933632, "lambda": 0.26741636854906875, "lambda_min": 0.09570103883743286, "lambda_max": 0.49809879064559937, "S_mean": 0.3006388346354167, "G_mean": 0.35659385030157864, "G_std": 0.16993269099233052, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.524723052978516, "grad_norm": 0.8730507493019104, "tokens_seen": 52928512, "tokens_per_sec": 18332.21912256454, "vram_gb": 5.684933632, "lambda": 0.26777064707130194, "lambda_min": 0.09580542147159576, "lambda_max": 0.49924930930137634, "S_mean": 0.297454833984375, "G_mean": 0.3613991611637175, "G_std": 0.1702465385509034, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.7103729248046875, "grad_norm": 0.8961475491523743, "tokens_seen": 53092352, "tokens_per_sec": 18331.356053905547, "vram_gb": 5.684933632, "lambda": 0.2681756444896261, "lambda_min": 0.09590914100408554, "lambda_max": 0.5003349781036377, "S_mean": 0.2976888020833333, "G_mean": 0.3612810665120681, "G_std": 0.16931156356198093, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.453229904174805, "grad_norm": 0.8758094310760498, "tokens_seen": 53256192, "tokens_per_sec": 18331.371553637513, "vram_gb": 5.684933632, "lambda": 0.2684893946473797, "lambda_min": 0.09599518775939941, "lambda_max": 0.5009340643882751, "S_mean": 0.3021952311197917, "G_mean": 0.35259960498660803, "G_std": 0.16959300559634963, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.5281805992126465, "grad_norm": 0.815061092376709, "tokens_seen": 53420032, "tokens_per_sec": 18330.874136022667, "vram_gb": 5.684933632, "lambda": 0.26887457600484294, "lambda_min": 0.09609492868185043, "lambda_max": 0.5021445751190186, "S_mean": 0.30438232421875, "G_mean": 0.350173053642114, "G_std": 0.16946091898716986, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.5998146533966064, "grad_norm": 0.8318444490432739, "tokens_seen": 53583872, "tokens_per_sec": 18329.945515971707, "vram_gb": 5.684933632, "lambda": 0.2692048115034898, "lambda_min": 0.0960843563079834, "lambda_max": 0.5030306577682495, "S_mean": 0.3047892252604167, "G_mean": 0.3562228417334457, "G_std": 0.16973368357867002, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.381149530410767, "grad_norm": 0.8409176468849182, "tokens_seen": 53747712, "tokens_per_sec": 18329.190510759214, "vram_gb": 5.684933632, "lambda": 0.2695869154607256, "lambda_min": 0.0961274579167366, "lambda_max": 0.503836989402771, "S_mean": 0.2893880208333333, "G_mean": 0.35485830266649526, "G_std": 0.16969317011535168, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.674598217010498, "grad_norm": 0.9392586350440979, "tokens_seen": 53911552, "tokens_per_sec": 18328.216691149315, "vram_gb": 5.684933632, "lambda": 0.26994226935009163, "lambda_min": 0.09601596742868423, "lambda_max": 0.5047003626823425, "S_mean": 0.2870686848958333, "G_mean": 0.3564275673901041, "G_std": 0.16984588319125274, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.493455410003662, "grad_norm": 0.9062103033065796, "tokens_seen": 54075392, "tokens_per_sec": 18326.983815903335, "vram_gb": 5.684933632, "lambda": 0.2703241373722752, "lambda_min": 0.0960843563079834, "lambda_max": 0.5058194398880005, "S_mean": 0.2924245198567708, "G_mean": 0.35579354758374393, "G_std": 0.16995018107506135, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.766320705413818, "grad_norm": 0.9532743096351624, "tokens_seen": 54239232, "tokens_per_sec": 18326.69001739204, "vram_gb": 5.684933632, "lambda": 0.2706438070163131, "lambda_min": 0.09614817798137665, "lambda_max": 0.5064541101455688, "S_mean": 0.3048807779947917, "G_mean": 0.3596278147306293, "G_std": 0.1710796591360122, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.435535192489624, "grad_norm": 0.8780087232589722, "tokens_seen": 54403072, "tokens_per_sec": 18326.446948093748, "vram_gb": 5.684933632, "lambda": 0.2710222726066907, "lambda_min": 0.09630485624074936, "lambda_max": 0.507520318031311, "S_mean": 0.29254150390625, "G_mean": 0.3555768684794505, "G_std": 0.16897200567958257, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.452692747116089, "grad_norm": 0.8369721174240112, "tokens_seen": 54566912, "tokens_per_sec": 18326.14209171968, "vram_gb": 5.684933632, "lambda": 0.27137570021053153, "lambda_min": 0.0963607132434845, "lambda_max": 0.5086318850517273, "S_mean": 0.276702880859375, "G_mean": 0.353838828081886, "G_std": 0.17084396118298173, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.42890191078186, "grad_norm": 0.8583846092224121, "tokens_seen": 54730752, "tokens_per_sec": 18326.25024264617, "vram_gb": 5.684933632, "lambda": 0.27170228337248165, "lambda_min": 0.09645355492830276, "lambda_max": 0.5096454620361328, "S_mean": 0.2938639322916667, "G_mean": 0.35187217003355425, "G_std": 0.16822658494735757, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.60208797454834, "grad_norm": 0.9203568696975708, "tokens_seen": 54894592, "tokens_per_sec": 18326.395490007864, "vram_gb": 5.684933632, "lambda": 0.27201268666734296, "lambda_min": 0.09638699889183044, "lambda_max": 0.5104402303695679, "S_mean": 0.2989501953125, "G_mean": 0.3551136131864041, "G_std": 0.16918375917399922, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.483799934387207, "grad_norm": 0.867340087890625, "tokens_seen": 55058432, "tokens_per_sec": 18327.004214512246, "vram_gb": 5.684933632, "lambda": 0.2723744288086891, "lambda_min": 0.0962277427315712, "lambda_max": 0.511378288269043, "S_mean": 0.2939249674479167, "G_mean": 0.35324171899507445, "G_std": 0.17038303256655732, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.478208780288696, "grad_norm": 0.8349854350090027, "tokens_seen": 55222272, "tokens_per_sec": 18326.447362602452, "vram_gb": 5.684933632, "lambda": 0.2727000055213769, "lambda_min": 0.09614511579275131, "lambda_max": 0.5121943950653076, "S_mean": 0.2941996256510417, "G_mean": 0.3596903880437215, "G_std": 0.16898885842723152, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.578546047210693, "grad_norm": 0.8693476319313049, "tokens_seen": 55386112, "tokens_per_sec": 18326.425541683475, "vram_gb": 5.684933632, "lambda": 0.2730521271005273, "lambda_min": 0.09604644775390625, "lambda_max": 0.5130585432052612, "S_mean": 0.2973429361979167, "G_mean": 0.3546933950856328, "G_std": 0.17037763446569443, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.4449663162231445, "grad_norm": 0.8391931056976318, "tokens_seen": 55549952, "tokens_per_sec": 18326.623501723592, "vram_gb": 5.684933632, "lambda": 0.27343341584006947, "lambda_min": 0.0960756167769432, "lambda_max": 0.5140879154205322, "S_mean": 0.3015696207682292, "G_mean": 0.3533131289295852, "G_std": 0.16808762525518736, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.378257513046265, "grad_norm": 0.8294309973716736, "tokens_seen": 55713792, "tokens_per_sec": 18326.69035795114, "vram_gb": 5.684933632, "lambda": 0.2737687186648448, "lambda_min": 0.096056267619133, "lambda_max": 0.5151908993721008, "S_mean": 0.3081156412760417, "G_mean": 0.35417149293546873, "G_std": 0.16936674892591932, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.59564208984375, "grad_norm": 0.8484142422676086, "tokens_seen": 55877632, "tokens_per_sec": 18327.053906885714, "vram_gb": 5.684933632, "lambda": 0.27411294883737963, "lambda_min": 0.09608551114797592, "lambda_max": 0.515854001045227, "S_mean": 0.302520751953125, "G_mean": 0.3540860185554872, "G_std": 0.16993586307701966, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.411941051483154, "grad_norm": 0.8295544981956482, "tokens_seen": 56041472, "tokens_per_sec": 18327.52651576866, "vram_gb": 5.684933632, "lambda": 0.27452017987767857, "lambda_min": 0.0961444228887558, "lambda_max": 0.516830563545227, "S_mean": 0.3010762532552083, "G_mean": 0.3474682103842497, "G_std": 0.16741058385620514, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.608802795410156, "grad_norm": 0.8196229338645935, "tokens_seen": 56205312, "tokens_per_sec": 18328.00553154248, "vram_gb": 5.684933632, "lambda": 0.27485172760983306, "lambda_min": 0.096145398914814, "lambda_max": 0.5177969336509705, "S_mean": 0.2969156901041667, "G_mean": 0.35698158650969464, "G_std": 0.1698688045144081, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.515182018280029, "grad_norm": 0.8597305417060852, "tokens_seen": 56369152, "tokens_per_sec": 18328.30892583414, "vram_gb": 5.684933632, "lambda": 0.2751132156699896, "lambda_min": 0.09624384343624115, "lambda_max": 0.5186829566955566, "S_mean": 0.31536865234375, "G_mean": 0.35282760408396524, "G_std": 0.16921437648124993, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.454292297363281, "grad_norm": 0.8509847521781921, "tokens_seen": 56532992, "tokens_per_sec": 18328.71514765606, "vram_gb": 5.684933632, "lambda": 0.2753921998664737, "lambda_min": 0.09641770273447037, "lambda_max": 0.5194081664085388, "S_mean": 0.3081766764322917, "G_mean": 0.3543674040120095, "G_std": 0.16986109471569458, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.477827548980713, "grad_norm": 0.85821133852005, "tokens_seen": 56696832, "tokens_per_sec": 18328.954275154498, "vram_gb": 5.684933632, "lambda": 0.2757540304834644, "lambda_min": 0.09654688835144043, "lambda_max": 0.5202457904815674, "S_mean": 0.312408447265625, "G_mean": 0.3563378950736175, "G_std": 0.16679848629670838, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.54169225692749, "grad_norm": 0.8933723568916321, "tokens_seen": 56860672, "tokens_per_sec": 18329.659191731655, "vram_gb": 5.684933632, "lambda": 0.27611275172481936, "lambda_min": 0.09654907137155533, "lambda_max": 0.5209624767303467, "S_mean": 0.3134053548177083, "G_mean": 0.3531963441055268, "G_std": 0.16866571091425916, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.506178855895996, "grad_norm": 0.8782209157943726, "tokens_seen": 57024512, "tokens_per_sec": 18330.069492581504, "vram_gb": 5.684933632, "lambda": 0.27656918826202553, "lambda_min": 0.09658647328615189, "lambda_max": 0.5219686031341553, "S_mean": 0.3113505045572917, "G_mean": 0.3506823836360127, "G_std": 0.16800539707764983, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.480563163757324, "grad_norm": 0.802361786365509, "tokens_seen": 57188352, "tokens_per_sec": 18330.77923852975, "vram_gb": 5.684933632, "lambda": 0.2768572609250744, "lambda_min": 0.09656879305839539, "lambda_max": 0.5229272842407227, "S_mean": 0.31121826171875, "G_mean": 0.35123895900323987, "G_std": 0.1685114479623735, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.3347861766815186, "grad_norm": 0.874756395816803, "tokens_seen": 57352192, "tokens_per_sec": 18330.962859503907, "vram_gb": 5.684933632, "lambda": 0.27720694181819755, "lambda_min": 0.09660448879003525, "lambda_max": 0.5238324403762817, "S_mean": 0.3059590657552083, "G_mean": 0.35148461527811986, "G_std": 0.17067184229381382, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.605238533020019, "val_ppl": 100.00683493676394, "lambda": 0.27722141705453396, "lambda_min": 0.09661221504211426, "lambda_max": 0.5238764882087708, "lambda_per_layer": [0.17271636426448822, 0.1424594670534134, 0.14832183718681335, 0.1309552788734436, 0.11932422965765, 0.09661221504211426, 0.12045169621706009, 0.16581012308597565, 0.22083780169487, 0.46085256338119507, 0.3955264091491699, 0.3319457471370697, 0.3124484717845917, 0.5238764882087708, 0.39019522070884705, 0.47065091133117676, 0.46500247716903687, 0.36182501912117004, 0.2743753492832184, 0.3702913820743561, 0.32015442848205566, 0.19357985258102417, 0.264886736869812, 0.2002139389514923], "S_mean": 0.3069051106770833, "G_mean": 0.35780623702642816, "G_std": 0.17075583230083188, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.674036026000977, "grad_norm": 0.8810184001922607, "tokens_seen": 57516032, "tokens_per_sec": 18321.091089615406, "vram_gb": 5.684933632, "lambda": 0.27754557970911264, "lambda_min": 0.09668924659490585, "lambda_max": 0.5246123671531677, "S_mean": 0.3031412760416667, "G_mean": 0.35182596595647436, "G_std": 0.16929303055318692, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.663238525390625, "grad_norm": 0.8990249633789062, "tokens_seen": 57679872, "tokens_per_sec": 18321.169483172107, "vram_gb": 5.684933632, "lambda": 0.27790394456436235, "lambda_min": 0.09668301045894623, "lambda_max": 0.5254614353179932, "S_mean": 0.3240763346354167, "G_mean": 0.3506757887856414, "G_std": 0.16891097859479487, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.56295919418335, "grad_norm": 0.8047946691513062, "tokens_seen": 57843712, "tokens_per_sec": 18321.07357839316, "vram_gb": 5.684933632, "lambda": 0.27821611799299717, "lambda_min": 0.09658251702785492, "lambda_max": 0.5262360572814941, "S_mean": 0.2996622721354167, "G_mean": 0.35237662587314844, "G_std": 0.17048376395056644, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.537740468978882, "grad_norm": 0.8775050640106201, "tokens_seen": 58007552, "tokens_per_sec": 18320.96693953279, "vram_gb": 5.684933632, "lambda": 0.27848599168161553, "lambda_min": 0.0965249314904213, "lambda_max": 0.5273041725158691, "S_mean": 0.3131103515625, "G_mean": 0.35679119220003486, "G_std": 0.17036455225509903, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.555546998977661, "grad_norm": 0.8682630062103271, "tokens_seen": 58171392, "tokens_per_sec": 18321.092468138435, "vram_gb": 5.684933632, "lambda": 0.2788785866772135, "lambda_min": 0.09657693654298782, "lambda_max": 0.5285927057266235, "S_mean": 0.294464111328125, "G_mean": 0.35246510120729607, "G_std": 0.17052766657434404, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.73469352722168, "grad_norm": 0.919460117816925, "tokens_seen": 58335232, "tokens_per_sec": 18321.261901664442, "vram_gb": 5.684933632, "lambda": 0.2792750112712383, "lambda_min": 0.09653594344854355, "lambda_max": 0.5297330021858215, "S_mean": 0.3124796549479167, "G_mean": 0.35737204101557535, "G_std": 0.16885581077076495, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.430366277694702, "grad_norm": 0.8748482465744019, "tokens_seen": 58499072, "tokens_per_sec": 18321.11484408015, "vram_gb": 5.684933632, "lambda": 0.27957310372342664, "lambda_min": 0.09652043133974075, "lambda_max": 0.5305981636047363, "S_mean": 0.3034006754557292, "G_mean": 0.34507559371801716, "G_std": 0.16957159154117107, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.482868671417236, "grad_norm": 0.9166277050971985, "tokens_seen": 58662912, "tokens_per_sec": 18321.33525466511, "vram_gb": 5.684933632, "lambda": 0.27983555756509304, "lambda_min": 0.09645840525627136, "lambda_max": 0.531312108039856, "S_mean": 0.3001607259114583, "G_mean": 0.3532575760812809, "G_std": 0.16967576858587563, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.543825626373291, "grad_norm": 1.0162073373794556, "tokens_seen": 58826752, "tokens_per_sec": 18321.08794444333, "vram_gb": 5.684933632, "lambda": 0.28015957586467266, "lambda_min": 0.0964200347661972, "lambda_max": 0.5320830941200256, "S_mean": 0.3065999348958333, "G_mean": 0.35498759523034096, "G_std": 0.16968584603940448, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.538828134536743, "grad_norm": 0.9060901403427124, "tokens_seen": 58990592, "tokens_per_sec": 18321.352312369174, "vram_gb": 5.684933632, "lambda": 0.2805316358183821, "lambda_min": 0.09631721675395966, "lambda_max": 0.5332205295562744, "S_mean": 0.310699462890625, "G_mean": 0.3545356549633046, "G_std": 0.16718805177758136, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.3321850299835205, "grad_norm": 0.8865309953689575, "tokens_seen": 59154432, "tokens_per_sec": 18321.664518126196, "vram_gb": 5.684933632, "lambda": 0.2808396841088931, "lambda_min": 0.09628876298666, "lambda_max": 0.5343836545944214, "S_mean": 0.2997538248697917, "G_mean": 0.3546012109145522, "G_std": 0.16898667028484246, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.407914161682129, "grad_norm": 0.81253981590271, "tokens_seen": 59318272, "tokens_per_sec": 18321.920140870217, "vram_gb": 5.684933632, "lambda": 0.28116820845752954, "lambda_min": 0.09632070362567902, "lambda_max": 0.5356515645980835, "S_mean": 0.302001953125, "G_mean": 0.3467804682441056, "G_std": 0.16860986415607235, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.522027969360352, "grad_norm": 0.8564781546592712, "tokens_seen": 59482112, "tokens_per_sec": 18322.08322180477, "vram_gb": 5.684933632, "lambda": 0.2814496175075571, "lambda_min": 0.0963512435555458, "lambda_max": 0.5366395711898804, "S_mean": 0.2975209554036458, "G_mean": 0.3451229579125841, "G_std": 0.16877101020266613, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.42173433303833, "grad_norm": 0.8447866439819336, "tokens_seen": 59645952, "tokens_per_sec": 18322.386739847898, "vram_gb": 5.684933632, "lambda": 0.28178582650919753, "lambda_min": 0.09635361284017563, "lambda_max": 0.5373758673667908, "S_mean": 0.2947998046875, "G_mean": 0.35021370959778625, "G_std": 0.16769595133761564, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.575928211212158, "grad_norm": 0.8365619778633118, "tokens_seen": 59809792, "tokens_per_sec": 18322.57594946521, "vram_gb": 5.684933632, "lambda": 0.2821484698603551, "lambda_min": 0.09629152715206146, "lambda_max": 0.5384105443954468, "S_mean": 0.2982126871744792, "G_mean": 0.35073534756277996, "G_std": 0.16706638154573739, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.396445035934448, "grad_norm": 0.8222199082374573, "tokens_seen": 59973632, "tokens_per_sec": 18322.95592855577, "vram_gb": 5.684933632, "lambda": 0.28248908339689177, "lambda_min": 0.09621350467205048, "lambda_max": 0.5390745401382446, "S_mean": 0.3011474609375, "G_mean": 0.3511496167629957, "G_std": 0.1683927800040692, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.477864742279053, "grad_norm": 0.8981546759605408, "tokens_seen": 60137472, "tokens_per_sec": 18323.52874886583, "vram_gb": 5.684933632, "lambda": 0.2827464962999026, "lambda_min": 0.09618192911148071, "lambda_max": 0.5399271249771118, "S_mean": 0.2964426676432292, "G_mean": 0.35127765328312915, "G_std": 0.16870075254701078, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.543715715408325, "grad_norm": 0.8570763468742371, "tokens_seen": 60301312, "tokens_per_sec": 18323.46965792323, "vram_gb": 5.684933632, "lambda": 0.28300652373582125, "lambda_min": 0.096251480281353, "lambda_max": 0.5410611629486084, "S_mean": 0.3065439860026042, "G_mean": 0.34860042622312903, "G_std": 0.16761517974858484, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.522404193878174, "grad_norm": 0.8833906650543213, "tokens_seen": 60465152, "tokens_per_sec": 18324.259608745815, "vram_gb": 5.684933632, "lambda": 0.2832796877870957, "lambda_min": 0.09629824012517929, "lambda_max": 0.5418787002563477, "S_mean": 0.2955525716145833, "G_mean": 0.35020428981321555, "G_std": 0.16776703437790275, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.394702196121216, "grad_norm": 0.8391278982162476, "tokens_seen": 60628992, "tokens_per_sec": 18324.618585309065, "vram_gb": 5.684933632, "lambda": 0.2835684133072694, "lambda_min": 0.09635557979345322, "lambda_max": 0.5427585244178772, "S_mean": 0.2913920084635417, "G_mean": 0.3501892539982994, "G_std": 0.16766607800188163, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.466414451599121, "grad_norm": 0.8855609893798828, "tokens_seen": 60792832, "tokens_per_sec": 18324.965550313864, "vram_gb": 5.684933632, "lambda": 0.28387489076703787, "lambda_min": 0.09643704444169998, "lambda_max": 0.5436509847640991, "S_mean": 0.3112080891927083, "G_mean": 0.3570984903878222, "G_std": 0.1691896466848751, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.375182151794434, "grad_norm": 0.8586657047271729, "tokens_seen": 60956672, "tokens_per_sec": 18325.52870250733, "vram_gb": 5.684933632, "lambda": 0.28414782260855037, "lambda_min": 0.09646496921777725, "lambda_max": 0.544175922870636, "S_mean": 0.291839599609375, "G_mean": 0.3515256590520342, "G_std": 0.16856055124662817, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.541701316833496, "grad_norm": 0.9347270131111145, "tokens_seen": 61120512, "tokens_per_sec": 18325.40070177284, "vram_gb": 5.684933632, "lambda": 0.28445135491589707, "lambda_min": 0.0964452251791954, "lambda_max": 0.5452515482902527, "S_mean": 0.2853495279947917, "G_mean": 0.3482652798605462, "G_std": 0.1684170727773259, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.40548849105835, "grad_norm": 0.908216655254364, "tokens_seen": 61284352, "tokens_per_sec": 18326.20961848227, "vram_gb": 5.684933632, "lambda": 0.2847857366626461, "lambda_min": 0.09633835405111313, "lambda_max": 0.5462756156921387, "S_mean": 0.2986043294270833, "G_mean": 0.3437471923728784, "G_std": 0.16745847223016122, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.465708017349243, "grad_norm": 0.8552470803260803, "tokens_seen": 61448192, "tokens_per_sec": 18326.710490492234, "vram_gb": 5.684933632, "lambda": 0.28506167605519295, "lambda_min": 0.0963582992553711, "lambda_max": 0.5474915504455566, "S_mean": 0.2861531575520833, "G_mean": 0.3495621548499912, "G_std": 0.16981246531940997, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.4983532428741455, "grad_norm": 0.8345365524291992, "tokens_seen": 61612032, "tokens_per_sec": 18326.53632766953, "vram_gb": 5.684933632, "lambda": 0.2853330882887046, "lambda_min": 0.09646804630756378, "lambda_max": 0.5483404994010925, "S_mean": 0.30023193359375, "G_mean": 0.3481289000871281, "G_std": 0.16837740425641337, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.387411832809448, "grad_norm": 0.9186060428619385, "tokens_seen": 61775872, "tokens_per_sec": 18327.067470486323, "vram_gb": 5.684933632, "lambda": 0.28560480816910666, "lambda_min": 0.09665609896183014, "lambda_max": 0.5489232540130615, "S_mean": 0.3131510416666667, "G_mean": 0.35537044362475473, "G_std": 0.16880838829092681, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.309545993804932, "grad_norm": 0.9012508392333984, "tokens_seen": 61939712, "tokens_per_sec": 18327.142788804955, "vram_gb": 5.684933632, "lambda": 0.28587047072748345, "lambda_min": 0.09667782485485077, "lambda_max": 0.5498606562614441, "S_mean": 0.3064473470052083, "G_mean": 0.34432211029343307, "G_std": 0.16811029876892766, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.5787553787231445, "grad_norm": 0.8977927565574646, "tokens_seen": 62103552, "tokens_per_sec": 18327.26917774334, "vram_gb": 5.684933632, "lambda": 0.2860917557651798, "lambda_min": 0.09665259718894958, "lambda_max": 0.550513505935669, "S_mean": 0.30841064453125, "G_mean": 0.35248891978214186, "G_std": 0.16987245234971246, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.3381195068359375, "grad_norm": 0.8628621697425842, "tokens_seen": 62267392, "tokens_per_sec": 18327.974654858932, "vram_gb": 5.684933632, "lambda": 0.2863617753610015, "lambda_min": 0.09664300084114075, "lambda_max": 0.5508347749710083, "S_mean": 0.3093465169270833, "G_mean": 0.3448391465159754, "G_std": 0.16780321625992656, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.4279584884643555, "grad_norm": 0.8893287777900696, "tokens_seen": 62431232, "tokens_per_sec": 18328.253343950884, "vram_gb": 5.684933632, "lambda": 0.286680169403553, "lambda_min": 0.09672051668167114, "lambda_max": 0.5516281127929688, "S_mean": 0.2974853515625, "G_mean": 0.3443421679548919, "G_std": 0.16689639592853686, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.3886988162994385, "grad_norm": 0.8880696892738342, "tokens_seen": 62595072, "tokens_per_sec": 18328.28388976093, "vram_gb": 5.684933632, "lambda": 0.28696960924814147, "lambda_min": 0.09681624174118042, "lambda_max": 0.5522632002830505, "S_mean": 0.3029378255208333, "G_mean": 0.3516507416497916, "G_std": 0.16679601006520292, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.422152996063232, "grad_norm": 0.8666843771934509, "tokens_seen": 62758912, "tokens_per_sec": 18328.424149004208, "vram_gb": 5.684933632, "lambda": 0.28722730465233326, "lambda_min": 0.09684103727340698, "lambda_max": 0.5529271960258484, "S_mean": 0.288604736328125, "G_mean": 0.350923239486292, "G_std": 0.1679200625512749, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.617302417755127, "grad_norm": 0.9162853360176086, "tokens_seen": 62922752, "tokens_per_sec": 18328.53637533497, "vram_gb": 5.684933632, "lambda": 0.28749151310573023, "lambda_min": 0.09686560183763504, "lambda_max": 0.553759753704071, "S_mean": 0.290802001953125, "G_mean": 0.3470952081649254, "G_std": 0.16791068902239203, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.56340765953064, "grad_norm": 0.926842451095581, "tokens_seen": 63086592, "tokens_per_sec": 18328.817595893302, "vram_gb": 5.684933632, "lambda": 0.28774813562631607, "lambda_min": 0.09691017866134644, "lambda_max": 0.5546374320983887, "S_mean": 0.3030802408854167, "G_mean": 0.35302536911331117, "G_std": 0.16799757070839405, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.549899339675903, "grad_norm": 0.8780343532562256, "tokens_seen": 63250432, "tokens_per_sec": 18328.479781638307, "vram_gb": 5.684933632, "lambda": 0.2880437957743804, "lambda_min": 0.0969633087515831, "lambda_max": 0.5556440353393555, "S_mean": 0.3084716796875, "G_mean": 0.3478489266708493, "G_std": 0.16748909993718067, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.474154472351074, "grad_norm": 0.9115309119224548, "tokens_seen": 63414272, "tokens_per_sec": 18328.547981634572, "vram_gb": 5.684933632, "lambda": 0.2883159515137474, "lambda_min": 0.09686954319477081, "lambda_max": 0.5566485524177551, "S_mean": 0.3020121256510417, "G_mean": 0.3537479277389745, "G_std": 0.16729347330207625, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.466527462005615, "grad_norm": 0.839059054851532, "tokens_seen": 63578112, "tokens_per_sec": 18329.124863034078, "vram_gb": 5.684933632, "lambda": 0.28854901747157174, "lambda_min": 0.09687651693820953, "lambda_max": 0.5572624802589417, "S_mean": 0.303863525390625, "G_mean": 0.350671809942772, "G_std": 0.16915020005156597, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.557050704956055, "grad_norm": 0.9143555760383606, "tokens_seen": 63741952, "tokens_per_sec": 18329.085658773027, "vram_gb": 5.684933632, "lambda": 0.288831552490592, "lambda_min": 0.09684178978204727, "lambda_max": 0.5582695007324219, "S_mean": 0.302764892578125, "G_mean": 0.3543751132674515, "G_std": 0.16882271187690398, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.467103004455566, "grad_norm": 0.912281334400177, "tokens_seen": 63905792, "tokens_per_sec": 18329.374455668985, "vram_gb": 5.684933632, "lambda": 0.28906052745878696, "lambda_min": 0.09682121127843857, "lambda_max": 0.5589512586593628, "S_mean": 0.3117472330729167, "G_mean": 0.34827418019995093, "G_std": 0.1671352230478078, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.343945741653442, "grad_norm": 0.8506284952163696, "tokens_seen": 64069632, "tokens_per_sec": 18329.624703481295, "vram_gb": 5.684933632, "lambda": 0.2892943893869718, "lambda_min": 0.09690821915864944, "lambda_max": 0.5595951676368713, "S_mean": 0.3068898518880208, "G_mean": 0.34121584706008434, "G_std": 0.16726436698809266, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.381585121154785, "grad_norm": 0.8955078721046448, "tokens_seen": 64233472, "tokens_per_sec": 18329.822528486315, "vram_gb": 5.684933632, "lambda": 0.2895163893699646, "lambda_min": 0.09691207110881805, "lambda_max": 0.5604269504547119, "S_mean": 0.3028361002604167, "G_mean": 0.34896242277075845, "G_std": 0.16755424455429116, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.370029926300049, "grad_norm": 0.9164637327194214, "tokens_seen": 64397312, "tokens_per_sec": 18330.115841286388, "vram_gb": 5.684933632, "lambda": 0.28979839974393445, "lambda_min": 0.09682799130678177, "lambda_max": 0.5613601207733154, "S_mean": 0.3121744791666667, "G_mean": 0.34943079265455407, "G_std": 0.16638895364788672, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.372319936752319, "grad_norm": 0.8573859930038452, "tokens_seen": 64561152, "tokens_per_sec": 18330.676959101067, "vram_gb": 5.684933632, "lambda": 0.29002845752984285, "lambda_min": 0.09670568257570267, "lambda_max": 0.5620765686035156, "S_mean": 0.3091837565104167, "G_mean": 0.3485886383180817, "G_std": 0.16681090711305538, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.352648973464966, "grad_norm": 0.9284193515777588, "tokens_seen": 64724992, "tokens_per_sec": 18330.958924996714, "vram_gb": 5.684933632, "lambda": 0.2903066373740633, "lambda_min": 0.0966983214020729, "lambda_max": 0.5628877878189087, "S_mean": 0.3069559733072917, "G_mean": 0.35036820895038545, "G_std": 0.16806357873914143, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.494656085968018, "grad_norm": 0.9108421802520752, "tokens_seen": 64888832, "tokens_per_sec": 18331.072840009398, "vram_gb": 5.684933632, "lambda": 0.2905640384803216, "lambda_min": 0.09670335799455643, "lambda_max": 0.5634546875953674, "S_mean": 0.3053792317708333, "G_mean": 0.34873160867330927, "G_std": 0.16762282544126114, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.489526987075806, "grad_norm": 0.8766506314277649, "tokens_seen": 65052672, "tokens_per_sec": 18331.166190704833, "vram_gb": 5.684933632, "lambda": 0.2908319712926944, "lambda_min": 0.0966021865606308, "lambda_max": 0.5640028715133667, "S_mean": 0.3033548990885417, "G_mean": 0.34590139767775935, "G_std": 0.1684516117287179, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.478013277053833, "grad_norm": 0.8479313850402832, "tokens_seen": 65216512, "tokens_per_sec": 18331.227704190936, "vram_gb": 5.684933632, "lambda": 0.2910601558784644, "lambda_min": 0.09655135124921799, "lambda_max": 0.5647762417793274, "S_mean": 0.3072408040364583, "G_mean": 0.3462152830325067, "G_std": 0.16939784803738198, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.489288806915283, "grad_norm": 0.878939688205719, "tokens_seen": 65380352, "tokens_per_sec": 18331.799166983878, "vram_gb": 5.684933632, "lambda": 0.2913198660438259, "lambda_min": 0.09652124345302582, "lambda_max": 0.5655355453491211, "S_mean": 0.295684814453125, "G_mean": 0.34892640309408307, "G_std": 0.16762543353252113, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.550002813339233, "grad_norm": 0.9625228047370911, "tokens_seen": 65544192, "tokens_per_sec": 18332.494070816963, "vram_gb": 5.684933632, "lambda": 0.2915290417149663, "lambda_min": 0.09655745327472687, "lambda_max": 0.5662276744842529, "S_mean": 0.3177642822265625, "G_mean": 0.3398551610298455, "G_std": 0.1663513499467323, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.533394241333008, "val_ppl": 93.07394095904635, "lambda": 0.2915420491869251, "lambda_min": 0.09655968099832535, "lambda_max": 0.566272497177124, "lambda_per_layer": [0.1750088632106781, 0.14305569231510162, 0.15047436952590942, 0.12921226024627686, 0.11903625726699829, 0.09655968099832535, 0.1194465309381485, 0.16796593368053436, 0.22550970315933228, 0.478562593460083, 0.41514286398887634, 0.3536949157714844, 0.3280144929885864, 0.566272497177124, 0.4129827618598938, 0.5052571892738342, 0.496984601020813, 0.3902761936187744, 0.28694725036621094, 0.40087589621543884, 0.34481480717658997, 0.2013206034898758, 0.28253173828125, 0.20706148445606232], "S_mean": 0.3156941731770833, "G_mean": 0.3487743477647503, "G_std": 0.16817266354337335, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.509157657623291, "grad_norm": 0.9200628995895386, "tokens_seen": 65708032, "tokens_per_sec": 18324.1782887801, "vram_gb": 5.684933632, "lambda": 0.291743409819901, "lambda_min": 0.09661928564310074, "lambda_max": 0.5669023394584656, "S_mean": 0.2986043294270833, "G_mean": 0.34269097819924355, "G_std": 0.16769722856891653, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.427502870559692, "grad_norm": 0.8860676884651184, "tokens_seen": 65871872, "tokens_per_sec": 18324.79241350191, "vram_gb": 5.684933632, "lambda": 0.2920057000592351, "lambda_min": 0.09664107114076614, "lambda_max": 0.5673008561134338, "S_mean": 0.3032073974609375, "G_mean": 0.3484524735249579, "G_std": 0.16710635422108075, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.444600343704224, "grad_norm": 0.8867190480232239, "tokens_seen": 66035712, "tokens_per_sec": 18325.174353683302, "vram_gb": 5.684933632, "lambda": 0.29225503249714774, "lambda_min": 0.09664071351289749, "lambda_max": 0.568029522895813, "S_mean": 0.3043924967447917, "G_mean": 0.34696300451954204, "G_std": 0.1683206919891139, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.546114921569824, "grad_norm": 0.8548710346221924, "tokens_seen": 66199552, "tokens_per_sec": 18325.587768208145, "vram_gb": 5.684933632, "lambda": 0.29250295273959637, "lambda_min": 0.09665627032518387, "lambda_max": 0.5688470602035522, "S_mean": 0.3021138509114583, "G_mean": 0.3471501438568036, "G_std": 0.16706748873305818, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.508975267410278, "grad_norm": 0.9126932621002197, "tokens_seen": 66363392, "tokens_per_sec": 18326.160880663392, "vram_gb": 5.684933632, "lambda": 0.2927549481391907, "lambda_min": 0.0966654121875763, "lambda_max": 0.5696718096733093, "S_mean": 0.301300048828125, "G_mean": 0.3518143813901891, "G_std": 0.1685207523405552, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.6400957107543945, "grad_norm": 0.8965667486190796, "tokens_seen": 66527232, "tokens_per_sec": 18326.499020313826, "vram_gb": 5.684933632, "lambda": 0.29296088094512623, "lambda_min": 0.09665638208389282, "lambda_max": 0.5704090595245361, "S_mean": 0.2872772216796875, "G_mean": 0.3529688556833814, "G_std": 0.1684802616170297, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.313551664352417, "grad_norm": 0.8799646496772766, "tokens_seen": 66691072, "tokens_per_sec": 18326.926512731083, "vram_gb": 5.684933632, "lambda": 0.2931942070523898, "lambda_min": 0.09665866941213608, "lambda_max": 0.5712360143661499, "S_mean": 0.2999369303385417, "G_mean": 0.34224673577894765, "G_std": 0.16635564613776901, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.372881174087524, "grad_norm": 0.9311012029647827, "tokens_seen": 66854912, "tokens_per_sec": 18327.313435290496, "vram_gb": 5.684933632, "lambda": 0.2934224000200629, "lambda_min": 0.09666676819324493, "lambda_max": 0.5720582008361816, "S_mean": 0.3299967447916667, "G_mean": 0.35625831759534776, "G_std": 0.16732687673841914, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.477740287780762, "grad_norm": 0.9714037179946899, "tokens_seen": 67018752, "tokens_per_sec": 18327.573450255397, "vram_gb": 5.684933632, "lambda": 0.29367736044029397, "lambda_min": 0.09658525884151459, "lambda_max": 0.5728349685668945, "S_mean": 0.3028106689453125, "G_mean": 0.3520342997896175, "G_std": 0.16703884575205544, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.341613292694092, "grad_norm": 0.9525511264801025, "tokens_seen": 67182592, "tokens_per_sec": 18328.163230768143, "vram_gb": 5.684933632, "lambda": 0.2938813157379627, "lambda_min": 0.09641313552856445, "lambda_max": 0.5733968019485474, "S_mean": 0.3084055582682292, "G_mean": 0.3471233159458886, "G_std": 0.16701326736559471, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.421045303344727, "grad_norm": 0.9123339056968689, "tokens_seen": 67346432, "tokens_per_sec": 18328.16955677605, "vram_gb": 5.684933632, "lambda": 0.2940479554235935, "lambda_min": 0.09635532647371292, "lambda_max": 0.5739959478378296, "S_mean": 0.3047739664713542, "G_mean": 0.3441978127229959, "G_std": 0.1664834312784175, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.371746778488159, "grad_norm": 0.9116755127906799, "tokens_seen": 67510272, "tokens_per_sec": 18328.290556337826, "vram_gb": 5.684933632, "lambda": 0.29426657905181247, "lambda_min": 0.09630800038576126, "lambda_max": 0.5747841000556946, "S_mean": 0.313934326171875, "G_mean": 0.3442156333476305, "G_std": 0.16685472157162926, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.317218780517578, "grad_norm": 0.9200761914253235, "tokens_seen": 67674112, "tokens_per_sec": 18328.417543563453, "vram_gb": 5.684933632, "lambda": 0.2945227064192295, "lambda_min": 0.09631475061178207, "lambda_max": 0.5755949020385742, "S_mean": 0.3319091796875, "G_mean": 0.34589773214732605, "G_std": 0.16569071006961167, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.310061931610107, "grad_norm": 0.8722288608551025, "tokens_seen": 67837952, "tokens_per_sec": 18328.990644133213, "vram_gb": 5.684933632, "lambda": 0.2947280090302229, "lambda_min": 0.0963348001241684, "lambda_max": 0.5762749910354614, "S_mean": 0.30865478515625, "G_mean": 0.34697897468383115, "G_std": 0.1680263984017074, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.518495559692383, "grad_norm": 0.892818033695221, "tokens_seen": 68001792, "tokens_per_sec": 18329.333275876696, "vram_gb": 5.684933632, "lambda": 0.29491738229990005, "lambda_min": 0.09635727107524872, "lambda_max": 0.5769813656806946, "S_mean": 0.3112080891927083, "G_mean": 0.35269507641593617, "G_std": 0.16602528483296433, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.378342866897583, "grad_norm": 0.8926124572753906, "tokens_seen": 68165632, "tokens_per_sec": 18329.318573310466, "vram_gb": 5.684933632, "lambda": 0.2951261727139354, "lambda_min": 0.09642621874809265, "lambda_max": 0.5778791904449463, "S_mean": 0.3002421061197917, "G_mean": 0.3524777806984882, "G_std": 0.1686014315734307, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.44179105758667, "grad_norm": 0.9070444703102112, "tokens_seen": 68329472, "tokens_per_sec": 18329.482439539166, "vram_gb": 5.684933632, "lambda": 0.29532100539654493, "lambda_min": 0.09646613150835037, "lambda_max": 0.5785543322563171, "S_mean": 0.322845458984375, "G_mean": 0.3465559324249625, "G_std": 0.16783104999922216, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.538575649261475, "grad_norm": 0.9308229088783264, "tokens_seen": 68493312, "tokens_per_sec": 18329.570934343625, "vram_gb": 5.684933632, "lambda": 0.2955704800163706, "lambda_min": 0.09648874402046204, "lambda_max": 0.5792238712310791, "S_mean": 0.316619873046875, "G_mean": 0.35293608081216615, "G_std": 0.16695881572862467, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.365241765975952, "grad_norm": 0.9284332394599915, "tokens_seen": 68657152, "tokens_per_sec": 18329.611381474362, "vram_gb": 5.684933632, "lambda": 0.2958135822166999, "lambda_min": 0.09651130437850952, "lambda_max": 0.5800639986991882, "S_mean": 0.3097330729166667, "G_mean": 0.3480631695905079, "G_std": 0.16820396071610352, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.4663496017456055, "grad_norm": 0.9054991602897644, "tokens_seen": 68820992, "tokens_per_sec": 18329.52557741715, "vram_gb": 5.684933632, "lambda": 0.2960570255915324, "lambda_min": 0.09656732529401779, "lambda_max": 0.5803014039993286, "S_mean": 0.3019256591796875, "G_mean": 0.34780398616567254, "G_std": 0.16665909253060818, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.333061218261719, "grad_norm": 0.9042491316795349, "tokens_seen": 68984832, "tokens_per_sec": 18329.802450885778, "vram_gb": 5.684933632, "lambda": 0.29628593878199655, "lambda_min": 0.09654408693313599, "lambda_max": 0.5808206796646118, "S_mean": 0.31488037109375, "G_mean": 0.3485206082308044, "G_std": 0.16768697765655816, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.2976133823394775, "grad_norm": 0.8821161389350891, "tokens_seen": 69148672, "tokens_per_sec": 18329.956026405864, "vram_gb": 5.684933632, "lambda": 0.2965079980591933, "lambda_min": 0.09648381173610687, "lambda_max": 0.5816903114318848, "S_mean": 0.305328369140625, "G_mean": 0.3410729637059073, "G_std": 0.16494595003314316, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.347478866577148, "grad_norm": 1.00315523147583, "tokens_seen": 69312512, "tokens_per_sec": 18330.152734095187, "vram_gb": 5.684933632, "lambda": 0.2967175059020519, "lambda_min": 0.09650444984436035, "lambda_max": 0.582417368888855, "S_mean": 0.3061319986979167, "G_mean": 0.34998001468678314, "G_std": 0.1671259889844805, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.362738847732544, "grad_norm": 0.8826786279678345, "tokens_seen": 69476352, "tokens_per_sec": 18329.942599327518, "vram_gb": 5.684933632, "lambda": 0.29693707699577016, "lambda_min": 0.0964929535984993, "lambda_max": 0.5832706689834595, "S_mean": 0.3086903889973958, "G_mean": 0.34131693560630083, "G_std": 0.16614567286645374, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.381798267364502, "grad_norm": 1.0183563232421875, "tokens_seen": 69640192, "tokens_per_sec": 18329.98289331465, "vram_gb": 5.684933632, "lambda": 0.2971234715854128, "lambda_min": 0.09643522650003433, "lambda_max": 0.5840684175491333, "S_mean": 0.3066151936848958, "G_mean": 0.3461358911978702, "G_std": 0.1675059519863377, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.483554124832153, "grad_norm": 0.9463692307472229, "tokens_seen": 69804032, "tokens_per_sec": 18330.346773902904, "vram_gb": 5.684933632, "lambda": 0.2973256617163618, "lambda_min": 0.0964033454656601, "lambda_max": 0.5845258235931396, "S_mean": 0.3066304524739583, "G_mean": 0.350596243981272, "G_std": 0.16690274506496885, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.554882287979126, "grad_norm": 0.9414798021316528, "tokens_seen": 69967872, "tokens_per_sec": 18330.586237948286, "vram_gb": 5.684933632, "lambda": 0.29755385778844357, "lambda_min": 0.0963720753788948, "lambda_max": 0.5850896239280701, "S_mean": 0.3090616861979167, "G_mean": 0.35058673610910773, "G_std": 0.16771664563566446, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.344965219497681, "grad_norm": 0.892549455165863, "tokens_seen": 70131712, "tokens_per_sec": 18330.86584107742, "vram_gb": 5.684933632, "lambda": 0.29772287607192993, "lambda_min": 0.09631039202213287, "lambda_max": 0.5854331254959106, "S_mean": 0.3144734700520833, "G_mean": 0.3433020083854596, "G_std": 0.16542166781922182, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.458066940307617, "grad_norm": 0.8994714021682739, "tokens_seen": 70295552, "tokens_per_sec": 18330.954029584023, "vram_gb": 5.684933632, "lambda": 0.29789695298920077, "lambda_min": 0.09636764973402023, "lambda_max": 0.5859419107437134, "S_mean": 0.3094431559244792, "G_mean": 0.34225006867200136, "G_std": 0.16635831569631895, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.277991056442261, "grad_norm": 0.9361839890480042, "tokens_seen": 70459392, "tokens_per_sec": 18330.999075551066, "vram_gb": 5.684933632, "lambda": 0.2980712124456962, "lambda_min": 0.09643621742725372, "lambda_max": 0.5868390202522278, "S_mean": 0.3030344645182292, "G_mean": 0.34428219497203827, "G_std": 0.16629081002126136, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.409350872039795, "grad_norm": 0.9525179862976074, "tokens_seen": 70623232, "tokens_per_sec": 18331.00216343829, "vram_gb": 5.684933632, "lambda": 0.2982683734347423, "lambda_min": 0.09642527997493744, "lambda_max": 0.5874794721603394, "S_mean": 0.3091939290364583, "G_mean": 0.3452728179593881, "G_std": 0.16569762211292982, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.478156566619873, "grad_norm": 0.9353529214859009, "tokens_seen": 70787072, "tokens_per_sec": 18330.451225652545, "vram_gb": 5.684933632, "lambda": 0.29846715088933706, "lambda_min": 0.09641990810632706, "lambda_max": 0.5878602266311646, "S_mean": 0.3192545572916667, "G_mean": 0.3466002275235951, "G_std": 0.16623864439316094, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.277865409851074, "grad_norm": 0.9830000996589661, "tokens_seen": 70950912, "tokens_per_sec": 18330.151810056264, "vram_gb": 5.684933632, "lambda": 0.2986529854436715, "lambda_min": 0.09642858803272247, "lambda_max": 0.588141679763794, "S_mean": 0.3232218424479167, "G_mean": 0.3484489099743466, "G_std": 0.16637988085858524, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.374173402786255, "grad_norm": 0.8715775012969971, "tokens_seen": 71114752, "tokens_per_sec": 18329.623236608557, "vram_gb": 5.684933632, "lambda": 0.29891638333598775, "lambda_min": 0.09647402912378311, "lambda_max": 0.5886983871459961, "S_mean": 0.3148244222005208, "G_mean": 0.3482598649182667, "G_std": 0.16616199452740452, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.244588375091553, "grad_norm": 0.933786153793335, "tokens_seen": 71278592, "tokens_per_sec": 18329.493859152128, "vram_gb": 5.684933632, "lambda": 0.29907733233024675, "lambda_min": 0.09652284532785416, "lambda_max": 0.5892209410667419, "S_mean": 0.3096516927083333, "G_mean": 0.34462310932576656, "G_std": 0.16612643003463745, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.496762037277222, "grad_norm": 0.9761075973510742, "tokens_seen": 71442432, "tokens_per_sec": 18329.646953693366, "vram_gb": 5.684933632, "lambda": 0.29926438132921857, "lambda_min": 0.09652762860059738, "lambda_max": 0.5897584557533264, "S_mean": 0.2974955240885417, "G_mean": 0.3491075331500421, "G_std": 0.16645223298110068, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.402791500091553, "grad_norm": 0.987503707408905, "tokens_seen": 71606272, "tokens_per_sec": 18330.391959058004, "vram_gb": 5.684933632, "lambda": 0.2994595052053531, "lambda_min": 0.0964888334274292, "lambda_max": 0.5902677774429321, "S_mean": 0.298919677734375, "G_mean": 0.34809320551964146, "G_std": 0.16605237040979168, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.328028202056885, "grad_norm": 0.9609997868537903, "tokens_seen": 71770112, "tokens_per_sec": 18330.822011846838, "vram_gb": 5.684933632, "lambda": 0.2996217406665285, "lambda_min": 0.0964450091123581, "lambda_max": 0.5906467437744141, "S_mean": 0.31121826171875, "G_mean": 0.34545919850158197, "G_std": 0.16531531481693187, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.386709213256836, "grad_norm": 0.8788910508155823, "tokens_seen": 71933952, "tokens_per_sec": 18330.98464888434, "vram_gb": 5.684933632, "lambda": 0.2998264009753863, "lambda_min": 0.09644478559494019, "lambda_max": 0.5913601517677307, "S_mean": 0.3137003580729167, "G_mean": 0.35151597174505395, "G_std": 0.16549068068464598, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.347972869873047, "grad_norm": 0.9851137399673462, "tokens_seen": 72097792, "tokens_per_sec": 18331.084774145944, "vram_gb": 5.684933632, "lambda": 0.3000129001835982, "lambda_min": 0.09643545001745224, "lambda_max": 0.5919579863548279, "S_mean": 0.3034159342447917, "G_mean": 0.3494852335813145, "G_std": 0.16687496386778852, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.280038356781006, "grad_norm": 0.9335250854492188, "tokens_seen": 72261632, "tokens_per_sec": 18331.038281278084, "vram_gb": 5.684933632, "lambda": 0.3002150778969129, "lambda_min": 0.09641548246145248, "lambda_max": 0.5925268530845642, "S_mean": 0.3169657389322917, "G_mean": 0.34914830008832115, "G_std": 0.16564610740169883, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.476439952850342, "grad_norm": 0.9034263491630554, "tokens_seen": 72425472, "tokens_per_sec": 18331.183012796886, "vram_gb": 5.684933632, "lambda": 0.30038159899413586, "lambda_min": 0.09638399630784988, "lambda_max": 0.5928270220756531, "S_mean": 0.3232014973958333, "G_mean": 0.34759856822590035, "G_std": 0.16495417547412217, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.3415257930755615, "grad_norm": 0.9135031700134277, "tokens_seen": 72589312, "tokens_per_sec": 18331.235515518663, "vram_gb": 5.684933632, "lambda": 0.30054401761541766, "lambda_min": 0.0963149443268776, "lambda_max": 0.5933934450149536, "S_mean": 0.3104960123697917, "G_mean": 0.3435556300294896, "G_std": 0.16597099454763034, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.369323253631592, "grad_norm": 0.920274555683136, "tokens_seen": 72753152, "tokens_per_sec": 18330.92291538007, "vram_gb": 5.684933632, "lambda": 0.3007311848923564, "lambda_min": 0.09632864594459534, "lambda_max": 0.5937654376029968, "S_mean": 0.3067728678385417, "G_mean": 0.34510134455437463, "G_std": 0.16603708329300085, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.172791004180908, "grad_norm": 0.917733371257782, "tokens_seen": 72916992, "tokens_per_sec": 18330.237141867532, "vram_gb": 5.684933632, "lambda": 0.3009409497802456, "lambda_min": 0.09637179225683212, "lambda_max": 0.5943714380264282, "S_mean": 0.3191935221354167, "G_mean": 0.34943100227974355, "G_std": 0.1670802537507067, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.37410831451416, "grad_norm": 0.9214547872543335, "tokens_seen": 73080832, "tokens_per_sec": 18330.382435701486, "vram_gb": 5.684933632, "lambda": 0.30108196288347244, "lambda_min": 0.0963926687836647, "lambda_max": 0.5948249101638794, "S_mean": 0.2981363932291667, "G_mean": 0.3474421286955476, "G_std": 0.16632016631774604, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.387206554412842, "grad_norm": 0.8912387490272522, "tokens_seen": 73244672, "tokens_per_sec": 18330.536089856112, "vram_gb": 5.684933632, "lambda": 0.3012527246028185, "lambda_min": 0.09643454849720001, "lambda_max": 0.595421314239502, "S_mean": 0.31097412109375, "G_mean": 0.3392605792420606, "G_std": 0.1646188800999274, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.389317035675049, "grad_norm": 0.9530846476554871, "tokens_seen": 73408512, "tokens_per_sec": 18330.543882735965, "vram_gb": 5.684933632, "lambda": 0.30139413351813954, "lambda_min": 0.0965087041258812, "lambda_max": 0.595774233341217, "S_mean": 0.3086140950520833, "G_mean": 0.3409686650459965, "G_std": 0.166153059185793, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.442312955856323, "grad_norm": 0.9505814909934998, "tokens_seen": 73572352, "tokens_per_sec": 18330.57366553904, "vram_gb": 5.684933632, "lambda": 0.3016029406959812, "lambda_min": 0.09650935977697372, "lambda_max": 0.5960533618927002, "S_mean": 0.3094685872395833, "G_mean": 0.34582372987642884, "G_std": 0.16670540433066586, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.382970809936523, "grad_norm": 0.9667299389839172, "tokens_seen": 73736192, "tokens_per_sec": 18330.783123374316, "vram_gb": 5.684933632, "lambda": 0.30180559245248634, "lambda_min": 0.09649330377578735, "lambda_max": 0.5966157913208008, "S_mean": 0.30950927734375, "G_mean": 0.34311852324754, "G_std": 0.16730748442932963, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.445945143699646, "val_ppl": 85.28044203144407, "lambda": 0.3018144238740206, "lambda_min": 0.09649284183979034, "lambda_max": 0.5966469049453735, "lambda_per_layer": [0.1770734190940857, 0.1435774862766266, 0.15202395617961884, 0.12880855798721313, 0.11854971945285797, 0.09649284183979034, 0.1192006766796112, 0.16886121034622192, 0.22785989940166473, 0.49118703603744507, 0.4271978735923767, 0.36891260743141174, 0.33785733580589294, 0.5966469049453735, 0.42900606989860535, 0.531028687953949, 0.5241233706474304, 0.4114586114883423, 0.29600369930267334, 0.42147600650787354, 0.3630968928337097, 0.20688819885253906, 0.2941163182258606, 0.21209879219532013], "S_mean": 0.3104197184244792, "G_mean": 0.34168482820192975, "G_std": 0.1673733504333844, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.4482948780059814, "grad_norm": 0.9660794734954834, "tokens_seen": 73900032, "tokens_per_sec": 18323.25886338058, "vram_gb": 5.684933632, "lambda": 0.30196432396769524, "lambda_min": 0.09644907712936401, "lambda_max": 0.5972947478294373, "S_mean": 0.3246866861979167, "G_mean": 0.3493578031969567, "G_std": 0.16544445930048823, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.394543170928955, "grad_norm": 0.9199790358543396, "tokens_seen": 74063872, "tokens_per_sec": 18323.245109319876, "vram_gb": 5.684933632, "lambda": 0.30213461723178625, "lambda_min": 0.09643233567476273, "lambda_max": 0.5980154275894165, "S_mean": 0.3062286376953125, "G_mean": 0.3451639513174693, "G_std": 0.16774680224868158, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.126825571060181, "grad_norm": 0.9532187581062317, "tokens_seen": 74227712, "tokens_per_sec": 18323.489305022034, "vram_gb": 5.684933632, "lambda": 0.30233525422712165, "lambda_min": 0.09644542634487152, "lambda_max": 0.5985507369041443, "S_mean": 0.30047607421875, "G_mean": 0.3462749949345986, "G_std": 0.16656022681854665, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.4717230796813965, "grad_norm": 0.9930275082588196, "tokens_seen": 74391552, "tokens_per_sec": 18323.366110211115, "vram_gb": 5.684933632, "lambda": 0.3025022568181157, "lambda_min": 0.09651017189025879, "lambda_max": 0.5988656878471375, "S_mean": 0.3098805745442708, "G_mean": 0.3491213978268206, "G_std": 0.16714121626379588, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.4248998165130615, "grad_norm": 0.9399372339248657, "tokens_seen": 74555392, "tokens_per_sec": 18323.6799482306, "vram_gb": 5.684933632, "lambda": 0.30264529772102833, "lambda_min": 0.0965157151222229, "lambda_max": 0.5993637442588806, "S_mean": 0.2993977864583333, "G_mean": 0.3452247752963255, "G_std": 0.16490211500786245, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.264117240905762, "grad_norm": 0.910805344581604, "tokens_seen": 74719232, "tokens_per_sec": 18323.725976015467, "vram_gb": 5.684933632, "lambda": 0.30278461674849194, "lambda_min": 0.09657309949398041, "lambda_max": 0.5997434854507446, "S_mean": 0.3043416341145833, "G_mean": 0.34526406282869476, "G_std": 0.167099326926594, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.206659555435181, "grad_norm": 0.8695467710494995, "tokens_seen": 74883072, "tokens_per_sec": 18323.744383341804, "vram_gb": 5.684933632, "lambda": 0.3029381713519494, "lambda_min": 0.09657333791255951, "lambda_max": 0.6003913283348083, "S_mean": 0.3050028483072917, "G_mean": 0.345937874323378, "G_std": 0.16606468963436782, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.453244686126709, "grad_norm": 0.9750369787216187, "tokens_seen": 75046912, "tokens_per_sec": 18323.770058734975, "vram_gb": 5.684933632, "lambda": 0.3031132124985258, "lambda_min": 0.0965750515460968, "lambda_max": 0.6008741855621338, "S_mean": 0.3072916666666667, "G_mean": 0.34319375238070887, "G_std": 0.16589667888668677, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.4546990394592285, "grad_norm": 0.8775314688682556, "tokens_seen": 75210752, "tokens_per_sec": 18323.904989801333, "vram_gb": 5.684933632, "lambda": 0.3032781636963288, "lambda_min": 0.09660618007183075, "lambda_max": 0.6014065742492676, "S_mean": 0.3021138509114583, "G_mean": 0.3460802521246175, "G_std": 0.1657711830145369, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.2482640743255615, "grad_norm": 0.9302464127540588, "tokens_seen": 75374592, "tokens_per_sec": 18323.68077523342, "vram_gb": 5.684933632, "lambda": 0.30346440865347785, "lambda_min": 0.09661030024290085, "lambda_max": 0.6020782589912415, "S_mean": 0.3143819173177083, "G_mean": 0.3488367274403572, "G_std": 0.16622686331781247, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.447941541671753, "grad_norm": 0.9630886912345886, "tokens_seen": 75538432, "tokens_per_sec": 18323.790568161457, "vram_gb": 5.684933632, "lambda": 0.3036352414637804, "lambda_min": 0.09659922122955322, "lambda_max": 0.6025406122207642, "S_mean": 0.30364990234375, "G_mean": 0.3469193447381258, "G_std": 0.1654435209929943, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.468387126922607, "grad_norm": 0.9776013493537903, "tokens_seen": 75702272, "tokens_per_sec": 18323.947683749357, "vram_gb": 5.684933632, "lambda": 0.30379374977201223, "lambda_min": 0.09655694663524628, "lambda_max": 0.6029907464981079, "S_mean": 0.3152262369791667, "G_mean": 0.3420507526025176, "G_std": 0.16597705023984113, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.3262619972229, "grad_norm": 0.914797842502594, "tokens_seen": 75866112, "tokens_per_sec": 18324.07019538631, "vram_gb": 5.684933632, "lambda": 0.30395191473265487, "lambda_min": 0.09648904949426651, "lambda_max": 0.6036463379859924, "S_mean": 0.316619873046875, "G_mean": 0.344086202327162, "G_std": 0.1648786417208612, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.342578411102295, "grad_norm": 0.9436485767364502, "tokens_seen": 76029952, "tokens_per_sec": 18324.14449506069, "vram_gb": 5.684933632, "lambda": 0.30414252324650687, "lambda_min": 0.09646183997392654, "lambda_max": 0.6041256785392761, "S_mean": 0.3155415852864583, "G_mean": 0.3478749340089659, "G_std": 0.16644776690130433, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.494268178939819, "grad_norm": 0.9560967087745667, "tokens_seen": 76193792, "tokens_per_sec": 18324.517726572427, "vram_gb": 5.684933632, "lambda": 0.30428841927399236, "lambda_min": 0.0964798852801323, "lambda_max": 0.6045872569084167, "S_mean": 0.31683349609375, "G_mean": 0.3504452692189564, "G_std": 0.1653730688461413, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.276040315628052, "grad_norm": 0.8866302371025085, "tokens_seen": 76357632, "tokens_per_sec": 18324.349355097747, "vram_gb": 5.684933632, "lambda": 0.304451921954751, "lambda_min": 0.09651308506727219, "lambda_max": 0.6050044894218445, "S_mean": 0.3094889322916667, "G_mean": 0.34756031710033614, "G_std": 0.16560138234247765, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.484017848968506, "grad_norm": 0.9658684730529785, "tokens_seen": 76521472, "tokens_per_sec": 18324.30372330916, "vram_gb": 5.684933632, "lambda": 0.30461473297327757, "lambda_min": 0.09651574492454529, "lambda_max": 0.6054872274398804, "S_mean": 0.3030598958333333, "G_mean": 0.3437775594647974, "G_std": 0.16464868381929895, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.295005798339844, "grad_norm": 0.9792251586914062, "tokens_seen": 76685312, "tokens_per_sec": 18324.58835077133, "vram_gb": 5.684933632, "lambda": 0.30475818552076817, "lambda_min": 0.09650148451328278, "lambda_max": 0.6058830618858337, "S_mean": 0.3044230143229167, "G_mean": 0.33928323568155366, "G_std": 0.16555223753675818, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.439047336578369, "grad_norm": 0.8929833173751831, "tokens_seen": 76849152, "tokens_per_sec": 18324.644320773376, "vram_gb": 5.684933632, "lambda": 0.30490000763287145, "lambda_min": 0.09656142443418503, "lambda_max": 0.6062915921211243, "S_mean": 0.3050028483072917, "G_mean": 0.34434664839257795, "G_std": 0.16560348394947746, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.214151859283447, "grad_norm": 0.9825838804244995, "tokens_seen": 77012992, "tokens_per_sec": 18324.908181621457, "vram_gb": 5.684933632, "lambda": 0.3050534601012866, "lambda_min": 0.09654795378446579, "lambda_max": 0.6067785620689392, "S_mean": 0.3060709635416667, "G_mean": 0.34464448085054755, "G_std": 0.16472669108770788, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.241849899291992, "grad_norm": 0.9776525497436523, "tokens_seen": 77176832, "tokens_per_sec": 18324.94820630022, "vram_gb": 5.684933632, "lambda": 0.30519053371002275, "lambda_min": 0.09657593071460724, "lambda_max": 0.6072090864181519, "S_mean": 0.3077392578125, "G_mean": 0.33988005854189396, "G_std": 0.16272712809344134, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.288963317871094, "grad_norm": 1.0879182815551758, "tokens_seen": 77340672, "tokens_per_sec": 18325.133696194676, "vram_gb": 5.684933632, "lambda": 0.30533194666107494, "lambda_min": 0.09650973230600357, "lambda_max": 0.6073725819587708, "S_mean": 0.320404052734375, "G_mean": 0.34042999140607816, "G_std": 0.16460196875656644, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.383628606796265, "grad_norm": 0.9353068470954895, "tokens_seen": 77504512, "tokens_per_sec": 18325.365372770055, "vram_gb": 5.684933632, "lambda": 0.30548406516512233, "lambda_min": 0.09652072191238403, "lambda_max": 0.6077187657356262, "S_mean": 0.3057149251302083, "G_mean": 0.34283583890646696, "G_std": 0.16501355341946086, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.377029180526733, "grad_norm": 0.9519214630126953, "tokens_seen": 77668352, "tokens_per_sec": 18325.50788347513, "vram_gb": 5.684933632, "lambda": 0.3055927200863759, "lambda_min": 0.09649538993835449, "lambda_max": 0.608041524887085, "S_mean": 0.316650390625, "G_mean": 0.34548574173823, "G_std": 0.16626230735952655, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.520817756652832, "grad_norm": 1.005191445350647, "tokens_seen": 77832192, "tokens_per_sec": 18325.912199285824, "vram_gb": 5.684933632, "lambda": 0.30577461887151003, "lambda_min": 0.09650492668151855, "lambda_max": 0.6085212230682373, "S_mean": 0.316680908203125, "G_mean": 0.34325661789625883, "G_std": 0.16544967757848403, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.212263584136963, "grad_norm": 0.9686437845230103, "tokens_seen": 77996032, "tokens_per_sec": 18326.169732205533, "vram_gb": 5.684933632, "lambda": 0.30591389785210293, "lambda_min": 0.09653650969266891, "lambda_max": 0.6088768243789673, "S_mean": 0.314178466796875, "G_mean": 0.34661122985805076, "G_std": 0.16470904679348072, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.360706329345703, "grad_norm": 0.9308199882507324, "tokens_seen": 78159872, "tokens_per_sec": 18326.348320740108, "vram_gb": 5.684933632, "lambda": 0.30605667643249035, "lambda_min": 0.09650992602109909, "lambda_max": 0.6092351675033569, "S_mean": 0.3004201253255208, "G_mean": 0.3461743829927097, "G_std": 0.16646231105551124, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.548252105712891, "grad_norm": 0.9641470313072205, "tokens_seen": 78323712, "tokens_per_sec": 18326.704136993925, "vram_gb": 5.684933632, "lambda": 0.3062013577048977, "lambda_min": 0.09651223570108414, "lambda_max": 0.6096255779266357, "S_mean": 0.3069712320963542, "G_mean": 0.34228607142965, "G_std": 0.16538382748452327, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.273358583450317, "grad_norm": 0.9698442816734314, "tokens_seen": 78487552, "tokens_per_sec": 18326.937378611026, "vram_gb": 5.684933632, "lambda": 0.3063488354285558, "lambda_min": 0.09646687656641006, "lambda_max": 0.6099415421485901, "S_mean": 0.3116099039713542, "G_mean": 0.34344879499015707, "G_std": 0.16548492309326926, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.409652471542358, "grad_norm": 0.9729011654853821, "tokens_seen": 78651392, "tokens_per_sec": 18327.255881388708, "vram_gb": 5.684933632, "lambda": 0.30649709173788625, "lambda_min": 0.09649481624364853, "lambda_max": 0.6103224158287048, "S_mean": 0.3169911702473958, "G_mean": 0.34274767820412916, "G_std": 0.16351175114202002, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.33239483833313, "grad_norm": 0.9827784299850464, "tokens_seen": 78815232, "tokens_per_sec": 18327.798520666074, "vram_gb": 5.684933632, "lambda": 0.3066350327183803, "lambda_min": 0.09650415927171707, "lambda_max": 0.6107423305511475, "S_mean": 0.3079376220703125, "G_mean": 0.34378385093684, "G_std": 0.16509286236638823, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.510993957519531, "grad_norm": 1.028762936592102, "tokens_seen": 78979072, "tokens_per_sec": 18327.911373773302, "vram_gb": 5.684933632, "lambda": 0.3067915004988511, "lambda_min": 0.0964801236987114, "lambda_max": 0.6111825704574585, "S_mean": 0.33453369140625, "G_mean": 0.353617789534231, "G_std": 0.1666640175972134, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.329723834991455, "grad_norm": 0.9559319019317627, "tokens_seen": 79142912, "tokens_per_sec": 18328.1004884434, "vram_gb": 5.684933632, "lambda": 0.30691477687408525, "lambda_min": 0.09642687439918518, "lambda_max": 0.6115337014198303, "S_mean": 0.298919677734375, "G_mean": 0.3446069451359411, "G_std": 0.16532630814860264, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.512345790863037, "grad_norm": 0.9957384467124939, "tokens_seen": 79306752, "tokens_per_sec": 18328.25227071708, "vram_gb": 5.684933632, "lambda": 0.30705568070213, "lambda_min": 0.09644345194101334, "lambda_max": 0.6120824217796326, "S_mean": 0.3127339680989583, "G_mean": 0.3446501987054944, "G_std": 0.16640178222830096, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.279970169067383, "grad_norm": 0.9302194714546204, "tokens_seen": 79470592, "tokens_per_sec": 18328.198115859126, "vram_gb": 5.684933632, "lambda": 0.3071918698648612, "lambda_min": 0.09645693004131317, "lambda_max": 0.6125447154045105, "S_mean": 0.3071950276692708, "G_mean": 0.3455254330765456, "G_std": 0.16434271990632018, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.323121547698975, "grad_norm": 1.0724040269851685, "tokens_seen": 79634432, "tokens_per_sec": 18328.521439734603, "vram_gb": 5.684933632, "lambda": 0.3073451512803634, "lambda_min": 0.0965379849076271, "lambda_max": 0.6128765940666199, "S_mean": 0.332763671875, "G_mean": 0.3464735358332594, "G_std": 0.16532101140667996, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.365035772323608, "grad_norm": 0.936407744884491, "tokens_seen": 79798272, "tokens_per_sec": 18328.727452991512, "vram_gb": 5.684933632, "lambda": 0.30748864977310103, "lambda_min": 0.09660514444112778, "lambda_max": 0.6133334040641785, "S_mean": 0.3116048177083333, "G_mean": 0.34357699705287814, "G_std": 0.16556776664219797, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.384572505950928, "grad_norm": 0.9622747302055359, "tokens_seen": 79962112, "tokens_per_sec": 18328.845518701022, "vram_gb": 5.684933632, "lambda": 0.30758994600425166, "lambda_min": 0.09657428413629532, "lambda_max": 0.6137314438819885, "S_mean": 0.303985595703125, "G_mean": 0.3405146359000355, "G_std": 0.16567775909788907, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.112149715423584, "grad_norm": 0.8988760709762573, "tokens_seen": 80125952, "tokens_per_sec": 18329.06638125054, "vram_gb": 5.684933632, "lambda": 0.3077298669765393, "lambda_min": 0.09655001014471054, "lambda_max": 0.6141331195831299, "S_mean": 0.3038075764973958, "G_mean": 0.3379040729875366, "G_std": 0.16485343935588995, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.274424076080322, "grad_norm": 0.9598637223243713, "tokens_seen": 80289792, "tokens_per_sec": 18329.30290643264, "vram_gb": 5.684933632, "lambda": 0.307878908701241, "lambda_min": 0.09652793407440186, "lambda_max": 0.6146101951599121, "S_mean": 0.3141988118489583, "G_mean": 0.33987252491836745, "G_std": 0.163630241372933, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.165339231491089, "grad_norm": 0.9066839814186096, "tokens_seen": 80453632, "tokens_per_sec": 18329.08635944776, "vram_gb": 5.684933632, "lambda": 0.3080027125154932, "lambda_min": 0.09651166200637817, "lambda_max": 0.6150568723678589, "S_mean": 0.3050435384114583, "G_mean": 0.33754552823180956, "G_std": 0.16380121690842012, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.486155271530151, "grad_norm": 0.9497828483581543, "tokens_seen": 80617472, "tokens_per_sec": 18329.452422861443, "vram_gb": 5.684933632, "lambda": 0.3081222552185257, "lambda_min": 0.09647586941719055, "lambda_max": 0.6153920888900757, "S_mean": 0.3350830078125, "G_mean": 0.35026859018641215, "G_std": 0.16381320830745003, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.30491304397583, "grad_norm": 1.0586720705032349, "tokens_seen": 80781312, "tokens_per_sec": 18329.36837816277, "vram_gb": 5.684933632, "lambda": 0.30823838245123625, "lambda_min": 0.09644495695829391, "lambda_max": 0.6156004667282104, "S_mean": 0.3189595540364583, "G_mean": 0.350689245465522, "G_std": 0.1643438764537374, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.333050727844238, "grad_norm": 0.9498305320739746, "tokens_seen": 80945152, "tokens_per_sec": 18329.599846469835, "vram_gb": 5.684933632, "lambda": 0.3083571546400587, "lambda_min": 0.09643780440092087, "lambda_max": 0.6158258318901062, "S_mean": 0.3089192708333333, "G_mean": 0.3399057885011037, "G_std": 0.16541464091278613, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.250339031219482, "grad_norm": 0.9720369577407837, "tokens_seen": 81108992, "tokens_per_sec": 18329.861666262004, "vram_gb": 5.684933632, "lambda": 0.30845462065190077, "lambda_min": 0.09642985463142395, "lambda_max": 0.6161914467811584, "S_mean": 0.3199971516927083, "G_mean": 0.34824877333206433, "G_std": 0.16402810951694846, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.296525478363037, "grad_norm": 0.9488085508346558, "tokens_seen": 81272832, "tokens_per_sec": 18330.224536645033, "vram_gb": 5.684933632, "lambda": 0.3085564511517684, "lambda_min": 0.09647299349308014, "lambda_max": 0.6165478229522705, "S_mean": 0.3106536865234375, "G_mean": 0.3420624987532695, "G_std": 0.163927923499917, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.501045227050781, "grad_norm": 0.9489713311195374, "tokens_seen": 81436672, "tokens_per_sec": 18330.6013102539, "vram_gb": 5.684933632, "lambda": 0.3086963541184862, "lambda_min": 0.09651684015989304, "lambda_max": 0.6168718338012695, "S_mean": 0.3108418782552083, "G_mean": 0.34542154070610803, "G_std": 0.16481794882565737, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.388270378112793, "grad_norm": 0.9945991039276123, "tokens_seen": 81600512, "tokens_per_sec": 18330.639661909638, "vram_gb": 5.684933632, "lambda": 0.3088552625849843, "lambda_min": 0.09657546877861023, "lambda_max": 0.6171128153800964, "S_mean": 0.2929026285807292, "G_mean": 0.34844274586066604, "G_std": 0.16491761757060885, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.356157302856445, "grad_norm": 0.9300527572631836, "tokens_seen": 81764352, "tokens_per_sec": 18330.93756986871, "vram_gb": 5.684933632, "lambda": 0.3089638880143563, "lambda_min": 0.09658916294574738, "lambda_max": 0.6175580024719238, "S_mean": 0.3016153971354167, "G_mean": 0.34624715537453693, "G_std": 0.16488171283466121, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.3272178173065186, "grad_norm": 0.9146011471748352, "tokens_seen": 81928192, "tokens_per_sec": 18329.267507035107, "vram_gb": 5.684933632, "lambda": 0.30906392354518175, "lambda_min": 0.09660229086875916, "lambda_max": 0.6179835796356201, "S_mean": 0.3016459147135417, "G_mean": 0.34430481151988107, "G_std": 0.1643871960695833, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.414562916755676, "val_ppl": 82.64570995553005, "lambda": 0.3090699032569925, "lambda_min": 0.09660259634256363, "lambda_max": 0.6180070638656616, "lambda_per_layer": [0.17853683233261108, 0.1439497172832489, 0.15315034985542297, 0.12866173684597015, 0.11776396632194519, 0.09660259634256363, 0.11903209984302521, 0.1698368489742279, 0.22981968522071838, 0.4975568652153015, 0.4362199008464813, 0.38036686182022095, 0.3453608453273773, 0.6180070638656616, 0.4393342435359955, 0.5474819540977478, 0.5420327186584473, 0.4257532060146332, 0.302354633808136, 0.43669411540031433, 0.37671536207199097, 0.2113509625196457, 0.304377019405365, 0.21671809256076813], "S_mean": 0.3101603190104167, "G_mean": 0.3446166791642706, "G_std": 0.16470846883021295, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.491995096206665, "grad_norm": 1.073291540145874, "tokens_seen": 82092032, "tokens_per_sec": 18322.391716936778, "vram_gb": 5.684933632, "lambda": 0.3091837642714381, "lambda_min": 0.09659931063652039, "lambda_max": 0.6184446215629578, "S_mean": 0.3284200032552083, "G_mean": 0.3492072701919824, "G_std": 0.16364619803304473, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.315975189208984, "grad_norm": 0.9439072012901306, "tokens_seen": 82255872, "tokens_per_sec": 18322.18363137904, "vram_gb": 5.684933632, "lambda": 0.30929168468962115, "lambda_min": 0.096615731716156, "lambda_max": 0.6186097860336304, "S_mean": 0.3144327799479167, "G_mean": 0.3435382464279731, "G_std": 0.16417353701156875, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.302910327911377, "grad_norm": 0.960668683052063, "tokens_seen": 82419712, "tokens_per_sec": 18322.59113675546, "vram_gb": 5.684933632, "lambda": 0.3094137602796157, "lambda_min": 0.096635602414608, "lambda_max": 0.6190271377563477, "S_mean": 0.3082173665364583, "G_mean": 0.34232755719373625, "G_std": 0.16489323771869144, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.33703875541687, "grad_norm": 1.04775071144104, "tokens_seen": 82583552, "tokens_per_sec": 18322.938495362087, "vram_gb": 5.684933632, "lambda": 0.3095495185504357, "lambda_min": 0.09661311656236649, "lambda_max": 0.6193682551383972, "S_mean": 0.30035400390625, "G_mean": 0.3441431905763845, "G_std": 0.1643485598421345, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.384470462799072, "grad_norm": 0.966421365737915, "tokens_seen": 82747392, "tokens_per_sec": 18323.335202141283, "vram_gb": 5.684933632, "lambda": 0.3096608966588974, "lambda_min": 0.09659267216920853, "lambda_max": 0.6197003126144409, "S_mean": 0.2938588460286458, "G_mean": 0.3388237490629156, "G_std": 0.16460222999254862, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.352104425430298, "grad_norm": 0.9458378553390503, "tokens_seen": 82911232, "tokens_per_sec": 18323.83545202835, "vram_gb": 5.684933632, "lambda": 0.3097817956780394, "lambda_min": 0.09657417237758636, "lambda_max": 0.6200799942016602, "S_mean": 0.306182861328125, "G_mean": 0.3417983838977913, "G_std": 0.1648665729444474, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.146273851394653, "grad_norm": 0.928722083568573, "tokens_seen": 83075072, "tokens_per_sec": 18324.050468032143, "vram_gb": 5.684933632, "lambda": 0.30990101117640734, "lambda_min": 0.09658556431531906, "lambda_max": 0.6203038692474365, "S_mean": 0.316253662109375, "G_mean": 0.34474145000179607, "G_std": 0.16307816561311483, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.272965908050537, "grad_norm": 0.9861852526664734, "tokens_seen": 83238912, "tokens_per_sec": 18324.343638094106, "vram_gb": 5.684933632, "lambda": 0.31000628632803756, "lambda_min": 0.0965779647231102, "lambda_max": 0.6207275986671448, "S_mean": 0.3049723307291667, "G_mean": 0.3488178439438343, "G_std": 0.1650071470066905, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.4452009201049805, "grad_norm": 1.0283094644546509, "tokens_seen": 83402752, "tokens_per_sec": 18324.51766566172, "vram_gb": 5.684933632, "lambda": 0.3101037899032235, "lambda_min": 0.09659307450056076, "lambda_max": 0.6208716630935669, "S_mean": 0.3067067464192708, "G_mean": 0.3443381539545953, "G_std": 0.16528439847752452, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.328649282455444, "grad_norm": 0.9853503108024597, "tokens_seen": 83566592, "tokens_per_sec": 18324.462388011652, "vram_gb": 5.684933632, "lambda": 0.31020110721389454, "lambda_min": 0.0966453105211258, "lambda_max": 0.621201753616333, "S_mean": 0.3117574055989583, "G_mean": 0.3475480879812191, "G_std": 0.1648317128419876, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.404174566268921, "grad_norm": 1.072756290435791, "tokens_seen": 83730432, "tokens_per_sec": 18324.74714837887, "vram_gb": 5.684933632, "lambda": 0.3103204909712076, "lambda_min": 0.09666775912046432, "lambda_max": 0.6215286254882812, "S_mean": 0.31243896484375, "G_mean": 0.3401738195680082, "G_std": 0.16346687668313584, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.23376989364624, "grad_norm": 1.053867220878601, "tokens_seen": 83894272, "tokens_per_sec": 18324.6881367475, "vram_gb": 5.684933632, "lambda": 0.3104028108840187, "lambda_min": 0.09663959592580795, "lambda_max": 0.6216808557510376, "S_mean": 0.332183837890625, "G_mean": 0.3470736830495298, "G_std": 0.1640051600212852, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.385745286941528, "grad_norm": 1.0128484964370728, "tokens_seen": 84058112, "tokens_per_sec": 18324.67422153856, "vram_gb": 5.684933632, "lambda": 0.3105178310846289, "lambda_min": 0.09665468335151672, "lambda_max": 0.6218811273574829, "S_mean": 0.2979685465494792, "G_mean": 0.34524595194185775, "G_std": 0.16519380166816214, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.388201713562012, "grad_norm": 0.9951669573783875, "tokens_seen": 84221952, "tokens_per_sec": 18324.95039425007, "vram_gb": 5.684933632, "lambda": 0.31062601196269196, "lambda_min": 0.09666389226913452, "lambda_max": 0.622151792049408, "S_mean": 0.3187510172526042, "G_mean": 0.3456831614021212, "G_std": 0.16455372523826858, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.27171516418457, "grad_norm": 0.9572778344154358, "tokens_seen": 84385792, "tokens_per_sec": 18324.859711357065, "vram_gb": 5.684933632, "lambda": 0.3107094143827756, "lambda_min": 0.09663746505975723, "lambda_max": 0.622381329536438, "S_mean": 0.3034871419270833, "G_mean": 0.3422182679641992, "G_std": 0.1640058062815418, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.4325950145721436, "grad_norm": 0.9945404529571533, "tokens_seen": 84549632, "tokens_per_sec": 18325.003583167432, "vram_gb": 5.684933632, "lambda": 0.31081789173185825, "lambda_min": 0.09666387736797333, "lambda_max": 0.6228225827217102, "S_mean": 0.3083292643229167, "G_mean": 0.34578449713687104, "G_std": 0.16541651884714761, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.3311872482299805, "grad_norm": 1.0124262571334839, "tokens_seen": 84713472, "tokens_per_sec": 18325.374144931557, "vram_gb": 5.684933632, "lambda": 0.3109361423800389, "lambda_min": 0.09667109698057175, "lambda_max": 0.623160719871521, "S_mean": 0.3053334554036458, "G_mean": 0.342937253570805, "G_std": 0.1647067836796244, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.423341274261475, "grad_norm": 0.9721639156341553, "tokens_seen": 84877312, "tokens_per_sec": 18325.249283633453, "vram_gb": 5.684933632, "lambda": 0.31103919198115665, "lambda_min": 0.09664755314588547, "lambda_max": 0.623386800289154, "S_mean": 0.3072052001953125, "G_mean": 0.3393843368006249, "G_std": 0.16482744451301792, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.376962661743164, "grad_norm": 1.0026655197143555, "tokens_seen": 85041152, "tokens_per_sec": 18325.642149169074, "vram_gb": 5.684933632, "lambda": 0.3111378081763784, "lambda_min": 0.09663235396146774, "lambda_max": 0.6237361431121826, "S_mean": 0.3098246256510417, "G_mean": 0.3428345713764429, "G_std": 0.16257365249718228, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.445582866668701, "grad_norm": 1.0244626998901367, "tokens_seen": 85204992, "tokens_per_sec": 18325.92054743231, "vram_gb": 5.684933632, "lambda": 0.311218217946589, "lambda_min": 0.09661983698606491, "lambda_max": 0.6241509318351746, "S_mean": 0.314361572265625, "G_mean": 0.3456390833792587, "G_std": 0.16425636418474218, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.390727758407593, "grad_norm": 0.9650494456291199, "tokens_seen": 85368832, "tokens_per_sec": 18325.66496568417, "vram_gb": 5.684933632, "lambda": 0.31130138598382473, "lambda_min": 0.0965961217880249, "lambda_max": 0.6244810819625854, "S_mean": 0.3114827473958333, "G_mean": 0.3446885173519452, "G_std": 0.16488051849106947, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.257154941558838, "grad_norm": 0.9479361176490784, "tokens_seen": 85532672, "tokens_per_sec": 18326.146843428993, "vram_gb": 5.684933632, "lambda": 0.3113922458142042, "lambda_min": 0.09657641500234604, "lambda_max": 0.6246678829193115, "S_mean": 0.32794189453125, "G_mean": 0.3449616488069296, "G_std": 0.16360821512838206, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.392410039901733, "grad_norm": 0.9493452906608582, "tokens_seen": 85696512, "tokens_per_sec": 18326.163890873384, "vram_gb": 5.684933632, "lambda": 0.3115001205975811, "lambda_min": 0.0965815931558609, "lambda_max": 0.624888002872467, "S_mean": 0.3082377115885417, "G_mean": 0.34174589610969025, "G_std": 0.16302334299931923, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.252959251403809, "grad_norm": 0.9626095294952393, "tokens_seen": 85860352, "tokens_per_sec": 18326.290065461377, "vram_gb": 5.684933632, "lambda": 0.311597784049809, "lambda_min": 0.0966382771730423, "lambda_max": 0.6252557635307312, "S_mean": 0.310943603515625, "G_mean": 0.34365716700752574, "G_std": 0.16453609033487737, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.347954750061035, "grad_norm": 1.023903489112854, "tokens_seen": 86024192, "tokens_per_sec": 18326.228550871983, "vram_gb": 5.684933632, "lambda": 0.31169732473790646, "lambda_min": 0.09662897139787674, "lambda_max": 0.6256486773490906, "S_mean": 0.301666259765625, "G_mean": 0.3431276755097012, "G_std": 0.1638877586616824, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.419975519180298, "grad_norm": 1.0040589570999146, "tokens_seen": 86188032, "tokens_per_sec": 18326.389024047097, "vram_gb": 5.684933632, "lambda": 0.31178807312001783, "lambda_min": 0.09661988168954849, "lambda_max": 0.6259704232215881, "S_mean": 0.31378173828125, "G_mean": 0.3437375579960644, "G_std": 0.16524459902818003, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.32434868812561, "grad_norm": 0.9870880246162415, "tokens_seen": 86351872, "tokens_per_sec": 18326.46895177929, "vram_gb": 5.684933632, "lambda": 0.3118666124840577, "lambda_min": 0.09664032608270645, "lambda_max": 0.626168429851532, "S_mean": 0.3052571614583333, "G_mean": 0.3414178915942709, "G_std": 0.16377110624065003, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.320682048797607, "grad_norm": 0.9674662947654724, "tokens_seen": 86515712, "tokens_per_sec": 18326.46797231008, "vram_gb": 5.684933632, "lambda": 0.3119821383928259, "lambda_min": 0.09668096154928207, "lambda_max": 0.6264805197715759, "S_mean": 0.3146260579427083, "G_mean": 0.3433294215550025, "G_std": 0.16334762734671435, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.370916843414307, "grad_norm": 1.0273921489715576, "tokens_seen": 86679552, "tokens_per_sec": 18326.630860928777, "vram_gb": 5.684933632, "lambda": 0.3120898411919673, "lambda_min": 0.0967128723859787, "lambda_max": 0.6268102526664734, "S_mean": 0.3037618001302083, "G_mean": 0.3374267515415947, "G_std": 0.16356565112558505, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.228014945983887, "grad_norm": 0.9560291171073914, "tokens_seen": 86843392, "tokens_per_sec": 18326.71213016578, "vram_gb": 5.684933632, "lambda": 0.31219553605963785, "lambda_min": 0.0967000350356102, "lambda_max": 0.6271587610244751, "S_mean": 0.316650390625, "G_mean": 0.34344355249777436, "G_std": 0.1624954853517314, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.251536846160889, "grad_norm": 1.039231300354004, "tokens_seen": 87007232, "tokens_per_sec": 18326.889363282186, "vram_gb": 5.684933632, "lambda": 0.31229190311084193, "lambda_min": 0.09670454263687134, "lambda_max": 0.6274394989013672, "S_mean": 0.3215230305989583, "G_mean": 0.33835393050685525, "G_std": 0.1636600192481031, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.250829458236694, "grad_norm": 0.9863668084144592, "tokens_seen": 87171072, "tokens_per_sec": 18326.9559553281, "vram_gb": 5.684933632, "lambda": 0.31238084472715855, "lambda_min": 0.09671974927186966, "lambda_max": 0.6277832388877869, "S_mean": 0.2982076009114583, "G_mean": 0.33978710882365704, "G_std": 0.16342705092392862, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.4239277839660645, "grad_norm": 1.0054434537887573, "tokens_seen": 87334912, "tokens_per_sec": 18326.87453047618, "vram_gb": 5.684933632, "lambda": 0.3124683502440651, "lambda_min": 0.096749447286129, "lambda_max": 0.6280303001403809, "S_mean": 0.3106587727864583, "G_mean": 0.3451198427937925, "G_std": 0.16446199112882218, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.3162853717803955, "grad_norm": 1.0175509452819824, "tokens_seen": 87498752, "tokens_per_sec": 18326.84633045393, "vram_gb": 5.684933632, "lambda": 0.3125674460704128, "lambda_min": 0.09675536304712296, "lambda_max": 0.6282386779785156, "S_mean": 0.3218892415364583, "G_mean": 0.3408746011555195, "G_std": 0.1634577123913914, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.297738313674927, "grad_norm": 1.0293329954147339, "tokens_seen": 87662592, "tokens_per_sec": 18327.006202280794, "vram_gb": 5.684933632, "lambda": 0.3126651793718338, "lambda_min": 0.09675659239292145, "lambda_max": 0.628441333770752, "S_mean": 0.320587158203125, "G_mean": 0.34602868432799977, "G_std": 0.16435814765281975, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.427701473236084, "grad_norm": 1.0207016468048096, "tokens_seen": 87826432, "tokens_per_sec": 18327.36240662405, "vram_gb": 5.684933632, "lambda": 0.3127626981586218, "lambda_min": 0.09673009812831879, "lambda_max": 0.6287856101989746, "S_mean": 0.3215738932291667, "G_mean": 0.3488659185046951, "G_std": 0.16370712919160724, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.404854774475098, "grad_norm": 1.0044996738433838, "tokens_seen": 87990272, "tokens_per_sec": 18327.48564393847, "vram_gb": 5.684933632, "lambda": 0.3128064988801877, "lambda_min": 0.096720851957798, "lambda_max": 0.6289165616035461, "S_mean": 0.3120524088541667, "G_mean": 0.34118162755233544, "G_std": 0.16345904488116503, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.183818340301514, "grad_norm": 1.0429884195327759, "tokens_seen": 88154112, "tokens_per_sec": 18327.553028484253, "vram_gb": 5.684933632, "lambda": 0.3128965475286047, "lambda_min": 0.09670091420412064, "lambda_max": 0.6292402744293213, "S_mean": 0.321533203125, "G_mean": 0.34672831821565825, "G_std": 0.16548720809320608, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.238893032073975, "grad_norm": 1.043016791343689, "tokens_seen": 88317952, "tokens_per_sec": 18327.71745184305, "vram_gb": 5.684933632, "lambda": 0.3129932287459572, "lambda_min": 0.09669489413499832, "lambda_max": 0.6295941472053528, "S_mean": 0.2996724446614583, "G_mean": 0.34127421335627633, "G_std": 0.1645365491664658, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.31777286529541, "grad_norm": 0.9625322818756104, "tokens_seen": 88481792, "tokens_per_sec": 18327.797066312924, "vram_gb": 5.684933632, "lambda": 0.31309217121452093, "lambda_min": 0.09672737121582031, "lambda_max": 0.6299339532852173, "S_mean": 0.3122965494791667, "G_mean": 0.34093608140634996, "G_std": 0.16309790002803007, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.278476238250732, "grad_norm": 0.9733418226242065, "tokens_seen": 88645632, "tokens_per_sec": 18327.931591218123, "vram_gb": 5.684933632, "lambda": 0.3131910466278593, "lambda_min": 0.09674384444952011, "lambda_max": 0.6303261518478394, "S_mean": 0.3069661458333333, "G_mean": 0.3434507339261472, "G_std": 0.16443087067455053, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.285468101501465, "grad_norm": 1.008037805557251, "tokens_seen": 88809472, "tokens_per_sec": 18327.936464801933, "vram_gb": 5.684933632, "lambda": 0.31325112500538427, "lambda_min": 0.0967637449502945, "lambda_max": 0.6305528879165649, "S_mean": 0.3160603841145833, "G_mean": 0.3398363692685962, "G_std": 0.16438295350720486, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.260097026824951, "grad_norm": 1.0174897909164429, "tokens_seen": 88973312, "tokens_per_sec": 18328.0820774503, "vram_gb": 5.684933632, "lambda": 0.3133311967055003, "lambda_min": 0.0967479944229126, "lambda_max": 0.6307384371757507, "S_mean": 0.3156840006510417, "G_mean": 0.3400708402817448, "G_std": 0.16501105902716517, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.234312295913696, "grad_norm": 1.0639009475708008, "tokens_seen": 89137152, "tokens_per_sec": 18328.342581153942, "vram_gb": 5.684933632, "lambda": 0.3134084139019251, "lambda_min": 0.09671236574649811, "lambda_max": 0.6308884024620056, "S_mean": 0.3087259928385417, "G_mean": 0.3391596705963214, "G_std": 0.16246813054506978, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.175869464874268, "grad_norm": 0.9718746542930603, "tokens_seen": 89300992, "tokens_per_sec": 18328.319934774074, "vram_gb": 5.684933632, "lambda": 0.31347896965841454, "lambda_min": 0.09669129550457001, "lambda_max": 0.6309928894042969, "S_mean": 0.3188273111979167, "G_mean": 0.33873204157377285, "G_std": 0.16238340774240592, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.347942590713501, "grad_norm": 1.0098644495010376, "tokens_seen": 89464832, "tokens_per_sec": 18328.37922303859, "vram_gb": 5.684933632, "lambda": 0.31355453344682854, "lambda_min": 0.09671471267938614, "lambda_max": 0.6311845779418945, "S_mean": 0.3077494303385417, "G_mean": 0.34670761227607727, "G_std": 0.1645458055039247, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.321042537689209, "grad_norm": 1.0221185684204102, "tokens_seen": 89628672, "tokens_per_sec": 18328.589056629404, "vram_gb": 5.684933632, "lambda": 0.3136347591256102, "lambda_min": 0.0967085212469101, "lambda_max": 0.6314699649810791, "S_mean": 0.3194376627604167, "G_mean": 0.3475120921308796, "G_std": 0.16311049244056144, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.296908140182495, "grad_norm": 1.0261740684509277, "tokens_seen": 89792512, "tokens_per_sec": 18328.357197033114, "vram_gb": 5.684933632, "lambda": 0.3137175490458806, "lambda_min": 0.09671493619680405, "lambda_max": 0.6317927837371826, "S_mean": 0.308319091796875, "G_mean": 0.34767933391655487, "G_std": 0.1638470673157523, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.464309930801392, "grad_norm": 1.1273211240768433, "tokens_seen": 89956352, "tokens_per_sec": 18328.484196430807, "vram_gb": 5.684933632, "lambda": 0.3137996320923169, "lambda_min": 0.09670988470315933, "lambda_max": 0.6319735050201416, "S_mean": 0.3049062093098958, "G_mean": 0.34267903631553054, "G_std": 0.16533329306791225, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.385815143585205, "grad_norm": 0.9385438561439514, "tokens_seen": 90120192, "tokens_per_sec": 18328.788657135054, "vram_gb": 5.684933632, "lambda": 0.31387865729629993, "lambda_min": 0.09666501730680466, "lambda_max": 0.6322678923606873, "S_mean": 0.3084309895833333, "G_mean": 0.3399026563080649, "G_std": 0.16356763880078992, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.366402578353882, "val_ppl": 78.75978928443558, "lambda": 0.313883235057195, "lambda_min": 0.09666404873132706, "lambda_max": 0.6322798728942871, "lambda_per_layer": [0.17965523898601532, 0.14416886866092682, 0.15400739014148712, 0.12860499322414398, 0.11803209036588669, 0.09666404873132706, 0.11891736090183258, 0.170328751206398, 0.2306562215089798, 0.5014661550521851, 0.4420604705810547, 0.38680464029312134, 0.3502376079559326, 0.6322798728942871, 0.4454370141029358, 0.5592616200447083, 0.5530309677124023, 0.43579357862472534, 0.30619195103645325, 0.44863277673721313, 0.3863782584667206, 0.21405702829360962, 0.31069526076316833, 0.21983547508716583], "S_mean": 0.3096110026041667, "G_mean": 0.3387455741564433, "G_std": 0.16373787773773074, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.188735246658325, "grad_norm": 0.9930898547172546, "tokens_seen": 90284032, "tokens_per_sec": 18322.555936076475, "vram_gb": 5.684933632, "lambda": 0.3139682011678815, "lambda_min": 0.09665203094482422, "lambda_max": 0.6325637698173523, "S_mean": 0.3124796549479167, "G_mean": 0.3439560338544349, "G_std": 0.16346966529575488, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.285310983657837, "grad_norm": 0.9971203207969666, "tokens_seen": 90447872, "tokens_per_sec": 18322.181892644447, "vram_gb": 5.684933632, "lambda": 0.3140761209651828, "lambda_min": 0.09664236754179001, "lambda_max": 0.6328388452529907, "S_mean": 0.3110402425130208, "G_mean": 0.34518624384266633, "G_std": 0.16340296397296092, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.457995414733887, "grad_norm": 1.0868464708328247, "tokens_seen": 90611712, "tokens_per_sec": 18322.434241431078, "vram_gb": 5.684933632, "lambda": 0.31416218914091587, "lambda_min": 0.09666112810373306, "lambda_max": 0.6328867673873901, "S_mean": 0.3081512451171875, "G_mean": 0.34230645226004225, "G_std": 0.163868409426262, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.307094097137451, "grad_norm": 1.02173912525177, "tokens_seen": 90775552, "tokens_per_sec": 18322.59645183974, "vram_gb": 5.684933632, "lambda": 0.3142455769702792, "lambda_min": 0.09670445322990417, "lambda_max": 0.6329461336135864, "S_mean": 0.318603515625, "G_mean": 0.3450187637936324, "G_std": 0.1648382229420046, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.3306884765625, "grad_norm": 1.1012436151504517, "tokens_seen": 90939392, "tokens_per_sec": 18322.617787725063, "vram_gb": 5.684933632, "lambda": 0.3143402102092902, "lambda_min": 0.09673148393630981, "lambda_max": 0.6332376003265381, "S_mean": 0.32916259765625, "G_mean": 0.3488812033707897, "G_std": 0.16232434931832054, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.39089035987854, "grad_norm": 0.9960228800773621, "tokens_seen": 91103232, "tokens_per_sec": 18322.10677779399, "vram_gb": 5.684933632, "lambda": 0.31443373517443735, "lambda_min": 0.09671425819396973, "lambda_max": 0.6335026621818542, "S_mean": 0.3099873860677083, "G_mean": 0.34203995174417895, "G_std": 0.16443041626674434, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.347765207290649, "grad_norm": 1.076553463935852, "tokens_seen": 91267072, "tokens_per_sec": 18320.668044539314, "vram_gb": 5.684933632, "lambda": 0.31451515449831885, "lambda_min": 0.09672526270151138, "lambda_max": 0.6338212490081787, "S_mean": 0.3122049967447917, "G_mean": 0.34011839842423797, "G_std": 0.16329994859794775, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.233980894088745, "grad_norm": 0.9766248464584351, "tokens_seen": 91430912, "tokens_per_sec": 18319.004847163207, "vram_gb": 5.684933632, "lambda": 0.31458022352308035, "lambda_min": 0.09670865535736084, "lambda_max": 0.6340433359146118, "S_mean": 0.3236134847005208, "G_mean": 0.3415016852474461, "G_std": 0.1634085854360213, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.355536460876465, "grad_norm": 1.0180878639221191, "tokens_seen": 91594752, "tokens_per_sec": 18317.91810159497, "vram_gb": 5.684933632, "lambda": 0.31467237168302137, "lambda_min": 0.09669281542301178, "lambda_max": 0.6343089938163757, "S_mean": 0.3117421468098958, "G_mean": 0.3431042606631915, "G_std": 0.16327958647161722, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.327190637588501, "grad_norm": 0.9869999289512634, "tokens_seen": 91758592, "tokens_per_sec": 18316.46394508477, "vram_gb": 5.684933632, "lambda": 0.314742907260855, "lambda_min": 0.09669395536184311, "lambda_max": 0.6345527768135071, "S_mean": 0.32177734375, "G_mean": 0.34321274670461815, "G_std": 0.164107065104569, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.195974826812744, "grad_norm": 1.0451666116714478, "tokens_seen": 91922432, "tokens_per_sec": 18315.507954556022, "vram_gb": 5.684933632, "lambda": 0.3148076043774684, "lambda_min": 0.09667830169200897, "lambda_max": 0.6347417235374451, "S_mean": 0.3169097900390625, "G_mean": 0.3416780672656993, "G_std": 0.16380869139296314, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.399794816970825, "grad_norm": 1.037936806678772, "tokens_seen": 92086272, "tokens_per_sec": 18313.43632229755, "vram_gb": 5.684933632, "lambda": 0.31487283514191705, "lambda_min": 0.09668546915054321, "lambda_max": 0.634939968585968, "S_mean": 0.3090413411458333, "G_mean": 0.3391622257574151, "G_std": 0.16266148157107332, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.380116939544678, "grad_norm": 1.0557671785354614, "tokens_seen": 92250112, "tokens_per_sec": 18311.655790050678, "vram_gb": 5.684933632, "lambda": 0.3149531055241823, "lambda_min": 0.09666112810373306, "lambda_max": 0.6351205110549927, "S_mean": 0.3099619547526042, "G_mean": 0.3387857341828446, "G_std": 0.1630151094092677, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.396600008010864, "grad_norm": 1.084825038909912, "tokens_seen": 92413952, "tokens_per_sec": 18309.87968838479, "vram_gb": 5.684933632, "lambda": 0.3150419980908434, "lambda_min": 0.09666358679533005, "lambda_max": 0.635374903678894, "S_mean": 0.319488525390625, "G_mean": 0.3416655626303206, "G_std": 0.16425674447479346, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.28109335899353, "grad_norm": 1.0711168050765991, "tokens_seen": 92577792, "tokens_per_sec": 18308.71522805408, "vram_gb": 5.684933632, "lambda": 0.3151340528080861, "lambda_min": 0.09669765830039978, "lambda_max": 0.6355445384979248, "S_mean": 0.319732666015625, "G_mean": 0.33846419599528116, "G_std": 0.16342533046069244, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.346818447113037, "grad_norm": 0.9546015858650208, "tokens_seen": 92741632, "tokens_per_sec": 18308.187521140084, "vram_gb": 5.684933632, "lambda": 0.3152175440142552, "lambda_min": 0.09670300036668777, "lambda_max": 0.6357834935188293, "S_mean": 0.3060302734375, "G_mean": 0.3387984541089584, "G_std": 0.164003596175462, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.565330743789673, "grad_norm": 1.0995886325836182, "tokens_seen": 92905472, "tokens_per_sec": 18307.201675398504, "vram_gb": 5.684933632, "lambda": 0.3152738371863961, "lambda_min": 0.09673155844211578, "lambda_max": 0.636000394821167, "S_mean": 0.3095347086588542, "G_mean": 0.34136111894622445, "G_std": 0.16369908481525877, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.263268232345581, "grad_norm": 1.0178292989730835, "tokens_seen": 93069312, "tokens_per_sec": 18306.940908503973, "vram_gb": 5.684933632, "lambda": 0.3153664267932375, "lambda_min": 0.09677807986736298, "lambda_max": 0.6362282037734985, "S_mean": 0.3263498942057292, "G_mean": 0.33609875307107967, "G_std": 0.16348220501095057, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.347934007644653, "grad_norm": 1.024178147315979, "tokens_seen": 93233152, "tokens_per_sec": 18306.777181210502, "vram_gb": 5.684933632, "lambda": 0.3154604847853382, "lambda_min": 0.09680230170488358, "lambda_max": 0.6364225149154663, "S_mean": 0.3228658040364583, "G_mean": 0.3378335024851064, "G_std": 0.16248643266347548, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.2996439933776855, "grad_norm": 1.0182768106460571, "tokens_seen": 93396992, "tokens_per_sec": 18306.87458029552, "vram_gb": 5.684933632, "lambda": 0.3155368436127901, "lambda_min": 0.09675860404968262, "lambda_max": 0.6367354989051819, "S_mean": 0.3116861979166667, "G_mean": 0.33700776876260835, "G_std": 0.16282064684977135, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.269179821014404, "grad_norm": 0.9751576781272888, "tokens_seen": 93560832, "tokens_per_sec": 18306.8413664145, "vram_gb": 5.684933632, "lambda": 0.31560262230535346, "lambda_min": 0.09672608226537704, "lambda_max": 0.6369382739067078, "S_mean": 0.3062744140625, "G_mean": 0.34021452922994894, "G_std": 0.16360171705794832, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.34670352935791, "grad_norm": 0.994849443435669, "tokens_seen": 93724672, "tokens_per_sec": 18307.023649394854, "vram_gb": 5.684933632, "lambda": 0.3156785052269697, "lambda_min": 0.09671726822853088, "lambda_max": 0.6370927095413208, "S_mean": 0.3255716959635417, "G_mean": 0.34232314753656584, "G_std": 0.16163598264877996, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.265100002288818, "grad_norm": 0.9598180651664734, "tokens_seen": 93888512, "tokens_per_sec": 18307.021056727528, "vram_gb": 5.684933632, "lambda": 0.31572730621943873, "lambda_min": 0.09672705084085464, "lambda_max": 0.6372888088226318, "S_mean": 0.3043721516927083, "G_mean": 0.3392535323897998, "G_std": 0.1625559014112999, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.485845565795898, "grad_norm": 1.02244234085083, "tokens_seen": 94052352, "tokens_per_sec": 18307.03336459199, "vram_gb": 5.684933632, "lambda": 0.3157919067889452, "lambda_min": 0.0967479795217514, "lambda_max": 0.6374050378799438, "S_mean": 0.3048807779947917, "G_mean": 0.341283911994348, "G_std": 0.16338044831839701, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.244711399078369, "grad_norm": 1.00771164894104, "tokens_seen": 94216192, "tokens_per_sec": 18307.142107530206, "vram_gb": 5.684933632, "lambda": 0.31585169211030006, "lambda_min": 0.09673720598220825, "lambda_max": 0.6375705003738403, "S_mean": 0.3163045247395833, "G_mean": 0.34187414256545406, "G_std": 0.16321113364150128, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.430334091186523, "grad_norm": 1.026511311531067, "tokens_seen": 94380032, "tokens_per_sec": 18307.256071441312, "vram_gb": 5.684933632, "lambda": 0.3159155234073599, "lambda_min": 0.09671049565076828, "lambda_max": 0.6378306150436401, "S_mean": 0.3242594401041667, "G_mean": 0.34371421734491986, "G_std": 0.16233071576183042, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.374675273895264, "grad_norm": 1.0145174264907837, "tokens_seen": 94543872, "tokens_per_sec": 18307.61251272756, "vram_gb": 5.684933632, "lambda": 0.31598049992074567, "lambda_min": 0.09670509397983551, "lambda_max": 0.6380419731140137, "S_mean": 0.3194478352864583, "G_mean": 0.3412140649743378, "G_std": 0.16360029322095215, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.330874443054199, "grad_norm": 1.0239849090576172, "tokens_seen": 94707712, "tokens_per_sec": 18307.747631729988, "vram_gb": 5.684933632, "lambda": 0.31603652487198514, "lambda_min": 0.09672284871339798, "lambda_max": 0.6382094025611877, "S_mean": 0.3085123697916667, "G_mean": 0.33828476110162836, "G_std": 0.1627563440706581, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.443542003631592, "grad_norm": 1.0348660945892334, "tokens_seen": 94871552, "tokens_per_sec": 18307.630213269975, "vram_gb": 5.684933632, "lambda": 0.31612058356404305, "lambda_min": 0.09670693427324295, "lambda_max": 0.6384315490722656, "S_mean": 0.3063151041666667, "G_mean": 0.33860448251167935, "G_std": 0.16417615565781793, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.293273687362671, "grad_norm": 1.031428575515747, "tokens_seen": 95035392, "tokens_per_sec": 18307.83455740921, "vram_gb": 5.684933632, "lambda": 0.31620523519814014, "lambda_min": 0.09671120345592499, "lambda_max": 0.6386176943778992, "S_mean": 0.3118387858072917, "G_mean": 0.34238786886756617, "G_std": 0.16374414996244013, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.199270009994507, "grad_norm": 1.0496140718460083, "tokens_seen": 95199232, "tokens_per_sec": 18308.082790591317, "vram_gb": 5.684933632, "lambda": 0.31628548229734105, "lambda_min": 0.09671036154031754, "lambda_max": 0.6387790441513062, "S_mean": 0.3154093424479167, "G_mean": 0.33999405421006185, "G_std": 0.16477281646803021, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.295637607574463, "grad_norm": 1.0615944862365723, "tokens_seen": 95363072, "tokens_per_sec": 18308.291881490582, "vram_gb": 5.684933632, "lambda": 0.3163602016866207, "lambda_min": 0.09671694040298462, "lambda_max": 0.6391307711601257, "S_mean": 0.3236338297526042, "G_mean": 0.33763236133381724, "G_std": 0.16321454585219422, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.186154127120972, "grad_norm": 1.0141853094100952, "tokens_seen": 95526912, "tokens_per_sec": 18308.450498843653, "vram_gb": 5.684933632, "lambda": 0.3164376424004634, "lambda_min": 0.09670058637857437, "lambda_max": 0.6393966674804688, "S_mean": 0.31524658203125, "G_mean": 0.3383478192457308, "G_std": 0.16347413028900823, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.338939905166626, "grad_norm": 1.141696810722351, "tokens_seen": 95690752, "tokens_per_sec": 18308.40137462109, "vram_gb": 5.684933632, "lambda": 0.3165389262139797, "lambda_min": 0.09672290086746216, "lambda_max": 0.6396684050559998, "S_mean": 0.3200887044270833, "G_mean": 0.34029906437111396, "G_std": 0.16278917691670358, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.236794948577881, "grad_norm": 1.04707932472229, "tokens_seen": 95854592, "tokens_per_sec": 18308.734337667083, "vram_gb": 5.684933632, "lambda": 0.3166080095494787, "lambda_min": 0.09673041105270386, "lambda_max": 0.6400037407875061, "S_mean": 0.316802978515625, "G_mean": 0.33812305765847367, "G_std": 0.16319536254741251, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.276752471923828, "grad_norm": 1.1669235229492188, "tokens_seen": 96018432, "tokens_per_sec": 18308.793088183575, "vram_gb": 5.684933632, "lambda": 0.3166909177477161, "lambda_min": 0.09671889990568161, "lambda_max": 0.6403067111968994, "S_mean": 0.3040568033854167, "G_mean": 0.3399431367094318, "G_std": 0.1637212100904435, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.2602925300598145, "grad_norm": 1.00978422164917, "tokens_seen": 96182272, "tokens_per_sec": 18308.946533068465, "vram_gb": 5.684933632, "lambda": 0.3167643683652083, "lambda_min": 0.09672737121582031, "lambda_max": 0.6405587792396545, "S_mean": 0.3077901204427083, "G_mean": 0.34166207428400713, "G_std": 0.1640460342168808, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.309720516204834, "grad_norm": 1.0701998472213745, "tokens_seen": 96346112, "tokens_per_sec": 18308.904225678467, "vram_gb": 5.684933632, "lambda": 0.3168382979929447, "lambda_min": 0.09672803431749344, "lambda_max": 0.640635073184967, "S_mean": 0.3077646891276042, "G_mean": 0.33923713870656985, "G_std": 0.163046066959699, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.575554847717285, "grad_norm": 1.2029314041137695, "tokens_seen": 96509952, "tokens_per_sec": 18309.07691248692, "vram_gb": 5.684933632, "lambda": 0.3169066058471799, "lambda_min": 0.09671622514724731, "lambda_max": 0.6408030390739441, "S_mean": 0.3169657389322917, "G_mean": 0.3403239132215579, "G_std": 0.16387350492489836, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.429588794708252, "grad_norm": 1.0270118713378906, "tokens_seen": 96673792, "tokens_per_sec": 18309.296340303204, "vram_gb": 5.684933632, "lambda": 0.316969799498717, "lambda_min": 0.09668560326099396, "lambda_max": 0.6410444974899292, "S_mean": 0.3143513997395833, "G_mean": 0.3423341928670804, "G_std": 0.16280285455286503, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.443935871124268, "grad_norm": 1.0643770694732666, "tokens_seen": 96837632, "tokens_per_sec": 18309.22343907252, "vram_gb": 5.684933632, "lambda": 0.3170322844137748, "lambda_min": 0.09669764339923859, "lambda_max": 0.6412525177001953, "S_mean": 0.3310953776041667, "G_mean": 0.34544164842615527, "G_std": 0.16254492577475807, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.4206578731536865, "grad_norm": 1.1039292812347412, "tokens_seen": 97001472, "tokens_per_sec": 18309.085207970296, "vram_gb": 5.684933632, "lambda": 0.31711076758801937, "lambda_min": 0.09668560326099396, "lambda_max": 0.6415305137634277, "S_mean": 0.3095499674479167, "G_mean": 0.34383001178503036, "G_std": 0.16340605355799198, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.257090091705322, "grad_norm": 1.0750523805618286, "tokens_seen": 97165312, "tokens_per_sec": 18309.308834011346, "vram_gb": 5.684933632, "lambda": 0.31718006016065675, "lambda_min": 0.0966896191239357, "lambda_max": 0.641734778881073, "S_mean": 0.3114522298177083, "G_mean": 0.3408999626214306, "G_std": 0.16324198339134455, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.2917320728302, "grad_norm": 1.0890796184539795, "tokens_seen": 97329152, "tokens_per_sec": 18309.49026893817, "vram_gb": 5.684933632, "lambda": 0.3172741200154026, "lambda_min": 0.09670662879943848, "lambda_max": 0.6421252489089966, "S_mean": 0.304229736328125, "G_mean": 0.34025574033148587, "G_std": 0.16243583235579231, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.305729150772095, "grad_norm": 1.0462244749069214, "tokens_seen": 97492992, "tokens_per_sec": 18309.306434610262, "vram_gb": 5.684933632, "lambda": 0.31733305379748344, "lambda_min": 0.09669454395771027, "lambda_max": 0.642337441444397, "S_mean": 0.32940673828125, "G_mean": 0.3419168271745245, "G_std": 0.16276114239978293, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.287417888641357, "grad_norm": 1.0246376991271973, "tokens_seen": 97656832, "tokens_per_sec": 18309.293697009547, "vram_gb": 5.684933632, "lambda": 0.3173962002620101, "lambda_min": 0.09667716175317764, "lambda_max": 0.6423997282981873, "S_mean": 0.3179728190104167, "G_mean": 0.34694111820620793, "G_std": 0.16378684396234652, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.462307453155518, "grad_norm": 1.1623731851577759, "tokens_seen": 97820672, "tokens_per_sec": 18309.374230338417, "vram_gb": 5.684933632, "lambda": 0.3174550735081236, "lambda_min": 0.096715047955513, "lambda_max": 0.6425905227661133, "S_mean": 0.3232269287109375, "G_mean": 0.3404680302677055, "G_std": 0.1639274999809762, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.360137462615967, "grad_norm": 1.0239651203155518, "tokens_seen": 97984512, "tokens_per_sec": 18309.347163500453, "vram_gb": 5.684933632, "lambda": 0.31750936961422366, "lambda_min": 0.09670644998550415, "lambda_max": 0.6428459882736206, "S_mean": 0.3197123209635417, "G_mean": 0.34163496022423107, "G_std": 0.1635090624137471, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.2870683670043945, "grad_norm": 0.961722195148468, "tokens_seen": 98148352, "tokens_per_sec": 18309.52740438414, "vram_gb": 5.684933632, "lambda": 0.3175949805105726, "lambda_min": 0.09672381728887558, "lambda_max": 0.6430150270462036, "S_mean": 0.3038228352864583, "G_mean": 0.3396819991370042, "G_std": 0.16162662215841314, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.335920524597168, "val_ppl": 76.39525023929815, "lambda": 0.3176627963160475, "lambda_min": 0.09670225530862808, "lambda_max": 0.6432420015335083, "lambda_per_layer": [0.18085598945617676, 0.1443270444869995, 0.1546369343996048, 0.12867967784404755, 0.11799386888742447, 0.09670225530862808, 0.11857030540704727, 0.17088893055915833, 0.2310791015625, 0.5038225650787354, 0.44472718238830566, 0.391945481300354, 0.3538212180137634, 0.6432420015335083, 0.451026976108551, 0.5686854720115662, 0.5633997917175293, 0.44344663619995117, 0.3102770745754242, 0.45657312870025635, 0.3940618634223938, 0.21701130270957947, 0.31596189737319946, 0.2221704125404358], "S_mean": 0.3223673502604167, "G_mean": 0.3413576711900532, "G_std": 0.1630307863621662, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5370.658619880676, "nan_failures": 0} diff --git a/g_only_s0/config.json b/g_only_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..eb1798b38e8b7c0e825820b92dcedd7285c13cf2 --- /dev/null +++ b/g_only_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s0", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/g_only_s0/final.pt b/g_only_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..16ab5a36560819f52bba646f734997f6d460d272 --- /dev/null +++ b/g_only_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0fb40abef9f7c1c7d9b7557e45a4bf5793c3ed6a875cbe9faef05d3b58c23b64 +size 253629735 diff --git a/g_only_s0/metrics.jsonl b/g_only_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..7597b43f7c310c75bbf4a2f738fe25b2c4a0d47e --- /dev/null +++ b/g_only_s0/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 0, "config": {"variant": "g_only", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s0", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.920852661132812, "grad_norm": 9.508766174316406, "tokens_seen": 8192, "tokens_per_sec": 8285.362591785324, "vram_gb": 5.768167424, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4967447916666667, "G_std": 0.09092203776041667, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.776525497436523, "grad_norm": 1.9907896518707275, "tokens_seen": 172032, "tokens_per_sec": 16924.340256323754, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4545084635416667, "G_std": 0.07177734375, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.03318977355957, "grad_norm": 1.742348074913025, "tokens_seen": 335872, "tokens_per_sec": 17292.17320286601, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4269205729166667, "G_std": 0.07720947265625, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.080991744995117, "grad_norm": 1.5474638938903809, "tokens_seen": 499712, "tokens_per_sec": 17410.622872176722, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4256184895833333, "G_std": 0.0850372314453125, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.551734924316406, "grad_norm": 0.7812139987945557, "tokens_seen": 663552, "tokens_per_sec": 17503.056201981963, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.419189453125, "G_std": 0.09417724609375, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.325989246368408, "grad_norm": 0.7770202159881592, "tokens_seen": 827392, "tokens_per_sec": 17564.615388666032, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4297688802083333, "G_std": 0.11756388346354167, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.094098329544067, "grad_norm": 0.7293755412101746, "tokens_seen": 991232, "tokens_per_sec": 17606.38435425747, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4077962239583333, "G_std": 0.12464396158854167, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.985921382904053, "grad_norm": 0.6033894419670105, "tokens_seen": 1155072, "tokens_per_sec": 17627.173205887277, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4068196614583333, "G_std": 0.134246826171875, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.76617956161499, "grad_norm": 0.7019789218902588, "tokens_seen": 1318912, "tokens_per_sec": 17651.217708638378, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3974609375, "G_std": 0.14316813151041666, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.837290287017822, "grad_norm": 0.5393996238708496, "tokens_seen": 1482752, "tokens_per_sec": 17670.19312500158, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3920084635416667, "G_std": 0.14719645182291666, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.682165145874023, "grad_norm": 0.5372763276100159, "tokens_seen": 1646592, "tokens_per_sec": 17686.528976094123, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3943684895833333, "G_std": 0.16119384765625, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.844391822814941, "grad_norm": 0.6566638350486755, "tokens_seen": 1810432, "tokens_per_sec": 17698.464175634945, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3870442708333333, "G_std": 0.16133626302083334, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.541489362716675, "grad_norm": 0.46757692098617554, "tokens_seen": 1974272, "tokens_per_sec": 17707.29003882251, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388671875, "G_std": 0.158447265625, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.650808811187744, "grad_norm": 0.5063909888267517, "tokens_seen": 2138112, "tokens_per_sec": 17715.906913843377, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.391845703125, "G_std": 0.1656494140625, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.406080722808838, "grad_norm": 0.4914308786392212, "tokens_seen": 2301952, "tokens_per_sec": 17722.879414168026, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846842447916667, "G_std": 0.16707356770833334, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.5533928871154785, "grad_norm": 0.45962774753570557, "tokens_seen": 2465792, "tokens_per_sec": 17730.314271454008, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3966471354166667, "G_std": 0.172607421875, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.348932266235352, "grad_norm": 0.43990591168403625, "tokens_seen": 2629632, "tokens_per_sec": 17735.522011219997, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3968912760416667, "G_std": 0.17628987630208334, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.400684833526611, "grad_norm": 0.49522334337234497, "tokens_seen": 2793472, "tokens_per_sec": 17740.359802393577, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41357421875, "G_std": 0.18705240885416666, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.547396659851074, "grad_norm": 0.5029144287109375, "tokens_seen": 2957312, "tokens_per_sec": 17744.110643951757, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4055989583333333, "G_std": 0.18393961588541666, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.404249906539917, "grad_norm": 0.5134385228157043, "tokens_seen": 3121152, "tokens_per_sec": 17749.1856816722, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3932291666666667, "G_std": 0.1875, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.296067476272583, "grad_norm": 0.44473886489868164, "tokens_seen": 3284992, "tokens_per_sec": 17748.8434147315, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4051106770833333, "G_std": 0.19150797526041666, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.414352893829346, "grad_norm": 0.4371470808982849, "tokens_seen": 3448832, "tokens_per_sec": 17748.444851685275, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4239908854166667, "G_std": 0.2012939453125, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.362046241760254, "grad_norm": 0.4741216003894806, "tokens_seen": 3612672, "tokens_per_sec": 17750.02622889167, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4229329427083333, "G_std": 0.20027669270833334, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.4023778438568115, "grad_norm": 0.465218186378479, "tokens_seen": 3776512, "tokens_per_sec": 17750.07836128109, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4178466796875, "G_std": 0.1964111328125, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.175987482070923, "grad_norm": 0.4350946843624115, "tokens_seen": 3940352, "tokens_per_sec": 17750.45915108982, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4530436197916667, "G_std": 0.20735677083333334, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.06322169303894, "grad_norm": 0.4852936863899231, "tokens_seen": 4104192, "tokens_per_sec": 17749.702406045853, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48193359375, "G_std": 0.21756998697916666, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.2870381593704225, "val_ppl": 537.5588053772019, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4619140625, "G_std": 0.21162923177083334, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.252084016799927, "grad_norm": 0.512508749961853, "tokens_seen": 4268032, "tokens_per_sec": 17500.885303019128, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4698079427083333, "G_std": 0.21586100260416666, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.176004409790039, "grad_norm": 0.494626522064209, "tokens_seen": 4431872, "tokens_per_sec": 17510.657281713775, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4944661458333333, "G_std": 0.21553548177083334, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.14801549911499, "grad_norm": 0.45634740591049194, "tokens_seen": 4595712, "tokens_per_sec": 17517.041921837244, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4838053385416667, "G_std": 0.21248372395833334, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.200341701507568, "grad_norm": 0.4855337142944336, "tokens_seen": 4759552, "tokens_per_sec": 17526.083263795732, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.49560546875, "G_std": 0.21207682291666666, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.143208742141724, "grad_norm": 0.46201616525650024, "tokens_seen": 4923392, "tokens_per_sec": 17535.06817579709, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5352376302083334, "G_std": 0.21673583984375, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.034154415130615, "grad_norm": 0.43268874287605286, "tokens_seen": 5087232, "tokens_per_sec": 17542.678888404793, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5078125, "G_std": 0.21577962239583334, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.126905918121338, "grad_norm": 0.4942730665206909, "tokens_seen": 5251072, "tokens_per_sec": 17550.54044508145, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5270182291666666, "G_std": 0.21199544270833334, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.009082794189453, "grad_norm": 0.5162107348442078, "tokens_seen": 5414912, "tokens_per_sec": 17558.53557945352, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5542805989583334, "G_std": 0.21461995442708334, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.001175403594971, "grad_norm": 0.4639512300491333, "tokens_seen": 5578752, "tokens_per_sec": 17565.384150060923, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5082194010416666, "G_std": 0.21272786458333334, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.976337671279907, "grad_norm": 0.5128575563430786, "tokens_seen": 5742592, "tokens_per_sec": 17571.49634011417, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.54541015625, "G_std": 0.21075439453125, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.084268569946289, "grad_norm": 0.515507698059082, "tokens_seen": 5906432, "tokens_per_sec": 17577.59408986415, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.528076171875, "G_std": 0.20998128255208334, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.909168243408203, "grad_norm": 0.46901413798332214, "tokens_seen": 6070272, "tokens_per_sec": 17584.095467398703, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5430501302083334, "G_std": 0.21232096354166666, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.903528213500977, "grad_norm": 0.491560161113739, "tokens_seen": 6234112, "tokens_per_sec": 17590.72709382811, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5521647135416666, "G_std": 0.2083740234375, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.915542840957642, "grad_norm": 0.48613440990448, "tokens_seen": 6397952, "tokens_per_sec": 17596.91853021462, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5515950520833334, "G_std": 0.2042236328125, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.808903694152832, "grad_norm": 0.5154011845588684, "tokens_seen": 6561792, "tokens_per_sec": 17601.81133401528, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5675455729166666, "G_std": 0.20033772786458334, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.790546417236328, "grad_norm": 0.48544734716415405, "tokens_seen": 6725632, "tokens_per_sec": 17606.84561267368, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.569091796875, "G_std": 0.19783528645833334, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.785609722137451, "grad_norm": 0.5089153051376343, "tokens_seen": 6889472, "tokens_per_sec": 17611.34234937454, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.559326171875, "G_std": 0.19891357421875, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.731289625167847, "grad_norm": 0.5239635705947876, "tokens_seen": 7053312, "tokens_per_sec": 17615.153728854773, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.570068359375, "G_std": 0.19657389322916666, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.892175912857056, "grad_norm": 0.4708206057548523, "tokens_seen": 7217152, "tokens_per_sec": 17619.684355001944, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5540364583333334, "G_std": 0.1973876953125, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.723673343658447, "grad_norm": 0.49590393900871277, "tokens_seen": 7380992, "tokens_per_sec": 17623.916443715978, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5313720703125, "G_std": 0.20235188802083334, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.763762474060059, "grad_norm": 0.5124049782752991, "tokens_seen": 7544832, "tokens_per_sec": 17626.50066336328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5430501302083334, "G_std": 0.20235188802083334, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.727604389190674, "grad_norm": 0.4937100410461426, "tokens_seen": 7708672, "tokens_per_sec": 17629.441044753934, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5469156901041666, "G_std": 0.19694010416666666, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.767548322677612, "grad_norm": 0.5273558497428894, "tokens_seen": 7872512, "tokens_per_sec": 17632.160710418946, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5422770182291666, "G_std": 0.19755045572916666, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.886250734329224, "grad_norm": 0.5367128849029541, "tokens_seen": 8036352, "tokens_per_sec": 17635.28210690449, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5409342447916666, "G_std": 0.19673665364583334, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.809383153915405, "grad_norm": 0.5610921382904053, "tokens_seen": 8200192, "tokens_per_sec": 17637.21598174071, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5491943359375, "G_std": 0.19441731770833334, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.805833256244659, "val_ppl": 332.23191235425827, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.542236328125, "G_std": 0.19594319661458334, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.61547064781189, "grad_norm": 0.4913500249385834, "tokens_seen": 8364032, "tokens_per_sec": 17512.536250595233, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5557454427083334, "G_std": 0.19657389322916666, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.684388637542725, "grad_norm": 0.5149298906326294, "tokens_seen": 8527872, "tokens_per_sec": 17517.492577186404, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5449625651041666, "G_std": 0.19718424479166666, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.750096797943115, "grad_norm": 0.5181910991668701, "tokens_seen": 8691712, "tokens_per_sec": 17522.75970698532, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5424397786458334, "G_std": 0.19045003255208334, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.718190670013428, "grad_norm": 0.5265754461288452, "tokens_seen": 8855552, "tokens_per_sec": 17526.03310282762, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5396728515625, "G_std": 0.19549560546875, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.600366592407227, "grad_norm": 0.5097454190254211, "tokens_seen": 9019392, "tokens_per_sec": 17530.87556710276, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.531982421875, "G_std": 0.19242350260416666, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.714572906494141, "grad_norm": 0.5396753549575806, "tokens_seen": 9183232, "tokens_per_sec": 17534.93705739281, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5465087890625, "G_std": 0.19594319661458334, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.529923915863037, "grad_norm": 0.5366814136505127, "tokens_seen": 9347072, "tokens_per_sec": 17538.901040852765, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5310465494791666, "G_std": 0.19209798177083334, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.571640968322754, "grad_norm": 0.49884483218193054, "tokens_seen": 9510912, "tokens_per_sec": 17542.74447997542, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5498860677083334, "G_std": 0.19063313802083334, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.62973690032959, "grad_norm": 0.528828501701355, "tokens_seen": 9674752, "tokens_per_sec": 17547.322395107625, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5210774739583334, "G_std": 0.18682861328125, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.510051965713501, "grad_norm": 0.5292899012565613, "tokens_seen": 9838592, "tokens_per_sec": 17551.519108352448, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5207112630208334, "G_std": 0.19028727213541666, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.6987464427948, "grad_norm": 0.5158385634422302, "tokens_seen": 10002432, "tokens_per_sec": 17555.233019766107, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5294189453125, "G_std": 0.1943359375, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.655746698379517, "grad_norm": 0.5440499782562256, "tokens_seen": 10166272, "tokens_per_sec": 17559.48229744122, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5279134114583334, "G_std": 0.19327799479166666, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.624995231628418, "grad_norm": 0.592824399471283, "tokens_seen": 10330112, "tokens_per_sec": 17563.517274674814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5297444661458334, "G_std": 0.19722493489583334, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.480140209197998, "grad_norm": 0.5634276270866394, "tokens_seen": 10493952, "tokens_per_sec": 17566.98396442125, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5185139973958334, "G_std": 0.19063313802083334, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.567570924758911, "grad_norm": 0.5553045272827148, "tokens_seen": 10657792, "tokens_per_sec": 17570.343489706705, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.522705078125, "G_std": 0.18896484375, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.526724338531494, "grad_norm": 0.539531946182251, "tokens_seen": 10821632, "tokens_per_sec": 17573.16914096183, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5242919921875, "G_std": 0.19317626953125, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.613200902938843, "grad_norm": 0.5260668396949768, "tokens_seen": 10985472, "tokens_per_sec": 17576.47854934943, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5221761067708334, "G_std": 0.19004313151041666, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.427840709686279, "grad_norm": 0.5245338678359985, "tokens_seen": 11149312, "tokens_per_sec": 17579.016736401692, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.520751953125, "G_std": 0.18880208333333334, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.2791547775268555, "grad_norm": 0.5502124428749084, "tokens_seen": 11313152, "tokens_per_sec": 17582.620828160118, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5063069661458334, "G_std": 0.187286376953125, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.566000938415527, "grad_norm": 0.5442852973937988, "tokens_seen": 11476992, "tokens_per_sec": 17585.5896940899, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5053304036458334, "G_std": 0.186248779296875, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.50579571723938, "grad_norm": 0.5430960059165955, "tokens_seen": 11640832, "tokens_per_sec": 17588.675724785455, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5075887044270834, "G_std": 0.18381754557291666, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.409761190414429, "grad_norm": 0.5532015562057495, "tokens_seen": 11804672, "tokens_per_sec": 17591.005542967814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.506591796875, "G_std": 0.18995157877604166, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.499024868011475, "grad_norm": 0.5853569507598877, "tokens_seen": 11968512, "tokens_per_sec": 17592.673137152306, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5242513020833334, "G_std": 0.192718505859375, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.412709712982178, "grad_norm": 0.5538255572319031, "tokens_seen": 12132352, "tokens_per_sec": 17595.009807543523, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4999186197916667, "G_std": 0.18854777018229166, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.649762392044067, "grad_norm": 0.6010509133338928, "tokens_seen": 12296192, "tokens_per_sec": 17597.12425751674, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.49700927734375, "G_std": 0.18772379557291666, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.519615292549133, "val_ppl": 249.5390192014915, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5020345052083334, "G_std": 0.18675740559895834, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.434514045715332, "grad_norm": 0.5838738679885864, "tokens_seen": 12460032, "tokens_per_sec": 17513.27983202102, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5090128580729166, "G_std": 0.189056396484375, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.283675909042358, "grad_norm": 0.5540887117385864, "tokens_seen": 12623872, "tokens_per_sec": 17516.663331622483, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4920247395833333, "G_std": 0.18678792317708334, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.523560523986816, "grad_norm": 0.6488651633262634, "tokens_seen": 12787712, "tokens_per_sec": 17520.293005892596, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5075480143229166, "G_std": 0.19283040364583334, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.425835132598877, "grad_norm": 0.5926249623298645, "tokens_seen": 12951552, "tokens_per_sec": 17523.867994272863, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4908040364583333, "G_std": 0.18861897786458334, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.537646055221558, "grad_norm": 0.6044854521751404, "tokens_seen": 13115392, "tokens_per_sec": 17527.147480706444, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4931844075520833, "G_std": 0.188873291015625, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.332539319992065, "grad_norm": 0.5604832768440247, "tokens_seen": 13279232, "tokens_per_sec": 17530.230352272883, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4916585286458333, "G_std": 0.18675740559895834, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.3900511264801025, "grad_norm": 0.5199568271636963, "tokens_seen": 13443072, "tokens_per_sec": 17533.34029325328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4882405598958333, "G_std": 0.188201904296875, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.3794896602630615, "grad_norm": 0.5837953686714172, "tokens_seen": 13606912, "tokens_per_sec": 17536.110351240375, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4827473958333333, "G_std": 0.18982950846354166, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.472980260848999, "grad_norm": 0.5330631732940674, "tokens_seen": 13770752, "tokens_per_sec": 17539.14211355013, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4774576822916667, "G_std": 0.18693033854166666, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.404017686843872, "grad_norm": 0.5771147012710571, "tokens_seen": 13934592, "tokens_per_sec": 17542.09027849432, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.484619140625, "G_std": 0.189453125, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.3435728549957275, "grad_norm": 0.658695638179779, "tokens_seen": 14098432, "tokens_per_sec": 17545.04277895969, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.486328125, "G_std": 0.1881103515625, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.428950071334839, "grad_norm": 0.6910367012023926, "tokens_seen": 14262272, "tokens_per_sec": 17548.124473230728, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4794108072916667, "G_std": 0.188232421875, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.397101163864136, "grad_norm": 0.5540364980697632, "tokens_seen": 14426112, "tokens_per_sec": 17551.205989215345, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4796142578125, "G_std": 0.18641153971354166, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.316443920135498, "grad_norm": 0.540654718875885, "tokens_seen": 14589952, "tokens_per_sec": 17554.07904767505, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4614054361979167, "G_std": 0.18194580078125, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.361908912658691, "grad_norm": 0.680677592754364, "tokens_seen": 14753792, "tokens_per_sec": 17557.19145330057, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4594523111979167, "G_std": 0.185089111328125, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.297167778015137, "grad_norm": 0.5530894994735718, "tokens_seen": 14917632, "tokens_per_sec": 17559.89871846751, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4691975911458333, "G_std": 0.19173177083333334, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.405107736587524, "grad_norm": 0.6714523434638977, "tokens_seen": 15081472, "tokens_per_sec": 17562.67132613, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4773966471354167, "G_std": 0.18779500325520834, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.359262466430664, "grad_norm": 0.5947059392929077, "tokens_seen": 15245312, "tokens_per_sec": 17565.374586472277, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4798787434895833, "G_std": 0.18738810221354166, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.302817344665527, "grad_norm": 0.5822960734367371, "tokens_seen": 15409152, "tokens_per_sec": 17567.805523633207, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.46502685546875, "G_std": 0.186004638671875, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.247166633605957, "grad_norm": 0.5683612823486328, "tokens_seen": 15572992, "tokens_per_sec": 17570.178102120633, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4611612955729167, "G_std": 0.18365478515625, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.20725154876709, "grad_norm": 0.6057959198951721, "tokens_seen": 15736832, "tokens_per_sec": 17571.952199156243, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4652506510416667, "G_std": 0.18320719401041666, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.19442892074585, "grad_norm": 0.629124641418457, "tokens_seen": 15900672, "tokens_per_sec": 17572.737063265067, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44989013671875, "G_std": 0.179779052734375, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.17106819152832, "grad_norm": 0.6192393898963928, "tokens_seen": 16064512, "tokens_per_sec": 17573.458345983803, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.45770263671875, "G_std": 0.18756103515625, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.052915573120117, "grad_norm": 0.6046925187110901, "tokens_seen": 16228352, "tokens_per_sec": 17573.66858115642, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4552408854166667, "G_std": 0.18155924479166666, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.219227313995361, "grad_norm": 0.6224650144577026, "tokens_seen": 16392192, "tokens_per_sec": 17573.725459083653, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.45849609375, "G_std": 0.18231201171875, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.3300078272819515, "val_ppl": 206.43959001086134, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4610392252604167, "G_std": 0.18220011393229166, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.110260009765625, "grad_norm": 0.7038714289665222, "tokens_seen": 16556032, "tokens_per_sec": 17508.721751518857, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4582926432291667, "G_std": 0.18379720052083334, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.331557512283325, "grad_norm": 0.9993065595626831, "tokens_seen": 16719872, "tokens_per_sec": 17508.224259262675, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47747802734375, "G_std": 0.19037882486979166, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.301686525344849, "grad_norm": 0.5749878883361816, "tokens_seen": 16883712, "tokens_per_sec": 17505.662005796134, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4581705729166667, "G_std": 0.18604532877604166, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.223100900650024, "grad_norm": 0.7211536169052124, "tokens_seen": 17047552, "tokens_per_sec": 17503.505715946507, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.46868896484375, "G_std": 0.18378702799479166, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.243688106536865, "grad_norm": 0.6006498336791992, "tokens_seen": 17211392, "tokens_per_sec": 17503.699029694115, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4566446940104167, "G_std": 0.18339029947916666, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.209092140197754, "grad_norm": 0.6924734115600586, "tokens_seen": 17375232, "tokens_per_sec": 17503.019342924763, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4367167154947917, "G_std": 0.17888387044270834, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.268516778945923, "grad_norm": 0.6320501565933228, "tokens_seen": 17539072, "tokens_per_sec": 17502.474799504467, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4484456380208333, "G_std": 0.182647705078125, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.295008420944214, "grad_norm": 0.6199290752410889, "tokens_seen": 17702912, "tokens_per_sec": 17499.74593043852, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4414265950520833, "G_std": 0.17948404947916666, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.270703315734863, "grad_norm": 0.6309139132499695, "tokens_seen": 17866752, "tokens_per_sec": 17501.882905032762, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.441680908203125, "G_std": 0.17822265625, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.207284212112427, "grad_norm": 0.5874049067497253, "tokens_seen": 18030592, "tokens_per_sec": 17504.15664970328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4357808430989583, "G_std": 0.178741455078125, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.200459003448486, "grad_norm": 0.600984513759613, "tokens_seen": 18194432, "tokens_per_sec": 17506.60974614089, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4326883951822917, "G_std": 0.17540486653645834, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.101091384887695, "grad_norm": 0.632282018661499, "tokens_seen": 18358272, "tokens_per_sec": 17509.05627121941, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.449371337890625, "G_std": 0.18125406901041666, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.216426610946655, "grad_norm": 0.6213477253913879, "tokens_seen": 18522112, "tokens_per_sec": 17511.415220941482, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.443023681640625, "G_std": 0.18242390950520834, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.1462554931640625, "grad_norm": 0.7338560819625854, "tokens_seen": 18685952, "tokens_per_sec": 17513.750275021153, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4376627604166667, "G_std": 0.17533365885416666, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.147456169128418, "grad_norm": 0.6771952509880066, "tokens_seen": 18849792, "tokens_per_sec": 17516.05520842749, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4483235677083333, "G_std": 0.18105061848958334, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.12248158454895, "grad_norm": 0.7123768925666809, "tokens_seen": 19013632, "tokens_per_sec": 17518.11263507456, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4344380696614583, "G_std": 0.17751057942708334, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.055939674377441, "grad_norm": 0.6662951707839966, "tokens_seen": 19177472, "tokens_per_sec": 17520.07089332392, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.434173583984375, "G_std": 0.17543538411458334, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.172410726547241, "grad_norm": 0.7263441681861877, "tokens_seen": 19341312, "tokens_per_sec": 17522.298362059664, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4397684733072917, "G_std": 0.177886962890625, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.244019269943237, "grad_norm": 0.6788997650146484, "tokens_seen": 19505152, "tokens_per_sec": 17524.490276198976, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4410909016927083, "G_std": 0.1815185546875, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.125135898590088, "grad_norm": 0.6262181997299194, "tokens_seen": 19668992, "tokens_per_sec": 17526.566026203538, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4363505045572917, "G_std": 0.17742919921875, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.222487926483154, "grad_norm": 0.6817317605018616, "tokens_seen": 19832832, "tokens_per_sec": 17528.563670255622, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.43060302734375, "G_std": 0.181427001953125, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.034891366958618, "grad_norm": 0.6267676949501038, "tokens_seen": 19996672, "tokens_per_sec": 17530.482357352223, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4239095052083333, "G_std": 0.17470296223958334, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.082193851470947, "grad_norm": 0.6946831941604614, "tokens_seen": 20160512, "tokens_per_sec": 17531.97844344791, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.434173583984375, "G_std": 0.17523193359375, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.0274577140808105, "grad_norm": 0.6562761664390564, "tokens_seen": 20324352, "tokens_per_sec": 17533.573080970695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.436767578125, "G_std": 0.1796875, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.195958614349365, "grad_norm": 0.6614639163017273, "tokens_seen": 20488192, "tokens_per_sec": 17535.329882318096, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4343058268229167, "G_std": 0.17918904622395834, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.174068176746369, "val_ppl": 176.6319478120215, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4325968424479167, "G_std": 0.17877197265625, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.096902370452881, "grad_norm": 0.6298574805259705, "tokens_seen": 20652032, "tokens_per_sec": 17485.781777883745, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.430694580078125, "G_std": 0.17490641276041666, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 4.983757495880127, "grad_norm": 0.7001813054084778, "tokens_seen": 20815872, "tokens_per_sec": 17487.625887366903, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4347737630208333, "G_std": 0.18032328287760416, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.2068703174591064, "grad_norm": 0.692969799041748, "tokens_seen": 20979712, "tokens_per_sec": 17489.331121406278, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4235331217447917, "G_std": 0.17801920572916666, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.080707550048828, "grad_norm": 0.7240349054336548, "tokens_seen": 21143552, "tokens_per_sec": 17491.146451392837, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4281209309895833, "G_std": 0.175201416015625, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 4.943805456161499, "grad_norm": 0.6491481065750122, "tokens_seen": 21307392, "tokens_per_sec": 17493.072765351913, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4200948079427083, "G_std": 0.177886962890625, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.96563720703125, "grad_norm": 0.6505531072616577, "tokens_seen": 21471232, "tokens_per_sec": 17494.92007768691, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4268290201822917, "G_std": 0.179534912109375, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.1027467250823975, "grad_norm": 0.6397854685783386, "tokens_seen": 21635072, "tokens_per_sec": 17496.85187487887, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41314697265625, "G_std": 0.1744384765625, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.051153898239136, "grad_norm": 0.6779762506484985, "tokens_seen": 21798912, "tokens_per_sec": 17498.933043430323, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.421112060546875, "G_std": 0.17713419596354166, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.042128562927246, "grad_norm": 0.6739419102668762, "tokens_seen": 21962752, "tokens_per_sec": 17500.850889974394, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.415191650390625, "G_std": 0.17584228515625, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.044853687286377, "grad_norm": 0.6376251578330994, "tokens_seen": 22126592, "tokens_per_sec": 17502.943947282714, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4313252766927083, "G_std": 0.17764790852864584, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.0199806690216064, "grad_norm": 0.614287257194519, "tokens_seen": 22290432, "tokens_per_sec": 17505.06602912965, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4196573893229167, "G_std": 0.176605224609375, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.084626913070679, "grad_norm": 0.6810340285301208, "tokens_seen": 22454272, "tokens_per_sec": 17507.225436045235, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4194844563802083, "G_std": 0.17483011881510416, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.239521026611328, "grad_norm": 0.6853621602058411, "tokens_seen": 22618112, "tokens_per_sec": 17509.239014561354, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4131571451822917, "G_std": 0.17423502604166666, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.203991889953613, "grad_norm": 0.6762627363204956, "tokens_seen": 22781952, "tokens_per_sec": 17511.3193949814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.415252685546875, "G_std": 0.1770172119140625, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.986602544784546, "grad_norm": 0.6445426940917969, "tokens_seen": 22945792, "tokens_per_sec": 17513.475765080646, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.425384521484375, "G_std": 0.17578125, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.26681923866272, "grad_norm": 0.7124970555305481, "tokens_seen": 23109632, "tokens_per_sec": 17515.28855552391, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.420623779296875, "G_std": 0.17322794596354166, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.006627082824707, "grad_norm": 0.6906690001487732, "tokens_seen": 23273472, "tokens_per_sec": 17517.345657500264, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4202779134114583, "G_std": 0.17325337727864584, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.030837535858154, "grad_norm": 0.6489294767379761, "tokens_seen": 23437312, "tokens_per_sec": 17519.48673060537, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4146931966145833, "G_std": 0.17571512858072916, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.077397584915161, "grad_norm": 0.7096554636955261, "tokens_seen": 23601152, "tokens_per_sec": 17521.381737815624, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.419219970703125, "G_std": 0.17529296875, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.883606910705566, "grad_norm": 0.6299237012863159, "tokens_seen": 23764992, "tokens_per_sec": 17523.1027199322, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4025675455729167, "G_std": 0.17490641276041666, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.036141872406006, "grad_norm": 0.6960007548332214, "tokens_seen": 23928832, "tokens_per_sec": 17525.052129222342, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4069112141927083, "G_std": 0.1750946044921875, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.070178031921387, "grad_norm": 0.6852125525474548, "tokens_seen": 24092672, "tokens_per_sec": 17526.968813680185, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4210306803385417, "G_std": 0.17851765950520834, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.921377182006836, "grad_norm": 0.6937862038612366, "tokens_seen": 24256512, "tokens_per_sec": 17528.68607654121, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4150899251302083, "G_std": 0.1712493896484375, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.077439308166504, "grad_norm": 0.6599253416061401, "tokens_seen": 24420352, "tokens_per_sec": 17530.216131177625, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4201558430989583, "G_std": 0.17365519205729166, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.068130016326904, "grad_norm": 0.7045713067054749, "tokens_seen": 24584192, "tokens_per_sec": 17531.41567099475, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40338134765625, "G_std": 0.170196533203125, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.0173819899559025, "val_ppl": 151.01542591217697, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.409332275390625, "G_std": 0.17232259114583334, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.013175010681152, "grad_norm": 0.6942648887634277, "tokens_seen": 24748032, "tokens_per_sec": 17490.1997138607, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4050089518229167, "G_std": 0.17301432291666666, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.145960569381714, "grad_norm": 0.7472270131111145, "tokens_seen": 24911872, "tokens_per_sec": 17491.80387593007, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4163614908854167, "G_std": 0.17736307779947916, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.889183282852173, "grad_norm": 0.6182893514633179, "tokens_seen": 25075712, "tokens_per_sec": 17492.70793842404, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4075419108072917, "G_std": 0.1713714599609375, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.963860511779785, "grad_norm": 0.7061532139778137, "tokens_seen": 25239552, "tokens_per_sec": 17493.420969040388, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.414886474609375, "G_std": 0.1731414794921875, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.891984224319458, "grad_norm": 0.6967146396636963, "tokens_seen": 25403392, "tokens_per_sec": 17493.820662857695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4037577311197917, "G_std": 0.172607421875, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.859045743942261, "grad_norm": 0.6706007719039917, "tokens_seen": 25567232, "tokens_per_sec": 17494.257145084073, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.407501220703125, "G_std": 0.17411295572916666, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 5.032615900039673, "grad_norm": 0.6717294454574585, "tokens_seen": 25731072, "tokens_per_sec": 17494.683062072578, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.412384033203125, "G_std": 0.17479960123697916, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.823154449462891, "grad_norm": 0.6468985676765442, "tokens_seen": 25894912, "tokens_per_sec": 17495.310692928695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.400726318359375, "G_std": 0.1700286865234375, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.867225408554077, "grad_norm": 0.7700868248939514, "tokens_seen": 26058752, "tokens_per_sec": 17495.788769021307, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4073994954427083, "G_std": 0.17798868815104166, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.824709415435791, "grad_norm": 0.64405357837677, "tokens_seen": 26222592, "tokens_per_sec": 17496.01358178738, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38946533203125, "G_std": 0.17126973470052084, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.788614749908447, "grad_norm": 0.755489706993103, "tokens_seen": 26386432, "tokens_per_sec": 17496.571263444566, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4090067545572917, "G_std": 0.17290242513020834, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.861300945281982, "grad_norm": 0.7083934545516968, "tokens_seen": 26550272, "tokens_per_sec": 17497.073744588277, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.410736083984375, "G_std": 0.17502848307291666, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.98565936088562, "grad_norm": 0.7051801085472107, "tokens_seen": 26714112, "tokens_per_sec": 17497.65017950106, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.398529052734375, "G_std": 0.17084757486979166, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.840130805969238, "grad_norm": 0.690192699432373, "tokens_seen": 26877952, "tokens_per_sec": 17498.55241797207, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3934224446614583, "G_std": 0.17444356282552084, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.028089761734009, "grad_norm": 0.7338056564331055, "tokens_seen": 27041792, "tokens_per_sec": 17499.43156800621, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4038289388020833, "G_std": 0.1710968017578125, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.8798768520355225, "grad_norm": 0.7133718729019165, "tokens_seen": 27205632, "tokens_per_sec": 17499.953994825828, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40118408203125, "G_std": 0.17322794596354166, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.9654998779296875, "grad_norm": 0.7061001062393188, "tokens_seen": 27369472, "tokens_per_sec": 17500.46730285067, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4032185872395833, "G_std": 0.17032368977864584, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.766084432601929, "grad_norm": 0.7204606533050537, "tokens_seen": 27533312, "tokens_per_sec": 17501.12383562545, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4085286458333333, "G_std": 0.17437744140625, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.921878337860107, "grad_norm": 1.1281492710113525, "tokens_seen": 27697152, "tokens_per_sec": 17501.539830124595, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3979593912760417, "G_std": 0.17179361979166666, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.960080623626709, "grad_norm": 0.7410067915916443, "tokens_seen": 27860992, "tokens_per_sec": 17501.476692790235, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3922017415364583, "G_std": 0.17340087890625, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.804123163223267, "grad_norm": 0.7400864958763123, "tokens_seen": 28024832, "tokens_per_sec": 17501.97180229088, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3995157877604167, "G_std": 0.17311604817708334, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.896920680999756, "grad_norm": 0.7062498331069946, "tokens_seen": 28188672, "tokens_per_sec": 17502.524556864853, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961283365885417, "G_std": 0.17128499348958334, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.804619073867798, "grad_norm": 0.708270251750946, "tokens_seen": 28352512, "tokens_per_sec": 17503.062564164728, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3955281575520833, "G_std": 0.17346700032552084, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.778301000595093, "grad_norm": 0.705034077167511, "tokens_seen": 28516352, "tokens_per_sec": 17503.36019027591, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3877054850260417, "G_std": 0.17207845052083334, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.947410583496094, "grad_norm": 0.7295790910720825, "tokens_seen": 28680192, "tokens_per_sec": 17503.760882724757, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3987019856770833, "G_std": 0.17249552408854166, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.919444584846497, "val_ppl": 136.92654098681754, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3896687825520833, "G_std": 0.1695556640625, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.93131947517395, "grad_norm": 0.7015523314476013, "tokens_seen": 28844032, "tokens_per_sec": 17467.37273077815, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4000752766927083, "G_std": 0.17133076985677084, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.782741546630859, "grad_norm": 0.7116081714630127, "tokens_seen": 29007872, "tokens_per_sec": 17467.99081973273, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3926595052083333, "G_std": 0.17604573567708334, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.844819784164429, "grad_norm": 0.7230266332626343, "tokens_seen": 29171712, "tokens_per_sec": 17468.229260244854, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3927001953125, "G_std": 0.16935221354166666, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.902569055557251, "grad_norm": 0.837245762348175, "tokens_seen": 29335552, "tokens_per_sec": 17468.068028737624, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3882853190104167, "G_std": 0.16881306966145834, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.775797128677368, "grad_norm": 0.6964112520217896, "tokens_seen": 29499392, "tokens_per_sec": 17468.533592103813, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961385091145833, "G_std": 0.17087809244791666, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.854603290557861, "grad_norm": 0.7255520820617676, "tokens_seen": 29663232, "tokens_per_sec": 17469.01348256051, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3955485026041667, "G_std": 0.17081705729166666, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.912585020065308, "grad_norm": 0.721347451210022, "tokens_seen": 29827072, "tokens_per_sec": 17469.59158406838, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3892822265625, "G_std": 0.16980997721354166, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.802244186401367, "grad_norm": 0.698302686214447, "tokens_seen": 29990912, "tokens_per_sec": 17470.18128873489, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961588541666667, "G_std": 0.16751607259114584, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.926981687545776, "grad_norm": 0.7424869537353516, "tokens_seen": 30154752, "tokens_per_sec": 17470.543001078055, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.396026611328125, "G_std": 0.170166015625, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.764885663986206, "grad_norm": 0.7369509339332581, "tokens_seen": 30318592, "tokens_per_sec": 17471.04934691625, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3911234537760417, "G_std": 0.17249043782552084, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.901978015899658, "grad_norm": 0.8016358017921448, "tokens_seen": 30482432, "tokens_per_sec": 17471.604490508416, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3965555826822917, "G_std": 0.17075093587239584, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.869346857070923, "grad_norm": 0.7458731532096863, "tokens_seen": 30646272, "tokens_per_sec": 17472.01194702054, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388153076171875, "G_std": 0.16915384928385416, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.933675289154053, "grad_norm": 0.7847370505332947, "tokens_seen": 30810112, "tokens_per_sec": 17472.632342105124, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39404296875, "G_std": 0.171234130859375, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.896205425262451, "grad_norm": 0.7249737977981567, "tokens_seen": 30973952, "tokens_per_sec": 17473.02638008351, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4054158528645833, "G_std": 0.172393798828125, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.879766941070557, "grad_norm": 0.7409494519233704, "tokens_seen": 31137792, "tokens_per_sec": 17473.72643903109, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39794921875, "G_std": 0.17162068684895834, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.66268253326416, "grad_norm": 0.7690057754516602, "tokens_seen": 31301632, "tokens_per_sec": 17473.84268119309, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38873291015625, "G_std": 0.16758219401041666, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.955108165740967, "grad_norm": 0.8009628057479858, "tokens_seen": 31465472, "tokens_per_sec": 17474.579919243777, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388519287109375, "G_std": 0.16872151692708334, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.836324214935303, "grad_norm": 0.7066766619682312, "tokens_seen": 31629312, "tokens_per_sec": 17475.319101712237, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388458251953125, "G_std": 0.17099507649739584, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.740480899810791, "grad_norm": 0.725151777267456, "tokens_seen": 31793152, "tokens_per_sec": 17475.9962594447, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39044189453125, "G_std": 0.16925557454427084, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.71989631652832, "grad_norm": 0.7428955435752869, "tokens_seen": 31956992, "tokens_per_sec": 17476.915449758642, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3866984049479167, "G_std": 0.16900126139322916, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.829251527786255, "grad_norm": 0.7583534717559814, "tokens_seen": 32120832, "tokens_per_sec": 17477.368210207238, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.389434814453125, "G_std": 0.16963704427083334, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.66120171546936, "grad_norm": 0.7264285683631897, "tokens_seen": 32284672, "tokens_per_sec": 17478.15740321495, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388763427734375, "G_std": 0.172760009765625, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.776068687438965, "grad_norm": 0.7177941799163818, "tokens_seen": 32448512, "tokens_per_sec": 17478.915799975413, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3973592122395833, "G_std": 0.16993204752604166, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.687511682510376, "grad_norm": 0.7109467387199402, "tokens_seen": 32612352, "tokens_per_sec": 17479.545037548247, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3859456380208333, "G_std": 0.16822306315104166, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.760048866271973, "grad_norm": 0.7117899656295776, "tokens_seen": 32776192, "tokens_per_sec": 17480.103051980488, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4009653727213542, "G_std": 0.1686248779296875, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.87808735370636, "val_ppl": 131.3791415976826, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3928120930989583, "G_std": 0.16900634765625, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.924747943878174, "grad_norm": 0.7318551540374756, "tokens_seen": 32940032, "tokens_per_sec": 17448.057445230585, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3859659830729167, "G_std": 0.16927591959635416, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.900606155395508, "grad_norm": 0.7569137215614319, "tokens_seen": 33103872, "tokens_per_sec": 17448.85221796637, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737284342447917, "G_std": 0.16655985514322916, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.757431507110596, "grad_norm": 0.718655526638031, "tokens_seen": 33267712, "tokens_per_sec": 17449.725683269946, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3878987630208333, "G_std": 0.16925557454427084, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.792655944824219, "grad_norm": 0.7012903094291687, "tokens_seen": 33431552, "tokens_per_sec": 17450.4854976951, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3797810872395833, "G_std": 0.1682891845703125, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.688676834106445, "grad_norm": 0.7533533573150635, "tokens_seen": 33595392, "tokens_per_sec": 17450.984926252266, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3900553385416667, "G_std": 0.1695098876953125, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.625044345855713, "grad_norm": 0.7384224534034729, "tokens_seen": 33759232, "tokens_per_sec": 17451.74685255299, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846842447916667, "G_std": 0.16484578450520834, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.7402520179748535, "grad_norm": 0.7653489708900452, "tokens_seen": 33923072, "tokens_per_sec": 17452.570752218693, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388519287109375, "G_std": 0.16909281412760416, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.726226568222046, "grad_norm": 0.7675644755363464, "tokens_seen": 34086912, "tokens_per_sec": 17453.428336928395, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3886617024739583, "G_std": 0.17026774088541666, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.916859149932861, "grad_norm": 0.7358502745628357, "tokens_seen": 34250752, "tokens_per_sec": 17454.39574428586, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3912404378255208, "G_std": 0.16982523600260416, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.736868858337402, "grad_norm": 0.7652812600135803, "tokens_seen": 34414592, "tokens_per_sec": 17455.34063179303, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3883463541666667, "G_std": 0.16704813639322916, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.8178019523620605, "grad_norm": 0.75921630859375, "tokens_seen": 34578432, "tokens_per_sec": 17456.255331884033, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831532796223958, "G_std": 0.16552734375, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.83102011680603, "grad_norm": 0.7168455123901367, "tokens_seen": 34742272, "tokens_per_sec": 17457.39238238624, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846893310546875, "G_std": 0.16720072428385416, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.838382720947266, "grad_norm": 0.7694299817085266, "tokens_seen": 34906112, "tokens_per_sec": 17458.218046716556, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3757883707682292, "G_std": 0.16521708170572916, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.872309923171997, "grad_norm": 0.8575940728187561, "tokens_seen": 35069952, "tokens_per_sec": 17459.31159166962, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.390167236328125, "G_std": 0.16943868001302084, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.839525461196899, "grad_norm": 0.753997802734375, "tokens_seen": 35233792, "tokens_per_sec": 17460.363517856793, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.385894775390625, "G_std": 0.1664886474609375, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.687926292419434, "grad_norm": 0.6988971829414368, "tokens_seen": 35397632, "tokens_per_sec": 17461.32519955918, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3777669270833333, "G_std": 0.1685791015625, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.712590217590332, "grad_norm": 0.76909339427948, "tokens_seen": 35561472, "tokens_per_sec": 17462.097321036756, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3906809488932292, "G_std": 0.16772969563802084, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.866310119628906, "grad_norm": 0.747515857219696, "tokens_seen": 35725312, "tokens_per_sec": 17462.946381929152, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3986256917317708, "G_std": 0.17032368977864584, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.844698905944824, "grad_norm": 0.7479916214942932, "tokens_seen": 35889152, "tokens_per_sec": 17463.667951278865, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38287353515625, "G_std": 0.1688690185546875, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.583908557891846, "grad_norm": 0.7769603133201599, "tokens_seen": 36052992, "tokens_per_sec": 17464.0374559412, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3852030436197917, "G_std": 0.16667683919270834, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.873600959777832, "grad_norm": 0.7421557307243347, "tokens_seen": 36216832, "tokens_per_sec": 17464.91991350938, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3882954915364583, "G_std": 0.16606648763020834, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.715955495834351, "grad_norm": 0.8019494414329529, "tokens_seen": 36380672, "tokens_per_sec": 17465.66189196744, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3847859700520833, "G_std": 0.16759745279947916, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.732473850250244, "grad_norm": 0.7846154570579529, "tokens_seen": 36544512, "tokens_per_sec": 17466.636315265714, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3731180826822917, "G_std": 0.16639200846354166, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.702930212020874, "grad_norm": 0.7511685490608215, "tokens_seen": 36708352, "tokens_per_sec": 17467.490327891857, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3797200520833333, "G_std": 0.16707865397135416, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.719897747039795, "grad_norm": 0.8087331652641296, "tokens_seen": 36872192, "tokens_per_sec": 17467.961767205263, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3925984700520833, "G_std": 0.16803995768229166, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.812619996070862, "val_ppl": 123.05359547924705, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3877614339192708, "G_std": 0.166717529296875, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.710099458694458, "grad_norm": 0.7521389722824097, "tokens_seen": 37036032, "tokens_per_sec": 17440.331976203695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3822123209635417, "G_std": 0.16756184895833334, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.768276691436768, "grad_norm": 0.7590110301971436, "tokens_seen": 37199872, "tokens_per_sec": 17441.084210034427, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3768259684244792, "G_std": 0.16613006591796875, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.766818284988403, "grad_norm": 0.7291926741600037, "tokens_seen": 37363712, "tokens_per_sec": 17441.970615265633, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3874664306640625, "G_std": 0.1667938232421875, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.730780839920044, "grad_norm": 0.7527073621749878, "tokens_seen": 37527552, "tokens_per_sec": 17442.590452329587, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.382080078125, "G_std": 0.16548665364583334, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.737623453140259, "grad_norm": 0.7972307801246643, "tokens_seen": 37691392, "tokens_per_sec": 17443.128349426413, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3768107096354167, "G_std": 0.16634623209635416, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.76103401184082, "grad_norm": 0.7960112690925598, "tokens_seen": 37855232, "tokens_per_sec": 17443.728588695758, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3824462890625, "G_std": 0.16579691569010416, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.804909944534302, "grad_norm": 0.8064020872116089, "tokens_seen": 38019072, "tokens_per_sec": 17444.63213187838, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3794657389322917, "G_std": 0.16327412923177084, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.68573522567749, "grad_norm": 0.7761533260345459, "tokens_seen": 38182912, "tokens_per_sec": 17445.38534391656, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775278727213542, "G_std": 0.16646830240885416, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.733438491821289, "grad_norm": 0.7774620652198792, "tokens_seen": 38346752, "tokens_per_sec": 17446.257947067403, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3793538411458333, "G_std": 0.16681416829427084, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.695749282836914, "grad_norm": 0.7871179580688477, "tokens_seen": 38510592, "tokens_per_sec": 17447.18914467431, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3828175862630208, "G_std": 0.16428120930989584, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.732056617736816, "grad_norm": 0.7730251550674438, "tokens_seen": 38674432, "tokens_per_sec": 17447.754553028695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809000651041667, "G_std": 0.167724609375, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.611609220504761, "grad_norm": 0.8383278250694275, "tokens_seen": 38838272, "tokens_per_sec": 17448.164933030326, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.379638671875, "G_std": 0.16666158040364584, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.708132028579712, "grad_norm": 0.8171554207801819, "tokens_seen": 39002112, "tokens_per_sec": 17448.81632755239, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3828175862630208, "G_std": 0.16518147786458334, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.7373199462890625, "grad_norm": 0.7357147932052612, "tokens_seen": 39165952, "tokens_per_sec": 17449.769600949163, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3817342122395833, "G_std": 0.1644617716471354, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.8097381591796875, "grad_norm": 0.7676548361778259, "tokens_seen": 39329792, "tokens_per_sec": 17450.72777456517, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3801676432291667, "G_std": 0.16680908203125, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.71077561378479, "grad_norm": 0.7926000356674194, "tokens_seen": 39493632, "tokens_per_sec": 17451.57685831363, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775227864583333, "G_std": 0.168365478515625, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.469035625457764, "grad_norm": 0.8083292841911316, "tokens_seen": 39657472, "tokens_per_sec": 17452.59037870595, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770650227864583, "G_std": 0.16458638509114584, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.734117746353149, "grad_norm": 0.773259162902832, "tokens_seen": 39821312, "tokens_per_sec": 17453.52144408893, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3760121663411458, "G_std": 0.16399637858072916, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.7129435539245605, "grad_norm": 0.786052942276001, "tokens_seen": 39985152, "tokens_per_sec": 17454.171129401104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37689208984375, "G_std": 0.16583251953125, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.5291666984558105, "grad_norm": 0.7264812588691711, "tokens_seen": 40148992, "tokens_per_sec": 17454.918766868977, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3742116292317708, "G_std": 0.16569010416666666, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.7526397705078125, "grad_norm": 0.8039868474006653, "tokens_seen": 40312832, "tokens_per_sec": 17455.899343373738, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3795369466145833, "G_std": 0.16694132486979166, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.6448564529418945, "grad_norm": 0.753730297088623, "tokens_seen": 40476672, "tokens_per_sec": 17456.655221178848, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3755289713541667, "G_std": 0.16683451334635416, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.708796977996826, "grad_norm": 0.7361549735069275, "tokens_seen": 40640512, "tokens_per_sec": 17457.490998994108, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3860880533854167, "G_std": 0.16581217447916666, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.557488679885864, "grad_norm": 0.7348860502243042, "tokens_seen": 40804352, "tokens_per_sec": 17458.329194671485, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737131754557292, "G_std": 0.16476949055989584, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.751444101333618, "grad_norm": 0.8703740835189819, "tokens_seen": 40968192, "tokens_per_sec": 17458.718851789363, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3782806396484375, "G_std": 0.16645304361979166, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.769047403335572, "val_ppl": 117.80696596946468, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3728281656901042, "G_std": 0.16516621907552084, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.652173280715942, "grad_norm": 0.8487904667854309, "tokens_seen": 41132032, "tokens_per_sec": 17433.959499647917, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3793589274088542, "G_std": 0.16344197591145834, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.668290376663208, "grad_norm": 0.8002393841743469, "tokens_seen": 41295872, "tokens_per_sec": 17434.726763572162, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3814646402994792, "G_std": 0.1679534912109375, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.559422969818115, "grad_norm": 0.8241577744483948, "tokens_seen": 41459712, "tokens_per_sec": 17435.451145683783, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3766377766927083, "G_std": 0.16486612955729166, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.666868686676025, "grad_norm": 0.7770039439201355, "tokens_seen": 41623552, "tokens_per_sec": 17436.365286138287, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.378265380859375, "G_std": 0.16417439778645834, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.672878265380859, "grad_norm": 0.7448977828025818, "tokens_seen": 41787392, "tokens_per_sec": 17437.115435644868, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3864593505859375, "G_std": 0.16526285807291666, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.591899633407593, "grad_norm": 0.7591446042060852, "tokens_seen": 41951232, "tokens_per_sec": 17437.891254053106, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3751322428385417, "G_std": 0.16463216145833334, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.700247287750244, "grad_norm": 0.8002793192863464, "tokens_seen": 42115072, "tokens_per_sec": 17438.36075125381, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781382242838542, "G_std": 0.16622416178385416, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.550368309020996, "grad_norm": 0.820284903049469, "tokens_seen": 42278912, "tokens_per_sec": 17439.404445295066, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3792063395182292, "G_std": 0.1639582316080729, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.586405277252197, "grad_norm": 0.824609100818634, "tokens_seen": 42442752, "tokens_per_sec": 17440.32496456961, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753407796223958, "G_std": 0.16582743326822916, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.515306711196899, "grad_norm": 0.8093904852867126, "tokens_seen": 42606592, "tokens_per_sec": 17440.807547516375, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3815968831380208, "G_std": 0.16524251302083334, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.623230934143066, "grad_norm": 0.7763972282409668, "tokens_seen": 42770432, "tokens_per_sec": 17441.62258273047, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809661865234375, "G_std": 0.16634623209635416, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.588695287704468, "grad_norm": 0.7835832834243774, "tokens_seen": 42934272, "tokens_per_sec": 17442.40749420773, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3839111328125, "G_std": 0.16587320963541666, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.563920736312866, "grad_norm": 0.733352541923523, "tokens_seen": 43098112, "tokens_per_sec": 17443.43139870814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3776397705078125, "G_std": 0.16457112630208334, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.594788551330566, "grad_norm": 0.8492591381072998, "tokens_seen": 43261952, "tokens_per_sec": 17444.0802508414, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3789571126302083, "G_std": 0.16460164388020834, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.726199388504028, "grad_norm": 0.8098321557044983, "tokens_seen": 43425792, "tokens_per_sec": 17445.009323610695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3776652018229167, "G_std": 0.1663818359375, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.593683958053589, "grad_norm": 0.7411289215087891, "tokens_seen": 43589632, "tokens_per_sec": 17446.039393886076, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3704274495442708, "G_std": 0.16423543294270834, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.675260066986084, "grad_norm": 0.8025728464126587, "tokens_seen": 43753472, "tokens_per_sec": 17446.803581813456, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3780161539713542, "G_std": 0.16675313313802084, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.51931095123291, "grad_norm": 0.7739837169647217, "tokens_seen": 43917312, "tokens_per_sec": 17447.752453931666, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716684977213542, "G_std": 0.16595458984375, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.7928526401519775, "grad_norm": 0.8122173547744751, "tokens_seen": 44081152, "tokens_per_sec": 17448.542828994985, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.382568359375, "G_std": 0.16552225748697916, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.496184825897217, "grad_norm": 0.8141473531723022, "tokens_seen": 44244992, "tokens_per_sec": 17449.336222133043, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673553466796875, "G_std": 0.16440836588541666, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.648874998092651, "grad_norm": 0.8193737268447876, "tokens_seen": 44408832, "tokens_per_sec": 17450.295710044138, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753662109375, "G_std": 0.1648839314778646, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.530743837356567, "grad_norm": 0.7815042734146118, "tokens_seen": 44572672, "tokens_per_sec": 17450.805676631437, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3731892903645833, "G_std": 0.16421000162760416, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.656736850738525, "grad_norm": 0.8553867936134338, "tokens_seen": 44736512, "tokens_per_sec": 17451.685708120018, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3760223388671875, "G_std": 0.16489664713541666, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.545287609100342, "grad_norm": 0.8026161193847656, "tokens_seen": 44900352, "tokens_per_sec": 17452.498446634512, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3811747233072917, "G_std": 0.16371663411458334, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.617936611175537, "grad_norm": 0.7704712748527527, "tokens_seen": 45064192, "tokens_per_sec": 17453.169900729827, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729909261067708, "G_std": 0.16260274251302084, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.700141704082489, "val_ppl": 109.96275351924238, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3787689208984375, "G_std": 0.16499837239583334, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.446603298187256, "grad_norm": 0.7766019105911255, "tokens_seen": 45228032, "tokens_per_sec": 17430.820357395027, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3692728678385417, "G_std": 0.16379292805989584, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.651191711425781, "grad_norm": 0.840620756149292, "tokens_seen": 45391872, "tokens_per_sec": 17431.287046793896, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3766937255859375, "G_std": 0.16232808430989584, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.695247411727905, "grad_norm": 0.8062357306480408, "tokens_seen": 45555712, "tokens_per_sec": 17432.074219135953, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716888427734375, "G_std": 0.16426595052083334, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.719505548477173, "grad_norm": 0.7608460187911987, "tokens_seen": 45719552, "tokens_per_sec": 17432.915031612498, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762054443359375, "G_std": 0.1652806599934896, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.551390647888184, "grad_norm": 0.8085259795188904, "tokens_seen": 45883392, "tokens_per_sec": 17433.522306722803, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720906575520833, "G_std": 0.1640472412109375, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.630748748779297, "grad_norm": 0.8119193911552429, "tokens_seen": 46047232, "tokens_per_sec": 17434.259004676962, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732401529947917, "G_std": 0.1646906534830729, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.742065191268921, "grad_norm": 0.8158015012741089, "tokens_seen": 46211072, "tokens_per_sec": 17434.974523012323, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3818105061848958, "G_std": 0.16695658365885416, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.514405012130737, "grad_norm": 0.7824839949607849, "tokens_seen": 46374912, "tokens_per_sec": 17435.62968545543, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770192464192708, "G_std": 0.16460164388020834, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.567246198654175, "grad_norm": 0.8254954218864441, "tokens_seen": 46538752, "tokens_per_sec": 17436.395894259847, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734130859375, "G_std": 0.16493733723958334, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.719225168228149, "grad_norm": 0.8898035287857056, "tokens_seen": 46702592, "tokens_per_sec": 17437.11151899989, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664194742838542, "G_std": 0.16457112630208334, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.563431024551392, "grad_norm": 0.7939354181289673, "tokens_seen": 46866432, "tokens_per_sec": 17437.7797592323, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722381591796875, "G_std": 0.16358439127604166, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.720765113830566, "grad_norm": 0.8280303478240967, "tokens_seen": 47030272, "tokens_per_sec": 17438.50877160671, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37384033203125, "G_std": 0.1640472412109375, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.576084136962891, "grad_norm": 0.759697437286377, "tokens_seen": 47194112, "tokens_per_sec": 17439.425349091307, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3708445231119792, "G_std": 0.16310628255208334, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.619009017944336, "grad_norm": 0.7847826480865479, "tokens_seen": 47357952, "tokens_per_sec": 17440.10565902005, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3699442545572917, "G_std": 0.1629638671875, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.476453542709351, "grad_norm": 0.8174769282341003, "tokens_seen": 47521792, "tokens_per_sec": 17440.862553778785, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3669382731119792, "G_std": 0.16365559895833334, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.733098030090332, "grad_norm": 0.7721109390258789, "tokens_seen": 47685632, "tokens_per_sec": 17441.52243608748, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3724263509114583, "G_std": 0.16198221842447916, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.533630132675171, "grad_norm": 0.7809209227561951, "tokens_seen": 47849472, "tokens_per_sec": 17442.422753058367, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3685709635416667, "G_std": 0.163299560546875, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.630663871765137, "grad_norm": 0.7986549735069275, "tokens_seen": 48013312, "tokens_per_sec": 17443.242182868486, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37115478515625, "G_std": 0.16237386067708334, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.67105507850647, "grad_norm": 0.8173574209213257, "tokens_seen": 48177152, "tokens_per_sec": 17443.795782210436, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373016357421875, "G_std": 0.16617584228515625, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.592663288116455, "grad_norm": 0.8366840481758118, "tokens_seen": 48340992, "tokens_per_sec": 17444.64731345791, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680470784505208, "G_std": 0.16347757975260416, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.573770046234131, "grad_norm": 0.81350177526474, "tokens_seen": 48504832, "tokens_per_sec": 17445.264574398825, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3687591552734375, "G_std": 0.16354878743489584, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.498575687408447, "grad_norm": 0.7941420078277588, "tokens_seen": 48668672, "tokens_per_sec": 17446.045604355702, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3685557047526042, "G_std": 0.16101582845052084, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.544731616973877, "grad_norm": 0.8979393243789673, "tokens_seen": 48832512, "tokens_per_sec": 17446.843751973203, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680674235026042, "G_std": 0.16544087727864584, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.80899977684021, "grad_norm": 0.8125686049461365, "tokens_seen": 48996352, "tokens_per_sec": 17447.577948656482, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3764394124348958, "G_std": 0.1656494140625, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.563044786453247, "grad_norm": 0.7716858983039856, "tokens_seen": 49160192, "tokens_per_sec": 17448.283699074767, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3777872721354167, "G_std": 0.163848876953125, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.65535204410553, "val_ppl": 105.1462295943598, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3759613037109375, "G_std": 0.16626993815104166, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.48551869392395, "grad_norm": 0.8197864890098572, "tokens_seen": 49324032, "tokens_per_sec": 17427.74511571353, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36468505859375, "G_std": 0.1640625, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.511743068695068, "grad_norm": 0.7658717632293701, "tokens_seen": 49487872, "tokens_per_sec": 17428.417624922546, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725128173828125, "G_std": 0.1646270751953125, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.668689012527466, "grad_norm": 0.8313459157943726, "tokens_seen": 49651712, "tokens_per_sec": 17429.006645753136, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.370208740234375, "G_std": 0.16194661458333334, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.608194828033447, "grad_norm": 0.800979495048523, "tokens_seen": 49815552, "tokens_per_sec": 17429.67734165569, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3683929443359375, "G_std": 0.162872314453125, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.5237061977386475, "grad_norm": 0.8157055377960205, "tokens_seen": 49979392, "tokens_per_sec": 17430.462339887043, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3687744140625, "G_std": 0.16327921549479166, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.6825878620147705, "grad_norm": 0.8253299593925476, "tokens_seen": 50143232, "tokens_per_sec": 17431.287674460385, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3723347981770833, "G_std": 0.1625213623046875, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.580136299133301, "grad_norm": 0.8426761031150818, "tokens_seen": 50307072, "tokens_per_sec": 17431.981945120464, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3765513102213542, "G_std": 0.16497294108072916, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.477505922317505, "grad_norm": 0.8061447739601135, "tokens_seen": 50470912, "tokens_per_sec": 17432.934848186742, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671366373697917, "G_std": 0.16331990559895834, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.63377046585083, "grad_norm": 0.8550890684127808, "tokens_seen": 50634752, "tokens_per_sec": 17433.693859483632, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732961018880208, "G_std": 0.16364542643229166, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.556345701217651, "grad_norm": 0.8422642350196838, "tokens_seen": 50798592, "tokens_per_sec": 17434.340831206908, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662312825520833, "G_std": 0.16463216145833334, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.526722431182861, "grad_norm": 0.8710882663726807, "tokens_seen": 50962432, "tokens_per_sec": 17435.180525545165, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373443603515625, "G_std": 0.1635284423828125, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.485511302947998, "grad_norm": 0.8109205961227417, "tokens_seen": 51126272, "tokens_per_sec": 17435.75911283137, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649190266927083, "G_std": 0.16199747721354166, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.624010324478149, "grad_norm": 0.9027352333068848, "tokens_seen": 51290112, "tokens_per_sec": 17436.518818099317, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725941975911458, "G_std": 0.1655604044596354, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.56792688369751, "grad_norm": 0.782560408115387, "tokens_seen": 51453952, "tokens_per_sec": 17437.202459637334, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3739878336588542, "G_std": 0.16234842936197916, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.50810432434082, "grad_norm": 0.8371365666389465, "tokens_seen": 51617792, "tokens_per_sec": 17437.65234260413, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3640492757161458, "G_std": 0.161407470703125, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.465242862701416, "grad_norm": 0.8662558197975159, "tokens_seen": 51781632, "tokens_per_sec": 17437.787946788718, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3675638834635417, "G_std": 0.16216023763020834, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.563857555389404, "grad_norm": 0.808320939540863, "tokens_seen": 51945472, "tokens_per_sec": 17438.633598591026, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3742421468098958, "G_std": 0.16240437825520834, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.566922664642334, "grad_norm": 0.8354389071464539, "tokens_seen": 52109312, "tokens_per_sec": 17439.393627185833, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3691202799479167, "G_std": 0.1627655029296875, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.576031446456909, "grad_norm": 0.8413287997245789, "tokens_seen": 52273152, "tokens_per_sec": 17440.164293399073, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3714141845703125, "G_std": 0.1607513427734375, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.4337732791900635, "grad_norm": 0.8716790080070496, "tokens_seen": 52436992, "tokens_per_sec": 17440.943115156933, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3746236165364583, "G_std": 0.1638615926106771, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.609098672866821, "grad_norm": 0.8290572166442871, "tokens_seen": 52600832, "tokens_per_sec": 17441.622189896752, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670094807942708, "G_std": 0.16247049967447916, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.66242527961731, "grad_norm": 0.9145711660385132, "tokens_seen": 52764672, "tokens_per_sec": 17441.98031005425, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597666422526042, "G_std": 0.16159566243489584, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.50855278968811, "grad_norm": 0.8759003281593323, "tokens_seen": 52928512, "tokens_per_sec": 17442.43206847258, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36297607421875, "G_std": 0.16192626953125, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.50559139251709, "grad_norm": 0.8586622476577759, "tokens_seen": 53092352, "tokens_per_sec": 17443.185674530287, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3679962158203125, "G_std": 0.16331990559895834, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.527850866317749, "grad_norm": 0.8410255312919617, "tokens_seen": 53256192, "tokens_per_sec": 17443.779994062286, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372222900390625, "G_std": 0.16312662760416666, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.630793142318725, "val_ppl": 102.59540460603455, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.36993408203125, "G_std": 0.1639429728190104, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.678898572921753, "grad_norm": 0.8436530232429504, "tokens_seen": 53420032, "tokens_per_sec": 17424.689998432, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690846761067708, "G_std": 0.16515096028645834, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.490042686462402, "grad_norm": 0.878561794757843, "tokens_seen": 53583872, "tokens_per_sec": 17425.26008547706, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716278076171875, "G_std": 0.1638946533203125, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.6873087882995605, "grad_norm": 0.8728581070899963, "tokens_seen": 53747712, "tokens_per_sec": 17425.85782129393, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673146565755208, "G_std": 0.1627324422200521, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.62444543838501, "grad_norm": 0.8764206171035767, "tokens_seen": 53911552, "tokens_per_sec": 17426.34431172651, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3666839599609375, "G_std": 0.1639531453450521, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.38438606262207, "grad_norm": 0.8746634721755981, "tokens_seen": 54075392, "tokens_per_sec": 17426.949634904264, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3665262858072917, "G_std": 0.162811279296875, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.382456302642822, "grad_norm": 0.8382308483123779, "tokens_seen": 54239232, "tokens_per_sec": 17427.547878298607, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650054931640625, "G_std": 0.1645965576171875, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.679741144180298, "grad_norm": 0.8225433826446533, "tokens_seen": 54403072, "tokens_per_sec": 17428.159268694195, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3699900309244792, "G_std": 0.16260274251302084, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.644181251525879, "grad_norm": 0.8046494722366333, "tokens_seen": 54566912, "tokens_per_sec": 17428.690612930182, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3717091878255208, "G_std": 0.16341654459635416, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.527673244476318, "grad_norm": 0.8236086964607239, "tokens_seen": 54730752, "tokens_per_sec": 17429.493893343555, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710886637369792, "G_std": 0.16353352864583334, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.493317365646362, "grad_norm": 0.855007529258728, "tokens_seen": 54894592, "tokens_per_sec": 17430.085958421994, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.362701416015625, "G_std": 0.16174825032552084, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.566770792007446, "grad_norm": 0.8913280963897705, "tokens_seen": 55058432, "tokens_per_sec": 17430.875926601977, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676656087239583, "G_std": 0.16302998860677084, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.4762349128723145, "grad_norm": 0.8237499594688416, "tokens_seen": 55222272, "tokens_per_sec": 17431.465749700907, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36236572265625, "G_std": 0.16263580322265625, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.488595008850098, "grad_norm": 0.88705974817276, "tokens_seen": 55386112, "tokens_per_sec": 17432.110577431344, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676859537760417, "G_std": 0.16368865966796875, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.48572564125061, "grad_norm": 0.8782538175582886, "tokens_seen": 55549952, "tokens_per_sec": 17432.772154151316, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3684438069661458, "G_std": 0.16478474934895834, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.441995143890381, "grad_norm": 0.8040663599967957, "tokens_seen": 55713792, "tokens_per_sec": 17433.417909792046, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646901448567708, "G_std": 0.16083272298177084, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.425517559051514, "grad_norm": 0.817447304725647, "tokens_seen": 55877632, "tokens_per_sec": 17434.21901418432, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3678741455078125, "G_std": 0.1636505126953125, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.3634867668151855, "grad_norm": 0.7908456921577454, "tokens_seen": 56041472, "tokens_per_sec": 17434.92306148149, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3741963704427083, "G_std": 0.1607208251953125, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.563191175460815, "grad_norm": 0.8809279799461365, "tokens_seen": 56205312, "tokens_per_sec": 17435.603661857826, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365386962890625, "G_std": 0.16376241048177084, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.504535436630249, "grad_norm": 0.7975493669509888, "tokens_seen": 56369152, "tokens_per_sec": 17436.14665836113, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.371368408203125, "G_std": 0.16394551595052084, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.6533894538879395, "grad_norm": 0.8602167367935181, "tokens_seen": 56532992, "tokens_per_sec": 17436.68686031871, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3704427083333333, "G_std": 0.164306640625, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.568257570266724, "grad_norm": 0.836186408996582, "tokens_seen": 56696832, "tokens_per_sec": 17437.402346806182, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3639984130859375, "G_std": 0.1623814900716146, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.40092658996582, "grad_norm": 0.8241569995880127, "tokens_seen": 56860672, "tokens_per_sec": 17438.010210611454, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3621673583984375, "G_std": 0.16161600748697916, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.53031063079834, "grad_norm": 0.8607854843139648, "tokens_seen": 57024512, "tokens_per_sec": 17438.595357063103, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.361785888671875, "G_std": 0.16106160481770834, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.348714828491211, "grad_norm": 0.8370847702026367, "tokens_seen": 57188352, "tokens_per_sec": 17439.032470383107, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727823893229167, "G_std": 0.16183980305989584, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.643402099609375, "grad_norm": 0.8916448354721069, "tokens_seen": 57352192, "tokens_per_sec": 17439.67486960928, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3711751302083333, "G_std": 0.16540018717447916, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.57691410779953, "val_ppl": 97.21393912296652, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3676910400390625, "G_std": 0.16259765625, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.4471235275268555, "grad_norm": 0.838839590549469, "tokens_seen": 57516032, "tokens_per_sec": 17422.08927280893, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646748860677083, "G_std": 0.1608403523763021, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.45895528793335, "grad_norm": 0.8525413274765015, "tokens_seen": 57679872, "tokens_per_sec": 17422.728423496952, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36767578125, "G_std": 0.16207377115885416, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.549091100692749, "grad_norm": 0.8645190000534058, "tokens_seen": 57843712, "tokens_per_sec": 17423.10337991265, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3665974934895833, "G_std": 0.1637446085611979, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.564897060394287, "grad_norm": 0.8710461854934692, "tokens_seen": 58007552, "tokens_per_sec": 17423.697196114517, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367645263671875, "G_std": 0.1632054646809896, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.518671989440918, "grad_norm": 0.8577271699905396, "tokens_seen": 58171392, "tokens_per_sec": 17424.250484232973, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3633524576822917, "G_std": 0.16186014811197916, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.736307144165039, "grad_norm": 0.9034472107887268, "tokens_seen": 58335232, "tokens_per_sec": 17424.86511581927, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.374969482421875, "G_std": 0.16155242919921875, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.5118982791900635, "grad_norm": 0.861575186252594, "tokens_seen": 58499072, "tokens_per_sec": 17425.487938412185, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676503499348958, "G_std": 0.1617863972981771, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.450214862823486, "grad_norm": 0.8198933005332947, "tokens_seen": 58662912, "tokens_per_sec": 17426.153612863534, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582000732421875, "G_std": 0.16054534912109375, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.589114189147949, "grad_norm": 0.8486289978027344, "tokens_seen": 58826752, "tokens_per_sec": 17426.535555599577, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365814208984375, "G_std": 0.162628173828125, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.616713047027588, "grad_norm": 0.9320365786552429, "tokens_seen": 58990592, "tokens_per_sec": 17426.857796148455, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671875, "G_std": 0.16314188639322916, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.593067646026611, "grad_norm": 0.8360547423362732, "tokens_seen": 59154432, "tokens_per_sec": 17427.327794964123, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732757568359375, "G_std": 0.16148885091145834, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.394601821899414, "grad_norm": 0.9002556800842285, "tokens_seen": 59318272, "tokens_per_sec": 17427.959102529992, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3607279459635417, "G_std": 0.162200927734375, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.445653438568115, "grad_norm": 0.8861992359161377, "tokens_seen": 59482112, "tokens_per_sec": 17428.482485375782, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656056722005208, "G_std": 0.1626408894856771, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.504093647003174, "grad_norm": 0.9545297622680664, "tokens_seen": 59645952, "tokens_per_sec": 17429.10259345334, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3578592936197917, "G_std": 0.16257731119791666, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.365746021270752, "grad_norm": 0.8315410614013672, "tokens_seen": 59809792, "tokens_per_sec": 17429.787028023522, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3644154866536458, "G_std": 0.1613642374674479, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.595918655395508, "grad_norm": 0.8211652636528015, "tokens_seen": 59973632, "tokens_per_sec": 17430.44428813567, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.368621826171875, "G_std": 0.16184234619140625, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.424305200576782, "grad_norm": 0.8434770107269287, "tokens_seen": 60137472, "tokens_per_sec": 17431.087643261802, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3637339274088542, "G_std": 0.16022237141927084, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.387807369232178, "grad_norm": 0.8184190392494202, "tokens_seen": 60301312, "tokens_per_sec": 17431.675900661536, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3563486735026042, "G_std": 0.1606267293294271, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.347423791885376, "grad_norm": 0.9021689295768738, "tokens_seen": 60465152, "tokens_per_sec": 17432.35904851774, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36688232421875, "G_std": 0.1627655029296875, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.466980934143066, "grad_norm": 0.8790953755378723, "tokens_seen": 60628992, "tokens_per_sec": 17433.06961812038, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653157552083333, "G_std": 0.16146341959635416, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.590203285217285, "grad_norm": 0.9098681211471558, "tokens_seen": 60792832, "tokens_per_sec": 17433.815284652155, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3709920247395833, "G_std": 0.16373697916666666, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.614591598510742, "grad_norm": 0.8768489956855774, "tokens_seen": 60956672, "tokens_per_sec": 17434.539639981835, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664093017578125, "G_std": 0.16201019287109375, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.412691116333008, "grad_norm": 0.8618361949920654, "tokens_seen": 61120512, "tokens_per_sec": 17435.024753958984, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664754231770833, "G_std": 0.16209920247395834, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.459540128707886, "grad_norm": 0.8912962079048157, "tokens_seen": 61284352, "tokens_per_sec": 17435.528902171565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3641001383463542, "G_std": 0.15850830078125, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.527146339416504, "grad_norm": 0.8683500289916992, "tokens_seen": 61448192, "tokens_per_sec": 17436.183192234028, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3530680338541667, "G_std": 0.1590118408203125, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.595513761043549, "val_ppl": 99.03900486170282, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3676350911458333, "G_std": 0.1600621541341146, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.510590553283691, "grad_norm": 0.8730258941650391, "tokens_seen": 61612032, "tokens_per_sec": 17419.691023384923, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646647135416667, "G_std": 0.16050211588541666, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.588456392288208, "grad_norm": 0.891457200050354, "tokens_seen": 61775872, "tokens_per_sec": 17420.271220750255, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673146565755208, "G_std": 0.16204833984375, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.279013633728027, "grad_norm": 0.8348298072814941, "tokens_seen": 61939712, "tokens_per_sec": 17420.753998660508, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589426676432292, "G_std": 0.15853118896484375, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.529789924621582, "grad_norm": 0.902744710445404, "tokens_seen": 62103552, "tokens_per_sec": 17421.2789195886, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3621978759765625, "G_std": 0.162689208984375, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.370537519454956, "grad_norm": 0.8490675687789917, "tokens_seen": 62267392, "tokens_per_sec": 17421.817161277744, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.362823486328125, "G_std": 0.16274515787760416, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.551669597625732, "grad_norm": 0.9420745968818665, "tokens_seen": 62431232, "tokens_per_sec": 17422.371298120157, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3635508219401042, "G_std": 0.1610387166341146, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.369140625, "grad_norm": 0.8719332814216614, "tokens_seen": 62595072, "tokens_per_sec": 17422.898989078425, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3593343098958333, "G_std": 0.16170756022135416, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.50460958480835, "grad_norm": 0.8631729483604431, "tokens_seen": 62758912, "tokens_per_sec": 17423.439507278334, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3573201497395833, "G_std": 0.16018168131510416, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.487091541290283, "grad_norm": 0.9132981896400452, "tokens_seen": 62922752, "tokens_per_sec": 17424.028254837816, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3621063232421875, "G_std": 0.1616643269856771, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.535225868225098, "grad_norm": 0.8598403930664062, "tokens_seen": 63086592, "tokens_per_sec": 17424.66631637095, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613128662109375, "G_std": 0.15984090169270834, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.49652099609375, "grad_norm": 0.9186596274375916, "tokens_seen": 63250432, "tokens_per_sec": 17425.26101581087, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650868733723958, "G_std": 0.1595484415690104, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.340496063232422, "grad_norm": 0.8385794162750244, "tokens_seen": 63414272, "tokens_per_sec": 17425.83825639768, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590952555338542, "G_std": 0.16066487630208334, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.472184419631958, "grad_norm": 0.8770472407341003, "tokens_seen": 63578112, "tokens_per_sec": 17426.459969471292, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650054931640625, "G_std": 0.1620941162109375, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.411756992340088, "grad_norm": 0.9091048240661621, "tokens_seen": 63741952, "tokens_per_sec": 17426.959998195613, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3661600748697917, "G_std": 0.1602935791015625, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.498344421386719, "grad_norm": 0.9429047703742981, "tokens_seen": 63905792, "tokens_per_sec": 17427.53863761945, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3569234212239583, "G_std": 0.16031392415364584, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.564298391342163, "grad_norm": 0.9389160871505737, "tokens_seen": 64069632, "tokens_per_sec": 17428.25892968948, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673655192057292, "G_std": 0.16114298502604166, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.321805953979492, "grad_norm": 0.847284197807312, "tokens_seen": 64233472, "tokens_per_sec": 17428.89071276486, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3625996907552083, "G_std": 0.16168721516927084, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.468782186508179, "grad_norm": 0.960317075252533, "tokens_seen": 64397312, "tokens_per_sec": 17429.554969998208, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3611501057942708, "G_std": 0.1606903076171875, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.5203893184661865, "grad_norm": 0.9888290166854858, "tokens_seen": 64561152, "tokens_per_sec": 17430.208705358953, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3636525472005208, "G_std": 0.16036478678385416, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.299683094024658, "grad_norm": 0.907209038734436, "tokens_seen": 64724992, "tokens_per_sec": 17430.724917706644, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3564453125, "G_std": 0.1609980265299479, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.3810038566589355, "grad_norm": 0.9470270276069641, "tokens_seen": 64888832, "tokens_per_sec": 17431.321123108537, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36474609375, "G_std": 0.16060892740885416, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.477220058441162, "grad_norm": 0.887715756893158, "tokens_seen": 65052672, "tokens_per_sec": 17431.677223907955, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3643391927083333, "G_std": 0.1620635986328125, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.5209128856658936, "grad_norm": 0.9191022515296936, "tokens_seen": 65216512, "tokens_per_sec": 17432.15881771064, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3667704264322917, "G_std": 0.162322998046875, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.420341968536377, "grad_norm": 0.8826854825019836, "tokens_seen": 65380352, "tokens_per_sec": 17432.676130952474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.357940673828125, "G_std": 0.1607386271158854, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.391645908355713, "grad_norm": 0.9305632710456848, "tokens_seen": 65544192, "tokens_per_sec": 17433.172547056332, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3660227457682292, "G_std": 0.16049702962239584, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.519860088825226, "val_ppl": 91.82275005056285, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3662363688151042, "G_std": 0.160064697265625, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.432293891906738, "grad_norm": 0.8674457669258118, "tokens_seen": 65708032, "tokens_per_sec": 17417.76984445118, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3604888916015625, "G_std": 0.15977986653645834, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.577717542648315, "grad_norm": 0.875072181224823, "tokens_seen": 65871872, "tokens_per_sec": 17418.214347250367, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650004069010417, "G_std": 0.1621424357096354, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.503804922103882, "grad_norm": 0.8632972836494446, "tokens_seen": 66035712, "tokens_per_sec": 17418.734301401477, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3595123291015625, "G_std": 0.16019439697265625, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.402439117431641, "grad_norm": 0.9155343174934387, "tokens_seen": 66199552, "tokens_per_sec": 17419.30039521975, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3584696451822917, "G_std": 0.1603571573893229, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.521487474441528, "grad_norm": 0.8707441687583923, "tokens_seen": 66363392, "tokens_per_sec": 17419.743995959558, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3651173909505208, "G_std": 0.159698486328125, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.345368385314941, "grad_norm": 0.830064058303833, "tokens_seen": 66527232, "tokens_per_sec": 17420.300567605234, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3631693522135417, "G_std": 0.16025797526041666, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.509350061416626, "grad_norm": 0.9521583914756775, "tokens_seen": 66691072, "tokens_per_sec": 17420.831010347305, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3565470377604167, "G_std": 0.1596527099609375, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.492091178894043, "grad_norm": 0.9037269949913025, "tokens_seen": 66854912, "tokens_per_sec": 17421.418500652147, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3621317545572917, "G_std": 0.15941365559895834, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.557426691055298, "grad_norm": 0.9104162454605103, "tokens_seen": 67018752, "tokens_per_sec": 17421.96781524409, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35931396484375, "G_std": 0.15970611572265625, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.259566307067871, "grad_norm": 0.9179626107215881, "tokens_seen": 67182592, "tokens_per_sec": 17422.49104896912, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680521647135417, "G_std": 0.16182708740234375, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.397991895675659, "grad_norm": 0.9219797849655151, "tokens_seen": 67346432, "tokens_per_sec": 17423.082661657685, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3619893391927083, "G_std": 0.16101582845052084, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.257698059082031, "grad_norm": 0.9222451448440552, "tokens_seen": 67510272, "tokens_per_sec": 17423.434232760712, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566640218098958, "G_std": 0.159149169921875, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.410158157348633, "grad_norm": 0.8612946271896362, "tokens_seen": 67674112, "tokens_per_sec": 17423.965813266703, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3592631022135417, "G_std": 0.16019439697265625, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.336467981338501, "grad_norm": 0.8807442784309387, "tokens_seen": 67837952, "tokens_per_sec": 17424.53128981201, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590901692708333, "G_std": 0.1584294637044271, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.3763651847839355, "grad_norm": 0.895607590675354, "tokens_seen": 68001792, "tokens_per_sec": 17425.186060635166, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609771728515625, "G_std": 0.16005706787109375, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.4201819896698, "grad_norm": 0.8928666114807129, "tokens_seen": 68165632, "tokens_per_sec": 17425.68008580746, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3563385009765625, "G_std": 0.15899149576822916, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.442604303359985, "grad_norm": 0.9134415984153748, "tokens_seen": 68329472, "tokens_per_sec": 17426.32816269702, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580830891927083, "G_std": 0.15868377685546875, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.482726097106934, "grad_norm": 1.0102617740631104, "tokens_seen": 68493312, "tokens_per_sec": 17426.878931096177, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3618062337239583, "G_std": 0.1583226521809896, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.437734127044678, "grad_norm": 0.8729194402694702, "tokens_seen": 68657152, "tokens_per_sec": 17427.484313569963, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35784912109375, "G_std": 0.15958150227864584, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.370134353637695, "grad_norm": 0.8768892288208008, "tokens_seen": 68820992, "tokens_per_sec": 17428.150181013156, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35797119140625, "G_std": 0.15772501627604166, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.521360874176025, "grad_norm": 0.9229019284248352, "tokens_seen": 68984832, "tokens_per_sec": 17428.74557457285, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3648223876953125, "G_std": 0.1586761474609375, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.182230710983276, "grad_norm": 0.8797451257705688, "tokens_seen": 69148672, "tokens_per_sec": 17429.33715643511, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3554229736328125, "G_std": 0.1583226521809896, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.3496339321136475, "grad_norm": 1.0200073719024658, "tokens_seen": 69312512, "tokens_per_sec": 17429.98639353933, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358184814453125, "G_std": 0.1585871378580729, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.301823616027832, "grad_norm": 0.9432343244552612, "tokens_seen": 69476352, "tokens_per_sec": 17430.659080617632, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3598276774088542, "G_std": 0.15825907389322916, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.627017974853516, "grad_norm": 0.9315717220306396, "tokens_seen": 69640192, "tokens_per_sec": 17431.195603961456, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3565572102864583, "G_std": 0.15918986002604166, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.513514626026153, "val_ppl": 91.24193691893228, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3621826171875, "G_std": 0.16052754720052084, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.424755096435547, "grad_norm": 0.9325756430625916, "tokens_seen": 69804032, "tokens_per_sec": 17416.774909564327, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603363037109375, "G_std": 0.1583989461263021, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.464342832565308, "grad_norm": 1.003720760345459, "tokens_seen": 69967872, "tokens_per_sec": 17417.00833199489, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616841634114583, "G_std": 0.1604766845703125, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.529433012008667, "grad_norm": 0.8943272233009338, "tokens_seen": 70131712, "tokens_per_sec": 17417.3521932716, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626047770182292, "G_std": 0.16025288899739584, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.367405891418457, "grad_norm": 0.9202273488044739, "tokens_seen": 70295552, "tokens_per_sec": 17417.73522702062, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566741943359375, "G_std": 0.1592992146809896, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.505518436431885, "grad_norm": 0.9801192283630371, "tokens_seen": 70459392, "tokens_per_sec": 17418.164927430094, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3631947835286458, "G_std": 0.16013081868489584, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.445816278457642, "grad_norm": 0.9686642289161682, "tokens_seen": 70623232, "tokens_per_sec": 17418.546018794925, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3660685221354167, "G_std": 0.16010284423828125, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.486584901809692, "grad_norm": 0.9122170209884644, "tokens_seen": 70787072, "tokens_per_sec": 17418.80298722487, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589731852213542, "G_std": 0.1585413614908854, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.313748359680176, "grad_norm": 0.9394065141677856, "tokens_seen": 70950912, "tokens_per_sec": 17419.023208610903, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365570068359375, "G_std": 0.15872701009114584, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.401702404022217, "grad_norm": 0.8493726849555969, "tokens_seen": 71114752, "tokens_per_sec": 17419.485354850975, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.359710693359375, "G_std": 0.15814717610677084, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.378940582275391, "grad_norm": 0.9057627320289612, "tokens_seen": 71278592, "tokens_per_sec": 17419.86668441075, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609670003255208, "G_std": 0.15793355305989584, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.347088575363159, "grad_norm": 0.8920137286186218, "tokens_seen": 71442432, "tokens_per_sec": 17420.39340772377, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3640543619791667, "G_std": 0.15850321451822916, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.4343931674957275, "grad_norm": 0.917157769203186, "tokens_seen": 71606272, "tokens_per_sec": 17420.808614249137, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3587646484375, "G_std": 0.160888671875, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.330517768859863, "grad_norm": 1.091037631034851, "tokens_seen": 71770112, "tokens_per_sec": 17421.40534422198, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646494547526042, "G_std": 0.15968831380208334, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.427680253982544, "grad_norm": 0.9154757261276245, "tokens_seen": 71933952, "tokens_per_sec": 17421.889155657795, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3601125081380208, "G_std": 0.15976969401041666, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.428891897201538, "grad_norm": 0.931192934513092, "tokens_seen": 72097792, "tokens_per_sec": 17422.226820852637, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670603434244792, "G_std": 0.15746053059895834, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.4305291175842285, "grad_norm": 0.9761743545532227, "tokens_seen": 72261632, "tokens_per_sec": 17422.434987508666, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646748860677083, "G_std": 0.1591796875, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.43233847618103, "grad_norm": 0.9511836171150208, "tokens_seen": 72425472, "tokens_per_sec": 17422.821208866477, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3612213134765625, "G_std": 0.15839640299479166, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.453413486480713, "grad_norm": 0.8915706276893616, "tokens_seen": 72589312, "tokens_per_sec": 17423.420485218106, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653666178385417, "G_std": 0.1571985880533854, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.282673120498657, "grad_norm": 0.9252042174339294, "tokens_seen": 72753152, "tokens_per_sec": 17424.007525362704, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3569183349609375, "G_std": 0.15737406412760416, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.372627258300781, "grad_norm": 0.8999847769737244, "tokens_seen": 72916992, "tokens_per_sec": 17424.608273516347, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36083984375, "G_std": 0.158660888671875, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.470168590545654, "grad_norm": 1.1207985877990723, "tokens_seen": 73080832, "tokens_per_sec": 17424.993896879565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646952311197917, "G_std": 0.1583887736002604, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.282682418823242, "grad_norm": 0.9313703179359436, "tokens_seen": 73244672, "tokens_per_sec": 17425.55886128712, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3631642659505208, "G_std": 0.1586888631184896, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.520712852478027, "grad_norm": 0.9895561337471008, "tokens_seen": 73408512, "tokens_per_sec": 17425.99717975421, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632456461588542, "G_std": 0.1593805948893229, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.37779974937439, "grad_norm": 0.946652889251709, "tokens_seen": 73572352, "tokens_per_sec": 17426.398002354214, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3618672688802083, "G_std": 0.1582310994466146, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.323139667510986, "grad_norm": 0.9205039739608765, "tokens_seen": 73736192, "tokens_per_sec": 17426.95428337315, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3591156005859375, "G_std": 0.157379150390625, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.477811431884765, "val_ppl": 88.04177624485945, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3615773518880208, "G_std": 0.1573918660481771, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.348893404006958, "grad_norm": 0.9040362238883972, "tokens_seen": 73900032, "tokens_per_sec": 17413.57858458222, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671976725260417, "G_std": 0.15941874186197916, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.32036566734314, "grad_norm": 0.8822205066680908, "tokens_seen": 74063872, "tokens_per_sec": 17413.93183434292, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3591257731119792, "G_std": 0.1588160196940104, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.329006195068359, "grad_norm": 0.9282068610191345, "tokens_seen": 74227712, "tokens_per_sec": 17414.310133683935, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610687255859375, "G_std": 0.15912373860677084, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.308641672134399, "grad_norm": 0.90728360414505, "tokens_seen": 74391552, "tokens_per_sec": 17414.557956678836, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3585764567057292, "G_std": 0.15637715657552084, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.270162105560303, "grad_norm": 0.9155462384223938, "tokens_seen": 74555392, "tokens_per_sec": 17414.893978109503, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3639322916666667, "G_std": 0.1596221923828125, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.358302593231201, "grad_norm": 0.9237802624702454, "tokens_seen": 74719232, "tokens_per_sec": 17415.216654765492, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3607279459635417, "G_std": 0.1582310994466146, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.467899322509766, "grad_norm": 0.9556766152381897, "tokens_seen": 74883072, "tokens_per_sec": 17415.78719736412, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589579264322917, "G_std": 0.15818277994791666, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.318796396255493, "grad_norm": 0.9665893316268921, "tokens_seen": 75046912, "tokens_per_sec": 17416.336144853478, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613179524739583, "G_std": 0.15845235188802084, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.467423915863037, "grad_norm": 1.0131341218948364, "tokens_seen": 75210752, "tokens_per_sec": 17416.788876213643, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367584228515625, "G_std": 0.15825653076171875, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.427067279815674, "grad_norm": 1.1896653175354004, "tokens_seen": 75374592, "tokens_per_sec": 17417.21686165402, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3712107340494792, "G_std": 0.15688578287760416, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.466366291046143, "grad_norm": 1.008059024810791, "tokens_seen": 75538432, "tokens_per_sec": 17417.725664564485, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594614664713542, "G_std": 0.15847269694010416, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.4654929637908936, "grad_norm": 0.9376114010810852, "tokens_seen": 75702272, "tokens_per_sec": 17418.3082893089, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610382080078125, "G_std": 0.1579157511393229, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.414297580718994, "grad_norm": 0.9732314348220825, "tokens_seen": 75866112, "tokens_per_sec": 17418.805830726928, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3596140543619792, "G_std": 0.15932210286458334, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.346682786941528, "grad_norm": 0.8954914212226868, "tokens_seen": 76029952, "tokens_per_sec": 17419.37462121487, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3564402262369792, "G_std": 0.15877532958984375, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.464831352233887, "grad_norm": 1.0108672380447388, "tokens_seen": 76193792, "tokens_per_sec": 17419.85621833182, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3560333251953125, "G_std": 0.1584294637044271, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.32570743560791, "grad_norm": 0.8842485547065735, "tokens_seen": 76357632, "tokens_per_sec": 17420.413993636987, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603769938151042, "G_std": 0.1563262939453125, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.484833717346191, "grad_norm": 1.0168969631195068, "tokens_seen": 76521472, "tokens_per_sec": 17420.933333356716, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589884440104167, "G_std": 0.15586344401041666, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.562865972518921, "grad_norm": 1.0221633911132812, "tokens_seen": 76685312, "tokens_per_sec": 17421.5312635324, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3627878824869792, "G_std": 0.15887959798177084, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.431909561157227, "grad_norm": 0.9732652902603149, "tokens_seen": 76849152, "tokens_per_sec": 17422.11804127871, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616434733072917, "G_std": 0.15950520833333334, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.304504632949829, "grad_norm": 0.9459869265556335, "tokens_seen": 77012992, "tokens_per_sec": 17422.763302031617, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3605143229166667, "G_std": 0.15923055013020834, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.545870542526245, "grad_norm": 1.1004664897918701, "tokens_seen": 77176832, "tokens_per_sec": 17423.238877506417, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676859537760417, "G_std": 0.157501220703125, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.403050422668457, "grad_norm": 0.9309484958648682, "tokens_seen": 77340672, "tokens_per_sec": 17423.86665563796, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3585611979166667, "G_std": 0.15877787272135416, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.356770038604736, "grad_norm": 0.9613153338432312, "tokens_seen": 77504512, "tokens_per_sec": 17424.3851591881, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3561655680338542, "G_std": 0.15741729736328125, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.41221022605896, "grad_norm": 0.889461874961853, "tokens_seen": 77668352, "tokens_per_sec": 17424.924522726138, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3567708333333333, "G_std": 0.1580352783203125, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.351767301559448, "grad_norm": 0.9756267666816711, "tokens_seen": 77832192, "tokens_per_sec": 17425.460511116744, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3572184244791667, "G_std": 0.1559015909830729, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.445501708984375, "val_ppl": 85.24263410620651, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3564503987630208, "G_std": 0.15705108642578125, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.204564571380615, "grad_norm": 0.8904143571853638, "tokens_seen": 77996032, "tokens_per_sec": 17412.72089289956, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358001708984375, "G_std": 0.15667724609375, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.525515556335449, "grad_norm": 0.9963718056678772, "tokens_seen": 78159872, "tokens_per_sec": 17413.277901681562, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3573099772135417, "G_std": 0.1576080322265625, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.494694948196411, "grad_norm": 0.9173113703727722, "tokens_seen": 78323712, "tokens_per_sec": 17413.77505111623, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603973388671875, "G_std": 0.15587870279947916, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.2885026931762695, "grad_norm": 0.9693016409873962, "tokens_seen": 78487552, "tokens_per_sec": 17414.2355367366, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3542022705078125, "G_std": 0.157623291015625, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.351529359817505, "grad_norm": 0.9893356561660767, "tokens_seen": 78651392, "tokens_per_sec": 17414.694333077055, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594970703125, "G_std": 0.1576995849609375, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.459428787231445, "grad_norm": 0.9444233179092407, "tokens_seen": 78815232, "tokens_per_sec": 17414.9185716537, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3593088785807292, "G_std": 0.1581293741861979, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.303701877593994, "grad_norm": 1.0229166746139526, "tokens_seen": 78979072, "tokens_per_sec": 17415.388089666907, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3560333251953125, "G_std": 0.1580810546875, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.428112506866455, "grad_norm": 0.9108160138130188, "tokens_seen": 79142912, "tokens_per_sec": 17415.794876719174, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3557790120442708, "G_std": 0.15838623046875, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.608844995498657, "grad_norm": 1.0227813720703125, "tokens_seen": 79306752, "tokens_per_sec": 17416.26378252151, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3604329427083333, "G_std": 0.1586481730143229, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.2818098068237305, "grad_norm": 0.883969783782959, "tokens_seen": 79470592, "tokens_per_sec": 17416.50544728364, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3519134521484375, "G_std": 0.1569239298502604, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.269627809524536, "grad_norm": 0.9204914569854736, "tokens_seen": 79634432, "tokens_per_sec": 17416.999896972302, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3573048909505208, "G_std": 0.15695699055989584, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.4428629875183105, "grad_norm": 0.9609402418136597, "tokens_seen": 79798272, "tokens_per_sec": 17417.434517226822, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.361053466796875, "G_std": 0.15966796875, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.284149646759033, "grad_norm": 0.9481637477874756, "tokens_seen": 79962112, "tokens_per_sec": 17417.88716926589, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3581339518229167, "G_std": 0.15541585286458334, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.3586344718933105, "grad_norm": 0.9257392883300781, "tokens_seen": 80125952, "tokens_per_sec": 17418.262275283363, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3564809163411458, "G_std": 0.1560338338216146, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.330106973648071, "grad_norm": 0.9574484825134277, "tokens_seen": 80289792, "tokens_per_sec": 17418.532015527027, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613382975260417, "G_std": 0.1569646199544271, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.469878435134888, "grad_norm": 1.0338295698165894, "tokens_seen": 80453632, "tokens_per_sec": 17418.989854429925, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632456461588542, "G_std": 0.15778350830078125, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.135722637176514, "grad_norm": 0.960864782333374, "tokens_seen": 80617472, "tokens_per_sec": 17419.56595169777, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606058756510417, "G_std": 0.15581258138020834, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.445955514907837, "grad_norm": 0.9996866583824158, "tokens_seen": 80781312, "tokens_per_sec": 17420.058475688253, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594868977864583, "G_std": 0.1569391886393229, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.496888875961304, "grad_norm": 1.067959189414978, "tokens_seen": 80945152, "tokens_per_sec": 17420.433042547873, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3642832438151042, "G_std": 0.15901947021484375, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.310277223587036, "grad_norm": 0.9679005742073059, "tokens_seen": 81108992, "tokens_per_sec": 17420.966701937916, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3592987060546875, "G_std": 0.15706634521484375, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.4416210651397705, "grad_norm": 0.9979745745658875, "tokens_seen": 81272832, "tokens_per_sec": 17421.578529006496, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.355133056640625, "G_std": 0.1567230224609375, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.567537307739258, "grad_norm": 1.04815673828125, "tokens_seen": 81436672, "tokens_per_sec": 17422.059355370857, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3578135172526042, "G_std": 0.1563873291015625, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.461585760116577, "grad_norm": 0.9558714032173157, "tokens_seen": 81600512, "tokens_per_sec": 17422.590106437354, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3558756510416667, "G_std": 0.1555150349934896, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.337673664093018, "grad_norm": 0.9449090361595154, "tokens_seen": 81764352, "tokens_per_sec": 17422.95954808876, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3542633056640625, "G_std": 0.15650431315104166, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.480930805206299, "grad_norm": 0.9871834516525269, "tokens_seen": 81928192, "tokens_per_sec": 17423.437231911033, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3570454915364583, "G_std": 0.1553624471028646, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.436952984333038, "val_ppl": 84.51702423716584, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3591766357421875, "G_std": 0.15767669677734375, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.246593236923218, "grad_norm": 0.9313861131668091, "tokens_seen": 82092032, "tokens_per_sec": 17411.29707986914, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589986165364583, "G_std": 0.15695699055989584, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.236993312835693, "grad_norm": 0.9309512376785278, "tokens_seen": 82255872, "tokens_per_sec": 17411.894005568956, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3535308837890625, "G_std": 0.15805562337239584, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.262070894241333, "grad_norm": 0.9414401054382324, "tokens_seen": 82419712, "tokens_per_sec": 17412.37432609025, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35345458984375, "G_std": 0.15699259440104166, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.604310512542725, "grad_norm": 1.208353042602539, "tokens_seen": 82583552, "tokens_per_sec": 17412.789007916537, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3556722005208333, "G_std": 0.15604654947916666, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.5516357421875, "grad_norm": 0.9558333158493042, "tokens_seen": 82747392, "tokens_per_sec": 17413.14885706186, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3578999837239583, "G_std": 0.1561762491861979, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.328901290893555, "grad_norm": 0.9894897937774658, "tokens_seen": 82911232, "tokens_per_sec": 17413.59086138967, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.356231689453125, "G_std": 0.156829833984375, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.3455939292907715, "grad_norm": 1.0223734378814697, "tokens_seen": 83075072, "tokens_per_sec": 17413.96738417824, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3543752034505208, "G_std": 0.15548197428385416, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.294241666793823, "grad_norm": 0.9373534321784973, "tokens_seen": 83238912, "tokens_per_sec": 17414.29530994593, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3559417724609375, "G_std": 0.15622711181640625, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.296720027923584, "grad_norm": 0.9183921217918396, "tokens_seen": 83402752, "tokens_per_sec": 17414.56076530214, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3569793701171875, "G_std": 0.1566899617513021, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.328613519668579, "grad_norm": 0.9837591648101807, "tokens_seen": 83566592, "tokens_per_sec": 17414.964748122533, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3579508463541667, "G_std": 0.15664164225260416, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.366917133331299, "grad_norm": 1.0553652048110962, "tokens_seen": 83730432, "tokens_per_sec": 17415.320509983194, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594207763671875, "G_std": 0.15637969970703125, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.350370168685913, "grad_norm": 1.0488243103027344, "tokens_seen": 83894272, "tokens_per_sec": 17415.787878349285, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3587595621744792, "G_std": 0.1573460896809896, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.406488418579102, "grad_norm": 1.0814692974090576, "tokens_seen": 84058112, "tokens_per_sec": 17415.531175516106, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3570658365885417, "G_std": 0.15682474772135416, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.397033214569092, "grad_norm": 1.0710134506225586, "tokens_seen": 84221952, "tokens_per_sec": 17416.052336186982, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3599395751953125, "G_std": 0.15577189127604166, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.2846269607543945, "grad_norm": 0.9460333585739136, "tokens_seen": 84385792, "tokens_per_sec": 17416.619716504767, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3535970052083333, "G_std": 0.15535481770833334, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.247344493865967, "grad_norm": 0.9613156318664551, "tokens_seen": 84549632, "tokens_per_sec": 17417.068970804143, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3523966471354167, "G_std": 0.1556116739908854, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.323890686035156, "grad_norm": 1.0597470998764038, "tokens_seen": 84713472, "tokens_per_sec": 17417.559320410972, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3521931966145833, "G_std": 0.15680694580078125, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.420758247375488, "grad_norm": 1.0025889873504639, "tokens_seen": 84877312, "tokens_per_sec": 17417.99901798987, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3579559326171875, "G_std": 0.1561304728190104, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.255435466766357, "grad_norm": 0.9801077842712402, "tokens_seen": 85041152, "tokens_per_sec": 17418.438439248694, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3529408772786458, "G_std": 0.15578715006510416, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.356053590774536, "grad_norm": 1.0198184251785278, "tokens_seen": 85204992, "tokens_per_sec": 17418.968925082314, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3551890055338542, "G_std": 0.1553217569986979, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.414550542831421, "grad_norm": 1.2882000207901, "tokens_seen": 85368832, "tokens_per_sec": 17419.506793988498, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626047770182292, "G_std": 0.15726725260416666, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.431502103805542, "grad_norm": 1.1091591119766235, "tokens_seen": 85532672, "tokens_per_sec": 17419.993969695694, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3592529296875, "G_std": 0.15629069010416666, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.465605020523071, "grad_norm": 0.987034022808075, "tokens_seen": 85696512, "tokens_per_sec": 17420.517808179397, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.361572265625, "G_std": 0.156585693359375, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.348021984100342, "grad_norm": 1.0513635873794556, "tokens_seen": 85860352, "tokens_per_sec": 17420.995960727847, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616943359375, "G_std": 0.15677388509114584, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.435948133468628, "grad_norm": 0.9732281565666199, "tokens_seen": 86024192, "tokens_per_sec": 17421.457206220686, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580169677734375, "G_std": 0.15594482421875, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.4427901148796085, "val_ppl": 85.01180378259798, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3562876383463542, "G_std": 0.15531412760416666, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.249158143997192, "grad_norm": 0.9822227954864502, "tokens_seen": 86188032, "tokens_per_sec": 17409.844505133307, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.356842041015625, "G_std": 0.1540654500325521, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.409606456756592, "grad_norm": 0.9968631863594055, "tokens_seen": 86351872, "tokens_per_sec": 17410.362160790173, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3544972737630208, "G_std": 0.15428670247395834, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.366855144500732, "grad_norm": 1.0108469724655151, "tokens_seen": 86515712, "tokens_per_sec": 17410.826161237488, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.351898193359375, "G_std": 0.15496063232421875, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.364727258682251, "grad_norm": 1.0042154788970947, "tokens_seen": 86679552, "tokens_per_sec": 17411.31712650487, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3570963541666667, "G_std": 0.15537261962890625, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.390864849090576, "grad_norm": 0.9390903115272522, "tokens_seen": 86843392, "tokens_per_sec": 17411.751386189808, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606770833333333, "G_std": 0.15595499674479166, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.199881315231323, "grad_norm": 1.0210193395614624, "tokens_seen": 87007232, "tokens_per_sec": 17412.095392086412, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550618489583333, "G_std": 0.156829833984375, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.21670937538147, "grad_norm": 0.9818359613418579, "tokens_seen": 87171072, "tokens_per_sec": 17412.502214029006, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3514658610026042, "G_std": 0.1564661661783854, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.3422157764434814, "grad_norm": 0.9500545859336853, "tokens_seen": 87334912, "tokens_per_sec": 17412.833788604028, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550364176432292, "G_std": 0.15512847900390625, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.431038856506348, "grad_norm": 1.1751363277435303, "tokens_seen": 87498752, "tokens_per_sec": 17413.228650145295, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3586324055989583, "G_std": 0.1547266642252604, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.265010356903076, "grad_norm": 0.9968959093093872, "tokens_seen": 87662592, "tokens_per_sec": 17413.521903971807, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3561350504557292, "G_std": 0.15500640869140625, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.474937915802002, "grad_norm": 1.0034472942352295, "tokens_seen": 87826432, "tokens_per_sec": 17413.991500962416, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3627878824869792, "G_std": 0.15462239583333334, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.3986499309539795, "grad_norm": 0.9433746933937073, "tokens_seen": 87990272, "tokens_per_sec": 17414.450449609623, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590545654296875, "G_std": 0.15406036376953125, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.314216613769531, "grad_norm": 0.9695367813110352, "tokens_seen": 88154112, "tokens_per_sec": 17414.92034965837, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566029866536458, "G_std": 0.1558074951171875, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.378273010253906, "grad_norm": 1.0011584758758545, "tokens_seen": 88317952, "tokens_per_sec": 17415.34547783697, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3587849934895833, "G_std": 0.15675862630208334, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.474588632583618, "grad_norm": 1.0576070547103882, "tokens_seen": 88481792, "tokens_per_sec": 17415.857063257565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603769938151042, "G_std": 0.15579986572265625, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.257494926452637, "grad_norm": 1.0770671367645264, "tokens_seen": 88645632, "tokens_per_sec": 17416.31828835841, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.359039306640625, "G_std": 0.15670013427734375, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.3657190799713135, "grad_norm": 0.9530032873153687, "tokens_seen": 88809472, "tokens_per_sec": 17416.7170173383, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3487497965494792, "G_std": 0.1542816162109375, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.465880632400513, "grad_norm": 1.0279608964920044, "tokens_seen": 88973312, "tokens_per_sec": 17417.20889516608, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35577392578125, "G_std": 0.1560541788736979, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.29216194152832, "grad_norm": 0.9857645630836487, "tokens_seen": 89137152, "tokens_per_sec": 17417.659874020086, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3505655924479167, "G_std": 0.15521240234375, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.388333320617676, "grad_norm": 1.079091191291809, "tokens_seen": 89300992, "tokens_per_sec": 17417.97479285007, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3581492106119792, "G_std": 0.15587615966796875, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.325841188430786, "grad_norm": 0.9831328392028809, "tokens_seen": 89464832, "tokens_per_sec": 17418.52413207978, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3576304117838542, "G_std": 0.1554743448893229, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.4011359214782715, "grad_norm": 1.0547419786453247, "tokens_seen": 89628672, "tokens_per_sec": 17418.94430685318, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550465901692708, "G_std": 0.15631866455078125, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.435718059539795, "grad_norm": 1.0995477437973022, "tokens_seen": 89792512, "tokens_per_sec": 17419.441646216477, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3572642008463542, "G_std": 0.15445454915364584, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.32962703704834, "grad_norm": 1.038475751876831, "tokens_seen": 89956352, "tokens_per_sec": 17419.96159007803, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3551228841145833, "G_std": 0.15618896484375, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.363365650177002, "grad_norm": 0.9524005651473999, "tokens_seen": 90120192, "tokens_per_sec": 17420.37471821527, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3484141031901042, "G_std": 0.15537516276041666, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.401202297210693, "val_ppl": 81.54885571014458, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3533172607421875, "G_std": 0.1565729777018229, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.228069305419922, "grad_norm": 0.9897737503051758, "tokens_seen": 90284032, "tokens_per_sec": 17409.1942444138, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3536529541015625, "G_std": 0.15555572509765625, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.223143100738525, "grad_norm": 0.9737480878829956, "tokens_seen": 90447872, "tokens_per_sec": 17409.55458396684, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3521830240885417, "G_std": 0.15445454915364584, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.437063932418823, "grad_norm": 1.034919261932373, "tokens_seen": 90611712, "tokens_per_sec": 17410.046377762483, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35595703125, "G_std": 0.15532684326171875, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.372406959533691, "grad_norm": 0.9726337790489197, "tokens_seen": 90775552, "tokens_per_sec": 17410.539435765015, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35400390625, "G_std": 0.1556981404622396, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.437132835388184, "grad_norm": 1.0313483476638794, "tokens_seen": 90939392, "tokens_per_sec": 17410.94046282502, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3557027180989583, "G_std": 0.15676116943359375, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.3213911056518555, "grad_norm": 0.9375766515731812, "tokens_seen": 91103232, "tokens_per_sec": 17411.160776098346, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3508097330729167, "G_std": 0.1540349324544271, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.200969696044922, "grad_norm": 0.938848078250885, "tokens_seen": 91267072, "tokens_per_sec": 17411.51420302577, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3573455810546875, "G_std": 0.15546417236328125, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.284227609634399, "grad_norm": 1.0422972440719604, "tokens_seen": 91430912, "tokens_per_sec": 17411.879917595856, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550567626953125, "G_std": 0.1544367472330729, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.379925489425659, "grad_norm": 0.9823831915855408, "tokens_seen": 91594752, "tokens_per_sec": 17412.373551011246, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.355010986328125, "G_std": 0.15491485595703125, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.362433671951294, "grad_norm": 0.9683425426483154, "tokens_seen": 91758592, "tokens_per_sec": 17412.71258609175, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3535512288411458, "G_std": 0.15488179524739584, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.358548402786255, "grad_norm": 0.9807716012001038, "tokens_seen": 91922432, "tokens_per_sec": 17413.054983180587, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3558349609375, "G_std": 0.15655517578125, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.187801122665405, "grad_norm": 0.9437678456306458, "tokens_seen": 92086272, "tokens_per_sec": 17413.450918071918, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3530476888020833, "G_std": 0.1557159423828125, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.37833309173584, "grad_norm": 1.0295547246932983, "tokens_seen": 92250112, "tokens_per_sec": 17413.85715763415, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3552652994791667, "G_std": 0.1541620890299479, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.389370679855347, "grad_norm": 1.0349091291427612, "tokens_seen": 92413952, "tokens_per_sec": 17414.13038572987, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3554636637369792, "G_std": 0.1563542683919271, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.369303226470947, "grad_norm": 0.9888035655021667, "tokens_seen": 92577792, "tokens_per_sec": 17414.571593216937, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3562723795572917, "G_std": 0.1566441853841146, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.240929841995239, "grad_norm": 0.9980627298355103, "tokens_seen": 92741632, "tokens_per_sec": 17414.90089257612, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35369873046875, "G_std": 0.1575037638346354, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.292327880859375, "grad_norm": 1.0241273641586304, "tokens_seen": 92905472, "tokens_per_sec": 17415.288986814754, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3546040852864583, "G_std": 0.156646728515625, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.1849658489227295, "grad_norm": 1.0360416173934937, "tokens_seen": 93069312, "tokens_per_sec": 17415.683126448217, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3528493245442708, "G_std": 0.1553217569986979, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.3763792514801025, "grad_norm": 1.0319843292236328, "tokens_seen": 93233152, "tokens_per_sec": 17416.04714574994, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.357940673828125, "G_std": 0.15634918212890625, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.302528142929077, "grad_norm": 1.0406498908996582, "tokens_seen": 93396992, "tokens_per_sec": 17416.47470052538, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3508555094401042, "G_std": 0.15712992350260416, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.3179240226745605, "grad_norm": 0.9975664615631104, "tokens_seen": 93560832, "tokens_per_sec": 17416.834567196995, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550364176432292, "G_std": 0.1538848876953125, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.171195983886719, "grad_norm": 0.9951872825622559, "tokens_seen": 93724672, "tokens_per_sec": 17417.212545259346, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.352996826171875, "G_std": 0.15386962890625, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.403195381164551, "grad_norm": 1.021976351737976, "tokens_seen": 93888512, "tokens_per_sec": 17417.692988650913, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3579915364583333, "G_std": 0.1553827921549479, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.280333518981934, "grad_norm": 1.0702762603759766, "tokens_seen": 94052352, "tokens_per_sec": 17418.06589418144, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3531595865885417, "G_std": 0.1544367472330729, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.291314363479614, "grad_norm": 0.9464473128318787, "tokens_seen": 94216192, "tokens_per_sec": 17418.533474699794, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.349456787109375, "G_std": 0.1560490926106771, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.408435094356537, "val_ppl": 82.1408202400175, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3502553304036458, "G_std": 0.15534464518229166, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.313306093215942, "grad_norm": 1.0406005382537842, "tokens_seen": 94380032, "tokens_per_sec": 17407.905750510454, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3519795735677083, "G_std": 0.15607452392578125, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.438596248626709, "grad_norm": 1.1916536092758179, "tokens_seen": 94543872, "tokens_per_sec": 17408.21121608007, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3553517659505208, "G_std": 0.15472412109375, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.384734869003296, "grad_norm": 1.0286812782287598, "tokens_seen": 94707712, "tokens_per_sec": 17408.63726535103, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3567606608072917, "G_std": 0.1537755330403646, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.190080404281616, "grad_norm": 1.1217317581176758, "tokens_seen": 94871552, "tokens_per_sec": 17408.969698699286, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3483632405598958, "G_std": 0.154541015625, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.3141419887542725, "grad_norm": 1.0953819751739502, "tokens_seen": 95035392, "tokens_per_sec": 17409.444234113515, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3612111409505208, "G_std": 0.15535990397135416, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.380828380584717, "grad_norm": 1.102529764175415, "tokens_seen": 95199232, "tokens_per_sec": 17409.82140323713, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3581593831380208, "G_std": 0.1538263956705729, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.132693290710449, "grad_norm": 0.9775939583778381, "tokens_seen": 95363072, "tokens_per_sec": 17410.23662367109, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3538716634114583, "G_std": 0.15610504150390625, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.284368276596069, "grad_norm": 0.9745901823043823, "tokens_seen": 95526912, "tokens_per_sec": 17410.502910467992, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3467458089192708, "G_std": 0.1531041463216146, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.265275001525879, "grad_norm": 0.9446101188659668, "tokens_seen": 95690752, "tokens_per_sec": 17410.824950005157, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3534596761067708, "G_std": 0.156646728515625, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.284668922424316, "grad_norm": 1.0290879011154175, "tokens_seen": 95854592, "tokens_per_sec": 17411.065961354867, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3531951904296875, "G_std": 0.1550928751627604, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.257430791854858, "grad_norm": 1.0186113119125366, "tokens_seen": 96018432, "tokens_per_sec": 17411.395652731415, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3523203531901042, "G_std": 0.15468343098958334, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.4276347160339355, "grad_norm": 0.9836366176605225, "tokens_seen": 96182272, "tokens_per_sec": 17411.78240331991, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3551991780598958, "G_std": 0.15384674072265625, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.362913608551025, "grad_norm": 1.161536693572998, "tokens_seen": 96346112, "tokens_per_sec": 17412.08570409235, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3559926350911458, "G_std": 0.15540059407552084, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.363677024841309, "grad_norm": 1.0652074813842773, "tokens_seen": 96509952, "tokens_per_sec": 17412.472713429575, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3503316243489583, "G_std": 0.1546198527018229, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.395190238952637, "grad_norm": 0.9852145910263062, "tokens_seen": 96673792, "tokens_per_sec": 17412.870015642096, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3542531331380208, "G_std": 0.154632568359375, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.291146993637085, "grad_norm": 1.0443240404129028, "tokens_seen": 96837632, "tokens_per_sec": 17413.224843490036, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3536834716796875, "G_std": 0.15639495849609375, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.3419530391693115, "grad_norm": 1.0469512939453125, "tokens_seen": 97001472, "tokens_per_sec": 17413.62988764423, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3522084554036458, "G_std": 0.1549402872721354, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.141834735870361, "grad_norm": 0.9652126431465149, "tokens_seen": 97165312, "tokens_per_sec": 17414.04887125069, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3503316243489583, "G_std": 0.1565119425455729, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.4812233448028564, "grad_norm": 1.1412756443023682, "tokens_seen": 97329152, "tokens_per_sec": 17414.447346372923, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3575286865234375, "G_std": 0.1544621785481771, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.271178483963013, "grad_norm": 1.069743275642395, "tokens_seen": 97492992, "tokens_per_sec": 17414.80745292987, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3500213623046875, "G_std": 0.1555658976236979, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.441014289855957, "grad_norm": 1.0483310222625732, "tokens_seen": 97656832, "tokens_per_sec": 17415.21617882492, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3537851969401042, "G_std": 0.1557133992513021, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.1319379806518555, "grad_norm": 1.067238211631775, "tokens_seen": 97820672, "tokens_per_sec": 17415.578653124394, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3500620524088542, "G_std": 0.154632568359375, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.348500967025757, "grad_norm": 1.026289701461792, "tokens_seen": 97984512, "tokens_per_sec": 17415.986209546754, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3564249674479167, "G_std": 0.15525054931640625, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.2342729568481445, "grad_norm": 0.984787106513977, "tokens_seen": 98148352, "tokens_per_sec": 17416.351666773586, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3484141031901042, "G_std": 0.15384419759114584, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.359493756294251, "val_ppl": 78.21752726763506, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3498077392578125, "G_std": 0.15547943115234375, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5647.636380910873, "nan_failures": 0} diff --git a/g_only_s1/config.json b/g_only_s1/config.json new file mode 100644 index 0000000000000000000000000000000000000000..08165d2200f4bd1d918683e5a2cfa84b9f1d1998 --- /dev/null +++ b/g_only_s1/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 1, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s1", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1001340, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/g_only_s1/final.pt b/g_only_s1/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..defc6167bb011d7eeeb6eda2250cbd7a0452ed60 --- /dev/null +++ b/g_only_s1/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0904b072959de61365838c737bf14d2ec72dbffbc5a02fcfa7fa49895a8fed85 +size 253629735 diff --git a/g_only_s1/metrics.jsonl b/g_only_s1/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..5c32dead7f03b85aebd1ce21482e46b008502779 --- /dev/null +++ b/g_only_s1/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 1, "config": {"variant": "g_only", "seed": 1, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s1", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1001340, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.924972534179688, "grad_norm": 9.604771614074707, "tokens_seen": 8192, "tokens_per_sec": 8306.269266835614, "vram_gb": 5.768167424, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5144856770833334, "G_std": 0.08725992838541667, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.78232192993164, "grad_norm": 2.08304500579834, "tokens_seen": 172032, "tokens_per_sec": 16893.597125004228, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.462158203125, "G_std": 0.070892333984375, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.164056777954102, "grad_norm": 1.6950100660324097, "tokens_seen": 335872, "tokens_per_sec": 17298.78610272951, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4429524739583333, "G_std": 0.07520548502604167, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.267223358154297, "grad_norm": 1.5229833126068115, "tokens_seen": 499712, "tokens_per_sec": 17444.999429260286, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.438232421875, "G_std": 0.0796966552734375, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.410038948059082, "grad_norm": 0.8849597573280334, "tokens_seen": 663552, "tokens_per_sec": 17523.0666653386, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44091796875, "G_std": 0.091217041015625, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.28564715385437, "grad_norm": 0.6548714637756348, "tokens_seen": 827392, "tokens_per_sec": 17569.608325350982, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4335123697916667, "G_std": 0.1031494140625, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.07114315032959, "grad_norm": 0.771887481212616, "tokens_seen": 991232, "tokens_per_sec": 17609.415138100438, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.425537109375, "G_std": 0.11285400390625, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.801666498184204, "grad_norm": 0.6028391718864441, "tokens_seen": 1155072, "tokens_per_sec": 17633.730581402255, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.412353515625, "G_std": 0.124603271484375, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.786003112792969, "grad_norm": 0.5349136590957642, "tokens_seen": 1318912, "tokens_per_sec": 17653.004010680368, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.418212890625, "G_std": 0.13328043619791666, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.759461402893066, "grad_norm": 0.5907541513442993, "tokens_seen": 1482752, "tokens_per_sec": 17666.307746881343, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3993326822916667, "G_std": 0.14162190755208334, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.605790376663208, "grad_norm": 0.5877522230148315, "tokens_seen": 1646592, "tokens_per_sec": 17677.492488425232, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3927408854166667, "G_std": 0.14599609375, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.600616455078125, "grad_norm": 0.5300235152244568, "tokens_seen": 1810432, "tokens_per_sec": 17684.355185459473, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961588541666667, "G_std": 0.15142822265625, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.62768816947937, "grad_norm": 0.5005213618278503, "tokens_seen": 1974272, "tokens_per_sec": 17688.7367442338, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3956705729166667, "G_std": 0.15903727213541666, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.627918243408203, "grad_norm": 0.5446878671646118, "tokens_seen": 2138112, "tokens_per_sec": 17696.556109040273, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3970540364583333, "G_std": 0.16473388671875, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.691066741943359, "grad_norm": 0.5278946757316589, "tokens_seen": 2301952, "tokens_per_sec": 17703.657274405938, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4051920572916667, "G_std": 0.17161051432291666, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.549041748046875, "grad_norm": 0.5229331254959106, "tokens_seen": 2465792, "tokens_per_sec": 17711.274771168188, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4017740885416667, "G_std": 0.17696126302083334, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.464349031448364, "grad_norm": 0.44754549860954285, "tokens_seen": 2629632, "tokens_per_sec": 17714.250338455422, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4063313802083333, "G_std": 0.17948404947916666, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.48103141784668, "grad_norm": 0.5074949264526367, "tokens_seen": 2793472, "tokens_per_sec": 17718.952089253813, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4210611979166667, "G_std": 0.18589274088541666, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.224128484725952, "grad_norm": 0.4701739251613617, "tokens_seen": 2957312, "tokens_per_sec": 17716.446765502507, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40087890625, "G_std": 0.18182373046875, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.2943549156188965, "grad_norm": 0.4331626892089844, "tokens_seen": 3121152, "tokens_per_sec": 17718.46527896149, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.409912109375, "G_std": 0.19091796875, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.261551141738892, "grad_norm": 0.45040130615234375, "tokens_seen": 3284992, "tokens_per_sec": 17721.66250113226, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4324544270833333, "G_std": 0.19915771484375, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.350820541381836, "grad_norm": 0.4495505392551422, "tokens_seen": 3448832, "tokens_per_sec": 17724.11382202535, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4244791666666667, "G_std": 0.20192464192708334, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.257469654083252, "grad_norm": 0.48233091831207275, "tokens_seen": 3612672, "tokens_per_sec": 17726.22408939241, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4252522786458333, "G_std": 0.20253499348958334, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.3201210498809814, "grad_norm": 0.5071979761123657, "tokens_seen": 3776512, "tokens_per_sec": 17728.437832315256, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.437744140625, "G_std": 0.20943196614583334, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.198254108428955, "grad_norm": 0.4882130026817322, "tokens_seen": 3940352, "tokens_per_sec": 17730.71323313014, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4337565104166667, "G_std": 0.20725504557291666, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.321253061294556, "grad_norm": 0.49476808309555054, "tokens_seen": 4104192, "tokens_per_sec": 17732.908041724204, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4768880208333333, "G_std": 0.21956380208333334, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.244848597049713, "val_ppl": 515.3511933287893, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.48291015625, "G_std": 0.22176106770833334, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.160852909088135, "grad_norm": 0.5223144292831421, "tokens_seen": 4268032, "tokens_per_sec": 17482.000208421603, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4537760416666667, "G_std": 0.21307373046875, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.223294019699097, "grad_norm": 0.4483858048915863, "tokens_seen": 4431872, "tokens_per_sec": 17490.639603117736, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4691569010416667, "G_std": 0.22115071614583334, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.135289669036865, "grad_norm": 0.4404427707195282, "tokens_seen": 4595712, "tokens_per_sec": 17498.79245900515, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5189615885416666, "G_std": 0.22513834635416666, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.086121320724487, "grad_norm": 0.5293880105018616, "tokens_seen": 4759552, "tokens_per_sec": 17507.391129273758, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.470458984375, "G_std": 0.2210693359375, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.125059604644775, "grad_norm": 0.5011733770370483, "tokens_seen": 4923392, "tokens_per_sec": 17514.891790157697, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5110677083333334, "G_std": 0.21885172526041666, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.237567186355591, "grad_norm": 0.4539794325828552, "tokens_seen": 5087232, "tokens_per_sec": 17521.696691786117, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5538736979166666, "G_std": 0.221435546875, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.236908197402954, "grad_norm": 0.5821757316589355, "tokens_seen": 5251072, "tokens_per_sec": 17528.70616047177, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.560791015625, "G_std": 0.21677652994791666, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.962710380554199, "grad_norm": 0.473405122756958, "tokens_seen": 5414912, "tokens_per_sec": 17536.499922848176, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5425618489583334, "G_std": 0.2156982421875, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 5.982151746749878, "grad_norm": 0.5057028532028198, "tokens_seen": 5578752, "tokens_per_sec": 17543.752711898964, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5274251302083334, "G_std": 0.20961507161458334, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.97622013092041, "grad_norm": 0.4607587158679962, "tokens_seen": 5742592, "tokens_per_sec": 17549.69105725749, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5130208333333334, "G_std": 0.21282958984375, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.053247690200806, "grad_norm": 0.46982690691947937, "tokens_seen": 5906432, "tokens_per_sec": 17555.60456912452, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5751953125, "G_std": 0.20719401041666666, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.9972007274627686, "grad_norm": 0.5549790859222412, "tokens_seen": 6070272, "tokens_per_sec": 17560.72704986211, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.56884765625, "G_std": 0.20273844401041666, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.0233235359191895, "grad_norm": 0.4707152843475342, "tokens_seen": 6234112, "tokens_per_sec": 17566.27422969119, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5531412760416666, "G_std": 0.20623779296875, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.006253957748413, "grad_norm": 0.46883442997932434, "tokens_seen": 6397952, "tokens_per_sec": 17572.131918567477, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5902506510416666, "G_std": 0.198974609375, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.927348613739014, "grad_norm": 0.5060428380966187, "tokens_seen": 6561792, "tokens_per_sec": 17577.28646307627, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.586669921875, "G_std": 0.20298258463541666, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8404622077941895, "grad_norm": 0.5154346823692322, "tokens_seen": 6725632, "tokens_per_sec": 17582.89338665988, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5594075520833334, "G_std": 0.19856770833333334, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.759993553161621, "grad_norm": 0.486040860414505, "tokens_seen": 6889472, "tokens_per_sec": 17587.762726356556, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.559814453125, "G_std": 0.19864908854166666, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.66979718208313, "grad_norm": 0.4993176758289337, "tokens_seen": 7053312, "tokens_per_sec": 17592.473798277973, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5826009114583334, "G_std": 0.1917724609375, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.763062477111816, "grad_norm": 0.591308057308197, "tokens_seen": 7217152, "tokens_per_sec": 17597.69621907637, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5708821614583334, "G_std": 0.19862874348958334, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.685304403305054, "grad_norm": 0.5040788054466248, "tokens_seen": 7380992, "tokens_per_sec": 17601.900226592767, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5380045572916666, "G_std": 0.19687906901041666, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.865669250488281, "grad_norm": 0.5126530528068542, "tokens_seen": 7544832, "tokens_per_sec": 17606.40235582939, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5624186197916666, "G_std": 0.19612630208333334, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.770277500152588, "grad_norm": 0.47482597827911377, "tokens_seen": 7708672, "tokens_per_sec": 17609.777285061875, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.566162109375, "G_std": 0.1910400390625, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.916459560394287, "grad_norm": 0.5048354864120483, "tokens_seen": 7872512, "tokens_per_sec": 17612.467969313682, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5484619140625, "G_std": 0.18874104817708334, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.626199245452881, "grad_norm": 0.5115190148353577, "tokens_seen": 8036352, "tokens_per_sec": 17615.77499291391, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5350341796875, "G_std": 0.19112141927083334, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.745244741439819, "grad_norm": 0.4945279359817505, "tokens_seen": 8200192, "tokens_per_sec": 17618.59588610156, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5567220052083334, "G_std": 0.19142659505208334, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.774618482589721, "val_ppl": 322.02155444997834, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5567220052083334, "G_std": 0.19295247395833334, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.661242485046387, "grad_norm": 0.5313418507575989, "tokens_seen": 8364032, "tokens_per_sec": 17494.680772644508, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5634765625, "G_std": 0.19435628255208334, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.723890781402588, "grad_norm": 0.519260048866272, "tokens_seen": 8527872, "tokens_per_sec": 17498.75388442799, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5581868489583334, "G_std": 0.18878173828125, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.763526678085327, "grad_norm": 0.5314663648605347, "tokens_seen": 8691712, "tokens_per_sec": 17502.38298895048, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5459798177083334, "G_std": 0.19187418619791666, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.573713064193726, "grad_norm": 0.48958662152290344, "tokens_seen": 8855552, "tokens_per_sec": 17506.6956112764, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5531819661458334, "G_std": 0.18906656901041666, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.7831737995147705, "grad_norm": 0.5334565043449402, "tokens_seen": 9019392, "tokens_per_sec": 17510.102983041575, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5497639973958334, "G_std": 0.189697265625, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.630723714828491, "grad_norm": 0.5025845170021057, "tokens_seen": 9183232, "tokens_per_sec": 17513.839726648344, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5359700520833334, "G_std": 0.18638102213541666, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.676323890686035, "grad_norm": 0.48440954089164734, "tokens_seen": 9347072, "tokens_per_sec": 17517.03925110193, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5467122395833334, "G_std": 0.18900553385416666, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.669658184051514, "grad_norm": 0.5483078360557556, "tokens_seen": 9510912, "tokens_per_sec": 17520.671068552285, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5389811197916666, "G_std": 0.18977864583333334, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.714070796966553, "grad_norm": 0.5674976706504822, "tokens_seen": 9674752, "tokens_per_sec": 17524.039797272886, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5594482421875, "G_std": 0.1905517578125, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.644197225570679, "grad_norm": 0.5699310898780823, "tokens_seen": 9838592, "tokens_per_sec": 17527.569393832655, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5287272135416666, "G_std": 0.19163004557291666, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.812585353851318, "grad_norm": 0.5209757089614868, "tokens_seen": 10002432, "tokens_per_sec": 17530.88163483648, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5346272786458334, "G_std": 0.19215901692708334, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.593014240264893, "grad_norm": 0.5340203642845154, "tokens_seen": 10166272, "tokens_per_sec": 17534.04542173508, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5206705729166666, "G_std": 0.18617757161458334, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.571081876754761, "grad_norm": 0.5529677271842957, "tokens_seen": 10330112, "tokens_per_sec": 17535.790616054594, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5228271484375, "G_std": 0.1907958984375, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.4934751987457275, "grad_norm": 0.556979238986969, "tokens_seen": 10493952, "tokens_per_sec": 17537.33563278263, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5172119140625, "G_std": 0.18855794270833334, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.610503673553467, "grad_norm": 0.5419532060623169, "tokens_seen": 10657792, "tokens_per_sec": 17538.94114903279, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5262044270833334, "G_std": 0.1893310546875, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.4675140380859375, "grad_norm": 0.5247291922569275, "tokens_seen": 10821632, "tokens_per_sec": 17536.6989785473, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5109049479166666, "G_std": 0.19014485677083334, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.7245192527771, "grad_norm": 0.5646486282348633, "tokens_seen": 10985472, "tokens_per_sec": 17537.330832863146, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5124104817708334, "G_std": 0.18682861328125, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.548182249069214, "grad_norm": 0.5552080273628235, "tokens_seen": 11149312, "tokens_per_sec": 17534.622845076752, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5126953125, "G_std": 0.188720703125, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.452489852905273, "grad_norm": 0.5258857607841492, "tokens_seen": 11313152, "tokens_per_sec": 17531.94427987138, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5108235677083334, "G_std": 0.18754069010416666, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.66227650642395, "grad_norm": 0.5110881328582764, "tokens_seen": 11476992, "tokens_per_sec": 17530.164788465067, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5013834635416666, "G_std": 0.18686930338541666, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.432371377944946, "grad_norm": 0.48696762323379517, "tokens_seen": 11640832, "tokens_per_sec": 17523.85983330821, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4960530598958333, "G_std": 0.19049072265625, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.5740556716918945, "grad_norm": 0.5457733869552612, "tokens_seen": 11804672, "tokens_per_sec": 17519.700870408997, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5017903645833334, "G_std": 0.18611653645833334, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.587893009185791, "grad_norm": 0.5862368941307068, "tokens_seen": 11968512, "tokens_per_sec": 17518.431805991375, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5019938151041666, "G_std": 0.19095865885416666, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.471709728240967, "grad_norm": 0.5125485062599182, "tokens_seen": 12132352, "tokens_per_sec": 17516.39827258812, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5011189778645834, "G_std": 0.19095865885416666, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.291302442550659, "grad_norm": 0.5570720434188843, "tokens_seen": 12296192, "tokens_per_sec": 17513.690188489363, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5101725260416666, "G_std": 0.18986002604166666, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.495800340175629, "val_ppl": 243.66646420264576, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5062662760416666, "G_std": 0.18906656901041666, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.486184120178223, "grad_norm": 0.5541260242462158, "tokens_seen": 12460032, "tokens_per_sec": 17432.111656291265, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5014851888020834, "G_std": 0.19453938802083334, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.491183280944824, "grad_norm": 0.603393018245697, "tokens_seen": 12623872, "tokens_per_sec": 17435.49585964904, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4924519856770833, "G_std": 0.1922607421875, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.45814323425293, "grad_norm": 0.5327566266059875, "tokens_seen": 12787712, "tokens_per_sec": 17438.831576540822, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4734903971354167, "G_std": 0.18929036458333334, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.144908905029297, "grad_norm": 0.5120548009872437, "tokens_seen": 12951552, "tokens_per_sec": 17442.10061992096, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48455810546875, "G_std": 0.19158935546875, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.227498769760132, "grad_norm": 0.5707536339759827, "tokens_seen": 13115392, "tokens_per_sec": 17445.37288553014, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47247314453125, "G_std": 0.18963623046875, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.247098207473755, "grad_norm": 0.5485741496086121, "tokens_seen": 13279232, "tokens_per_sec": 17448.54125716467, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4798380533854167, "G_std": 0.18865966796875, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.2393958568573, "grad_norm": 0.5296325087547302, "tokens_seen": 13443072, "tokens_per_sec": 17451.19652728673, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4689127604166667, "G_std": 0.189239501953125, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.371848106384277, "grad_norm": 0.5355192422866821, "tokens_seen": 13606912, "tokens_per_sec": 17454.485741534772, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4813232421875, "G_std": 0.19029744466145834, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.180217266082764, "grad_norm": 0.5616884231567383, "tokens_seen": 13770752, "tokens_per_sec": 17457.73670947497, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4839884440104167, "G_std": 0.18623860677083334, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.2554755210876465, "grad_norm": 0.5202010869979858, "tokens_seen": 13934592, "tokens_per_sec": 17461.069520436446, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4751383463541667, "G_std": 0.188873291015625, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.337219715118408, "grad_norm": 0.5633550882339478, "tokens_seen": 14098432, "tokens_per_sec": 17464.289868910677, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47662353515625, "G_std": 0.18669637044270834, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.270479679107666, "grad_norm": 0.5252576470375061, "tokens_seen": 14262272, "tokens_per_sec": 17467.249926072254, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4689737955729167, "G_std": 0.18675740559895834, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.308837890625, "grad_norm": 0.5544828176498413, "tokens_seen": 14426112, "tokens_per_sec": 17470.44449691428, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4785359700520833, "G_std": 0.190521240234375, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.3200318813323975, "grad_norm": 0.5597090125083923, "tokens_seen": 14589952, "tokens_per_sec": 17473.501711459314, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4659830729166667, "G_std": 0.18822224934895834, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.415538549423218, "grad_norm": 0.5772337913513184, "tokens_seen": 14753792, "tokens_per_sec": 17476.80433019381, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4661051432291667, "G_std": 0.18690999348958334, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.4146504402160645, "grad_norm": 0.548040509223938, "tokens_seen": 14917632, "tokens_per_sec": 17479.54079755453, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4720052083333333, "G_std": 0.18853759765625, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.187805891036987, "grad_norm": 0.5757331252098083, "tokens_seen": 15081472, "tokens_per_sec": 17482.853429756135, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4651896158854167, "G_std": 0.18614705403645834, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.28761625289917, "grad_norm": 0.6147106289863586, "tokens_seen": 15245312, "tokens_per_sec": 17485.537190428095, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47296142578125, "G_std": 0.18267822265625, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.111154794692993, "grad_norm": 0.5824455618858337, "tokens_seen": 15409152, "tokens_per_sec": 17488.27292931843, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4586385091145833, "G_std": 0.18297322591145834, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.33219051361084, "grad_norm": 0.5562164187431335, "tokens_seen": 15572992, "tokens_per_sec": 17489.20002076731, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4619954427083333, "G_std": 0.18636067708333334, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.3681981563568115, "grad_norm": 0.5548241138458252, "tokens_seen": 15736832, "tokens_per_sec": 17491.489187494244, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4544474283854167, "G_std": 0.185516357421875, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.494680643081665, "grad_norm": 0.6499180793762207, "tokens_seen": 15900672, "tokens_per_sec": 17494.23074490164, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4643758138020833, "G_std": 0.18589274088541666, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.122730016708374, "grad_norm": 0.6280980706214905, "tokens_seen": 16064512, "tokens_per_sec": 17496.87578154129, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.43365478515625, "G_std": 0.18555704752604166, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.250172853469849, "grad_norm": 0.5901655554771423, "tokens_seen": 16228352, "tokens_per_sec": 17499.469427677886, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4436238606770833, "G_std": 0.189208984375, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.303261995315552, "grad_norm": 0.5771619081497192, "tokens_seen": 16392192, "tokens_per_sec": 17502.029161603943, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4567057291666667, "G_std": 0.18686930338541666, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.334175717830658, "val_ppl": 207.30180318386638, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4448445638020833, "G_std": 0.18638102213541666, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.205942153930664, "grad_norm": 0.6224868297576904, "tokens_seen": 16556032, "tokens_per_sec": 17440.976625445164, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4540812174479167, "G_std": 0.185760498046875, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.29200553894043, "grad_norm": 0.5995799899101257, "tokens_seen": 16719872, "tokens_per_sec": 17443.64612056237, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4493001302083333, "G_std": 0.18217976888020834, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.223070859909058, "grad_norm": 0.5518000721931458, "tokens_seen": 16883712, "tokens_per_sec": 17446.38409208501, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4518229166666667, "G_std": 0.182464599609375, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.262681722640991, "grad_norm": 0.6305819749832153, "tokens_seen": 17047552, "tokens_per_sec": 17448.907686821716, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4603271484375, "G_std": 0.18522135416666666, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.28721022605896, "grad_norm": 0.5763866305351257, "tokens_seen": 17211392, "tokens_per_sec": 17451.555376031116, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4472249348958333, "G_std": 0.184539794921875, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.15344500541687, "grad_norm": 0.5999931693077087, "tokens_seen": 17375232, "tokens_per_sec": 17454.085519877717, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4482014973958333, "G_std": 0.18377685546875, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.137406826019287, "grad_norm": 0.6476722955703735, "tokens_seen": 17539072, "tokens_per_sec": 17456.69297342573, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4360860188802083, "G_std": 0.183013916015625, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.185767412185669, "grad_norm": 0.5726574659347534, "tokens_seen": 17702912, "tokens_per_sec": 17459.151905961204, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4343363444010417, "G_std": 0.18415323893229166, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.203423023223877, "grad_norm": 0.6008636355400085, "tokens_seen": 17866752, "tokens_per_sec": 17461.563998197056, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.433197021484375, "G_std": 0.18311564127604166, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.149962663650513, "grad_norm": 0.5972684025764465, "tokens_seen": 18030592, "tokens_per_sec": 17463.883072657933, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4318033854166667, "G_std": 0.18393961588541666, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.290598154067993, "grad_norm": 0.5891042947769165, "tokens_seen": 18194432, "tokens_per_sec": 17466.105474199394, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4376118977864583, "G_std": 0.18006388346354166, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.074777364730835, "grad_norm": 0.6016299724578857, "tokens_seen": 18358272, "tokens_per_sec": 17468.601196034448, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4405517578125, "G_std": 0.18149820963541666, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.048860311508179, "grad_norm": 0.6008973121643066, "tokens_seen": 18522112, "tokens_per_sec": 17471.05320445661, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4287007649739583, "G_std": 0.18372599283854166, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.150762557983398, "grad_norm": 0.5839955806732178, "tokens_seen": 18685952, "tokens_per_sec": 17473.540572414295, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4318949381510417, "G_std": 0.17954508463541666, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.086086750030518, "grad_norm": 0.5948331952095032, "tokens_seen": 18849792, "tokens_per_sec": 17476.014041992636, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4255777994791667, "G_std": 0.18109130859375, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 4.965067148208618, "grad_norm": 0.6015940308570862, "tokens_seen": 19013632, "tokens_per_sec": 17478.557822474253, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4287109375, "G_std": 0.182281494140625, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.1455583572387695, "grad_norm": 0.5817144513130188, "tokens_seen": 19177472, "tokens_per_sec": 17480.900076656395, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.43035888671875, "G_std": 0.17786661783854166, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.205411672592163, "grad_norm": 0.5976775288581848, "tokens_seen": 19341312, "tokens_per_sec": 17482.293257339155, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4256795247395833, "G_std": 0.182586669921875, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.188444375991821, "grad_norm": 0.6570028066635132, "tokens_seen": 19505152, "tokens_per_sec": 17482.33429872916, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.432769775390625, "G_std": 0.17994181315104166, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.096336841583252, "grad_norm": 0.6272376775741577, "tokens_seen": 19668992, "tokens_per_sec": 17482.662053524364, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.440673828125, "G_std": 0.184417724609375, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.128993511199951, "grad_norm": 0.6233760714530945, "tokens_seen": 19832832, "tokens_per_sec": 17482.64189889359, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.43072509765625, "G_std": 0.17937723795572916, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.116847276687622, "grad_norm": 0.6335731744766235, "tokens_seen": 19996672, "tokens_per_sec": 17481.738826108318, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4240926106770833, "G_std": 0.17785135904947916, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.141054630279541, "grad_norm": 0.6129591464996338, "tokens_seen": 20160512, "tokens_per_sec": 17481.98167775879, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4329020182291667, "G_std": 0.18079121907552084, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.117124795913696, "grad_norm": 0.6569232940673828, "tokens_seen": 20324352, "tokens_per_sec": 17480.668222807428, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4241943359375, "G_std": 0.1824798583984375, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.142443656921387, "grad_norm": 0.640424907207489, "tokens_seen": 20488192, "tokens_per_sec": 17478.605269140066, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42279052734375, "G_std": 0.18093363444010416, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.1718221545219425, "val_ppl": 176.23567371849006, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4216410319010417, "G_std": 0.18098958333333334, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.2243492603302, "grad_norm": 0.6709285974502563, "tokens_seen": 20652032, "tokens_per_sec": 17426.544159703804, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4182942708333333, "G_std": 0.18030293782552084, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.192338705062866, "grad_norm": 0.6673791408538818, "tokens_seen": 20815872, "tokens_per_sec": 17426.141004795896, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4267171223958333, "G_std": 0.18131001790364584, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.158908367156982, "grad_norm": 0.6357905268669128, "tokens_seen": 20979712, "tokens_per_sec": 17427.43338101845, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.424591064453125, "G_std": 0.17816670735677084, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 4.986374855041504, "grad_norm": 0.6055315136909485, "tokens_seen": 21143552, "tokens_per_sec": 17425.58653580559, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.408050537109375, "G_std": 0.17885335286458334, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.067137718200684, "grad_norm": 0.640030562877655, "tokens_seen": 21307392, "tokens_per_sec": 17426.88751565043, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41888427734375, "G_std": 0.1824188232421875, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.987008810043335, "grad_norm": 0.6104980707168579, "tokens_seen": 21471232, "tokens_per_sec": 17427.67655512058, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4148457845052083, "G_std": 0.1824798583984375, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.159045219421387, "grad_norm": 0.6373855471611023, "tokens_seen": 21635072, "tokens_per_sec": 17429.624985644205, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4182332356770833, "G_std": 0.17870585123697916, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.0170347690582275, "grad_norm": 0.6367121934890747, "tokens_seen": 21798912, "tokens_per_sec": 17430.174411051514, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4131876627604167, "G_std": 0.17946370442708334, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.003350257873535, "grad_norm": 0.6184960007667542, "tokens_seen": 21962752, "tokens_per_sec": 17432.29539170626, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.416534423828125, "G_std": 0.17959086100260416, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.160585403442383, "grad_norm": 0.6426164507865906, "tokens_seen": 22126592, "tokens_per_sec": 17434.447666582717, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4107564290364583, "G_std": 0.17940266927083334, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.000765800476074, "grad_norm": 0.6222465634346008, "tokens_seen": 22290432, "tokens_per_sec": 17436.380857999993, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4187723795572917, "G_std": 0.1805572509765625, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.168351411819458, "grad_norm": 0.6968913674354553, "tokens_seen": 22454272, "tokens_per_sec": 17437.729558958596, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4188130696614583, "G_std": 0.18317159016927084, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 4.987234115600586, "grad_norm": 0.7643021941184998, "tokens_seen": 22618112, "tokens_per_sec": 17439.39682069899, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.417083740234375, "G_std": 0.17943318684895834, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.121123552322388, "grad_norm": 0.6847097873687744, "tokens_seen": 22781952, "tokens_per_sec": 17441.478727523558, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.405181884765625, "G_std": 0.17937723795572916, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.877793788909912, "grad_norm": 0.6412307620048523, "tokens_seen": 22945792, "tokens_per_sec": 17443.865832781616, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3957417805989583, "G_std": 0.1760711669921875, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.169278144836426, "grad_norm": 0.6203922629356384, "tokens_seen": 23109632, "tokens_per_sec": 17445.9151161691, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4107462565104167, "G_std": 0.1810302734375, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.053743124008179, "grad_norm": 0.6755492687225342, "tokens_seen": 23273472, "tokens_per_sec": 17447.88029096747, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4073588053385417, "G_std": 0.17876180013020834, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.995816707611084, "grad_norm": 0.6878148317337036, "tokens_seen": 23437312, "tokens_per_sec": 17449.307134400264, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4192403157552083, "G_std": 0.1797637939453125, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.039168834686279, "grad_norm": 0.7063972353935242, "tokens_seen": 23601152, "tokens_per_sec": 17450.807640135034, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4008585611979167, "G_std": 0.1788177490234375, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.028746128082275, "grad_norm": 0.634369969367981, "tokens_seen": 23764992, "tokens_per_sec": 17452.647577290965, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4063720703125, "G_std": 0.17670694986979166, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.986296892166138, "grad_norm": 0.7055433988571167, "tokens_seen": 23928832, "tokens_per_sec": 17454.893199024795, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4036966959635417, "G_std": 0.17479960123697916, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.066208839416504, "grad_norm": 0.6323748230934143, "tokens_seen": 24092672, "tokens_per_sec": 17456.364458095, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40277099609375, "G_std": 0.17532857259114584, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.849884510040283, "grad_norm": 0.6596122980117798, "tokens_seen": 24256512, "tokens_per_sec": 17458.258368530223, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961893717447917, "G_std": 0.178680419921875, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.947715759277344, "grad_norm": 0.6094458699226379, "tokens_seen": 24420352, "tokens_per_sec": 17460.14941563783, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3992106119791667, "G_std": 0.17442830403645834, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.937033653259277, "grad_norm": 0.6470036506652832, "tokens_seen": 24584192, "tokens_per_sec": 17461.748209601035, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4038798014322917, "G_std": 0.17696126302083334, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.028022515773773, "val_ppl": 152.6308888963167, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4158121744791667, "G_std": 0.1757965087890625, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.983754396438599, "grad_norm": 0.6852867603302002, "tokens_seen": 24748032, "tokens_per_sec": 17421.719889879696, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4064229329427083, "G_std": 0.17879740397135416, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.825466632843018, "grad_norm": 0.6265023946762085, "tokens_seen": 24911872, "tokens_per_sec": 17423.342522647454, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3988545735677083, "G_std": 0.17572021484375, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.976581573486328, "grad_norm": 0.6765649318695068, "tokens_seen": 25075712, "tokens_per_sec": 17425.237730721048, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3974812825520833, "G_std": 0.17594401041666666, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.830885648727417, "grad_norm": 0.6691678762435913, "tokens_seen": 25239552, "tokens_per_sec": 17426.886751461538, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3967793782552083, "G_std": 0.1775665283203125, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.987555027008057, "grad_norm": 0.6938115358352661, "tokens_seen": 25403392, "tokens_per_sec": 17428.769562966696, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3948262532552083, "G_std": 0.17676289876302084, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.953492641448975, "grad_norm": 0.6455568075180054, "tokens_seen": 25567232, "tokens_per_sec": 17430.512407198512, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4093221028645833, "G_std": 0.17710367838541666, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.924675941467285, "grad_norm": 0.6485337018966675, "tokens_seen": 25731072, "tokens_per_sec": 17432.347320276196, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4039408365885417, "G_std": 0.175506591796875, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.940512418746948, "grad_norm": 0.6478042006492615, "tokens_seen": 25894912, "tokens_per_sec": 17434.274141291316, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3929341634114583, "G_std": 0.1761932373046875, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.986968517303467, "grad_norm": 0.6973533034324646, "tokens_seen": 26058752, "tokens_per_sec": 17436.083908093067, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3896280924479167, "G_std": 0.1775665283203125, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.767622470855713, "grad_norm": 0.6530627608299255, "tokens_seen": 26222592, "tokens_per_sec": 17437.498931807528, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3953043619791667, "G_std": 0.1768341064453125, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.959375858306885, "grad_norm": 0.7956804633140564, "tokens_seen": 26386432, "tokens_per_sec": 17439.183997273893, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40289306640625, "G_std": 0.1756439208984375, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.003755331039429, "grad_norm": 0.6619188189506531, "tokens_seen": 26550272, "tokens_per_sec": 17441.14117037769, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39788818359375, "G_std": 0.17763264973958334, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.881757736206055, "grad_norm": 0.6954954862594604, "tokens_seen": 26714112, "tokens_per_sec": 17442.910711296474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.396728515625, "G_std": 0.17656453450520834, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.829447269439697, "grad_norm": 0.6937515735626221, "tokens_seen": 26877952, "tokens_per_sec": 17444.78804545654, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3914286295572917, "G_std": 0.1782989501953125, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.009927749633789, "grad_norm": 0.665688157081604, "tokens_seen": 27041792, "tokens_per_sec": 17446.49815934997, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3991800944010417, "G_std": 0.176116943359375, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 5.0286946296691895, "grad_norm": 0.6549315452575684, "tokens_seen": 27205632, "tokens_per_sec": 17448.336185602442, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.397796630859375, "G_std": 0.17588297526041666, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.86311936378479, "grad_norm": 0.6137987375259399, "tokens_seen": 27369472, "tokens_per_sec": 17450.30063483824, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3956197102864583, "G_std": 0.17572530110677084, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.794999122619629, "grad_norm": 0.6511764526367188, "tokens_seen": 27533312, "tokens_per_sec": 17452.27893966214, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.394775390625, "G_std": 0.17478434244791666, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.062919855117798, "grad_norm": 0.6526127457618713, "tokens_seen": 27697152, "tokens_per_sec": 17454.303665149244, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.395782470703125, "G_std": 0.17426554361979166, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 5.141180753707886, "grad_norm": 0.6652522683143616, "tokens_seen": 27860992, "tokens_per_sec": 17456.484730868477, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.395660400390625, "G_std": 0.17673746744791666, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.032778739929199, "grad_norm": 0.6993711590766907, "tokens_seen": 28024832, "tokens_per_sec": 17458.320047199235, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3959554036458333, "G_std": 0.17685953776041666, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.840836048126221, "grad_norm": 0.705437958240509, "tokens_seen": 28188672, "tokens_per_sec": 17460.167366055284, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3919779459635417, "G_std": 0.17601521809895834, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.97053074836731, "grad_norm": 0.726315975189209, "tokens_seen": 28352512, "tokens_per_sec": 17461.94315753759, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3860575358072917, "G_std": 0.173309326171875, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.852496385574341, "grad_norm": 0.7070013880729675, "tokens_seen": 28516352, "tokens_per_sec": 17463.819714917234, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3939615885416667, "G_std": 0.17284647623697916, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.881189823150635, "grad_norm": 0.7020925283432007, "tokens_seen": 28680192, "tokens_per_sec": 17465.59678903286, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3855438232421875, "G_std": 0.17434183756510416, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.98121919631958, "val_ppl": 145.65185164771393, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3837687174479167, "G_std": 0.17426045735677084, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.835445880889893, "grad_norm": 0.6988804340362549, "tokens_seen": 28844032, "tokens_per_sec": 17431.342033896297, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3933207194010417, "G_std": 0.17370096842447916, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.93360710144043, "grad_norm": 0.6939635276794434, "tokens_seen": 29007872, "tokens_per_sec": 17433.206095512716, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3863728841145833, "G_std": 0.17576090494791666, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.923235177993774, "grad_norm": 0.7194262146949768, "tokens_seen": 29171712, "tokens_per_sec": 17434.736979534468, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388702392578125, "G_std": 0.1745758056640625, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.9799277782440186, "grad_norm": 0.7965627312660217, "tokens_seen": 29335552, "tokens_per_sec": 17436.368855174176, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.380462646484375, "G_std": 0.17304484049479166, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 5.0464417934417725, "grad_norm": 0.7147142291069031, "tokens_seen": 29499392, "tokens_per_sec": 17438.019771580854, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3907674153645833, "G_std": 0.17563883463541666, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.892889022827148, "grad_norm": 0.6969085335731506, "tokens_seen": 29663232, "tokens_per_sec": 17439.61051189601, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3847096761067708, "G_std": 0.1717529296875, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.787437438964844, "grad_norm": 0.6941623687744141, "tokens_seen": 29827072, "tokens_per_sec": 17441.20538889553, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3949076334635417, "G_std": 0.17322794596354166, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.73260235786438, "grad_norm": 0.6998682618141174, "tokens_seen": 29990912, "tokens_per_sec": 17442.7371261446, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787282307942708, "G_std": 0.1734619140625, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.729490518569946, "grad_norm": 0.6973463296890259, "tokens_seen": 30154752, "tokens_per_sec": 17444.223123370455, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3822174072265625, "G_std": 0.17489115397135416, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.962517976760864, "grad_norm": 0.7101876139640808, "tokens_seen": 30318592, "tokens_per_sec": 17445.855034662814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38720703125, "G_std": 0.17196146647135416, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.879666566848755, "grad_norm": 0.7106400728225708, "tokens_seen": 30482432, "tokens_per_sec": 17447.578822215597, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.386627197265625, "G_std": 0.17378743489583334, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.8868184089660645, "grad_norm": 0.6816982626914978, "tokens_seen": 30646272, "tokens_per_sec": 17449.175623329415, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38140869140625, "G_std": 0.1734771728515625, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.874136686325073, "grad_norm": 0.7499358057975769, "tokens_seen": 30810112, "tokens_per_sec": 17450.83568722923, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845570882161458, "G_std": 0.17485555013020834, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.764174938201904, "grad_norm": 0.6880481243133545, "tokens_seen": 30973952, "tokens_per_sec": 17452.480436453676, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.385467529296875, "G_std": 0.17283121744791666, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.7743730545043945, "grad_norm": 0.7024186253547668, "tokens_seen": 31137792, "tokens_per_sec": 17454.102709901264, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767293294270833, "G_std": 0.17339579264322916, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.837926626205444, "grad_norm": 0.6745477318763733, "tokens_seen": 31301632, "tokens_per_sec": 17455.3284885484, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3859761555989583, "G_std": 0.17412821451822916, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.910677433013916, "grad_norm": 0.6698147654533386, "tokens_seen": 31465472, "tokens_per_sec": 17456.915306580395, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3854573567708333, "G_std": 0.17290242513020834, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.79474401473999, "grad_norm": 0.7066230773925781, "tokens_seen": 31629312, "tokens_per_sec": 17458.45678359527, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3789316813151042, "G_std": 0.17174275716145834, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.665973901748657, "grad_norm": 0.6713321805000305, "tokens_seen": 31793152, "tokens_per_sec": 17460.204969028087, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3786671956380208, "G_std": 0.17052205403645834, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.7874755859375, "grad_norm": 0.7274401187896729, "tokens_seen": 31956992, "tokens_per_sec": 17461.84527500566, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3799591064453125, "G_std": 0.17266845703125, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.897618293762207, "grad_norm": 0.7084488272666931, "tokens_seen": 32120832, "tokens_per_sec": 17463.438197505573, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781331380208333, "G_std": 0.17212931315104166, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.731750011444092, "grad_norm": 0.6809490323066711, "tokens_seen": 32284672, "tokens_per_sec": 17465.100783224523, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3866933186848958, "G_std": 0.17151896158854166, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.953547239303589, "grad_norm": 0.6888608932495117, "tokens_seen": 32448512, "tokens_per_sec": 17466.56064727634, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3754984537760417, "G_std": 0.170318603515625, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 5.018388986587524, "grad_norm": 0.7213529348373413, "tokens_seen": 32612352, "tokens_per_sec": 17468.038370889746, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3777720133463542, "G_std": 0.172760009765625, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.929891586303711, "grad_norm": 0.7264758348464966, "tokens_seen": 32776192, "tokens_per_sec": 17469.54988900611, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37908935546875, "G_std": 0.17091878255208334, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.852234256267548, "val_ppl": 128.02611368394042, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3852284749348958, "G_std": 0.17249043782552084, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.579616069793701, "grad_norm": 0.7264847159385681, "tokens_seen": 32940032, "tokens_per_sec": 17439.341671630715, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3863983154296875, "G_std": 0.174774169921875, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.826837539672852, "grad_norm": 0.6901677250862122, "tokens_seen": 33103872, "tokens_per_sec": 17441.069329319373, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796335856119792, "G_std": 0.1722259521484375, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.909882307052612, "grad_norm": 0.8198688626289368, "tokens_seen": 33267712, "tokens_per_sec": 17442.79216024142, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.375823974609375, "G_std": 0.1714630126953125, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.689335107803345, "grad_norm": 0.7031031847000122, "tokens_seen": 33431552, "tokens_per_sec": 17444.183850962643, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3745524088541667, "G_std": 0.17354329427083334, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.832671165466309, "grad_norm": 0.7678247094154358, "tokens_seen": 33595392, "tokens_per_sec": 17445.560659340368, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3698679606119792, "G_std": 0.17172749837239584, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.821068286895752, "grad_norm": 0.7760733962059021, "tokens_seen": 33759232, "tokens_per_sec": 17446.892600927007, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3814036051432292, "G_std": 0.1748504638671875, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.89771032333374, "grad_norm": 0.7188676595687866, "tokens_seen": 33923072, "tokens_per_sec": 17448.341137636573, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38043212890625, "G_std": 0.1719512939453125, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.72335147857666, "grad_norm": 0.7158476114273071, "tokens_seen": 34086912, "tokens_per_sec": 17449.635465636355, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803151448567708, "G_std": 0.17322794596354166, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.9574549198150635, "grad_norm": 0.7264237999916077, "tokens_seen": 34250752, "tokens_per_sec": 17450.974734546577, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775583902994792, "G_std": 0.173370361328125, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.6648454666137695, "grad_norm": 0.6771124601364136, "tokens_seen": 34414592, "tokens_per_sec": 17452.33389283476, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763885498046875, "G_std": 0.17238871256510416, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.736378908157349, "grad_norm": 0.6836021542549133, "tokens_seen": 34578432, "tokens_per_sec": 17453.55481480565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3742116292317708, "G_std": 0.17274983723958334, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.695959806442261, "grad_norm": 0.7185946702957153, "tokens_seen": 34742272, "tokens_per_sec": 17454.165111263555, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3782857259114583, "G_std": 0.1710357666015625, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.734888076782227, "grad_norm": 0.6968331933021545, "tokens_seen": 34906112, "tokens_per_sec": 17455.35519556956, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737030029296875, "G_std": 0.17169189453125, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.869872331619263, "grad_norm": 0.7436686158180237, "tokens_seen": 35069952, "tokens_per_sec": 17456.614361704593, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787027994791667, "G_std": 0.17039998372395834, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.917787551879883, "grad_norm": 0.7103937864303589, "tokens_seen": 35233792, "tokens_per_sec": 17458.050724423854, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3741607666015625, "G_std": 0.1699676513671875, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.708852291107178, "grad_norm": 0.7079433798789978, "tokens_seen": 35397632, "tokens_per_sec": 17459.597624277914, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753458658854167, "G_std": 0.1696039835611979, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.771491050720215, "grad_norm": 0.7413904666900635, "tokens_seen": 35561472, "tokens_per_sec": 17460.82173774382, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725229899088542, "G_std": 0.1700261433919271, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.827725410461426, "grad_norm": 0.7537403702735901, "tokens_seen": 35725312, "tokens_per_sec": 17462.27879068756, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810170491536458, "G_std": 0.16976420084635416, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.696723699569702, "grad_norm": 0.729514479637146, "tokens_seen": 35889152, "tokens_per_sec": 17463.780445800214, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3679911295572917, "G_std": 0.17125701904296875, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.813372611999512, "grad_norm": 0.705788791179657, "tokens_seen": 36052992, "tokens_per_sec": 17465.159504834588, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3666585286458333, "G_std": 0.17205301920572916, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.791592121124268, "grad_norm": 0.7435498833656311, "tokens_seen": 36216832, "tokens_per_sec": 17466.600938589912, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729044596354167, "G_std": 0.17136637369791666, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.703552961349487, "grad_norm": 0.7602213621139526, "tokens_seen": 36380672, "tokens_per_sec": 17467.95601876842, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37213134765625, "G_std": 0.16836039225260416, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.528384208679199, "grad_norm": 0.7112488746643066, "tokens_seen": 36544512, "tokens_per_sec": 17469.366891644182, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770345052083333, "G_std": 0.16936238606770834, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.671140193939209, "grad_norm": 0.7108644247055054, "tokens_seen": 36708352, "tokens_per_sec": 17470.69681503213, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3812459309895833, "G_std": 0.1707636515299479, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.937722206115723, "grad_norm": 0.726813018321991, "tokens_seen": 36872192, "tokens_per_sec": 17471.884877825873, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671213785807292, "G_std": 0.1687596638997396, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.756679737567902, "val_ppl": 116.35894158737695, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3696950276692708, "G_std": 0.17037455240885416, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.814798831939697, "grad_norm": 0.7180641889572144, "tokens_seen": 37036032, "tokens_per_sec": 17444.752305624494, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3761698404947917, "G_std": 0.1690546671549479, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.780610799789429, "grad_norm": 0.7244895696640015, "tokens_seen": 37199872, "tokens_per_sec": 17446.14877693524, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372344970703125, "G_std": 0.168731689453125, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.656058311462402, "grad_norm": 0.695054292678833, "tokens_seen": 37363712, "tokens_per_sec": 17447.513594978656, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3755340576171875, "G_std": 0.1695988972981771, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.836179733276367, "grad_norm": 0.753275990486145, "tokens_seen": 37527552, "tokens_per_sec": 17448.859598026378, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3760223388671875, "G_std": 0.17156473795572916, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.67812442779541, "grad_norm": 0.7626591324806213, "tokens_seen": 37691392, "tokens_per_sec": 17450.129954740667, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3758087158203125, "G_std": 0.17313385009765625, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.664065361022949, "grad_norm": 0.7265082597732544, "tokens_seen": 37855232, "tokens_per_sec": 17451.195062631265, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3749593098958333, "G_std": 0.1716130574544271, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.824448108673096, "grad_norm": 0.7187249660491943, "tokens_seen": 38019072, "tokens_per_sec": 17452.277339899694, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372528076171875, "G_std": 0.16985575358072916, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.703272104263306, "grad_norm": 0.7370670437812805, "tokens_seen": 38182912, "tokens_per_sec": 17453.468787117086, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3779500325520833, "G_std": 0.17052714029947916, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.7977190017700195, "grad_norm": 0.7348777055740356, "tokens_seen": 38346752, "tokens_per_sec": 17454.662179698964, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3743133544921875, "G_std": 0.1663970947265625, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.748681306838989, "grad_norm": 0.7466457486152649, "tokens_seen": 38510592, "tokens_per_sec": 17455.763264868354, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710988362630208, "G_std": 0.1672643025716146, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.793177843093872, "grad_norm": 0.7265517711639404, "tokens_seen": 38674432, "tokens_per_sec": 17456.922564528595, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3699188232421875, "G_std": 0.1690241495768229, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.723906517028809, "grad_norm": 0.7344738841056824, "tokens_seen": 38838272, "tokens_per_sec": 17457.987449560704, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3738861083984375, "G_std": 0.168975830078125, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.81748628616333, "grad_norm": 0.7027660608291626, "tokens_seen": 39002112, "tokens_per_sec": 17459.09271786852, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3666127522786458, "G_std": 0.1684697469075521, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.612379312515259, "grad_norm": 0.7287847995758057, "tokens_seen": 39165952, "tokens_per_sec": 17460.196237284348, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3711954752604167, "G_std": 0.1696319580078125, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.589370250701904, "grad_norm": 0.7021083831787109, "tokens_seen": 39329792, "tokens_per_sec": 17461.355888124755, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662261962890625, "G_std": 0.16924031575520834, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.764439582824707, "grad_norm": 0.7072136402130127, "tokens_seen": 39493632, "tokens_per_sec": 17462.39776176892, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3689422607421875, "G_std": 0.1673558553059896, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.7410125732421875, "grad_norm": 0.7475316524505615, "tokens_seen": 39657472, "tokens_per_sec": 17463.55889021244, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36505126953125, "G_std": 0.1678899129231771, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.837521076202393, "grad_norm": 0.6779477000236511, "tokens_seen": 39821312, "tokens_per_sec": 17464.724977747443, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3694712320963542, "G_std": 0.169708251953125, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.602471590042114, "grad_norm": 0.7329753637313843, "tokens_seen": 39985152, "tokens_per_sec": 17465.90662091382, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762054443359375, "G_std": 0.16823323567708334, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.725026607513428, "grad_norm": 0.7027779817581177, "tokens_seen": 40148992, "tokens_per_sec": 17467.16578740286, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3689676920572917, "G_std": 0.16917928059895834, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.752464771270752, "grad_norm": 0.7271302342414856, "tokens_seen": 40312832, "tokens_per_sec": 17468.39051500686, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3674570719401042, "G_std": 0.16698455810546875, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.508648872375488, "grad_norm": 0.7633376717567444, "tokens_seen": 40476672, "tokens_per_sec": 17469.695980097233, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3556365966796875, "G_std": 0.1667302449544271, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.721800327301025, "grad_norm": 0.7593448162078857, "tokens_seen": 40640512, "tokens_per_sec": 17470.91947303326, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737131754557292, "G_std": 0.16644287109375, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.835183620452881, "grad_norm": 0.7547889351844788, "tokens_seen": 40804352, "tokens_per_sec": 17472.110052188007, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632100423177083, "G_std": 0.16871134440104166, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.708332538604736, "grad_norm": 0.7717562913894653, "tokens_seen": 40968192, "tokens_per_sec": 17473.261393598517, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3760325113932292, "G_std": 0.16738128662109375, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.735227823257446, "val_ppl": 113.88940241745621, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.36016845703125, "G_std": 0.1679255167643229, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.660971641540527, "grad_norm": 0.7418161630630493, "tokens_seen": 41132032, "tokens_per_sec": 17448.946697751035, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3663889567057292, "G_std": 0.1690241495768229, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.612402439117432, "grad_norm": 0.7629133462905884, "tokens_seen": 41295872, "tokens_per_sec": 17449.422251307966, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3631846110026042, "G_std": 0.16791534423828125, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.748296737670898, "grad_norm": 0.7134037017822266, "tokens_seen": 41459712, "tokens_per_sec": 17450.542365748333, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3625996907552083, "G_std": 0.1691462198893229, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.755114555358887, "grad_norm": 0.7834659218788147, "tokens_seen": 41623552, "tokens_per_sec": 17451.636877255776, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662821451822917, "G_std": 0.16763814290364584, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.62279748916626, "grad_norm": 0.726311445236206, "tokens_seen": 41787392, "tokens_per_sec": 17452.79025900044, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3630472819010417, "G_std": 0.16879781087239584, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.728989839553833, "grad_norm": 0.769493043422699, "tokens_seen": 41951232, "tokens_per_sec": 17453.785554379152, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3684590657552083, "G_std": 0.16996256510416666, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.706695079803467, "grad_norm": 0.7398648858070374, "tokens_seen": 42115072, "tokens_per_sec": 17454.666622723995, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3562418619791667, "G_std": 0.1675287882486979, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.741857528686523, "grad_norm": 0.7889065146446228, "tokens_seen": 42278912, "tokens_per_sec": 17455.52867525852, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372283935546875, "G_std": 0.1697107950846354, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.776384592056274, "grad_norm": 0.7915240526199341, "tokens_seen": 42442752, "tokens_per_sec": 17456.492291297676, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670450846354167, "G_std": 0.16501617431640625, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.733515501022339, "grad_norm": 0.7760242223739624, "tokens_seen": 42606592, "tokens_per_sec": 17457.48513624504, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367523193359375, "G_std": 0.1686528523763021, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.549176454544067, "grad_norm": 0.7346403002738953, "tokens_seen": 42770432, "tokens_per_sec": 17458.459206235573, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.361053466796875, "G_std": 0.16861979166666666, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.883486747741699, "grad_norm": 0.8084875345230103, "tokens_seen": 42934272, "tokens_per_sec": 17459.32796786495, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613026936848958, "G_std": 0.16667683919270834, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.635869979858398, "grad_norm": 0.7467740774154663, "tokens_seen": 43098112, "tokens_per_sec": 17460.46658507151, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3608042399088542, "G_std": 0.1683171590169271, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.518739461898804, "grad_norm": 0.721835196018219, "tokens_seen": 43261952, "tokens_per_sec": 17461.65993008616, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3667551676432292, "G_std": 0.1668116251627604, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.655506610870361, "grad_norm": 0.7405825853347778, "tokens_seen": 43425792, "tokens_per_sec": 17462.66486439135, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676961263020833, "G_std": 0.1658935546875, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.63976788520813, "grad_norm": 0.7931942939758301, "tokens_seen": 43589632, "tokens_per_sec": 17463.219830657024, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649851481119792, "G_std": 0.16808573404947916, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.7309253215789795, "grad_norm": 0.7482976317405701, "tokens_seen": 43753472, "tokens_per_sec": 17464.25247886583, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657023111979167, "G_std": 0.16808573404947916, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.485138654708862, "grad_norm": 0.7556707262992859, "tokens_seen": 43917312, "tokens_per_sec": 17465.217502258856, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590443929036458, "G_std": 0.1657842000325521, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.539077520370483, "grad_norm": 0.8186208605766296, "tokens_seen": 44081152, "tokens_per_sec": 17466.421668428633, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3706258138020833, "G_std": 0.1667760213216146, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.554165601730347, "grad_norm": 0.7465534210205078, "tokens_seen": 44244992, "tokens_per_sec": 17467.557271421447, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594411214192708, "G_std": 0.16700236002604166, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.630130052566528, "grad_norm": 0.6994841694831848, "tokens_seen": 44408832, "tokens_per_sec": 17468.70881255585, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3555755615234375, "G_std": 0.16645304361979166, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.757309675216675, "grad_norm": 0.7838468551635742, "tokens_seen": 44572672, "tokens_per_sec": 17469.715863681104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597005208333333, "G_std": 0.16759490966796875, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.661609172821045, "grad_norm": 0.757672905921936, "tokens_seen": 44736512, "tokens_per_sec": 17470.96155394463, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613026936848958, "G_std": 0.1667963663736979, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.665363550186157, "grad_norm": 0.7842923402786255, "tokens_seen": 44900352, "tokens_per_sec": 17472.126599601583, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580271402994792, "G_std": 0.1685333251953125, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.6484997272491455, "grad_norm": 0.8094078898429871, "tokens_seen": 45064192, "tokens_per_sec": 17473.15960801912, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36041259765625, "G_std": 0.1668116251627604, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.6725511193275455, "val_ppl": 106.9702886064662, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3586629231770833, "G_std": 0.16786702473958334, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.488605737686157, "grad_norm": 0.7595173716545105, "tokens_seen": 45228032, "tokens_per_sec": 17451.306241806484, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3600362141927083, "G_std": 0.16535441080729166, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.631130933761597, "grad_norm": 0.8148356676101685, "tokens_seen": 45391872, "tokens_per_sec": 17452.338019074174, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36376953125, "G_std": 0.16732025146484375, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.631294250488281, "grad_norm": 0.7361170649528503, "tokens_seen": 45555712, "tokens_per_sec": 17453.27626474317, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3683878580729167, "G_std": 0.1690495808919271, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.491864204406738, "grad_norm": 0.7467951774597168, "tokens_seen": 45719552, "tokens_per_sec": 17454.473241761712, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597768147786458, "G_std": 0.1661097208658854, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.6803576946258545, "grad_norm": 0.7850157022476196, "tokens_seen": 45883392, "tokens_per_sec": 17455.502597744857, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.361114501953125, "G_std": 0.16634623209635416, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.692323684692383, "grad_norm": 0.7976621389389038, "tokens_seen": 46047232, "tokens_per_sec": 17456.49604547536, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3608500162760417, "G_std": 0.16770172119140625, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.6291022300720215, "grad_norm": 0.7453889846801758, "tokens_seen": 46211072, "tokens_per_sec": 17457.491941432436, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3607381184895833, "G_std": 0.16907246907552084, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.741025924682617, "grad_norm": 0.7772088050842285, "tokens_seen": 46374912, "tokens_per_sec": 17458.403653748548, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626505533854167, "G_std": 0.1674321492513021, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.780766010284424, "grad_norm": 0.8103025555610657, "tokens_seen": 46538752, "tokens_per_sec": 17459.255366131565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603922526041667, "G_std": 0.1681035359700521, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.600702285766602, "grad_norm": 0.7652459740638733, "tokens_seen": 46702592, "tokens_per_sec": 17460.08698833302, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3546142578125, "G_std": 0.16835530598958334, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.4941346645355225, "grad_norm": 0.7771524786949158, "tokens_seen": 46866432, "tokens_per_sec": 17460.999178739403, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3608500162760417, "G_std": 0.1649144490559896, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.620588302612305, "grad_norm": 0.7455837726593018, "tokens_seen": 47030272, "tokens_per_sec": 17462.007257796628, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3602701822916667, "G_std": 0.16649627685546875, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.672779083251953, "grad_norm": 0.7686507701873779, "tokens_seen": 47194112, "tokens_per_sec": 17463.072603611567, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3581136067708333, "G_std": 0.1684850056966146, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.60406231880188, "grad_norm": 0.7894642353057861, "tokens_seen": 47357952, "tokens_per_sec": 17464.05108532441, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3518727620442708, "G_std": 0.16729227701822916, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.586153984069824, "grad_norm": 0.7662554383277893, "tokens_seen": 47521792, "tokens_per_sec": 17465.002440518565, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3592529296875, "G_std": 0.16925048828125, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.581692457199097, "grad_norm": 0.7429575324058533, "tokens_seen": 47685632, "tokens_per_sec": 17465.91563759093, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.355010986328125, "G_std": 0.16617329915364584, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.549290657043457, "grad_norm": 0.7771679759025574, "tokens_seen": 47849472, "tokens_per_sec": 17466.850702812575, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3570709228515625, "G_std": 0.16728719075520834, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.566472053527832, "grad_norm": 0.7805683612823486, "tokens_seen": 48013312, "tokens_per_sec": 17467.772587081836, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35968017578125, "G_std": 0.1662877400716146, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.529274225234985, "grad_norm": 0.7685331106185913, "tokens_seen": 48177152, "tokens_per_sec": 17468.815460375936, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.354766845703125, "G_std": 0.1670074462890625, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.510153770446777, "grad_norm": 0.7472619414329529, "tokens_seen": 48340992, "tokens_per_sec": 17469.892619175804, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3643239339192708, "G_std": 0.1651891072591146, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.619209289550781, "grad_norm": 0.8082845211029053, "tokens_seen": 48504832, "tokens_per_sec": 17470.928180443076, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3586680094401042, "G_std": 0.1649449666341146, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.5841383934021, "grad_norm": 0.8086077570915222, "tokens_seen": 48668672, "tokens_per_sec": 17471.87610827229, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606923421223958, "G_std": 0.16754659016927084, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.743818521499634, "grad_norm": 0.7718925476074219, "tokens_seen": 48832512, "tokens_per_sec": 17472.91012083111, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3596750895182292, "G_std": 0.1673558553059896, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.52582573890686, "grad_norm": 0.7460461854934692, "tokens_seen": 48996352, "tokens_per_sec": 17473.97179630621, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3533681233723958, "G_std": 0.1672185262044271, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.559886932373047, "grad_norm": 0.8004333972930908, "tokens_seen": 49160192, "tokens_per_sec": 17474.971446700667, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3537038167317708, "G_std": 0.1646092732747396, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.690562057495117, "val_ppl": 108.91437875004515, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.362396240234375, "G_std": 0.1663665771484375, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.64157772064209, "grad_norm": 0.7885446548461914, "tokens_seen": 49324032, "tokens_per_sec": 17454.783052006413, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3661041259765625, "G_std": 0.16628265380859375, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.521833896636963, "grad_norm": 0.7832602858543396, "tokens_seen": 49487872, "tokens_per_sec": 17455.813050472843, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3648732503255208, "G_std": 0.16689554850260416, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.43265700340271, "grad_norm": 0.730434775352478, "tokens_seen": 49651712, "tokens_per_sec": 17456.697778182795, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3562571207682292, "G_std": 0.16529083251953125, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.443321466445923, "grad_norm": 0.7841930389404297, "tokens_seen": 49815552, "tokens_per_sec": 17457.791206967115, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3553212483723958, "G_std": 0.16684977213541666, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.460804462432861, "grad_norm": 0.7769560217857361, "tokens_seen": 49979392, "tokens_per_sec": 17458.77183790437, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3536580403645833, "G_std": 0.1680170694986979, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.570755958557129, "grad_norm": 0.7441126108169556, "tokens_seen": 50143232, "tokens_per_sec": 17459.73587128654, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35491943359375, "G_std": 0.16526031494140625, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.486354827880859, "grad_norm": 0.7836323976516724, "tokens_seen": 50307072, "tokens_per_sec": 17460.542219738076, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.351837158203125, "G_std": 0.16675567626953125, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.4913649559021, "grad_norm": 0.7851389646530151, "tokens_seen": 50470912, "tokens_per_sec": 17461.332912719226, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3553568522135417, "G_std": 0.16607666015625, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.61723780632019, "grad_norm": 0.8151231408119202, "tokens_seen": 50634752, "tokens_per_sec": 17462.19785627055, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35418701171875, "G_std": 0.16558329264322916, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.54160737991333, "grad_norm": 0.7793050408363342, "tokens_seen": 50798592, "tokens_per_sec": 17463.046102572986, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566029866536458, "G_std": 0.1680170694986979, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.581684350967407, "grad_norm": 0.8172875642776489, "tokens_seen": 50962432, "tokens_per_sec": 17463.726470682206, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3485972086588542, "G_std": 0.16522979736328125, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.5330305099487305, "grad_norm": 0.8064216375350952, "tokens_seen": 51126272, "tokens_per_sec": 17464.40592863555, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582051595052083, "G_std": 0.16544596354166666, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.551082611083984, "grad_norm": 0.8050402402877808, "tokens_seen": 51290112, "tokens_per_sec": 17464.74387673175, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3601837158203125, "G_std": 0.1635284423828125, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.61382532119751, "grad_norm": 0.7735973000526428, "tokens_seen": 51453952, "tokens_per_sec": 17465.008164028153, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35284423828125, "G_std": 0.16407012939453125, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.447723865509033, "grad_norm": 0.8045572638511658, "tokens_seen": 51617792, "tokens_per_sec": 17465.3115562071, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3530222574869792, "G_std": 0.1655120849609375, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.639336585998535, "grad_norm": 0.7847527861595154, "tokens_seen": 51781632, "tokens_per_sec": 17465.76859939438, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35357666015625, "G_std": 0.16577657063802084, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.476409435272217, "grad_norm": 0.7860195636749268, "tokens_seen": 51945472, "tokens_per_sec": 17466.469189829102, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3550313313802083, "G_std": 0.16440836588541666, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.538524627685547, "grad_norm": 0.8011198043823242, "tokens_seen": 52109312, "tokens_per_sec": 17467.01714695702, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3601735432942708, "G_std": 0.16508992513020834, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.589054584503174, "grad_norm": 0.7925305962562561, "tokens_seen": 52273152, "tokens_per_sec": 17467.51123425181, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3587544759114583, "G_std": 0.1644312540690104, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.524555921554565, "grad_norm": 0.8448556661605835, "tokens_seen": 52436992, "tokens_per_sec": 17468.11106180718, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36456298828125, "G_std": 0.1647516886393229, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.584098815917969, "grad_norm": 0.8233609199523926, "tokens_seen": 52600832, "tokens_per_sec": 17468.428958131335, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3588612874348958, "G_std": 0.1647822062174479, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.553751707077026, "grad_norm": 0.7925133109092712, "tokens_seen": 52764672, "tokens_per_sec": 17468.92659015801, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3623758951822917, "G_std": 0.16512298583984375, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.618341684341431, "grad_norm": 0.8001401424407959, "tokens_seen": 52928512, "tokens_per_sec": 17469.400371651358, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3533681233723958, "G_std": 0.1632664998372396, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.411961555480957, "grad_norm": 0.7808105945587158, "tokens_seen": 53092352, "tokens_per_sec": 17469.791653275708, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3542887369791667, "G_std": 0.1644770304361979, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.503306865692139, "grad_norm": 0.7896780967712402, "tokens_seen": 53256192, "tokens_per_sec": 17470.080477300948, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3512420654296875, "G_std": 0.1652654012044271, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.631434428691864, "val_ppl": 102.66121874155365, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.35064697265625, "G_std": 0.16472880045572916, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.491121768951416, "grad_norm": 0.7660262584686279, "tokens_seen": 53420032, "tokens_per_sec": 17450.80466103181, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3529052734375, "G_std": 0.16448974609375, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.620938777923584, "grad_norm": 0.808737576007843, "tokens_seen": 53583872, "tokens_per_sec": 17451.30043723016, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3534444173177083, "G_std": 0.16473388671875, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.579558372497559, "grad_norm": 0.8040254712104797, "tokens_seen": 53747712, "tokens_per_sec": 17451.795846502235, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35638427734375, "G_std": 0.16515604654947916, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.5347819328308105, "grad_norm": 0.7730528116226196, "tokens_seen": 53911552, "tokens_per_sec": 17452.191982402222, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.355010986328125, "G_std": 0.16360982259114584, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.32498025894165, "grad_norm": 0.7778370976448059, "tokens_seen": 54075392, "tokens_per_sec": 17452.6442576099, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34930419921875, "G_std": 0.1639378865559896, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.755758047103882, "grad_norm": 0.7988825440406799, "tokens_seen": 54239232, "tokens_per_sec": 17453.177470984105, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3445994059244792, "G_std": 0.16396077473958334, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.5436320304870605, "grad_norm": 0.8193172216415405, "tokens_seen": 54403072, "tokens_per_sec": 17453.446301659555, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3557027180989583, "G_std": 0.16365814208984375, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.506994724273682, "grad_norm": 0.8324943780899048, "tokens_seen": 54566912, "tokens_per_sec": 17453.746264152815, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3510894775390625, "G_std": 0.16376241048177084, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.487002372741699, "grad_norm": 0.7749890089035034, "tokens_seen": 54730752, "tokens_per_sec": 17454.055601796474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3523661295572917, "G_std": 0.16439056396484375, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.507817983627319, "grad_norm": 0.7751393914222717, "tokens_seen": 54894592, "tokens_per_sec": 17454.42592590199, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3486378987630208, "G_std": 0.16241455078125, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.5127623081207275, "grad_norm": 0.8294209241867065, "tokens_seen": 55058432, "tokens_per_sec": 17454.706369054307, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3490498860677083, "G_std": 0.1643091837565104, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.5102832317352295, "grad_norm": 0.800025999546051, "tokens_seen": 55222272, "tokens_per_sec": 17455.04248984985, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3537546793619792, "G_std": 0.1626103719075521, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.56234884262085, "grad_norm": 0.8089288473129272, "tokens_seen": 55386112, "tokens_per_sec": 17455.19197302508, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597819010416667, "G_std": 0.1661809285481771, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.550036430358887, "grad_norm": 0.8087611198425293, "tokens_seen": 55549952, "tokens_per_sec": 17455.599394571505, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3502858479817708, "G_std": 0.1640192667643229, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.506173610687256, "grad_norm": 0.8913390040397644, "tokens_seen": 55713792, "tokens_per_sec": 17455.97079901346, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3533884684244792, "G_std": 0.1658935546875, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.657083034515381, "grad_norm": 0.8718472719192505, "tokens_seen": 55877632, "tokens_per_sec": 17456.261677415474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3542226155598958, "G_std": 0.16533406575520834, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.572826385498047, "grad_norm": 0.8302167057991028, "tokens_seen": 56041472, "tokens_per_sec": 17456.570753112726, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3551228841145833, "G_std": 0.1636988321940104, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.470807313919067, "grad_norm": 0.8221389651298523, "tokens_seen": 56205312, "tokens_per_sec": 17456.818681281737, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.346466064453125, "G_std": 0.1628087361653646, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.51199197769165, "grad_norm": 0.8163458108901978, "tokens_seen": 56369152, "tokens_per_sec": 17457.17381111776, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3535868326822917, "G_std": 0.16457621256510416, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.498992919921875, "grad_norm": 0.8232035040855408, "tokens_seen": 56532992, "tokens_per_sec": 17457.52179936513, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566538492838542, "G_std": 0.1636505126953125, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.4122302532196045, "grad_norm": 0.8090710043907166, "tokens_seen": 56696832, "tokens_per_sec": 17458.023699655412, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3524932861328125, "G_std": 0.16511027018229166, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.431572914123535, "grad_norm": 0.7863472700119019, "tokens_seen": 56860672, "tokens_per_sec": 17458.433019786997, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3551025390625, "G_std": 0.1637700398763021, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.4941418170928955, "grad_norm": 0.8291234374046326, "tokens_seen": 57024512, "tokens_per_sec": 17458.810794886045, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3448232014973958, "G_std": 0.1627985636393229, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.357986927032471, "grad_norm": 0.8038017749786377, "tokens_seen": 57188352, "tokens_per_sec": 17459.255356947928, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3525950113932292, "G_std": 0.1646143595377604, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.37216854095459, "grad_norm": 0.8010163307189941, "tokens_seen": 57352192, "tokens_per_sec": 17459.677111492925, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3452301025390625, "G_std": 0.1624120076497396, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.5722506761550905, "val_ppl": 96.76164400635356, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3495890299479167, "G_std": 0.16264851888020834, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.521562576293945, "grad_norm": 0.8437648415565491, "tokens_seen": 57516032, "tokens_per_sec": 17441.869516635114, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3577372233072917, "G_std": 0.1639582316080729, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.614372968673706, "grad_norm": 0.8085127472877502, "tokens_seen": 57679872, "tokens_per_sec": 17442.355845390615, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582814534505208, "G_std": 0.1648712158203125, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.511358261108398, "grad_norm": 0.8002929091453552, "tokens_seen": 57843712, "tokens_per_sec": 17442.811995948225, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3539683024088542, "G_std": 0.1629486083984375, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.41559362411499, "grad_norm": 0.787678599357605, "tokens_seen": 58007552, "tokens_per_sec": 17443.281825936218, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3539021809895833, "G_std": 0.1636937459309896, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.566171646118164, "grad_norm": 0.846364438533783, "tokens_seen": 58171392, "tokens_per_sec": 17443.731735598627, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35186767578125, "G_std": 0.1641693115234375, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.463183403015137, "grad_norm": 0.8098052740097046, "tokens_seen": 58335232, "tokens_per_sec": 17444.27783295055, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3457234700520833, "G_std": 0.1619695027669271, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.4194066524505615, "grad_norm": 0.8507726192474365, "tokens_seen": 58499072, "tokens_per_sec": 17444.703953903343, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3402506510416667, "G_std": 0.16149139404296875, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.3620076179504395, "grad_norm": 0.8245987296104431, "tokens_seen": 58662912, "tokens_per_sec": 17445.05204376386, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3391316731770833, "G_std": 0.1616185506184896, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.574235200881958, "grad_norm": 0.824911892414093, "tokens_seen": 58826752, "tokens_per_sec": 17445.396984853392, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.349273681640625, "G_std": 0.16501617431640625, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.474890470504761, "grad_norm": 0.8281317949295044, "tokens_seen": 58990592, "tokens_per_sec": 17445.6873856365, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3492482503255208, "G_std": 0.16463216145833334, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.556081056594849, "grad_norm": 0.8592404127120972, "tokens_seen": 59154432, "tokens_per_sec": 17445.99091285046, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3484649658203125, "G_std": 0.16345723470052084, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.464112043380737, "grad_norm": 0.8263806104660034, "tokens_seen": 59318272, "tokens_per_sec": 17446.29413910599, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3487955729166667, "G_std": 0.16412353515625, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.518261909484863, "grad_norm": 0.8462309241294861, "tokens_seen": 59482112, "tokens_per_sec": 17446.611459805907, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3523712158203125, "G_std": 0.1634241739908854, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.597245693206787, "grad_norm": 0.820014238357544, "tokens_seen": 59645952, "tokens_per_sec": 17446.91458685339, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3527984619140625, "G_std": 0.1607233683268229, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.42025351524353, "grad_norm": 0.8194311857223511, "tokens_seen": 59809792, "tokens_per_sec": 17447.253896474398, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34539794921875, "G_std": 0.1625035603841146, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.478841304779053, "grad_norm": 0.7872413396835327, "tokens_seen": 59973632, "tokens_per_sec": 17447.53101734829, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35272216796875, "G_std": 0.1625493367513021, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.501894235610962, "grad_norm": 0.8516227006912231, "tokens_seen": 60137472, "tokens_per_sec": 17447.93806252537, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3497823079427083, "G_std": 0.16363016764322916, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.507527828216553, "grad_norm": 0.8285000324249268, "tokens_seen": 60301312, "tokens_per_sec": 17448.283787422355, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3462575276692708, "G_std": 0.16431427001953125, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.354349136352539, "grad_norm": 0.8383818864822388, "tokens_seen": 60465152, "tokens_per_sec": 17448.576031523025, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3486124674479167, "G_std": 0.1617406209309896, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.37073016166687, "grad_norm": 0.8605540990829468, "tokens_seen": 60628992, "tokens_per_sec": 17449.064902739443, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3422088623046875, "G_std": 0.163909912109375, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.544785022735596, "grad_norm": 0.8192263245582581, "tokens_seen": 60792832, "tokens_per_sec": 17449.434329529202, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3496500651041667, "G_std": 0.1648279825846354, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.570145130157471, "grad_norm": 0.8401603698730469, "tokens_seen": 60956672, "tokens_per_sec": 17449.824108266544, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3588765462239583, "G_std": 0.1647516886393229, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.434834003448486, "grad_norm": 0.8311923742294312, "tokens_seen": 61120512, "tokens_per_sec": 17450.315910749272, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34765625, "G_std": 0.1631596883138021, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.674493312835693, "grad_norm": 0.8678951859474182, "tokens_seen": 61284352, "tokens_per_sec": 17450.763506863907, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3489278157552083, "G_std": 0.1633478800455729, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.310656309127808, "grad_norm": 0.8270685076713562, "tokens_seen": 61448192, "tokens_per_sec": 17451.194772024566, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3455912272135417, "G_std": 0.1623560587565104, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.55007541179657, "val_ppl": 94.63954498393997, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3484446207682292, "G_std": 0.1627222696940104, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.542169809341431, "grad_norm": 0.8542649745941162, "tokens_seen": 61612032, "tokens_per_sec": 17434.653203113536, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3443654378255208, "G_std": 0.1631952921549479, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.394088983535767, "grad_norm": 0.8298646807670593, "tokens_seen": 61775872, "tokens_per_sec": 17435.16273706814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3415374755859375, "G_std": 0.1631596883138021, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.423027038574219, "grad_norm": 0.8590152859687805, "tokens_seen": 61939712, "tokens_per_sec": 17435.586009757182, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3556772867838542, "G_std": 0.16425323486328125, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.373261213302612, "grad_norm": 0.7999911308288574, "tokens_seen": 62103552, "tokens_per_sec": 17436.000212639254, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.351470947265625, "G_std": 0.1631495157877604, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.406196117401123, "grad_norm": 0.8183586001396179, "tokens_seen": 62267392, "tokens_per_sec": 17436.485360510695, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3468373616536458, "G_std": 0.1623713175455729, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.383992910385132, "grad_norm": 0.8440758585929871, "tokens_seen": 62431232, "tokens_per_sec": 17436.796316536198, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3490397135416667, "G_std": 0.1629816691080729, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.41497278213501, "grad_norm": 0.8479061722755432, "tokens_seen": 62595072, "tokens_per_sec": 17437.069140694377, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.353485107421875, "G_std": 0.16404215494791666, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.501626253128052, "grad_norm": 0.8810231685638428, "tokens_seen": 62758912, "tokens_per_sec": 17437.236481694006, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3591054280598958, "G_std": 0.162109375, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.373688220977783, "grad_norm": 0.8821227550506592, "tokens_seen": 62922752, "tokens_per_sec": 17437.266929826765, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35015869140625, "G_std": 0.16285196940104166, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.4779181480407715, "grad_norm": 0.8757103085517883, "tokens_seen": 63086592, "tokens_per_sec": 17437.584824901118, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3482462565104167, "G_std": 0.1634368896484375, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.534169912338257, "grad_norm": 0.8883322477340698, "tokens_seen": 63250432, "tokens_per_sec": 17437.90399135988, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3534393310546875, "G_std": 0.16307576497395834, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.352992296218872, "grad_norm": 0.8441953063011169, "tokens_seen": 63414272, "tokens_per_sec": 17438.31223636777, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.33978271484375, "G_std": 0.16144816080729166, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.571167469024658, "grad_norm": 0.8396545648574829, "tokens_seen": 63578112, "tokens_per_sec": 17438.73339040277, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3506266276041667, "G_std": 0.16191609700520834, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.430476188659668, "grad_norm": 0.8242312073707581, "tokens_seen": 63741952, "tokens_per_sec": 17439.023717814754, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3449910481770833, "G_std": 0.1617101033528646, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.487955093383789, "grad_norm": 0.8878821730613708, "tokens_seen": 63905792, "tokens_per_sec": 17439.39067521066, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3510945638020833, "G_std": 0.1635309855143229, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.3597493171691895, "grad_norm": 0.8559828400611877, "tokens_seen": 64069632, "tokens_per_sec": 17439.79044270015, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3475240071614583, "G_std": 0.16199747721354166, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.48970103263855, "grad_norm": 0.8629156351089478, "tokens_seen": 64233472, "tokens_per_sec": 17440.041860379806, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3510487874348958, "G_std": 0.1629002888997396, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.441333055496216, "grad_norm": 0.8891997337341309, "tokens_seen": 64397312, "tokens_per_sec": 17440.435360602834, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.347442626953125, "G_std": 0.1624170939127604, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.407573223114014, "grad_norm": 0.8696302175521851, "tokens_seen": 64561152, "tokens_per_sec": 17440.760884271607, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3477986653645833, "G_std": 0.1629308064778646, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.572624921798706, "grad_norm": 0.8858615756034851, "tokens_seen": 64724992, "tokens_per_sec": 17441.216434947295, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3448435465494792, "G_std": 0.16167704264322916, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.423422336578369, "grad_norm": 0.8871817588806152, "tokens_seen": 64888832, "tokens_per_sec": 17441.740716621985, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.342498779296875, "G_std": 0.16219584147135416, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.479530334472656, "grad_norm": 0.8846254944801331, "tokens_seen": 65052672, "tokens_per_sec": 17442.14318551958, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3476460774739583, "G_std": 0.16192118326822916, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.535192489624023, "grad_norm": 0.8868570923805237, "tokens_seen": 65216512, "tokens_per_sec": 17442.621473971838, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3520151774088542, "G_std": 0.1625391642252604, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.2546892166137695, "grad_norm": 0.9131051898002625, "tokens_seen": 65380352, "tokens_per_sec": 17442.989550972536, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3500162760416667, "G_std": 0.16206868489583334, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.337379455566406, "grad_norm": 0.8514596223831177, "tokens_seen": 65544192, "tokens_per_sec": 17443.47292584558, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3435160319010417, "G_std": 0.1616388956705729, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.535138034820557, "val_ppl": 93.23638428379378, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3512522379557292, "G_std": 0.16191864013671875, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.616724491119385, "grad_norm": 0.8735825419425964, "tokens_seen": 65708032, "tokens_per_sec": 17428.198675261527, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3466695149739583, "G_std": 0.16178131103515625, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.410238265991211, "grad_norm": 0.8421277403831482, "tokens_seen": 65871872, "tokens_per_sec": 17428.532722073527, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3445485432942708, "G_std": 0.1612726847330729, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.292625665664673, "grad_norm": 0.8435711860656738, "tokens_seen": 66035712, "tokens_per_sec": 17429.129016422004, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.345306396484375, "G_std": 0.1623662312825521, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.40082311630249, "grad_norm": 0.8887073993682861, "tokens_seen": 66199552, "tokens_per_sec": 17429.60142557829, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3501739501953125, "G_std": 0.16246541341145834, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.403462886810303, "grad_norm": 0.8617547154426575, "tokens_seen": 66363392, "tokens_per_sec": 17430.136188374083, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3456268310546875, "G_std": 0.1614964803059896, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.431220531463623, "grad_norm": 0.8120085597038269, "tokens_seen": 66527232, "tokens_per_sec": 17430.60061039144, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3487701416015625, "G_std": 0.1629765828450521, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.396325588226318, "grad_norm": 0.8603227138519287, "tokens_seen": 66691072, "tokens_per_sec": 17430.90679770788, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3452504475911458, "G_std": 0.1634038289388021, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.509687185287476, "grad_norm": 0.9085901379585266, "tokens_seen": 66854912, "tokens_per_sec": 17431.330727766643, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3496297200520833, "G_std": 0.1619695027669271, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.449852228164673, "grad_norm": 0.8274161219596863, "tokens_seen": 67018752, "tokens_per_sec": 17431.734567986645, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34393310546875, "G_std": 0.16276041666666666, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.504842758178711, "grad_norm": 0.9385164380073547, "tokens_seen": 67182592, "tokens_per_sec": 17432.003998929755, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3425801595052083, "G_std": 0.1626001993815104, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.368203639984131, "grad_norm": 0.8520752787590027, "tokens_seen": 67346432, "tokens_per_sec": 17432.37924574814, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3408304850260417, "G_std": 0.1615753173828125, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.430076360702515, "grad_norm": 0.8930606245994568, "tokens_seen": 67510272, "tokens_per_sec": 17432.810777260165, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3484039306640625, "G_std": 0.16254425048828125, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.339820861816406, "grad_norm": 0.9486882090568542, "tokens_seen": 67674112, "tokens_per_sec": 17433.199265410527, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3397216796875, "G_std": 0.16141764322916666, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.411877870559692, "grad_norm": 0.8716352581977844, "tokens_seen": 67837952, "tokens_per_sec": 17433.54733333215, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.343505859375, "G_std": 0.1620025634765625, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.575273513793945, "grad_norm": 0.8852729201316833, "tokens_seen": 68001792, "tokens_per_sec": 17433.868165810385, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.347320556640625, "G_std": 0.1614049275716146, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.229082345962524, "grad_norm": 0.8300482630729675, "tokens_seen": 68165632, "tokens_per_sec": 17434.14482013236, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.346160888671875, "G_std": 0.16347249348958334, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.409663438796997, "grad_norm": 0.8565188050270081, "tokens_seen": 68329472, "tokens_per_sec": 17434.5484345531, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3459218343098958, "G_std": 0.16234334309895834, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.399889945983887, "grad_norm": 0.8595541715621948, "tokens_seen": 68493312, "tokens_per_sec": 17434.970287377328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3469441731770833, "G_std": 0.16094462076822916, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.3662636280059814, "grad_norm": 0.828242301940918, "tokens_seen": 68657152, "tokens_per_sec": 17435.331113547072, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3463541666666667, "G_std": 0.16097005208333334, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.560265302658081, "grad_norm": 0.8408443331718445, "tokens_seen": 68820992, "tokens_per_sec": 17435.817790794328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3504435221354167, "G_std": 0.16080220540364584, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.368492841720581, "grad_norm": 0.8650238513946533, "tokens_seen": 68984832, "tokens_per_sec": 17436.186025806634, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3453623453776042, "G_std": 0.1623992919921875, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.370121479034424, "grad_norm": 0.8564128279685974, "tokens_seen": 69148672, "tokens_per_sec": 17436.65330096765, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3419189453125, "G_std": 0.16167449951171875, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.415262699127197, "grad_norm": 0.8493146896362305, "tokens_seen": 69312512, "tokens_per_sec": 17437.077367747905, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3439381917317708, "G_std": 0.16051737467447916, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.532857894897461, "grad_norm": 0.8422931432723999, "tokens_seen": 69476352, "tokens_per_sec": 17437.499026072983, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3443094889322917, "G_std": 0.1607844034830729, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.4950690269470215, "grad_norm": 0.9323274493217468, "tokens_seen": 69640192, "tokens_per_sec": 17437.776344313323, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3422648111979167, "G_std": 0.1616643269856771, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.487946736812591, "val_ppl": 88.9386438292095, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3404083251953125, "G_std": 0.16184234619140625, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.429226398468018, "grad_norm": 0.8826199769973755, "tokens_seen": 69804032, "tokens_per_sec": 17423.28940098282, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3441619873046875, "G_std": 0.16190083821614584, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.3848090171813965, "grad_norm": 0.8793484568595886, "tokens_seen": 69967872, "tokens_per_sec": 17423.66851314801, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3461507161458333, "G_std": 0.1602300008138021, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.354198217391968, "grad_norm": 0.8760618567466736, "tokens_seen": 70131712, "tokens_per_sec": 17424.08989900759, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.345916748046875, "G_std": 0.16265360514322916, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.492360591888428, "grad_norm": 0.8611981272697449, "tokens_seen": 70295552, "tokens_per_sec": 17424.517564450623, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3447774251302083, "G_std": 0.16091410319010416, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.527599096298218, "grad_norm": 0.9138822555541992, "tokens_seen": 70459392, "tokens_per_sec": 17424.98217508635, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3455607096354167, "G_std": 0.16034444173177084, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.509983539581299, "grad_norm": 0.8542786240577698, "tokens_seen": 70623232, "tokens_per_sec": 17425.41114769452, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3460489908854167, "G_std": 0.16259002685546875, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.399398326873779, "grad_norm": 0.8907527327537537, "tokens_seen": 70787072, "tokens_per_sec": 17425.765699454554, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3389689127604167, "G_std": 0.1597773234049479, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.458123445510864, "grad_norm": 0.9186338782310486, "tokens_seen": 70950912, "tokens_per_sec": 17426.27917695608, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3431650797526042, "G_std": 0.16001129150390625, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.368613958358765, "grad_norm": 0.8817214965820312, "tokens_seen": 71114752, "tokens_per_sec": 17426.654542097865, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3427225748697917, "G_std": 0.160064697265625, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.3485565185546875, "grad_norm": 0.9064164757728577, "tokens_seen": 71278592, "tokens_per_sec": 17427.032899015256, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.343109130859375, "G_std": 0.16056315104166666, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.455223321914673, "grad_norm": 0.8593478202819824, "tokens_seen": 71442432, "tokens_per_sec": 17427.446182211585, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3496602376302083, "G_std": 0.16168975830078125, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.563718795776367, "grad_norm": 0.8736574053764343, "tokens_seen": 71606272, "tokens_per_sec": 17427.92839960907, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.344146728515625, "G_std": 0.16287994384765625, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.391044616699219, "grad_norm": 0.8558555245399475, "tokens_seen": 71770112, "tokens_per_sec": 17428.235602687848, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34295654296875, "G_std": 0.16016133626302084, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.154800176620483, "grad_norm": 0.8874867558479309, "tokens_seen": 71933952, "tokens_per_sec": 17428.72876810208, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3359273274739583, "G_std": 0.16255696614583334, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.399204969406128, "grad_norm": 0.9057822227478027, "tokens_seen": 72097792, "tokens_per_sec": 17429.07086804833, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3452962239583333, "G_std": 0.1617609659830729, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.380121946334839, "grad_norm": 0.8979665040969849, "tokens_seen": 72261632, "tokens_per_sec": 17429.4056618005, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.342376708984375, "G_std": 0.16049702962239584, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.237180471420288, "grad_norm": 0.9029502868652344, "tokens_seen": 72425472, "tokens_per_sec": 17429.697127755833, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.346954345703125, "G_std": 0.1600468953450521, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.378175258636475, "grad_norm": 0.8877645134925842, "tokens_seen": 72589312, "tokens_per_sec": 17430.098464882645, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.340118408203125, "G_std": 0.16108194986979166, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.502631187438965, "grad_norm": 0.9262354373931885, "tokens_seen": 72753152, "tokens_per_sec": 17430.56939237585, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3446451822916667, "G_std": 0.1609013875325521, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.545889139175415, "grad_norm": 0.9382127523422241, "tokens_seen": 72916992, "tokens_per_sec": 17430.983456647846, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3467203776041667, "G_std": 0.16269938151041666, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.374941825866699, "grad_norm": 0.8687764406204224, "tokens_seen": 73080832, "tokens_per_sec": 17431.394799647052, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3452351888020833, "G_std": 0.162200927734375, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.375345230102539, "grad_norm": 0.9085054397583008, "tokens_seen": 73244672, "tokens_per_sec": 17431.80357489053, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.348388671875, "G_std": 0.16034698486328125, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.410640239715576, "grad_norm": 0.8600000143051147, "tokens_seen": 73408512, "tokens_per_sec": 17432.252247508615, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34912109375, "G_std": 0.1602935791015625, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.452922582626343, "grad_norm": 0.8599798083305359, "tokens_seen": 73572352, "tokens_per_sec": 17432.62898277911, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3380228678385417, "G_std": 0.1611353556315104, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.424991130828857, "grad_norm": 0.8635755777359009, "tokens_seen": 73736192, "tokens_per_sec": 17433.21365929417, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3426462809244792, "G_std": 0.16040802001953125, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.461307227611542, "val_ppl": 86.60064188531582, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3449910481770833, "G_std": 0.1613006591796875, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.45179009437561, "grad_norm": 0.8863084316253662, "tokens_seen": 73900032, "tokens_per_sec": 17419.663772369284, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3425140380859375, "G_std": 0.16027323404947916, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.407166004180908, "grad_norm": 0.9125377535820007, "tokens_seen": 74063872, "tokens_per_sec": 17420.10812051648, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3379770914713542, "G_std": 0.16207122802734375, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.436578989028931, "grad_norm": 0.8606238961219788, "tokens_seen": 74227712, "tokens_per_sec": 17420.51803602866, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3402506510416667, "G_std": 0.16057840983072916, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.445177316665649, "grad_norm": 0.9164729714393616, "tokens_seen": 74391552, "tokens_per_sec": 17420.985872731948, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3455607096354167, "G_std": 0.16079203287760416, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.358701229095459, "grad_norm": 0.9460729956626892, "tokens_seen": 74555392, "tokens_per_sec": 17421.405771028054, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3425394694010417, "G_std": 0.16040802001953125, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.2216198444366455, "grad_norm": 0.8751262426376343, "tokens_seen": 74719232, "tokens_per_sec": 17421.85440774844, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3385874430338542, "G_std": 0.1598332722981771, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.481666088104248, "grad_norm": 0.9019984006881714, "tokens_seen": 74883072, "tokens_per_sec": 17422.330562810523, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3408559163411458, "G_std": 0.1593755086263021, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.549889087677002, "grad_norm": 0.8650379776954651, "tokens_seen": 75046912, "tokens_per_sec": 17422.745625221254, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3458709716796875, "G_std": 0.16187540690104166, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.33720588684082, "grad_norm": 0.904819130897522, "tokens_seen": 75210752, "tokens_per_sec": 17423.019201837942, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3390146891276042, "G_std": 0.1585540771484375, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.458536386489868, "grad_norm": 0.892065167427063, "tokens_seen": 75374592, "tokens_per_sec": 17423.3962106044, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3442586263020833, "G_std": 0.16045379638671875, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.34470009803772, "grad_norm": 0.8987054228782654, "tokens_seen": 75538432, "tokens_per_sec": 17423.820791432052, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3439534505208333, "G_std": 0.1618169148763021, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.383154392242432, "grad_norm": 0.8655301928520203, "tokens_seen": 75702272, "tokens_per_sec": 17424.23964614795, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.340789794921875, "G_std": 0.15944925944010416, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.458834409713745, "grad_norm": 0.8722665905952454, "tokens_seen": 75866112, "tokens_per_sec": 17424.657983653622, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3490702311197917, "G_std": 0.16058349609375, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.450650215148926, "grad_norm": 0.8892578482627869, "tokens_seen": 76029952, "tokens_per_sec": 17425.011903176328, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3438008626302083, "G_std": 0.15995025634765625, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.524979591369629, "grad_norm": 0.9624994397163391, "tokens_seen": 76193792, "tokens_per_sec": 17425.263683145076, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.346954345703125, "G_std": 0.15979766845703125, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.282761812210083, "grad_norm": 0.8862881660461426, "tokens_seen": 76357632, "tokens_per_sec": 17425.605440209245, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3429412841796875, "G_std": 0.1595916748046875, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.286465644836426, "grad_norm": 0.9087539315223694, "tokens_seen": 76521472, "tokens_per_sec": 17425.86787269254, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.343109130859375, "G_std": 0.16082255045572916, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.466412544250488, "grad_norm": 0.9484125375747681, "tokens_seen": 76685312, "tokens_per_sec": 17426.15823771532, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3453114827473958, "G_std": 0.15996551513671875, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.389178037643433, "grad_norm": 0.9322476983070374, "tokens_seen": 76849152, "tokens_per_sec": 17426.48274441442, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3511098225911458, "G_std": 0.1594670613606771, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.341750860214233, "grad_norm": 0.9528585076332092, "tokens_seen": 77012992, "tokens_per_sec": 17426.822171989174, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3474833170572917, "G_std": 0.1594874064127604, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.238593101501465, "grad_norm": 0.9223253726959229, "tokens_seen": 77176832, "tokens_per_sec": 17427.001717329913, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3439687093098958, "G_std": 0.1608912150065104, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.338257789611816, "grad_norm": 0.9103731513023376, "tokens_seen": 77340672, "tokens_per_sec": 17427.35610546222, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.343475341796875, "G_std": 0.159820556640625, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.307660818099976, "grad_norm": 0.918352484703064, "tokens_seen": 77504512, "tokens_per_sec": 17427.730021459523, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3419545491536458, "G_std": 0.16062164306640625, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.388834476470947, "grad_norm": 0.8763156533241272, "tokens_seen": 77668352, "tokens_per_sec": 17428.183022117217, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3418680826822917, "G_std": 0.15943400065104166, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.358845233917236, "grad_norm": 0.9185782670974731, "tokens_seen": 77832192, "tokens_per_sec": 17428.602588311474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3478240966796875, "G_std": 0.15883890787760416, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.453887045383453, "val_ppl": 85.96042753250518, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3425038655598958, "G_std": 0.15934244791666666, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.411218881607056, "grad_norm": 0.9468597769737244, "tokens_seen": 77996032, "tokens_per_sec": 17415.569900098177, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.347320556640625, "G_std": 0.1604334513346354, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.288104295730591, "grad_norm": 0.9095145463943481, "tokens_seen": 78159872, "tokens_per_sec": 17416.009862617684, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.344085693359375, "G_std": 0.1596094767252604, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.41070556640625, "grad_norm": 0.9761458039283752, "tokens_seen": 78323712, "tokens_per_sec": 17416.056903996378, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3399200439453125, "G_std": 0.160552978515625, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.43468451499939, "grad_norm": 0.951148271560669, "tokens_seen": 78487552, "tokens_per_sec": 17416.514099724216, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3471527099609375, "G_std": 0.15915934244791666, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.342880487442017, "grad_norm": 0.8972079753875732, "tokens_seen": 78651392, "tokens_per_sec": 17416.933921517575, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.343017578125, "G_std": 0.15917460123697916, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.356731414794922, "grad_norm": 0.9286532402038574, "tokens_seen": 78815232, "tokens_per_sec": 17417.24403543876, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3414306640625, "G_std": 0.1594518025716146, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.283584833145142, "grad_norm": 0.8958113193511963, "tokens_seen": 78979072, "tokens_per_sec": 17417.583236293893, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3373311360677083, "G_std": 0.15888468424479166, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.544259071350098, "grad_norm": 0.9385562539100647, "tokens_seen": 79142912, "tokens_per_sec": 17417.89513918136, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3383433024088542, "G_std": 0.15968831380208334, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.320009469985962, "grad_norm": 0.8790403604507446, "tokens_seen": 79306752, "tokens_per_sec": 17418.319938554934, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3417612711588542, "G_std": 0.1603418986002604, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.413118362426758, "grad_norm": 0.918368399143219, "tokens_seen": 79470592, "tokens_per_sec": 17418.638460141483, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3448994954427083, "G_std": 0.15943400065104166, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.482676029205322, "grad_norm": 0.9846792221069336, "tokens_seen": 79634432, "tokens_per_sec": 17419.023186190407, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3452606201171875, "G_std": 0.16010538736979166, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.2265942096710205, "grad_norm": 0.9032095074653625, "tokens_seen": 79798272, "tokens_per_sec": 17419.379537709898, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.344390869140625, "G_std": 0.15856424967447916, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.461095333099365, "grad_norm": 0.9216921925544739, "tokens_seen": 79962112, "tokens_per_sec": 17419.687558281406, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3394775390625, "G_std": 0.15863800048828125, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.345035552978516, "grad_norm": 0.8867546916007996, "tokens_seen": 80125952, "tokens_per_sec": 17420.076897805466, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.339080810546875, "G_std": 0.15998077392578125, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.4436070919036865, "grad_norm": 0.8989778161048889, "tokens_seen": 80289792, "tokens_per_sec": 17420.462070324204, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3434193929036458, "G_std": 0.15851593017578125, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.288878440856934, "grad_norm": 0.910026490688324, "tokens_seen": 80453632, "tokens_per_sec": 17420.855822282207, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3376057942708333, "G_std": 0.15825653076171875, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.476208448410034, "grad_norm": 0.9734244346618652, "tokens_seen": 80617472, "tokens_per_sec": 17421.180013205812, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3425852457682292, "G_std": 0.1579462687174479, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.520376920700073, "grad_norm": 1.0126404762268066, "tokens_seen": 80781312, "tokens_per_sec": 17421.518015556063, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3434906005859375, "G_std": 0.16021219889322916, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.361896276473999, "grad_norm": 0.929239809513092, "tokens_seen": 80945152, "tokens_per_sec": 17421.797795429968, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3449045817057292, "G_std": 0.1603546142578125, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.338681697845459, "grad_norm": 0.878925085067749, "tokens_seen": 81108992, "tokens_per_sec": 17422.196038709426, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3357696533203125, "G_std": 0.15824381510416666, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.304879188537598, "grad_norm": 0.8963487148284912, "tokens_seen": 81272832, "tokens_per_sec": 17422.586268784722, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3383839925130208, "G_std": 0.159698486328125, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.269306659698486, "grad_norm": 0.902246356010437, "tokens_seen": 81436672, "tokens_per_sec": 17422.988685665823, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3371683756510417, "G_std": 0.15900166829427084, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.3392014503479, "grad_norm": 0.9883899092674255, "tokens_seen": 81600512, "tokens_per_sec": 17423.42941735104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3377939860026042, "G_std": 0.1582183837890625, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.405208110809326, "grad_norm": 0.8834903240203857, "tokens_seen": 81764352, "tokens_per_sec": 17423.777783372578, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3376973470052083, "G_std": 0.1580963134765625, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.329387426376343, "grad_norm": 0.9523568153381348, "tokens_seen": 81928192, "tokens_per_sec": 17424.129448073258, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3370513916015625, "G_std": 0.15837860107421875, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.416970670223236, "val_ppl": 82.84494020266425, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.33935546875, "G_std": 0.15921529134114584, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.654148578643799, "grad_norm": 1.071079969406128, "tokens_seen": 82092032, "tokens_per_sec": 17411.994319309422, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3438873291015625, "G_std": 0.16080220540364584, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.239717483520508, "grad_norm": 0.9291374087333679, "tokens_seen": 82255872, "tokens_per_sec": 17412.43055945827, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3339945475260417, "G_std": 0.15862274169921875, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.475993394851685, "grad_norm": 0.9249173998832703, "tokens_seen": 82419712, "tokens_per_sec": 17412.73852925684, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3466949462890625, "G_std": 0.1575749715169271, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.245173454284668, "grad_norm": 0.9211347103118896, "tokens_seen": 82583552, "tokens_per_sec": 17412.857986879615, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3431752522786458, "G_std": 0.1587066650390625, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.308934450149536, "grad_norm": 0.954861581325531, "tokens_seen": 82747392, "tokens_per_sec": 17413.321009160998, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34588623046875, "G_std": 0.1585566202799479, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.33337664604187, "grad_norm": 0.8987537026405334, "tokens_seen": 82911232, "tokens_per_sec": 17413.814542091306, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34326171875, "G_std": 0.1605707804361979, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.255999565124512, "grad_norm": 0.9162140488624573, "tokens_seen": 83075072, "tokens_per_sec": 17414.010252240136, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3359425862630208, "G_std": 0.15849812825520834, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.367602825164795, "grad_norm": 0.9198970198631287, "tokens_seen": 83238912, "tokens_per_sec": 17414.472988820467, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3437093098958333, "G_std": 0.15836079915364584, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.273804187774658, "grad_norm": 0.8806013464927673, "tokens_seen": 83402752, "tokens_per_sec": 17414.71043190218, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3369140625, "G_std": 0.1596247355143229, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.466490745544434, "grad_norm": 0.9834569096565247, "tokens_seen": 83566592, "tokens_per_sec": 17414.948859149004, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3410288492838542, "G_std": 0.15867106119791666, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.391853094100952, "grad_norm": 0.9770525097846985, "tokens_seen": 83730432, "tokens_per_sec": 17415.201797539445, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3459879557291667, "G_std": 0.15924835205078125, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.323573589324951, "grad_norm": 0.9291623830795288, "tokens_seen": 83894272, "tokens_per_sec": 17415.556719884094, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3351491292317708, "G_std": 0.15766143798828125, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.4446282386779785, "grad_norm": 0.8963831067085266, "tokens_seen": 84058112, "tokens_per_sec": 17415.825430094323, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3447977701822917, "G_std": 0.15782928466796875, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.350457668304443, "grad_norm": 0.8979538083076477, "tokens_seen": 84221952, "tokens_per_sec": 17416.220115546745, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3380889892578125, "G_std": 0.15898895263671875, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.258896350860596, "grad_norm": 0.952529788017273, "tokens_seen": 84385792, "tokens_per_sec": 17416.46328933022, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3329315185546875, "G_std": 0.1580632527669271, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.295494318008423, "grad_norm": 0.9236435294151306, "tokens_seen": 84549632, "tokens_per_sec": 17416.646120450652, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3466084798177083, "G_std": 0.1586456298828125, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.468840599060059, "grad_norm": 0.959058403968811, "tokens_seen": 84713472, "tokens_per_sec": 17416.980787283468, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.342254638671875, "G_std": 0.15892537434895834, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.338995695114136, "grad_norm": 0.8977235555648804, "tokens_seen": 84877312, "tokens_per_sec": 17417.347307841108, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3377176920572917, "G_std": 0.15922037760416666, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.3567116260528564, "grad_norm": 0.9445017576217651, "tokens_seen": 85041152, "tokens_per_sec": 17417.722728918583, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.341552734375, "G_std": 0.15799967447916666, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.448739767074585, "grad_norm": 0.9552162289619446, "tokens_seen": 85204992, "tokens_per_sec": 17418.098730888494, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.344696044921875, "G_std": 0.15889994303385416, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.407594203948975, "grad_norm": 1.021801471710205, "tokens_seen": 85368832, "tokens_per_sec": 17418.41119464332, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3439280192057292, "G_std": 0.15842183430989584, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.420323133468628, "grad_norm": 0.9473506212234497, "tokens_seen": 85532672, "tokens_per_sec": 17418.766756770692, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3411966959635417, "G_std": 0.15952046712239584, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.243071794509888, "grad_norm": 0.9188619256019592, "tokens_seen": 85696512, "tokens_per_sec": 17419.182645590885, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3404286702473958, "G_std": 0.1595611572265625, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.336564540863037, "grad_norm": 0.9067156910896301, "tokens_seen": 85860352, "tokens_per_sec": 17419.5568776151, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3381144205729167, "G_std": 0.15967305501302084, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.518898963928223, "grad_norm": 1.1331822872161865, "tokens_seen": 86024192, "tokens_per_sec": 17419.922463044284, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3447113037109375, "G_std": 0.15894063313802084, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.412785685062408, "val_ppl": 82.49895982360368, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3380381266276042, "G_std": 0.15853118896484375, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.354318618774414, "grad_norm": 0.9504539370536804, "tokens_seen": 86188032, "tokens_per_sec": 17408.21047603895, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3406168619791667, "G_std": 0.15903472900390625, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.4604198932647705, "grad_norm": 0.9732409715652466, "tokens_seen": 86351872, "tokens_per_sec": 17408.65617364345, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3383127848307292, "G_std": 0.1595611572265625, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.436091661453247, "grad_norm": 0.9802422523498535, "tokens_seen": 86515712, "tokens_per_sec": 17409.048602309238, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3411916097005208, "G_std": 0.15904744466145834, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.3035290241241455, "grad_norm": 0.934047520160675, "tokens_seen": 86679552, "tokens_per_sec": 17409.42802375642, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3337809244791667, "G_std": 0.15702311197916666, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.243493318557739, "grad_norm": 0.9576987624168396, "tokens_seen": 86843392, "tokens_per_sec": 17409.787994438037, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3373921712239583, "G_std": 0.15784454345703125, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.339595794677734, "grad_norm": 0.9786390066146851, "tokens_seen": 87007232, "tokens_per_sec": 17410.11182649687, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3430328369140625, "G_std": 0.15984090169270834, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.444212198257446, "grad_norm": 0.992074191570282, "tokens_seen": 87171072, "tokens_per_sec": 17410.531536089315, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3413543701171875, "G_std": 0.15715535481770834, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.253251552581787, "grad_norm": 0.9088348746299744, "tokens_seen": 87334912, "tokens_per_sec": 17410.775839105867, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.341522216796875, "G_std": 0.15724945068359375, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.270699501037598, "grad_norm": 0.9285292625427246, "tokens_seen": 87498752, "tokens_per_sec": 17411.19051602123, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.337493896484375, "G_std": 0.15880839029947916, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.347136974334717, "grad_norm": 0.8917679190635681, "tokens_seen": 87662592, "tokens_per_sec": 17411.509992978186, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3366495768229167, "G_std": 0.15807342529296875, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.357941627502441, "grad_norm": 0.9786003828048706, "tokens_seen": 87826432, "tokens_per_sec": 17411.860376265744, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3453114827473958, "G_std": 0.1587244669596354, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.4034342765808105, "grad_norm": 0.9538366794586182, "tokens_seen": 87990272, "tokens_per_sec": 17412.31125121194, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3436838785807292, "G_std": 0.15833282470703125, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.311819314956665, "grad_norm": 0.9037075042724609, "tokens_seen": 88154112, "tokens_per_sec": 17412.566632253285, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3383026123046875, "G_std": 0.15857950846354166, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.3337531089782715, "grad_norm": 0.9302893280982971, "tokens_seen": 88317952, "tokens_per_sec": 17412.88736132916, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3419901529947917, "G_std": 0.15807596842447916, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.412298202514648, "grad_norm": 1.0104752779006958, "tokens_seen": 88481792, "tokens_per_sec": 17413.23260544611, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3407694498697917, "G_std": 0.15833028157552084, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.506742000579834, "grad_norm": 0.9387645721435547, "tokens_seen": 88645632, "tokens_per_sec": 17413.590188985298, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3417714436848958, "G_std": 0.1599248250325521, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.189962387084961, "grad_norm": 0.9184392094612122, "tokens_seen": 88809472, "tokens_per_sec": 17414.017794187774, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3349253336588542, "G_std": 0.1571324666341146, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.378664255142212, "grad_norm": 0.890150249004364, "tokens_seen": 88973312, "tokens_per_sec": 17414.320299464667, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3438618977864583, "G_std": 0.15743001302083334, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.209491968154907, "grad_norm": 0.9755824208259583, "tokens_seen": 89137152, "tokens_per_sec": 17414.63651665976, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3380126953125, "G_std": 0.15809122721354166, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.343323707580566, "grad_norm": 0.9394874572753906, "tokens_seen": 89300992, "tokens_per_sec": 17414.86894575061, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.340362548828125, "G_std": 0.158416748046875, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.352431297302246, "grad_norm": 0.9814360737800598, "tokens_seen": 89464832, "tokens_per_sec": 17415.201931936397, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3405253092447917, "G_std": 0.15749613444010416, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.29314398765564, "grad_norm": 0.9388853311538696, "tokens_seen": 89628672, "tokens_per_sec": 17415.48923938443, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3349507649739583, "G_std": 0.15723164876302084, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.301074743270874, "grad_norm": 0.960002601146698, "tokens_seen": 89792512, "tokens_per_sec": 17415.884618333654, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.33526611328125, "G_std": 0.15805816650390625, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.507322311401367, "grad_norm": 1.0254195928573608, "tokens_seen": 89956352, "tokens_per_sec": 17416.209776339758, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34539794921875, "G_std": 0.15982564290364584, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.307684659957886, "grad_norm": 0.9360846281051636, "tokens_seen": 90120192, "tokens_per_sec": 17416.64205326139, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3430938720703125, "G_std": 0.15771993001302084, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.397501420974732, "val_ppl": 81.24761126601173, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3382975260416667, "G_std": 0.157501220703125, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.116525173187256, "grad_norm": 0.963951051235199, "tokens_seen": 90284032, "tokens_per_sec": 17405.39409787463, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3385976155598958, "G_std": 0.1569366455078125, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.322408199310303, "grad_norm": 0.9988288283348083, "tokens_seen": 90447872, "tokens_per_sec": 17405.782811015208, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.341827392578125, "G_std": 0.15940093994140625, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.387725830078125, "grad_norm": 0.9807114601135254, "tokens_seen": 90611712, "tokens_per_sec": 17406.258040782104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.338714599609375, "G_std": 0.1590906778971354, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.39532995223999, "grad_norm": 1.002564787864685, "tokens_seen": 90775552, "tokens_per_sec": 17406.545092776505, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3358357747395833, "G_std": 0.15865580240885416, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.408804416656494, "grad_norm": 0.945408284664154, "tokens_seen": 90939392, "tokens_per_sec": 17406.984124063674, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3376820882161458, "G_std": 0.1573003133138021, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.385237693786621, "grad_norm": 0.9370347857475281, "tokens_seen": 91103232, "tokens_per_sec": 17407.311228718092, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3446807861328125, "G_std": 0.1594975789388021, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.2661004066467285, "grad_norm": 0.9548856616020203, "tokens_seen": 91267072, "tokens_per_sec": 17407.476805639973, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3387400309244792, "G_std": 0.1580810546875, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.2860071659088135, "grad_norm": 0.9805667996406555, "tokens_seen": 91430912, "tokens_per_sec": 17407.77679190906, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3320973714192708, "G_std": 0.1567230224609375, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.306278705596924, "grad_norm": 0.9479236006736755, "tokens_seen": 91594752, "tokens_per_sec": 17408.170732378567, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3386789957682292, "G_std": 0.1586176554361979, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.496589660644531, "grad_norm": 1.1501573324203491, "tokens_seen": 91758592, "tokens_per_sec": 17408.417618249056, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3391825358072917, "G_std": 0.15807342529296875, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.369804859161377, "grad_norm": 1.0547856092453003, "tokens_seen": 91922432, "tokens_per_sec": 17408.790809893413, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.33642578125, "G_std": 0.1567509969075521, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.272549629211426, "grad_norm": 0.9771418571472168, "tokens_seen": 92086272, "tokens_per_sec": 17409.073520325346, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3370310465494792, "G_std": 0.1584447224934896, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.294916391372681, "grad_norm": 0.931003749370575, "tokens_seen": 92250112, "tokens_per_sec": 17409.409957722237, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3425750732421875, "G_std": 0.15708160400390625, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.195920705795288, "grad_norm": 0.995062530040741, "tokens_seen": 92413952, "tokens_per_sec": 17409.72973144158, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3360493977864583, "G_std": 0.15680694580078125, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.40294337272644, "grad_norm": 0.9563466310501099, "tokens_seen": 92577792, "tokens_per_sec": 17410.02906184431, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3442179361979167, "G_std": 0.15826924641927084, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.222583293914795, "grad_norm": 0.983289897441864, "tokens_seen": 92741632, "tokens_per_sec": 17410.33014395657, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3378550211588542, "G_std": 0.15769449869791666, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.3865647315979, "grad_norm": 1.0456124544143677, "tokens_seen": 92905472, "tokens_per_sec": 17410.73306733873, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3390299479166667, "G_std": 0.15711466471354166, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.274125576019287, "grad_norm": 0.9574071764945984, "tokens_seen": 93069312, "tokens_per_sec": 17411.066153790965, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3345591227213542, "G_std": 0.15764617919921875, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.3405327796936035, "grad_norm": 1.0472297668457031, "tokens_seen": 93233152, "tokens_per_sec": 17411.3553324326, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3398640950520833, "G_std": 0.15886688232421875, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.395733118057251, "grad_norm": 0.9517611861228943, "tokens_seen": 93396992, "tokens_per_sec": 17411.7529583511, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3379465738932292, "G_std": 0.15952046712239584, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.087195158004761, "grad_norm": 0.9578410387039185, "tokens_seen": 93560832, "tokens_per_sec": 17411.892495526165, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3337860107421875, "G_std": 0.15699005126953125, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.184998989105225, "grad_norm": 0.9509748220443726, "tokens_seen": 93724672, "tokens_per_sec": 17412.177975191917, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.334716796875, "G_std": 0.15668741861979166, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.315640926361084, "grad_norm": 0.9910479784011841, "tokens_seen": 93888512, "tokens_per_sec": 17412.43755387373, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3395589192708333, "G_std": 0.15802764892578125, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.372627258300781, "grad_norm": 0.9264960289001465, "tokens_seen": 94052352, "tokens_per_sec": 17412.89297452104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34063720703125, "G_std": 0.15652974446614584, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.5183000564575195, "grad_norm": 1.0173475742340088, "tokens_seen": 94216192, "tokens_per_sec": 17413.274114362604, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3369191487630208, "G_std": 0.15821075439453125, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.388315033912659, "val_ppl": 80.50465701849714, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3397878011067708, "G_std": 0.15782674153645834, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.293367385864258, "grad_norm": 0.9658580422401428, "tokens_seen": 94380032, "tokens_per_sec": 17402.51964407217, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3368581136067708, "G_std": 0.15776570638020834, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.195741176605225, "grad_norm": 0.9583472609519958, "tokens_seen": 94543872, "tokens_per_sec": 17402.77473603266, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3339894612630208, "G_std": 0.15807088216145834, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.329084873199463, "grad_norm": 1.0426201820373535, "tokens_seen": 94707712, "tokens_per_sec": 17403.173494714923, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3370259602864583, "G_std": 0.15732320149739584, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.36934494972229, "grad_norm": 0.9525707364082336, "tokens_seen": 94871552, "tokens_per_sec": 17403.63104355847, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3382822672526042, "G_std": 0.1575647989908854, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.189576148986816, "grad_norm": 0.983458399772644, "tokens_seen": 95035392, "tokens_per_sec": 17403.99004545045, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3362325032552083, "G_std": 0.15750630696614584, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.430707693099976, "grad_norm": 1.0173147916793823, "tokens_seen": 95199232, "tokens_per_sec": 17404.302563654142, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34210205078125, "G_std": 0.15852864583333334, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.3685173988342285, "grad_norm": 0.9917275905609131, "tokens_seen": 95363072, "tokens_per_sec": 17404.654147473168, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3374481201171875, "G_std": 0.1569671630859375, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.369790554046631, "grad_norm": 0.9647441506385803, "tokens_seen": 95526912, "tokens_per_sec": 17405.090153540317, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3376820882161458, "G_std": 0.1584930419921875, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.280210018157959, "grad_norm": 0.9416313767433167, "tokens_seen": 95690752, "tokens_per_sec": 17405.420687322698, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3402557373046875, "G_std": 0.15680440266927084, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.299653053283691, "grad_norm": 0.9389987587928772, "tokens_seen": 95854592, "tokens_per_sec": 17405.87528822474, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3365681966145833, "G_std": 0.1578191121419271, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.174186944961548, "grad_norm": 0.953036367893219, "tokens_seen": 96018432, "tokens_per_sec": 17406.15299016731, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3364308675130208, "G_std": 0.15768941243489584, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.392987251281738, "grad_norm": 1.0076746940612793, "tokens_seen": 96182272, "tokens_per_sec": 17406.390916262986, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3430684407552083, "G_std": 0.1582489013671875, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.282958984375, "grad_norm": 0.9714409708976746, "tokens_seen": 96346112, "tokens_per_sec": 17406.66943670676, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3402913411458333, "G_std": 0.1575902303059896, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.270713567733765, "grad_norm": 0.9822807312011719, "tokens_seen": 96509952, "tokens_per_sec": 17407.029262495915, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3309783935546875, "G_std": 0.1564814249674479, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.23492169380188, "grad_norm": 1.0182337760925293, "tokens_seen": 96673792, "tokens_per_sec": 17407.28829126803, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3384145100911458, "G_std": 0.157135009765625, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.323875427246094, "grad_norm": 0.9271125793457031, "tokens_seen": 96837632, "tokens_per_sec": 17407.489519907493, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3376719156901042, "G_std": 0.1574732462565104, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.294405460357666, "grad_norm": 0.9689651727676392, "tokens_seen": 97001472, "tokens_per_sec": 17407.741516670558, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.34051513671875, "G_std": 0.15740203857421875, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.304148197174072, "grad_norm": 0.968044638633728, "tokens_seen": 97165312, "tokens_per_sec": 17408.053089715104, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3346099853515625, "G_std": 0.158172607421875, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.264930248260498, "grad_norm": 0.9665790796279907, "tokens_seen": 97329152, "tokens_per_sec": 17408.3036633906, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3377126057942708, "G_std": 0.1577123006184896, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.407443284988403, "grad_norm": 1.1550449132919312, "tokens_seen": 97492992, "tokens_per_sec": 17408.597249399296, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3348744710286458, "G_std": 0.1573664347330729, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.244600296020508, "grad_norm": 1.1267788410186768, "tokens_seen": 97656832, "tokens_per_sec": 17408.956646279246, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.332000732421875, "G_std": 0.1570281982421875, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.299643039703369, "grad_norm": 0.97489994764328, "tokens_seen": 97820672, "tokens_per_sec": 17409.35131524226, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3362681070963542, "G_std": 0.1568145751953125, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.2211384773254395, "grad_norm": 0.981156587600708, "tokens_seen": 97984512, "tokens_per_sec": 17409.633125750133, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3381398518880208, "G_std": 0.15543111165364584, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.31032657623291, "grad_norm": 0.9810850620269775, "tokens_seen": 98148352, "tokens_per_sec": 17410.01410180047, "vram_gb": 6.28956416, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3359120686848958, "G_std": 0.1566187540690104, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.389214038848877, "val_ppl": 80.57706364462581, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3429412841796875, "G_std": 0.15882364908854166, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 5649.687728404999, "nan_failures": 0} diff --git a/g_only_s2/config.json b/g_only_s2/config.json new file mode 100644 index 0000000000000000000000000000000000000000..aebc7556f6361a2879550a6f669a6282dc58460a --- /dev/null +++ b/g_only_s2/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 2, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s2", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 2001343, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/g_only_s2/final.pt b/g_only_s2/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..f7301a0f76085694de55602b4bab328c10613a06 --- /dev/null +++ b/g_only_s2/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d0615178109f28740c0e41ddaaa40b9fab759336528f2808c12d5d8cf65c9223 +size 253629735 diff --git a/g_only_s2/metrics.jsonl b/g_only_s2/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..e555158b097a1aa8f2598f1e00e464f8c7c7c843 --- /dev/null +++ b/g_only_s2/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 2, "config": {"variant": "g_only", "seed": 2, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s2", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 2001343, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.890518188476562, "grad_norm": 10.160429954528809, "tokens_seen": 8192, "tokens_per_sec": 6591.940498650714, "vram_gb": 4.987702272, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5052897135416666, "G_std": 0.08565266927083333, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.880475997924805, "grad_norm": 2.1625030040740967, "tokens_seen": 172032, "tokens_per_sec": 20790.429164679743, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.436279296875, "G_std": 0.06296793619791667, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.112411499023438, "grad_norm": 1.8225091695785522, "tokens_seen": 335872, "tokens_per_sec": 19596.930175131474, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.427001953125, "G_std": 0.07282002766927083, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.191549301147461, "grad_norm": 1.5593948364257812, "tokens_seen": 499712, "tokens_per_sec": 20096.583734512667, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4217122395833333, "G_std": 0.0843048095703125, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.56137752532959, "grad_norm": 0.8810999989509583, "tokens_seen": 663552, "tokens_per_sec": 20765.312044530787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4251302083333333, "G_std": 0.10027058919270833, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.242398262023926, "grad_norm": 0.8697165250778198, "tokens_seen": 827392, "tokens_per_sec": 21152.76404953599, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4134114583333333, "G_std": 0.11545817057291667, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.138974666595459, "grad_norm": 0.629650890827179, "tokens_seen": 991232, "tokens_per_sec": 21437.293423324012, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4169108072916667, "G_std": 0.122589111328125, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 7.0134923458099365, "grad_norm": 0.5678006410598755, "tokens_seen": 1155072, "tokens_per_sec": 21638.815249309144, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40234375, "G_std": 0.13069661458333334, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.725820064544678, "grad_norm": 0.534537136554718, "tokens_seen": 1318912, "tokens_per_sec": 21793.337392033674, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4013671875, "G_std": 0.1385498046875, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.8416359424591064, "grad_norm": 0.5227241516113281, "tokens_seen": 1482752, "tokens_per_sec": 21906.466796590314, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.400634765625, "G_std": 0.14906819661458334, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.619797945022583, "grad_norm": 0.5500121712684631, "tokens_seen": 1646592, "tokens_per_sec": 22002.02658752967, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3798828125, "G_std": 0.154296875, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.621648788452148, "grad_norm": 0.5776386260986328, "tokens_seen": 1810432, "tokens_per_sec": 22080.784538054795, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3935546875, "G_std": 0.15299479166666666, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.591502666473389, "grad_norm": 0.5207910537719727, "tokens_seen": 1974272, "tokens_per_sec": 22140.88269330516, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3861490885416667, "G_std": 0.15456136067708334, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.411506652832031, "grad_norm": 0.44272828102111816, "tokens_seen": 2138112, "tokens_per_sec": 22188.549615713953, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4021809895833333, "G_std": 0.16650390625, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.513993740081787, "grad_norm": 0.4754171073436737, "tokens_seen": 2301952, "tokens_per_sec": 22233.632424957228, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4115397135416667, "G_std": 0.17140706380208334, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.6525163650512695, "grad_norm": 0.4709170460700989, "tokens_seen": 2465792, "tokens_per_sec": 22276.209326659507, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4124348958333333, "G_std": 0.182861328125, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.601083755493164, "grad_norm": 0.4852502644062042, "tokens_seen": 2629632, "tokens_per_sec": 22312.067443695403, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4034830729166667, "G_std": 0.18463134765625, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.4611735343933105, "grad_norm": 0.4648764729499817, "tokens_seen": 2793472, "tokens_per_sec": 22341.964527274307, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4058430989583333, "G_std": 0.19075520833333334, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.401877403259277, "grad_norm": 0.5370504260063171, "tokens_seen": 2957312, "tokens_per_sec": 22370.76876637971, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40625, "G_std": 0.19252522786458334, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.262488842010498, "grad_norm": 0.485965758562088, "tokens_seen": 3121152, "tokens_per_sec": 22399.150583171373, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4348958333333333, "G_std": 0.20391845703125, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.483797073364258, "grad_norm": 0.49092018604278564, "tokens_seen": 3284992, "tokens_per_sec": 22420.778234380454, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4185384114583333, "G_std": 0.19911702473958334, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.291632890701294, "grad_norm": 0.4546121060848236, "tokens_seen": 3448832, "tokens_per_sec": 22439.65089017359, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4432779947916667, "G_std": 0.21073404947916666, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.300347805023193, "grad_norm": 0.473076730966568, "tokens_seen": 3612672, "tokens_per_sec": 22458.71247311578, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4318033854166667, "G_std": 0.21321614583333334, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.247294187545776, "grad_norm": 0.4595493674278259, "tokens_seen": 3776512, "tokens_per_sec": 22470.39462528132, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4419759114583333, "G_std": 0.21443684895833334, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.495319843292236, "grad_norm": 0.5016887187957764, "tokens_seen": 3940352, "tokens_per_sec": 22488.83064878025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4442545572916667, "G_std": 0.21272786458333334, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.313615322113037, "grad_norm": 0.48372992873191833, "tokens_seen": 4104192, "tokens_per_sec": 22504.044938298925, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4357096354166667, "G_std": 0.2135009765625, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.357385158538818, "grad_norm": 0.5336084961891174, "tokens_seen": 4268032, "tokens_per_sec": 22517.274372371052, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.478515625, "G_std": 0.22591145833333334, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.08536434173584, "grad_norm": 0.4602265954017639, "tokens_seen": 4431872, "tokens_per_sec": 22526.74318206077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4805501302083333, "G_std": 0.22098795572916666, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.061105966567993, "grad_norm": 0.5294255018234253, "tokens_seen": 4595712, "tokens_per_sec": 22532.55948046129, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5020345052083334, "G_std": 0.22196451822916666, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 5.914686441421509, "grad_norm": 0.46137675642967224, "tokens_seen": 4759552, "tokens_per_sec": 22540.379987599328, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4993489583333333, "G_std": 0.21573893229166666, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.051862955093384, "grad_norm": 0.47104403376579285, "tokens_seen": 4923392, "tokens_per_sec": 22551.981180085797, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.513671875, "G_std": 0.227294921875, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.21113920211792, "grad_norm": 0.5012168288230896, "tokens_seen": 5087232, "tokens_per_sec": 22561.371204335584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.518310546875, "G_std": 0.22224934895833334, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.138489007949829, "grad_norm": 0.4710128903388977, "tokens_seen": 5251072, "tokens_per_sec": 22572.08595846394, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.528076171875, "G_std": 0.22725423177083334, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.121492862701416, "grad_norm": 0.4941888153553009, "tokens_seen": 5414912, "tokens_per_sec": 22583.724044104467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5537923177083334, "G_std": 0.21756998697916666, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.060635328292847, "grad_norm": 0.44145429134368896, "tokens_seen": 5578752, "tokens_per_sec": 22591.23004872781, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.532470703125, "G_std": 0.21284993489583334, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.977803945541382, "grad_norm": 0.5059952735900879, "tokens_seen": 5742592, "tokens_per_sec": 22599.92324411486, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5603841145833334, "G_std": 0.21071370442708334, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.026819229125977, "grad_norm": 0.5962278246879578, "tokens_seen": 5906432, "tokens_per_sec": 22605.322918067235, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5616861979166666, "G_std": 0.21333821614583334, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.011735200881958, "grad_norm": 0.517457127571106, "tokens_seen": 6070272, "tokens_per_sec": 22613.567284528923, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5357259114583334, "G_std": 0.21217854817708334, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.022727966308594, "grad_norm": 0.47577571868896484, "tokens_seen": 6234112, "tokens_per_sec": 22620.233184475677, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5782877604166666, "G_std": 0.20491536458333334, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.786971569061279, "grad_norm": 0.4781129062175751, "tokens_seen": 6397952, "tokens_per_sec": 22627.856108807882, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5655110677083334, "G_std": 0.20245361328125, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.889337778091431, "grad_norm": 0.4991913139820099, "tokens_seen": 6561792, "tokens_per_sec": 22631.75820298546, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5841471354166666, "G_std": 0.2000732421875, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8423614501953125, "grad_norm": 0.4568149149417877, "tokens_seen": 6725632, "tokens_per_sec": 22636.68659793985, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.55810546875, "G_std": 0.19826253255208334, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.911439657211304, "grad_norm": 0.4868650734424591, "tokens_seen": 6889472, "tokens_per_sec": 22641.38508401796, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5636393229166666, "G_std": 0.20302327473958334, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.880586385726929, "grad_norm": 0.4985743761062622, "tokens_seen": 7053312, "tokens_per_sec": 22646.285686332794, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5511067708333334, "G_std": 0.20188395182291666, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.97646951675415, "grad_norm": 0.5673156976699829, "tokens_seen": 7217152, "tokens_per_sec": 22649.116139029407, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5885416666666666, "G_std": 0.197265625, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.77367377281189, "grad_norm": 0.5038659572601318, "tokens_seen": 7380992, "tokens_per_sec": 22646.663182708275, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5504150390625, "G_std": 0.1993408203125, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.912355661392212, "grad_norm": 0.47141796350479126, "tokens_seen": 7544832, "tokens_per_sec": 22647.182011303903, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5569661458333334, "G_std": 0.19364420572916666, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.8328471183776855, "grad_norm": 0.46691325306892395, "tokens_seen": 7708672, "tokens_per_sec": 22648.217952770043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5612386067708334, "G_std": 0.19203694661458334, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.730938673019409, "grad_norm": 0.4832923114299774, "tokens_seen": 7872512, "tokens_per_sec": 22649.984867075436, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.572021484375, "G_std": 0.19732666015625, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.8104071617126465, "grad_norm": 0.4938339293003082, "tokens_seen": 8036352, "tokens_per_sec": 22651.568080790166, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.56982421875, "G_std": 0.19671630859375, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.732360363006592, "grad_norm": 0.4978537857532501, "tokens_seen": 8200192, "tokens_per_sec": 22653.439610612655, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5630696614583334, "G_std": 0.19608561197916666, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.731231069564819, "val_ppl": 308.34863343508647, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5623372395833334, "G_std": 0.20048014322916666, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.6849541664123535, "grad_norm": 0.49466538429260254, "tokens_seen": 8364032, "tokens_per_sec": 22578.76941997496, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5563557942708334, "G_std": 0.20109049479166666, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.819107532501221, "grad_norm": 0.487445205450058, "tokens_seen": 8527872, "tokens_per_sec": 22583.601656746243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5682373046875, "G_std": 0.1923828125, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.688319444656372, "grad_norm": 0.78382408618927, "tokens_seen": 8691712, "tokens_per_sec": 22587.9444074568, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5499267578125, "G_std": 0.19927978515625, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.764822959899902, "grad_norm": 0.5285676717758179, "tokens_seen": 8855552, "tokens_per_sec": 22589.540361858326, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5444742838541666, "G_std": 0.19891357421875, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.597487926483154, "grad_norm": 0.48836445808410645, "tokens_seen": 9019392, "tokens_per_sec": 22593.12406862024, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5621744791666666, "G_std": 0.1943359375, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.689857006072998, "grad_norm": 0.5198389291763306, "tokens_seen": 9183232, "tokens_per_sec": 22593.70075239418, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5421142578125, "G_std": 0.19911702473958334, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.741154193878174, "grad_norm": 0.5552091002464294, "tokens_seen": 9347072, "tokens_per_sec": 22597.01283922844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5581461588541666, "G_std": 0.20017496744791666, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.6624860763549805, "grad_norm": 0.5004135966300964, "tokens_seen": 9510912, "tokens_per_sec": 22599.63389719292, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5555826822916666, "G_std": 0.19893391927083334, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.48832893371582, "grad_norm": 0.4967702329158783, "tokens_seen": 9674752, "tokens_per_sec": 22603.747438801467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5484212239583334, "G_std": 0.19854736328125, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.554058074951172, "grad_norm": 0.4812192916870117, "tokens_seen": 9838592, "tokens_per_sec": 22603.45727332002, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5351969401041666, "G_std": 0.19816080729166666, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.604426383972168, "grad_norm": 0.4873007535934448, "tokens_seen": 10002432, "tokens_per_sec": 22606.488431017384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5479329427083334, "G_std": 0.20058186848958334, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.55021333694458, "grad_norm": 0.5750076174736023, "tokens_seen": 10166272, "tokens_per_sec": 22609.574303331578, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5462239583333334, "G_std": 0.20068359375, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.480369567871094, "grad_norm": 0.5269908308982849, "tokens_seen": 10330112, "tokens_per_sec": 22612.641264989175, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5277913411458334, "G_std": 0.19553629557291666, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.451297998428345, "grad_norm": 0.5328020453453064, "tokens_seen": 10493952, "tokens_per_sec": 22615.106223218507, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5511067708333334, "G_std": 0.19746907552083334, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.606449365615845, "grad_norm": 0.4956774413585663, "tokens_seen": 10657792, "tokens_per_sec": 22618.430097233842, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5299072265625, "G_std": 0.20517985026041666, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.472171783447266, "grad_norm": 0.5609950423240662, "tokens_seen": 10821632, "tokens_per_sec": 22622.942190546903, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.54345703125, "G_std": 0.19525146484375, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.666619777679443, "grad_norm": 0.5515887141227722, "tokens_seen": 10985472, "tokens_per_sec": 22626.59845528107, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5423583984375, "G_std": 0.19925944010416666, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.4766833782196045, "grad_norm": 0.5155245065689087, "tokens_seen": 11149312, "tokens_per_sec": 22629.742411861153, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5489095052083334, "G_std": 0.19578043619791666, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.494901180267334, "grad_norm": 0.5516836643218994, "tokens_seen": 11313152, "tokens_per_sec": 22630.88446333799, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5292561848958334, "G_std": 0.20314534505208334, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.393446445465088, "grad_norm": 0.5276585817337036, "tokens_seen": 11476992, "tokens_per_sec": 22633.983531783488, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5227457682291666, "G_std": 0.19932047526041666, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.4659483432769775, "grad_norm": 0.5682323575019836, "tokens_seen": 11640832, "tokens_per_sec": 22637.21057677041, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5234375, "G_std": 0.19878133138020834, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.4999308586120605, "grad_norm": 0.5132646560668945, "tokens_seen": 11804672, "tokens_per_sec": 22639.02603568677, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5244954427083334, "G_std": 0.19883219401041666, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.4337193965911865, "grad_norm": 0.5891045331954956, "tokens_seen": 11968512, "tokens_per_sec": 22640.320304653887, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5108235677083334, "G_std": 0.2012939453125, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.357651233673096, "grad_norm": 0.566250205039978, "tokens_seen": 12132352, "tokens_per_sec": 22643.331497631916, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5225626627604166, "G_std": 0.199188232421875, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.630319118499756, "grad_norm": 0.6014958620071411, "tokens_seen": 12296192, "tokens_per_sec": 22645.916312754, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5278523763020834, "G_std": 0.19713338216145834, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.416194677352905, "grad_norm": 0.6238242983818054, "tokens_seen": 12460032, "tokens_per_sec": 22648.516744989884, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5062459309895834, "G_std": 0.194549560546875, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.399068355560303, "grad_norm": 0.5099077820777893, "tokens_seen": 12623872, "tokens_per_sec": 22650.071670745914, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5170694986979166, "G_std": 0.19831339518229166, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.610186338424683, "grad_norm": 0.5133869647979736, "tokens_seen": 12787712, "tokens_per_sec": 22651.87019085987, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5211385091145834, "G_std": 0.19852701822916666, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.5002312660217285, "grad_norm": 0.5630433559417725, "tokens_seen": 12951552, "tokens_per_sec": 22653.768608164337, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5106404622395834, "G_std": 0.19378662109375, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.403512001037598, "grad_norm": 0.5581861734390259, "tokens_seen": 13115392, "tokens_per_sec": 22656.62522719541, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5182291666666666, "G_std": 0.198974609375, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.3833677768707275, "grad_norm": 0.5310951471328735, "tokens_seen": 13279232, "tokens_per_sec": 22658.38533940654, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5196126302083334, "G_std": 0.1959228515625, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.449187278747559, "grad_norm": 0.561756432056427, "tokens_seen": 13443072, "tokens_per_sec": 22659.8210421019, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5208943684895834, "G_std": 0.198150634765625, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.171303987503052, "grad_norm": 0.5506733655929565, "tokens_seen": 13606912, "tokens_per_sec": 22662.24908494909, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5033365885416666, "G_std": 0.19394938151041666, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.441040277481079, "grad_norm": 0.5672905445098877, "tokens_seen": 13770752, "tokens_per_sec": 22663.110802009844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5103352864583334, "G_std": 0.19341023763020834, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.509666681289673, "grad_norm": 0.5854852795600891, "tokens_seen": 13934592, "tokens_per_sec": 22665.24172462651, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.50445556640625, "G_std": 0.197662353515625, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.3031816482543945, "grad_norm": 0.5718931555747986, "tokens_seen": 14098432, "tokens_per_sec": 22667.163534556956, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5204264322916666, "G_std": 0.19853719075520834, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.3684821128845215, "grad_norm": 0.5489054918289185, "tokens_seen": 14262272, "tokens_per_sec": 22668.89901562679, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5171101888020834, "G_std": 0.1954345703125, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.244795322418213, "grad_norm": 0.5523470640182495, "tokens_seen": 14426112, "tokens_per_sec": 22670.372909176243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5015462239583334, "G_std": 0.19963582356770834, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.282311201095581, "grad_norm": 0.6078657507896423, "tokens_seen": 14589952, "tokens_per_sec": 22672.299880039463, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.50567626953125, "G_std": 0.19757080078125, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.4543962478637695, "grad_norm": 0.5608018040657043, "tokens_seen": 14753792, "tokens_per_sec": 22674.01599161701, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.50054931640625, "G_std": 0.19748942057291666, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.47899866104126, "grad_norm": 0.5640729069709778, "tokens_seen": 14917632, "tokens_per_sec": 22676.001190686897, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4886474609375, "G_std": 0.19352213541666666, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.4174089431762695, "grad_norm": 0.5715698599815369, "tokens_seen": 15081472, "tokens_per_sec": 22677.66004617637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5041097005208334, "G_std": 0.197479248046875, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.440298557281494, "grad_norm": 0.6205018162727356, "tokens_seen": 15245312, "tokens_per_sec": 22679.01689627104, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4883829752604167, "G_std": 0.19380696614583334, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.181677579879761, "grad_norm": 0.5692741870880127, "tokens_seen": 15409152, "tokens_per_sec": 22680.345471624572, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4864908854166667, "G_std": 0.1956787109375, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.243059873580933, "grad_norm": 0.6470932364463806, "tokens_seen": 15572992, "tokens_per_sec": 22682.49289555763, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5016682942708334, "G_std": 0.19207763671875, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.423802375793457, "grad_norm": 0.5787601470947266, "tokens_seen": 15736832, "tokens_per_sec": 22683.237555183998, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4935302734375, "G_std": 0.19354248046875, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.561117649078369, "grad_norm": 0.6567501425743103, "tokens_seen": 15900672, "tokens_per_sec": 22684.980836667884, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4972330729166667, "G_std": 0.19425455729166666, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.17571759223938, "grad_norm": 0.5662603378295898, "tokens_seen": 16064512, "tokens_per_sec": 22686.137553293294, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4930826822916667, "G_std": 0.19111124674479166, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.162286043167114, "grad_norm": 0.5962398648262024, "tokens_seen": 16228352, "tokens_per_sec": 22687.321103049566, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4913533528645833, "G_std": 0.19452921549479166, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.117932558059692, "grad_norm": 0.6098605990409851, "tokens_seen": 16392192, "tokens_per_sec": 22689.080928766238, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4912923177083333, "G_std": 0.19239298502604166, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.284428548812866, "val_ppl": 197.2414373832445, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4771525065104167, "G_std": 0.1900634765625, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.193026781082153, "grad_norm": 0.5936353206634521, "tokens_seen": 16556032, "tokens_per_sec": 22651.805195995636, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48016357421875, "G_std": 0.197723388671875, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.332354545593262, "grad_norm": 0.7963306307792664, "tokens_seen": 16719872, "tokens_per_sec": 22653.481588141112, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4796142578125, "G_std": 0.19336954752604166, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.3688108921051025, "grad_norm": 0.6992717981338501, "tokens_seen": 16883712, "tokens_per_sec": 22655.747903162046, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4782918294270833, "G_std": 0.1927490234375, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.39268684387207, "grad_norm": 0.580557644367218, "tokens_seen": 17047552, "tokens_per_sec": 22656.62149412167, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4844970703125, "G_std": 0.19554646809895834, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.2067177295684814, "grad_norm": 0.6192073225975037, "tokens_seen": 17211392, "tokens_per_sec": 22657.84726539105, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4794514973958333, "G_std": 0.19153849283854166, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.361120700836182, "grad_norm": 0.6827689409255981, "tokens_seen": 17375232, "tokens_per_sec": 22658.454583537423, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4737141927083333, "G_std": 0.19459025065104166, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.13432502746582, "grad_norm": 0.5788435339927673, "tokens_seen": 17539072, "tokens_per_sec": 22655.867550011546, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4730021158854167, "G_std": 0.19304402669270834, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.179521083831787, "grad_norm": 0.5977402925491333, "tokens_seen": 17702912, "tokens_per_sec": 22656.869073721035, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.475738525390625, "G_std": 0.19127400716145834, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.294147491455078, "grad_norm": 0.6067907810211182, "tokens_seen": 17866752, "tokens_per_sec": 22658.955543470707, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4795328776041667, "G_std": 0.19402058919270834, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.067959785461426, "grad_norm": 0.6101078987121582, "tokens_seen": 18030592, "tokens_per_sec": 22660.18217371092, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4567667643229167, "G_std": 0.19255574544270834, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.16466760635376, "grad_norm": 0.6146475076675415, "tokens_seen": 18194432, "tokens_per_sec": 22660.961035207172, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.475311279296875, "G_std": 0.196044921875, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.2876269817352295, "grad_norm": 0.5868560075759888, "tokens_seen": 18358272, "tokens_per_sec": 22662.288854463255, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4803873697916667, "G_std": 0.19316609700520834, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.072334289550781, "grad_norm": 0.636040985584259, "tokens_seen": 18522112, "tokens_per_sec": 22663.705309152832, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4657999674479167, "G_std": 0.19065348307291666, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.118056058883667, "grad_norm": 0.6453072428703308, "tokens_seen": 18685952, "tokens_per_sec": 22663.424726608166, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4778544108072917, "G_std": 0.193023681640625, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.3242199420928955, "grad_norm": 0.6402515172958374, "tokens_seen": 18849792, "tokens_per_sec": 22664.610453164078, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4591878255208333, "G_std": 0.19468180338541666, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.250844240188599, "grad_norm": 0.6476659774780273, "tokens_seen": 19013632, "tokens_per_sec": 22665.090819216242, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.466827392578125, "G_std": 0.19002278645833334, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.103859186172485, "grad_norm": 0.6146115064620972, "tokens_seen": 19177472, "tokens_per_sec": 22666.070540229324, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4684651692708333, "G_std": 0.18901570638020834, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.117624282836914, "grad_norm": 0.6772739887237549, "tokens_seen": 19341312, "tokens_per_sec": 22667.461929527948, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4695231119791667, "G_std": 0.19526163736979166, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.096008539199829, "grad_norm": 0.6639174818992615, "tokens_seen": 19505152, "tokens_per_sec": 22668.552876520906, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48406982421875, "G_std": 0.19109090169270834, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.081029176712036, "grad_norm": 0.6072087287902832, "tokens_seen": 19668992, "tokens_per_sec": 22669.820029217844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4647216796875, "G_std": 0.19626363118489584, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.232818365097046, "grad_norm": 0.6379767656326294, "tokens_seen": 19832832, "tokens_per_sec": 22671.67181698124, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4826558430989583, "G_std": 0.19432576497395834, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.106198310852051, "grad_norm": 0.6103435754776001, "tokens_seen": 19996672, "tokens_per_sec": 22672.379980372807, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4629007975260417, "G_std": 0.19400533040364584, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.044000864028931, "grad_norm": 0.621913731098175, "tokens_seen": 20160512, "tokens_per_sec": 22673.175461072486, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4675699869791667, "G_std": 0.1890411376953125, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.054963111877441, "grad_norm": 0.6049242615699768, "tokens_seen": 20324352, "tokens_per_sec": 22674.4054031524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4742838541666667, "G_std": 0.19133504231770834, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.142972230911255, "grad_norm": 0.6391798853874207, "tokens_seen": 20488192, "tokens_per_sec": 22675.768964743485, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.462432861328125, "G_std": 0.1898345947265625, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 4.973775386810303, "grad_norm": 0.6137213110923767, "tokens_seen": 20652032, "tokens_per_sec": 22676.613735991836, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.454742431640625, "G_std": 0.18654378255208334, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.070831298828125, "grad_norm": 0.6431013941764832, "tokens_seen": 20815872, "tokens_per_sec": 22677.980871776734, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4590047200520833, "G_std": 0.194122314453125, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.162186622619629, "grad_norm": 0.7337348461151123, "tokens_seen": 20979712, "tokens_per_sec": 22679.09480930336, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47381591796875, "G_std": 0.19249979654947916, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.150633335113525, "grad_norm": 0.7000817656517029, "tokens_seen": 21143552, "tokens_per_sec": 22679.349512246292, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4802144368489583, "G_std": 0.18940226236979166, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.018671751022339, "grad_norm": 0.5889642238616943, "tokens_seen": 21307392, "tokens_per_sec": 22680.618422585143, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47052001953125, "G_std": 0.19526163736979166, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.9771201610565186, "grad_norm": 0.6649610996246338, "tokens_seen": 21471232, "tokens_per_sec": 22681.599121800864, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4625040690104167, "G_std": 0.18910725911458334, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.093209981918335, "grad_norm": 0.6844274997711182, "tokens_seen": 21635072, "tokens_per_sec": 22683.333631965837, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4650675455729167, "G_std": 0.19228617350260416, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.1341776847839355, "grad_norm": 0.6411687135696411, "tokens_seen": 21798912, "tokens_per_sec": 22684.953653428896, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4666544596354167, "G_std": 0.18924967447916666, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.106099843978882, "grad_norm": 0.6916066408157349, "tokens_seen": 21962752, "tokens_per_sec": 22686.096337095627, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4715983072916667, "G_std": 0.19019063313802084, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.113662242889404, "grad_norm": 0.6402279138565063, "tokens_seen": 22126592, "tokens_per_sec": 22686.968174391393, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4589335123697917, "G_std": 0.18961588541666666, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 4.987201929092407, "grad_norm": 0.6673773527145386, "tokens_seen": 22290432, "tokens_per_sec": 22688.131924719568, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4524943033854167, "G_std": 0.18855794270833334, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.073860168457031, "grad_norm": 0.5868970155715942, "tokens_seen": 22454272, "tokens_per_sec": 22688.48493685936, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.450042724609375, "G_std": 0.19109598795572916, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.009464263916016, "grad_norm": 0.6323602199554443, "tokens_seen": 22618112, "tokens_per_sec": 22689.30387899519, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4508056640625, "G_std": 0.19169108072916666, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 4.9420647621154785, "grad_norm": 0.6300879716873169, "tokens_seen": 22781952, "tokens_per_sec": 22689.686543875658, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4373270670572917, "G_std": 0.19213358561197916, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.98596453666687, "grad_norm": 0.6598234176635742, "tokens_seen": 22945792, "tokens_per_sec": 22689.361888837393, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4506022135416667, "G_std": 0.19136555989583334, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.959754943847656, "grad_norm": 0.6562029719352722, "tokens_seen": 23109632, "tokens_per_sec": 22690.087536885985, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4437154134114583, "G_std": 0.1923980712890625, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.105981349945068, "grad_norm": 0.6985127925872803, "tokens_seen": 23273472, "tokens_per_sec": 22690.437873319308, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4638570149739583, "G_std": 0.18741861979166666, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.8671488761901855, "grad_norm": 0.694798469543457, "tokens_seen": 23437312, "tokens_per_sec": 22691.386420261846, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4420267740885417, "G_std": 0.19128926595052084, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.035286903381348, "grad_norm": 0.6557915806770325, "tokens_seen": 23601152, "tokens_per_sec": 22691.097785899856, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4574686686197917, "G_std": 0.19067891438802084, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.993108510971069, "grad_norm": 0.6856927871704102, "tokens_seen": 23764992, "tokens_per_sec": 22690.64094661468, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4514058430989583, "G_std": 0.19214375813802084, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.042448282241821, "grad_norm": 0.6115302443504333, "tokens_seen": 23928832, "tokens_per_sec": 22691.411349397196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.447021484375, "G_std": 0.19136555989583334, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.127472400665283, "grad_norm": 0.6616790890693665, "tokens_seen": 24092672, "tokens_per_sec": 22692.11051293204, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4496663411458333, "G_std": 0.19266255696614584, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.884561538696289, "grad_norm": 0.642409086227417, "tokens_seen": 24256512, "tokens_per_sec": 22692.70740890151, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4459431966145833, "G_std": 0.1902313232421875, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.98357367515564, "grad_norm": 0.6597146987915039, "tokens_seen": 24420352, "tokens_per_sec": 22693.23845073428, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4384256998697917, "G_std": 0.18948872884114584, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.880458831787109, "grad_norm": 0.6527066826820374, "tokens_seen": 24584192, "tokens_per_sec": 22693.74041773867, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4494120279947917, "G_std": 0.19113667805989584, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.039802861213684, "val_ppl": 154.43956599604746, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.446563720703125, "G_std": 0.19324747721354166, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.061682939529419, "grad_norm": 0.6643278002738953, "tokens_seen": 24748032, "tokens_per_sec": 22667.616149229732, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4556986490885417, "G_std": 0.1911468505859375, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.027410507202148, "grad_norm": 0.7005634903907776, "tokens_seen": 24911872, "tokens_per_sec": 22668.560116263077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4429524739583333, "G_std": 0.19234212239583334, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.951792478561401, "grad_norm": 0.6806544661521912, "tokens_seen": 25075712, "tokens_per_sec": 22669.824555002077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4340108235677083, "G_std": 0.1891937255859375, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.844560146331787, "grad_norm": 0.6414554715156555, "tokens_seen": 25239552, "tokens_per_sec": 22669.019829300873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4388326009114583, "G_std": 0.189361572265625, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.975668430328369, "grad_norm": 0.6731707453727722, "tokens_seen": 25403392, "tokens_per_sec": 22669.41887424667, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4311014811197917, "G_std": 0.18990071614583334, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.725084543228149, "grad_norm": 0.7105466723442078, "tokens_seen": 25567232, "tokens_per_sec": 22670.59653587319, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4437255859375, "G_std": 0.19202677408854166, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.869819641113281, "grad_norm": 0.6407650113105774, "tokens_seen": 25731072, "tokens_per_sec": 22671.333119560604, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4434814453125, "G_std": 0.1884002685546875, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.839755535125732, "grad_norm": 0.634108304977417, "tokens_seen": 25894912, "tokens_per_sec": 22672.151335100374, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.445220947265625, "G_std": 0.18971761067708334, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.947533369064331, "grad_norm": 0.6184845566749573, "tokens_seen": 26058752, "tokens_per_sec": 22672.70975592478, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4402974446614583, "G_std": 0.1888580322265625, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.882340908050537, "grad_norm": 0.6654408574104309, "tokens_seen": 26222592, "tokens_per_sec": 22673.17052159986, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4231160481770833, "G_std": 0.18932088216145834, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.937117099761963, "grad_norm": 0.6891507506370544, "tokens_seen": 26386432, "tokens_per_sec": 22673.79532396776, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4395345052083333, "G_std": 0.19141642252604166, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.988398313522339, "grad_norm": 0.6822013854980469, "tokens_seen": 26550272, "tokens_per_sec": 22675.190054718947, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4449666341145833, "G_std": 0.18713887532552084, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.075250864028931, "grad_norm": 0.6683701276779175, "tokens_seen": 26714112, "tokens_per_sec": 22676.321793572082, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4401652018229167, "G_std": 0.18953959147135416, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.840641260147095, "grad_norm": 0.7215304970741272, "tokens_seen": 26877952, "tokens_per_sec": 22677.335929552995, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4371134440104167, "G_std": 0.19055684407552084, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.761971950531006, "grad_norm": 0.6910153031349182, "tokens_seen": 27041792, "tokens_per_sec": 22678.54323403108, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.430328369140625, "G_std": 0.1894989013671875, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.84899115562439, "grad_norm": 0.7399301528930664, "tokens_seen": 27205632, "tokens_per_sec": 22679.465681140045, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44268798828125, "G_std": 0.18750508626302084, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.895944595336914, "grad_norm": 0.6606634259223938, "tokens_seen": 27369472, "tokens_per_sec": 22679.295847460246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4423828125, "G_std": 0.19053141276041666, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.852521896362305, "grad_norm": 0.6982266902923584, "tokens_seen": 27533312, "tokens_per_sec": 22680.356846524002, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4323628743489583, "G_std": 0.18779500325520834, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.9353721141815186, "grad_norm": 0.6562835574150085, "tokens_seen": 27697152, "tokens_per_sec": 22681.222163442108, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4411214192708333, "G_std": 0.18929545084635416, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.869904279708862, "grad_norm": 0.6905900239944458, "tokens_seen": 27860992, "tokens_per_sec": 22682.357764244833, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4502461751302083, "G_std": 0.19028727213541666, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.992946147918701, "grad_norm": 0.661593496799469, "tokens_seen": 28024832, "tokens_per_sec": 22683.295574592918, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4383341471354167, "G_std": 0.18997701009114584, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.918418884277344, "grad_norm": 0.6717556715011597, "tokens_seen": 28188672, "tokens_per_sec": 22683.617635460236, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4378763834635417, "G_std": 0.19129435221354166, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.9176130294799805, "grad_norm": 0.7716364860534668, "tokens_seen": 28352512, "tokens_per_sec": 22684.773328897827, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4361470540364583, "G_std": 0.19152323404947916, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.948995351791382, "grad_norm": 0.7063320279121399, "tokens_seen": 28516352, "tokens_per_sec": 22685.661328633087, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4353129069010417, "G_std": 0.189666748046875, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.848150730133057, "grad_norm": 0.6897032260894775, "tokens_seen": 28680192, "tokens_per_sec": 22686.23434144254, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4349873860677083, "G_std": 0.19019063313802084, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.903554439544678, "grad_norm": 0.6899807453155518, "tokens_seen": 28844032, "tokens_per_sec": 22686.706282785104, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4334055582682292, "G_std": 0.18796284993489584, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.872207403182983, "grad_norm": 0.6669788360595703, "tokens_seen": 29007872, "tokens_per_sec": 22687.440600406822, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.428955078125, "G_std": 0.18828328450520834, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.801256895065308, "grad_norm": 0.6807920336723328, "tokens_seen": 29171712, "tokens_per_sec": 22687.38934024873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4402364095052083, "G_std": 0.18755086263020834, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.885678768157959, "grad_norm": 0.7485401034355164, "tokens_seen": 29335552, "tokens_per_sec": 22687.81125190203, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4342702229817708, "G_std": 0.18924967447916666, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.744303226470947, "grad_norm": 0.6702672839164734, "tokens_seen": 29499392, "tokens_per_sec": 22688.658995278405, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4315745035807292, "G_std": 0.18845113118489584, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.852118015289307, "grad_norm": 0.7221682667732239, "tokens_seen": 29663232, "tokens_per_sec": 22688.927999867246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.426422119140625, "G_std": 0.19083150227864584, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.785208702087402, "grad_norm": 0.703192949295044, "tokens_seen": 29827072, "tokens_per_sec": 22689.92508442567, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4347381591796875, "G_std": 0.18971761067708334, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.8483076095581055, "grad_norm": 0.6631271839141846, "tokens_seen": 29990912, "tokens_per_sec": 22690.66809737534, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4323628743489583, "G_std": 0.18942769368489584, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.871558666229248, "grad_norm": 0.664389967918396, "tokens_seen": 30154752, "tokens_per_sec": 22691.176137296927, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4336903889973958, "G_std": 0.18546549479166666, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.79292631149292, "grad_norm": 0.7008147239685059, "tokens_seen": 30318592, "tokens_per_sec": 22691.40992908837, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4185384114583333, "G_std": 0.18821207682291666, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.96545934677124, "grad_norm": 0.7028505206108093, "tokens_seen": 30482432, "tokens_per_sec": 22692.001092449813, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4273681640625, "G_std": 0.18929545084635416, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.856369972229004, "grad_norm": 0.7076614499092102, "tokens_seen": 30646272, "tokens_per_sec": 22692.207079081338, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4201253255208333, "G_std": 0.18750508626302084, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.812169075012207, "grad_norm": 0.7376694083213806, "tokens_seen": 30810112, "tokens_per_sec": 22693.21448475239, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4113413492838542, "G_std": 0.18616231282552084, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.7866997718811035, "grad_norm": 0.7078622579574585, "tokens_seen": 30973952, "tokens_per_sec": 22694.11698129282, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4171905517578125, "G_std": 0.1859588623046875, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.892408132553101, "grad_norm": 0.6709064841270447, "tokens_seen": 31137792, "tokens_per_sec": 22694.821527920303, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4329630533854167, "G_std": 0.19036356608072916, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.831104278564453, "grad_norm": 0.6772249937057495, "tokens_seen": 31301632, "tokens_per_sec": 22695.508651324053, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42279052734375, "G_std": 0.18770853678385416, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.9876182079315186, "grad_norm": 0.6920614242553711, "tokens_seen": 31465472, "tokens_per_sec": 22695.993283308017, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4304351806640625, "G_std": 0.18618265787760416, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.640578031539917, "grad_norm": 0.7024469971656799, "tokens_seen": 31629312, "tokens_per_sec": 22696.865360672768, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41705322265625, "G_std": 0.18558247884114584, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.7461981773376465, "grad_norm": 0.6953504681587219, "tokens_seen": 31793152, "tokens_per_sec": 22697.595989920475, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4259745279947917, "G_std": 0.18996175130208334, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.774219512939453, "grad_norm": 0.6501219868659973, "tokens_seen": 31956992, "tokens_per_sec": 22698.296068673593, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4114583333333333, "G_std": 0.18615214029947916, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.79320502281189, "grad_norm": 0.6838524341583252, "tokens_seen": 32120832, "tokens_per_sec": 22699.119855736448, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4309844970703125, "G_std": 0.18825785319010416, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.917952537536621, "grad_norm": 0.7077226638793945, "tokens_seen": 32284672, "tokens_per_sec": 22699.793432805287, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4263661702473958, "G_std": 0.18976847330729166, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.7003583908081055, "grad_norm": 0.7152302265167236, "tokens_seen": 32448512, "tokens_per_sec": 22700.42385711783, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.421295166015625, "G_std": 0.18756103515625, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.6387012004852295, "grad_norm": 0.7336569428443909, "tokens_seen": 32612352, "tokens_per_sec": 22700.986643579563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4190572102864583, "G_std": 0.1857147216796875, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.866362810134888, "grad_norm": 0.7034605145454407, "tokens_seen": 32776192, "tokens_per_sec": 22701.934290405683, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4163258870442708, "G_std": 0.18742879231770834, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.864228057861328, "val_ppl": 129.57087877530427, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4175821940104167, "G_std": 0.188201904296875, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.88962459564209, "grad_norm": 0.7273709177970886, "tokens_seen": 32940032, "tokens_per_sec": 22682.781272026954, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4218953450520833, "G_std": 0.1876678466796875, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.673688650131226, "grad_norm": 0.7415040135383606, "tokens_seen": 33103872, "tokens_per_sec": 22682.974169222016, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4279937744140625, "G_std": 0.1876373291015625, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.704356908798218, "grad_norm": 0.7530417442321777, "tokens_seen": 33267712, "tokens_per_sec": 22683.511030595866, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4234059651692708, "G_std": 0.1846771240234375, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.823770999908447, "grad_norm": 0.7379795908927917, "tokens_seen": 33431552, "tokens_per_sec": 22683.720629520823, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4126180013020833, "G_std": 0.185760498046875, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.726752758026123, "grad_norm": 0.7273500561714172, "tokens_seen": 33595392, "tokens_per_sec": 22684.480859254247, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4149983723958333, "G_std": 0.18768310546875, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.90969705581665, "grad_norm": 0.72911137342453, "tokens_seen": 33759232, "tokens_per_sec": 22684.963697518997, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4195810953776042, "G_std": 0.18743387858072916, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.831843137741089, "grad_norm": 0.732568085193634, "tokens_seen": 33923072, "tokens_per_sec": 22685.681190293897, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4228159586588542, "G_std": 0.1890411376953125, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.749246835708618, "grad_norm": 0.7183780670166016, "tokens_seen": 34086912, "tokens_per_sec": 22685.81285047731, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4200541178385417, "G_std": 0.18668619791666666, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.818979740142822, "grad_norm": 0.6724742650985718, "tokens_seen": 34250752, "tokens_per_sec": 22686.400038593994, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4147135416666667, "G_std": 0.18646240234375, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.657435894012451, "grad_norm": 0.7274091839790344, "tokens_seen": 34414592, "tokens_per_sec": 22686.968985938558, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4092508951822917, "G_std": 0.18859354654947916, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.703033924102783, "grad_norm": 0.6884642243385315, "tokens_seen": 34578432, "tokens_per_sec": 22687.67164641083, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4293212890625, "G_std": 0.1851806640625, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.7373034954071045, "grad_norm": 0.7185440063476562, "tokens_seen": 34742272, "tokens_per_sec": 22688.500204503332, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4128824869791667, "G_std": 0.1849365234375, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.632127285003662, "grad_norm": 0.7160308361053467, "tokens_seen": 34906112, "tokens_per_sec": 22688.8488196213, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.413116455078125, "G_std": 0.18882242838541666, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.718679904937744, "grad_norm": 0.7104653716087341, "tokens_seen": 35069952, "tokens_per_sec": 22689.6418700654, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4183451334635417, "G_std": 0.18891398111979166, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.749356269836426, "grad_norm": 0.8152555823326111, "tokens_seen": 35233792, "tokens_per_sec": 22690.644331880983, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4118448893229167, "G_std": 0.18570963541666666, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.845882415771484, "grad_norm": 0.6946055889129639, "tokens_seen": 35397632, "tokens_per_sec": 22691.156718003556, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4234466552734375, "G_std": 0.18722025553385416, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.79676365852356, "grad_norm": 0.7342365384101868, "tokens_seen": 35561472, "tokens_per_sec": 22691.911663559807, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4102274576822917, "G_std": 0.18505859375, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.678110122680664, "grad_norm": 0.7224200963973999, "tokens_seen": 35725312, "tokens_per_sec": 22692.69018287484, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4152170817057292, "G_std": 0.18776448567708334, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.656900882720947, "grad_norm": 0.7216212749481201, "tokens_seen": 35889152, "tokens_per_sec": 22693.24241155331, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4015655517578125, "G_std": 0.18456013997395834, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.719619035720825, "grad_norm": 0.7531205415725708, "tokens_seen": 36052992, "tokens_per_sec": 22694.04580382456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4190521240234375, "G_std": 0.1859283447265625, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.745302200317383, "grad_norm": 0.7248569130897522, "tokens_seen": 36216832, "tokens_per_sec": 22694.232478105252, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4155324300130208, "G_std": 0.18402862548828125, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.742037057876587, "grad_norm": 0.7518880367279053, "tokens_seen": 36380672, "tokens_per_sec": 22695.107263500682, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4206492106119792, "G_std": 0.1866302490234375, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.891345739364624, "grad_norm": 0.7376968860626221, "tokens_seen": 36544512, "tokens_per_sec": 22696.041749080436, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4086659749348958, "G_std": 0.18563079833984375, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.832027912139893, "grad_norm": 0.7714853882789612, "tokens_seen": 36708352, "tokens_per_sec": 22696.82330376258, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.416015625, "G_std": 0.18850199381510416, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.691762924194336, "grad_norm": 0.7191632986068726, "tokens_seen": 36872192, "tokens_per_sec": 22697.57937452688, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.412353515625, "G_std": 0.18622843424479166, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.807344913482666, "grad_norm": 0.7483969330787659, "tokens_seen": 37036032, "tokens_per_sec": 22698.347223118726, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4203338623046875, "G_std": 0.18441263834635416, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.7653727531433105, "grad_norm": 0.7471747398376465, "tokens_seen": 37199872, "tokens_per_sec": 22698.951534240845, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4130605061848958, "G_std": 0.18645731608072916, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.771628379821777, "grad_norm": 0.7561524510383606, "tokens_seen": 37363712, "tokens_per_sec": 22699.564422853593, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4023030598958333, "G_std": 0.1868413289388021, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.604578971862793, "grad_norm": 0.784246563911438, "tokens_seen": 37527552, "tokens_per_sec": 22700.28002748325, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4112294514973958, "G_std": 0.18491109212239584, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.606788158416748, "grad_norm": 0.7088251709938049, "tokens_seen": 37691392, "tokens_per_sec": 22700.463574711666, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.413970947265625, "G_std": 0.1850433349609375, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.685198545455933, "grad_norm": 0.724109411239624, "tokens_seen": 37855232, "tokens_per_sec": 22701.359852690188, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.407440185546875, "G_std": 0.18493143717447916, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.693547487258911, "grad_norm": 0.7229140400886536, "tokens_seen": 38019072, "tokens_per_sec": 22702.085090904362, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4008026123046875, "G_std": 0.18555704752604166, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.850066661834717, "grad_norm": 0.7704138159751892, "tokens_seen": 38182912, "tokens_per_sec": 22702.60091871953, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4084421793619792, "G_std": 0.1829376220703125, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.874110698699951, "grad_norm": 0.7576395869255066, "tokens_seen": 38346752, "tokens_per_sec": 22703.545114896962, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4175923665364583, "G_std": 0.1877466837565104, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.702857732772827, "grad_norm": 0.7204440832138062, "tokens_seen": 38510592, "tokens_per_sec": 22704.249197615085, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4096221923828125, "G_std": 0.18656412760416666, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.571524620056152, "grad_norm": 0.7918305993080139, "tokens_seen": 38674432, "tokens_per_sec": 22704.619276184563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4029693603515625, "G_std": 0.18369038899739584, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.880276679992676, "grad_norm": 0.7229289412498474, "tokens_seen": 38838272, "tokens_per_sec": 22705.122694389982, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4084320068359375, "G_std": 0.18733978271484375, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.5275421142578125, "grad_norm": 0.7543634176254272, "tokens_seen": 39002112, "tokens_per_sec": 22705.8828532004, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4013570149739583, "G_std": 0.18418375651041666, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.513623952865601, "grad_norm": 0.7637909054756165, "tokens_seen": 39165952, "tokens_per_sec": 22706.45915593535, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.404266357421875, "G_std": 0.183441162109375, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.777431488037109, "grad_norm": 0.736571192741394, "tokens_seen": 39329792, "tokens_per_sec": 22706.992084433798, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4099934895833333, "G_std": 0.18551127115885416, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.682717323303223, "grad_norm": 0.7409370541572571, "tokens_seen": 39493632, "tokens_per_sec": 22707.60118191749, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4095611572265625, "G_std": 0.18314615885416666, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.7500598430633545, "grad_norm": 0.7326306700706482, "tokens_seen": 39657472, "tokens_per_sec": 22708.079336540824, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3989461263020833, "G_std": 0.18677266438802084, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.757736921310425, "grad_norm": 0.76226407289505, "tokens_seen": 39821312, "tokens_per_sec": 22708.346377865066, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4147796630859375, "G_std": 0.1862945556640625, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.630368947982788, "grad_norm": 0.7882533669471741, "tokens_seen": 39985152, "tokens_per_sec": 22708.977773023897, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4041035970052083, "G_std": 0.185333251953125, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.690674304962158, "grad_norm": 0.7366514205932617, "tokens_seen": 40148992, "tokens_per_sec": 22709.560925849524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4107716878255208, "G_std": 0.18295796712239584, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.669296741485596, "grad_norm": 0.8091769218444824, "tokens_seen": 40312832, "tokens_per_sec": 22710.14599940524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4105733235677083, "G_std": 0.18486531575520834, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.649401664733887, "grad_norm": 0.7030816674232483, "tokens_seen": 40476672, "tokens_per_sec": 22710.36317420124, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4005584716796875, "G_std": 0.184722900390625, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.6317009925842285, "grad_norm": 0.7208169102668762, "tokens_seen": 40640512, "tokens_per_sec": 22710.55065295037, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4045155843098958, "G_std": 0.18552144368489584, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.635048866271973, "grad_norm": 0.7662695646286011, "tokens_seen": 40804352, "tokens_per_sec": 22711.122437070062, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4083455403645833, "G_std": 0.18503316243489584, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.545424461364746, "grad_norm": 0.7204164266586304, "tokens_seen": 40968192, "tokens_per_sec": 22711.45816825704, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3948872884114583, "G_std": 0.1864039103190104, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.73875138759613, "val_ppl": 114.29140688274332, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4043731689453125, "G_std": 0.1851781209309896, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.808262825012207, "grad_norm": 0.7519007325172424, "tokens_seen": 41132032, "tokens_per_sec": 22695.941903398172, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4080098470052083, "G_std": 0.18535359700520834, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.616508722305298, "grad_norm": 0.7957213521003723, "tokens_seen": 41295872, "tokens_per_sec": 22695.840365512242, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4147288004557292, "G_std": 0.18891398111979166, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.636056661605835, "grad_norm": 0.7427785992622375, "tokens_seen": 41459712, "tokens_per_sec": 22695.602233980513, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4031219482421875, "G_std": 0.18231964111328125, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.5488762855529785, "grad_norm": 0.7388110756874084, "tokens_seen": 41623552, "tokens_per_sec": 22694.49931086051, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4016520182291667, "G_std": 0.18533833821614584, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.654137849807739, "grad_norm": 0.7392054200172424, "tokens_seen": 41787392, "tokens_per_sec": 22694.68672494184, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4025115966796875, "G_std": 0.1828180948893229, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.488805770874023, "grad_norm": 0.7558677196502686, "tokens_seen": 41951232, "tokens_per_sec": 22694.403149109392, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4011383056640625, "G_std": 0.184295654296875, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.793547630310059, "grad_norm": 0.8420838713645935, "tokens_seen": 42115072, "tokens_per_sec": 22694.471300701873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4087371826171875, "G_std": 0.1868616739908854, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.83801794052124, "grad_norm": 0.8174188137054443, "tokens_seen": 42278912, "tokens_per_sec": 22694.008950413154, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4083455403645833, "G_std": 0.18634541829427084, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.624372959136963, "grad_norm": 0.7260066866874695, "tokens_seen": 42442752, "tokens_per_sec": 22692.94289554341, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4011891682942708, "G_std": 0.18526458740234375, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.654522657394409, "grad_norm": 0.7940670847892761, "tokens_seen": 42606592, "tokens_per_sec": 22691.840381727565, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4001871744791667, "G_std": 0.1836090087890625, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.730146884918213, "grad_norm": 0.7654042840003967, "tokens_seen": 42770432, "tokens_per_sec": 22692.1367467438, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4016164143880208, "G_std": 0.18312835693359375, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.599946975708008, "grad_norm": 0.7555034756660461, "tokens_seen": 42934272, "tokens_per_sec": 22692.01696203732, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3951466878255208, "G_std": 0.18232218424479166, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.457312107086182, "grad_norm": 0.7642599940299988, "tokens_seen": 43098112, "tokens_per_sec": 22692.02026703619, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4032948811848958, "G_std": 0.18221537272135416, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.8022661209106445, "grad_norm": 0.7669584155082703, "tokens_seen": 43261952, "tokens_per_sec": 22692.172055263098, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.412322998046875, "G_std": 0.18359375, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.684141635894775, "grad_norm": 0.7798404097557068, "tokens_seen": 43425792, "tokens_per_sec": 22692.3674287146, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4116058349609375, "G_std": 0.18735758463541666, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.725244760513306, "grad_norm": 0.7885032892227173, "tokens_seen": 43589632, "tokens_per_sec": 22692.44035636758, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39617919921875, "G_std": 0.18520100911458334, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.471757888793945, "grad_norm": 0.7311140894889832, "tokens_seen": 43753472, "tokens_per_sec": 22693.06558234026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3972371419270833, "G_std": 0.18398539225260416, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.721540212631226, "grad_norm": 0.7621083855628967, "tokens_seen": 43917312, "tokens_per_sec": 22692.890899524205, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4118601481119792, "G_std": 0.18413798014322916, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.487545967102051, "grad_norm": 0.8374825716018677, "tokens_seen": 44081152, "tokens_per_sec": 22693.102679161027, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4015909830729167, "G_std": 0.18137613932291666, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.556821584701538, "grad_norm": 0.7848896980285645, "tokens_seen": 44244992, "tokens_per_sec": 22693.46779062638, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3969828287760417, "G_std": 0.1822382609049479, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.693598508834839, "grad_norm": 0.7609805464744568, "tokens_seen": 44408832, "tokens_per_sec": 22693.620887675344, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4098307291666667, "G_std": 0.18386332194010416, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.534003973007202, "grad_norm": 0.7286794185638428, "tokens_seen": 44572672, "tokens_per_sec": 22693.7934201035, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3980763753255208, "G_std": 0.18373870849609375, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.772885084152222, "grad_norm": 0.8379713892936707, "tokens_seen": 44736512, "tokens_per_sec": 22693.817204583036, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3990987141927083, "G_std": 0.18378702799479166, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.53066611289978, "grad_norm": 0.7768651247024536, "tokens_seen": 44900352, "tokens_per_sec": 22693.062779539876, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4024149576822917, "G_std": 0.1833826700846354, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.887481689453125, "grad_norm": 0.770111083984375, "tokens_seen": 45064192, "tokens_per_sec": 22693.085550259573, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4044138590494792, "G_std": 0.1833038330078125, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.606288433074951, "grad_norm": 0.8432204723358154, "tokens_seen": 45228032, "tokens_per_sec": 22693.006773659577, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4057871500651042, "G_std": 0.18172709147135416, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.580265283584595, "grad_norm": 0.7567086219787598, "tokens_seen": 45391872, "tokens_per_sec": 22692.833342457616, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4007008870442708, "G_std": 0.18291219075520834, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.650269508361816, "grad_norm": 0.7318214774131775, "tokens_seen": 45555712, "tokens_per_sec": 22692.42953427082, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4074554443359375, "G_std": 0.18462626139322916, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.597469329833984, "grad_norm": 0.775412380695343, "tokens_seen": 45719552, "tokens_per_sec": 22692.155201758425, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3935648600260417, "G_std": 0.18422953287760416, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.756185531616211, "grad_norm": 0.7552761435508728, "tokens_seen": 45883392, "tokens_per_sec": 22691.282715635618, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3907368977864583, "G_std": 0.18083953857421875, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.656281232833862, "grad_norm": 0.7907607555389404, "tokens_seen": 46047232, "tokens_per_sec": 22691.042549655813, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3994038899739583, "G_std": 0.1835454305013021, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.522066116333008, "grad_norm": 0.7929607033729553, "tokens_seen": 46211072, "tokens_per_sec": 22691.28581655934, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4011077880859375, "G_std": 0.1845830281575521, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.556985378265381, "grad_norm": 0.7641923427581787, "tokens_seen": 46374912, "tokens_per_sec": 22691.15033199694, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4034372965494792, "G_std": 0.18552398681640625, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.6599297523498535, "grad_norm": 0.7996082901954651, "tokens_seen": 46538752, "tokens_per_sec": 22691.47136696977, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4017232259114583, "G_std": 0.18456522623697916, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.56949257850647, "grad_norm": 0.7784314155578613, "tokens_seen": 46702592, "tokens_per_sec": 22691.595459539658, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3835652669270833, "G_std": 0.181243896484375, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.521953821182251, "grad_norm": 0.764277994632721, "tokens_seen": 46866432, "tokens_per_sec": 22691.75402249334, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3983052571614583, "G_std": 0.18394978841145834, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.5552451610565186, "grad_norm": 0.7720563411712646, "tokens_seen": 47030272, "tokens_per_sec": 22691.46308741266, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4013519287109375, "G_std": 0.18171183268229166, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.337948799133301, "grad_norm": 0.7440731525421143, "tokens_seen": 47194112, "tokens_per_sec": 22691.532291595584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.384979248046875, "G_std": 0.1828638712565104, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.528970956802368, "grad_norm": 0.726098358631134, "tokens_seen": 47357952, "tokens_per_sec": 22691.480558571613, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3918304443359375, "G_std": 0.18236287434895834, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.629834175109863, "grad_norm": 0.7714294195175171, "tokens_seen": 47521792, "tokens_per_sec": 22691.24399146171, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3967437744140625, "G_std": 0.18324788411458334, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.598567962646484, "grad_norm": 0.7990182042121887, "tokens_seen": 47685632, "tokens_per_sec": 22691.567919552468, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3949228922526042, "G_std": 0.18234761555989584, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.59889817237854, "grad_norm": 0.7605514526367188, "tokens_seen": 47849472, "tokens_per_sec": 22691.856171673422, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3910115559895833, "G_std": 0.18207804361979166, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.636566877365112, "grad_norm": 0.7971169352531433, "tokens_seen": 48013312, "tokens_per_sec": 22692.093832453676, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3939768473307292, "G_std": 0.1830317179361979, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.633886098861694, "grad_norm": 0.7807011604309082, "tokens_seen": 48177152, "tokens_per_sec": 22691.617064949747, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.398468017578125, "G_std": 0.17972310384114584, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.7860329151153564, "grad_norm": 0.7664968371391296, "tokens_seen": 48340992, "tokens_per_sec": 22692.130132428694, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4014078776041667, "G_std": 0.1824213663736979, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.62197208404541, "grad_norm": 0.8118278384208679, "tokens_seen": 48504832, "tokens_per_sec": 22692.157779932535, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3944803873697917, "G_std": 0.18346659342447916, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.6032936573028564, "grad_norm": 0.8031115531921387, "tokens_seen": 48668672, "tokens_per_sec": 22692.15357053282, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3946787516276042, "G_std": 0.1826324462890625, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.641209840774536, "grad_norm": 0.7362129092216492, "tokens_seen": 48832512, "tokens_per_sec": 22692.213697690026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3865254720052083, "G_std": 0.18047587076822916, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.508579254150391, "grad_norm": 0.84260493516922, "tokens_seen": 48996352, "tokens_per_sec": 22692.65391032177, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3955434163411458, "G_std": 0.1812260945638021, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.53187370300293, "grad_norm": 0.8052374720573425, "tokens_seen": 49160192, "tokens_per_sec": 22692.69131025891, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3934682210286458, "G_std": 0.18169148763020834, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.62666265964508, "val_ppl": 102.17251004548747, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3984375, "G_std": 0.18129730224609375, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.46746301651001, "grad_norm": 0.7654880881309509, "tokens_seen": 49324032, "tokens_per_sec": 22679.74695812947, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3888753255208333, "G_std": 0.18082427978515625, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.607300519943237, "grad_norm": 0.7759965658187866, "tokens_seen": 49487872, "tokens_per_sec": 22680.14782902568, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3993123372395833, "G_std": 0.18088277180989584, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.5062689781188965, "grad_norm": 0.8014824986457825, "tokens_seen": 49651712, "tokens_per_sec": 22680.14530079078, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4021504720052083, "G_std": 0.1826807657877604, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.600592613220215, "grad_norm": 0.8103574514389038, "tokens_seen": 49815552, "tokens_per_sec": 22680.188260184535, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3978627522786458, "G_std": 0.1821161905924479, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.5592591762542725, "grad_norm": 0.81181401014328, "tokens_seen": 49979392, "tokens_per_sec": 22680.265421778306, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4018046061197917, "G_std": 0.18320719401041666, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.453798770904541, "grad_norm": 0.7598201632499695, "tokens_seen": 50143232, "tokens_per_sec": 22680.495092055975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3939971923828125, "G_std": 0.18252054850260416, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.533521413803101, "grad_norm": 0.8361908197402954, "tokens_seen": 50307072, "tokens_per_sec": 22680.679817016662, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853607177734375, "G_std": 0.18095906575520834, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.623756170272827, "grad_norm": 0.7535346746444702, "tokens_seen": 50470912, "tokens_per_sec": 22680.783916773125, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4012400309244792, "G_std": 0.18213399251302084, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.515127658843994, "grad_norm": 0.7845277190208435, "tokens_seen": 50634752, "tokens_per_sec": 22680.84535489666, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3979288736979167, "G_std": 0.18251546223958334, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.5916595458984375, "grad_norm": 0.8036597371101379, "tokens_seen": 50798592, "tokens_per_sec": 22681.239545681034, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3930409749348958, "G_std": 0.18198140462239584, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.489260196685791, "grad_norm": 0.8002414107322693, "tokens_seen": 50962432, "tokens_per_sec": 22681.16693556863, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38916015625, "G_std": 0.1829401652018229, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.435343980789185, "grad_norm": 0.7831129431724548, "tokens_seen": 51126272, "tokens_per_sec": 22681.073685212985, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3851267496744792, "G_std": 0.18187967936197916, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.439287900924683, "grad_norm": 0.8299136161804199, "tokens_seen": 51290112, "tokens_per_sec": 22681.189543188095, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3953653971354167, "G_std": 0.18335978190104166, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.7338972091674805, "grad_norm": 0.8003802299499512, "tokens_seen": 51453952, "tokens_per_sec": 22680.78140688311, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3922932942708333, "G_std": 0.1824467976888021, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.720793724060059, "grad_norm": 0.8220890164375305, "tokens_seen": 51617792, "tokens_per_sec": 22680.77993853763, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3865966796875, "G_std": 0.1833038330078125, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.499884366989136, "grad_norm": 0.8183959126472473, "tokens_seen": 51781632, "tokens_per_sec": 22681.096033760252, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899993896484375, "G_std": 0.1824188232421875, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.618325233459473, "grad_norm": 0.8211345076560974, "tokens_seen": 51945472, "tokens_per_sec": 22680.93098167326, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3922627766927083, "G_std": 0.1815185546875, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.499445676803589, "grad_norm": 0.7566724419593811, "tokens_seen": 52109312, "tokens_per_sec": 22681.114334946124, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3840484619140625, "G_std": 0.18271891276041666, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.537006139755249, "grad_norm": 0.7773487567901611, "tokens_seen": 52273152, "tokens_per_sec": 22681.34513763292, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899383544921875, "G_std": 0.1817779541015625, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.374337434768677, "grad_norm": 0.8117662668228149, "tokens_seen": 52436992, "tokens_per_sec": 22681.459826562415, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3812357584635417, "G_std": 0.1805241902669271, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.572246551513672, "grad_norm": 0.8163625001907349, "tokens_seen": 52600832, "tokens_per_sec": 22681.498274984628, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4014180501302083, "G_std": 0.18225606282552084, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.728162527084351, "grad_norm": 0.8297780752182007, "tokens_seen": 52764672, "tokens_per_sec": 22681.287968184486, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3953908284505208, "G_std": 0.180023193359375, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.5161497592926025, "grad_norm": 0.8299847841262817, "tokens_seen": 52928512, "tokens_per_sec": 22681.42606349055, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39398193359375, "G_std": 0.18119303385416666, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.589750289916992, "grad_norm": 0.9654127359390259, "tokens_seen": 53092352, "tokens_per_sec": 22681.6038665199, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.390960693359375, "G_std": 0.18056488037109375, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.3882036209106445, "grad_norm": 0.8164151310920715, "tokens_seen": 53256192, "tokens_per_sec": 22682.081573184096, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.389617919921875, "G_std": 0.18152872721354166, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.659284353256226, "grad_norm": 0.8490450382232666, "tokens_seen": 53420032, "tokens_per_sec": 22682.158750237464, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3950093587239583, "G_std": 0.1802800496419271, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.580800771713257, "grad_norm": 0.871062695980072, "tokens_seen": 53583872, "tokens_per_sec": 22682.357496646193, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40057373046875, "G_std": 0.1813201904296875, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.561418533325195, "grad_norm": 0.8294814825057983, "tokens_seen": 53747712, "tokens_per_sec": 22682.810157222186, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3989969889322917, "G_std": 0.18233489990234375, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.435856103897095, "grad_norm": 0.8361157178878784, "tokens_seen": 53911552, "tokens_per_sec": 22682.863228347815, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3953501383463542, "G_std": 0.18302154541015625, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.6039719581604, "grad_norm": 0.7851086258888245, "tokens_seen": 54075392, "tokens_per_sec": 22683.162791003157, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4027608235677083, "G_std": 0.18321736653645834, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.463764667510986, "grad_norm": 0.8350525498390198, "tokens_seen": 54239232, "tokens_per_sec": 22683.35501655842, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3867085774739583, "G_std": 0.1826502482096354, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.588496685028076, "grad_norm": 0.8394814729690552, "tokens_seen": 54403072, "tokens_per_sec": 22683.124030278213, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845316569010417, "G_std": 0.18040974934895834, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.677242040634155, "grad_norm": 0.8130969405174255, "tokens_seen": 54566912, "tokens_per_sec": 22683.220100970437, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3979136149088542, "G_std": 0.1813507080078125, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.502873420715332, "grad_norm": 0.8224331140518188, "tokens_seen": 54730752, "tokens_per_sec": 22683.31804714556, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3922017415364583, "G_std": 0.18102010091145834, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.592590808868408, "grad_norm": 0.864209771156311, "tokens_seen": 54894592, "tokens_per_sec": 22683.20873571497, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3935597737630208, "G_std": 0.1821161905924479, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.60276460647583, "grad_norm": 0.8096357583999634, "tokens_seen": 55058432, "tokens_per_sec": 22683.40284696077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3894195556640625, "G_std": 0.18248748779296875, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.569863319396973, "grad_norm": 0.7769128680229187, "tokens_seen": 55222272, "tokens_per_sec": 22683.543342009456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3924001057942708, "G_std": 0.1795069376627604, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.578439950942993, "grad_norm": 0.7613825798034668, "tokens_seen": 55386112, "tokens_per_sec": 22683.653668220646, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3881734212239583, "G_std": 0.18109130859375, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.541999101638794, "grad_norm": 0.7732553482055664, "tokens_seen": 55549952, "tokens_per_sec": 22683.61056328263, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3876953125, "G_std": 0.1803716023763021, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.586518287658691, "grad_norm": 0.8567088842391968, "tokens_seen": 55713792, "tokens_per_sec": 22683.612688386667, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3860575358072917, "G_std": 0.1828460693359375, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.54621958732605, "grad_norm": 0.7837955951690674, "tokens_seen": 55877632, "tokens_per_sec": 22683.560164804527, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3911692301432292, "G_std": 0.18090057373046875, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.6051926612854, "grad_norm": 0.7806916832923889, "tokens_seen": 56041472, "tokens_per_sec": 22683.751213307543, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3895212809244792, "G_std": 0.18113199869791666, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.605039834976196, "grad_norm": 0.8737772703170776, "tokens_seen": 56205312, "tokens_per_sec": 22683.48710031384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3960622151692708, "G_std": 0.1819788614908854, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.48401951789856, "grad_norm": 0.8223187923431396, "tokens_seen": 56369152, "tokens_per_sec": 22683.467866222174, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.397918701171875, "G_std": 0.18166351318359375, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.556438446044922, "grad_norm": 0.8528813123703003, "tokens_seen": 56532992, "tokens_per_sec": 22683.454374755453, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.381988525390625, "G_std": 0.18077596028645834, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.411880970001221, "grad_norm": 0.817436933517456, "tokens_seen": 56696832, "tokens_per_sec": 22683.506154066585, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.390167236328125, "G_std": 0.17987823486328125, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.7639031410217285, "grad_norm": 0.8415563106536865, "tokens_seen": 56860672, "tokens_per_sec": 22683.70815191974, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3911234537760417, "G_std": 0.18324788411458334, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.52313232421875, "grad_norm": 0.8642857670783997, "tokens_seen": 57024512, "tokens_per_sec": 22683.7804018767, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3861185709635417, "G_std": 0.17986551920572916, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.454339027404785, "grad_norm": 0.808097243309021, "tokens_seen": 57188352, "tokens_per_sec": 22683.767975487557, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3850148518880208, "G_std": 0.1812566121419271, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.567883491516113, "grad_norm": 0.8332902789115906, "tokens_seen": 57352192, "tokens_per_sec": 22684.057757873677, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3797607421875, "G_std": 0.1799443562825521, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.568076038360596, "val_ppl": 96.35854117980071, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3932342529296875, "G_std": 0.18105316162109375, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.4617486000061035, "grad_norm": 0.816012442111969, "tokens_seen": 57516032, "tokens_per_sec": 22672.606218666544, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3901774088541667, "G_std": 0.1804682413736979, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.5696775913238525, "grad_norm": 0.8444259166717529, "tokens_seen": 57679872, "tokens_per_sec": 22672.77521176835, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3918406168619792, "G_std": 0.1823755900065104, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.606515407562256, "grad_norm": 0.8289660811424255, "tokens_seen": 57843712, "tokens_per_sec": 22672.92417830348, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3917338053385417, "G_std": 0.182037353515625, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.4696245193481445, "grad_norm": 0.8389130234718323, "tokens_seen": 58007552, "tokens_per_sec": 22673.029751812104, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3949228922526042, "G_std": 0.1795171101888021, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.474920272827148, "grad_norm": 0.8185527920722961, "tokens_seen": 58171392, "tokens_per_sec": 22672.771611281674, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3844807942708333, "G_std": 0.17844136555989584, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.429987907409668, "grad_norm": 0.8706984519958496, "tokens_seen": 58335232, "tokens_per_sec": 22672.066692565204, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3865102132161458, "G_std": 0.18218994140625, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.567702293395996, "grad_norm": 0.8194073438644409, "tokens_seen": 58499072, "tokens_per_sec": 22672.13715682376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775227864583333, "G_std": 0.18066914876302084, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.426828384399414, "grad_norm": 0.8215758800506592, "tokens_seen": 58662912, "tokens_per_sec": 22672.006113974003, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3889516194661458, "G_std": 0.18099212646484375, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.504711627960205, "grad_norm": 0.8519901633262634, "tokens_seen": 58826752, "tokens_per_sec": 22671.988167055217, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38446044921875, "G_std": 0.18123372395833334, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.5476603507995605, "grad_norm": 0.8792968988418579, "tokens_seen": 58990592, "tokens_per_sec": 22672.147197483064, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3812204996744792, "G_std": 0.1796696980794271, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.574070453643799, "grad_norm": 0.8365442156791687, "tokens_seen": 59154432, "tokens_per_sec": 22672.25468593318, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3904876708984375, "G_std": 0.18094635009765625, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.3680174350738525, "grad_norm": 0.8618782162666321, "tokens_seen": 59318272, "tokens_per_sec": 22672.61287848319, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38714599609375, "G_std": 0.1793950398763021, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.5805017948150635, "grad_norm": 0.909857988357544, "tokens_seen": 59482112, "tokens_per_sec": 22672.92754694984, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3844451904296875, "G_std": 0.1818567911783854, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.4715423583984375, "grad_norm": 0.8812128901481628, "tokens_seen": 59645952, "tokens_per_sec": 22673.200875311162, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3826904296875, "G_std": 0.18123372395833334, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.4653449058532715, "grad_norm": 0.8399589657783508, "tokens_seen": 59809792, "tokens_per_sec": 22673.338904979108, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3891042073567708, "G_std": 0.181365966796875, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.620862007141113, "grad_norm": 0.8626936078071594, "tokens_seen": 59973632, "tokens_per_sec": 22673.550303108637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3962148030598958, "G_std": 0.18124898274739584, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.403007984161377, "grad_norm": 0.8132584691047668, "tokens_seen": 60137472, "tokens_per_sec": 22673.66634501466, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846689860026042, "G_std": 0.18047078450520834, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.382238388061523, "grad_norm": 0.8976227641105652, "tokens_seen": 60301312, "tokens_per_sec": 22673.733260779656, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853556315104167, "G_std": 0.1792882283528646, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.6002891063690186, "grad_norm": 0.8323314189910889, "tokens_seen": 60465152, "tokens_per_sec": 22674.05273516357, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3907267252604167, "G_std": 0.18022664388020834, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.428555965423584, "grad_norm": 0.8295065760612488, "tokens_seen": 60628992, "tokens_per_sec": 22674.37590610471, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3799641927083333, "G_std": 0.18014017740885416, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.657737970352173, "grad_norm": 0.8314570784568787, "tokens_seen": 60792832, "tokens_per_sec": 22674.541401114086, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3931121826171875, "G_std": 0.18115743001302084, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.396164655685425, "grad_norm": 0.8047237396240234, "tokens_seen": 60956672, "tokens_per_sec": 22674.677049167032, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3898061116536458, "G_std": 0.18168131510416666, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.532175064086914, "grad_norm": 0.8660282492637634, "tokens_seen": 61120512, "tokens_per_sec": 22674.6627793077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.376434326171875, "G_std": 0.1805903116861979, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.509074926376343, "grad_norm": 0.8097493052482605, "tokens_seen": 61284352, "tokens_per_sec": 22674.69588039747, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3774363199869792, "G_std": 0.1798985799153646, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.4625563621521, "grad_norm": 0.8234096169471741, "tokens_seen": 61448192, "tokens_per_sec": 22674.93131438444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787078857421875, "G_std": 0.18039449055989584, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.346532106399536, "grad_norm": 0.7939028143882751, "tokens_seen": 61612032, "tokens_per_sec": 22675.096279567835, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3869578043619792, "G_std": 0.17923736572265625, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.447078227996826, "grad_norm": 0.8569576740264893, "tokens_seen": 61775872, "tokens_per_sec": 22675.27206608424, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3838399251302083, "G_std": 0.18016560872395834, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.594691038131714, "grad_norm": 0.8095682859420776, "tokens_seen": 61939712, "tokens_per_sec": 22675.286714814873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3857574462890625, "G_std": 0.18194580078125, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.468149662017822, "grad_norm": 0.8471786379814148, "tokens_seen": 62103552, "tokens_per_sec": 22675.53705978093, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809916178385417, "G_std": 0.17949422200520834, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.457032680511475, "grad_norm": 0.8075667023658752, "tokens_seen": 62267392, "tokens_per_sec": 22675.23872721451, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3815409342447917, "G_std": 0.17980194091796875, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.711150884628296, "grad_norm": 0.9254572987556458, "tokens_seen": 62431232, "tokens_per_sec": 22675.336942687965, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3878224690755208, "G_std": 0.18027750651041666, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.4253411293029785, "grad_norm": 0.865653932094574, "tokens_seen": 62595072, "tokens_per_sec": 22675.60571820205, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770090738932292, "G_std": 0.1803919474283854, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.353779554367065, "grad_norm": 0.7991268038749695, "tokens_seen": 62758912, "tokens_per_sec": 22675.597044586488, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3863983154296875, "G_std": 0.17959086100260416, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.321965217590332, "grad_norm": 0.8296690583229065, "tokens_seen": 62922752, "tokens_per_sec": 22675.531957322266, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846537272135417, "G_std": 0.17980448404947916, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.480785369873047, "grad_norm": 0.7995949387550354, "tokens_seen": 63086592, "tokens_per_sec": 22675.775334786973, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3836619059244792, "G_std": 0.18013509114583334, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.531430006027222, "grad_norm": 0.893939197063446, "tokens_seen": 63250432, "tokens_per_sec": 22675.19552811096, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38641357421875, "G_std": 0.18033599853515625, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.420652151107788, "grad_norm": 0.9033908247947693, "tokens_seen": 63414272, "tokens_per_sec": 22674.96551982953, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3866526285807292, "G_std": 0.1816126505533854, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.550413608551025, "grad_norm": 0.8888316750526428, "tokens_seen": 63578112, "tokens_per_sec": 22674.8456597896, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.390777587890625, "G_std": 0.1803461710611979, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.600499153137207, "grad_norm": 0.8700786828994751, "tokens_seen": 63741952, "tokens_per_sec": 22674.74330188803, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3752288818359375, "G_std": 0.1799138387044271, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.4857542514801025, "grad_norm": 0.8702519536018372, "tokens_seen": 63905792, "tokens_per_sec": 22674.89108413761, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3849538167317708, "G_std": 0.17926025390625, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.4474523067474365, "grad_norm": 0.8363479375839233, "tokens_seen": 64069632, "tokens_per_sec": 22675.05630035581, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.388336181640625, "G_std": 0.1785151163736979, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.350131511688232, "grad_norm": 0.8414722681045532, "tokens_seen": 64233472, "tokens_per_sec": 22675.202154597977, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37945556640625, "G_std": 0.18055216471354166, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.404013395309448, "grad_norm": 0.8349618315696716, "tokens_seen": 64397312, "tokens_per_sec": 22675.26949640926, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763427734375, "G_std": 0.1794586181640625, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.470919132232666, "grad_norm": 0.9065503478050232, "tokens_seen": 64561152, "tokens_per_sec": 22675.17285292542, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3832346598307292, "G_std": 0.1798248291015625, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.462644577026367, "grad_norm": 0.8501095771789551, "tokens_seen": 64724992, "tokens_per_sec": 22675.29384153321, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781483968098958, "G_std": 0.1804784138997396, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.534132242202759, "grad_norm": 0.8531330227851868, "tokens_seen": 64888832, "tokens_per_sec": 22675.29304706329, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3912760416666667, "G_std": 0.18194071451822916, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.441152095794678, "grad_norm": 0.9038145542144775, "tokens_seen": 65052672, "tokens_per_sec": 22675.484845417504, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3818511962890625, "G_std": 0.1785608927408854, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.502831935882568, "grad_norm": 0.8975228071212769, "tokens_seen": 65216512, "tokens_per_sec": 22675.471916989067, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3751118977864583, "G_std": 0.17875162760416666, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.468623399734497, "grad_norm": 0.8936397433280945, "tokens_seen": 65380352, "tokens_per_sec": 22675.89056092817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753611246744792, "G_std": 0.179656982421875, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.422874927520752, "grad_norm": 0.8503640294075012, "tokens_seen": 65544192, "tokens_per_sec": 22676.08745774597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3785858154296875, "G_std": 0.177032470703125, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.469820618629456, "val_ppl": 87.34105425012471, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3874308268229167, "G_std": 0.1785151163736979, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.4876627922058105, "grad_norm": 0.8608917593955994, "tokens_seen": 65708032, "tokens_per_sec": 22666.358845101404, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38470458984375, "G_std": 0.17836507161458334, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.505366802215576, "grad_norm": 0.8796061277389526, "tokens_seen": 65871872, "tokens_per_sec": 22666.70894071937, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3830464680989583, "G_std": 0.1786982218424479, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.3750691413879395, "grad_norm": 0.8913702964782715, "tokens_seen": 66035712, "tokens_per_sec": 22666.610823449715, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803151448567708, "G_std": 0.18109639485677084, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.443072319030762, "grad_norm": 0.8314475417137146, "tokens_seen": 66199552, "tokens_per_sec": 22666.80023411469, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3859507242838542, "G_std": 0.17816162109375, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.497866153717041, "grad_norm": 0.8844879865646362, "tokens_seen": 66363392, "tokens_per_sec": 22662.246347926808, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3805898030598958, "G_std": 0.1802241007486979, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.518361330032349, "grad_norm": 0.8635482788085938, "tokens_seen": 66527232, "tokens_per_sec": 22660.474813567278, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3861033121744792, "G_std": 0.1795654296875, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.438220262527466, "grad_norm": 0.8617628216743469, "tokens_seen": 66691072, "tokens_per_sec": 22660.572363714567, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38421630859375, "G_std": 0.1782557169596354, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.544872999191284, "grad_norm": 0.8444671630859375, "tokens_seen": 66854912, "tokens_per_sec": 22660.425466493118, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787587483723958, "G_std": 0.1796417236328125, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.530851602554321, "grad_norm": 0.9505759477615356, "tokens_seen": 67018752, "tokens_per_sec": 22660.62313262886, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3836568196614583, "G_std": 0.1798426310221354, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.494761228561401, "grad_norm": 0.8376728296279907, "tokens_seen": 67182592, "tokens_per_sec": 22660.571050172228, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3839314778645833, "G_std": 0.18111419677734375, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.494878053665161, "grad_norm": 0.8901426196098328, "tokens_seen": 67346432, "tokens_per_sec": 22660.822209661528, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3905893961588542, "G_std": 0.17995707194010416, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.41517448425293, "grad_norm": 0.909116268157959, "tokens_seen": 67510272, "tokens_per_sec": 22660.9094698531, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3793131510416667, "G_std": 0.18048604329427084, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.463053464889526, "grad_norm": 0.8794466257095337, "tokens_seen": 67674112, "tokens_per_sec": 22660.936939154308, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3801625569661458, "G_std": 0.179229736328125, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.438117027282715, "grad_norm": 0.8530974984169006, "tokens_seen": 67837952, "tokens_per_sec": 22661.08727041862, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3855387369791667, "G_std": 0.1792755126953125, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.488952875137329, "grad_norm": 0.9057617783546448, "tokens_seen": 68001792, "tokens_per_sec": 22661.19046852795, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3873291015625, "G_std": 0.18125406901041666, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.415828704833984, "grad_norm": 0.8760727643966675, "tokens_seen": 68165632, "tokens_per_sec": 22661.373975435046, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3872578938802083, "G_std": 0.1798985799153646, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.460235357284546, "grad_norm": 0.8803485035896301, "tokens_seen": 68329472, "tokens_per_sec": 22660.913478124945, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767547607421875, "G_std": 0.1789423624674479, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.470455169677734, "grad_norm": 0.8457365036010742, "tokens_seen": 68493312, "tokens_per_sec": 22660.74183667726, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3802388509114583, "G_std": 0.1780522664388021, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.430330038070679, "grad_norm": 0.8764951229095459, "tokens_seen": 68657152, "tokens_per_sec": 22661.060026371, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3799794514973958, "G_std": 0.17894490559895834, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.381040811538696, "grad_norm": 0.9327062964439392, "tokens_seen": 68820992, "tokens_per_sec": 22661.075295896902, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38995361328125, "G_std": 0.1789703369140625, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.577007532119751, "grad_norm": 0.9094229936599731, "tokens_seen": 68984832, "tokens_per_sec": 22661.26407802641, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3868509928385417, "G_std": 0.1779301961263021, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.460626840591431, "grad_norm": 0.9462320804595947, "tokens_seen": 69148672, "tokens_per_sec": 22661.46146466085, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803354899088542, "G_std": 0.18109130859375, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.400699138641357, "grad_norm": 0.9112773537635803, "tokens_seen": 69312512, "tokens_per_sec": 22661.629747794716, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3866068522135417, "G_std": 0.1786473592122396, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.358317136764526, "grad_norm": 0.833244264125824, "tokens_seen": 69476352, "tokens_per_sec": 22661.313651039025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737691243489583, "G_std": 0.1770757039388021, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.381990432739258, "grad_norm": 0.849780261516571, "tokens_seen": 69640192, "tokens_per_sec": 22660.895656500765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3739013671875, "G_std": 0.1774139404296875, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.514523983001709, "grad_norm": 0.8981420397758484, "tokens_seen": 69804032, "tokens_per_sec": 22660.67232668744, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845977783203125, "G_std": 0.1802520751953125, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.582767486572266, "grad_norm": 0.9995344281196594, "tokens_seen": 69967872, "tokens_per_sec": 22659.616101980915, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3956095377604167, "G_std": 0.17886861165364584, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.313868761062622, "grad_norm": 0.9382249116897583, "tokens_seen": 70131712, "tokens_per_sec": 22658.287378582354, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3802235921223958, "G_std": 0.17923482259114584, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.468908071517944, "grad_norm": 0.8707469701766968, "tokens_seen": 70295552, "tokens_per_sec": 22657.711751464096, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3821665445963542, "G_std": 0.1788508097330729, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.358637571334839, "grad_norm": 0.9239907264709473, "tokens_seen": 70459392, "tokens_per_sec": 22657.272672339786, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3863169352213542, "G_std": 0.1790440877278646, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.380643844604492, "grad_norm": 0.8727285265922546, "tokens_seen": 70623232, "tokens_per_sec": 22656.57225807076, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3836313883463542, "G_std": 0.18095906575520834, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.379086017608643, "grad_norm": 0.8517846465110779, "tokens_seen": 70787072, "tokens_per_sec": 22654.93194395079, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3756815592447917, "G_std": 0.1789830525716146, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.441310405731201, "grad_norm": 0.9091770052909851, "tokens_seen": 70950912, "tokens_per_sec": 22653.09905939464, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3820648193359375, "G_std": 0.179412841796875, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.318962574005127, "grad_norm": 0.9385361075401306, "tokens_seen": 71114752, "tokens_per_sec": 22651.571976178915, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899332682291667, "G_std": 0.17966206868489584, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.198628664016724, "grad_norm": 0.8361815810203552, "tokens_seen": 71278592, "tokens_per_sec": 22649.4343269158, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725077311197917, "G_std": 0.17917887369791666, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.378281354904175, "grad_norm": 0.9110616445541382, "tokens_seen": 71442432, "tokens_per_sec": 22647.936893283902, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720347086588542, "G_std": 0.17893473307291666, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.417831182479858, "grad_norm": 0.8759187459945679, "tokens_seen": 71606272, "tokens_per_sec": 22644.608176453305, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3832143147786458, "G_std": 0.1785430908203125, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.516528606414795, "grad_norm": 0.9635768532752991, "tokens_seen": 71770112, "tokens_per_sec": 22642.887888367222, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.390869140625, "G_std": 0.1787872314453125, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.529814004898071, "grad_norm": 0.8777604699134827, "tokens_seen": 71933952, "tokens_per_sec": 22641.464351897084, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3752237955729167, "G_std": 0.17739105224609375, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.357605218887329, "grad_norm": 0.886954128742218, "tokens_seen": 72097792, "tokens_per_sec": 22640.670522590444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3797251383463542, "G_std": 0.1791661580403646, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.373287916183472, "grad_norm": 0.9065393805503845, "tokens_seen": 72261632, "tokens_per_sec": 22640.057537599496, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725382486979167, "G_std": 0.17621358235677084, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.517886161804199, "grad_norm": 0.8680258989334106, "tokens_seen": 72425472, "tokens_per_sec": 22640.139807135212, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775227864583333, "G_std": 0.1797510782877604, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.38428807258606, "grad_norm": 0.913476288318634, "tokens_seen": 72589312, "tokens_per_sec": 22639.913820502297, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3813730875651042, "G_std": 0.17882792154947916, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.382097482681274, "grad_norm": 0.8802539110183716, "tokens_seen": 72753152, "tokens_per_sec": 22639.82838219995, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3733876546223958, "G_std": 0.1774571736653646, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.478176116943359, "grad_norm": 0.8801902532577515, "tokens_seen": 72916992, "tokens_per_sec": 22639.8120298528, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3791859944661458, "G_std": 0.1776123046875, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.36011528968811, "grad_norm": 0.9243143200874329, "tokens_seen": 73080832, "tokens_per_sec": 22639.165704701303, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3771158854166667, "G_std": 0.17685699462890625, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.443100452423096, "grad_norm": 0.9356406927108765, "tokens_seen": 73244672, "tokens_per_sec": 22638.659773726013, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3790435791015625, "G_std": 0.1798578898111979, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.4145238399505615, "grad_norm": 0.883924126625061, "tokens_seen": 73408512, "tokens_per_sec": 22638.45728167197, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3790842692057292, "G_std": 0.17950948079427084, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.313624620437622, "grad_norm": 0.9699910283088684, "tokens_seen": 73572352, "tokens_per_sec": 22638.48445783193, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834279378255208, "G_std": 0.17840576171875, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.352571487426758, "grad_norm": 0.8682926297187805, "tokens_seen": 73736192, "tokens_per_sec": 22638.148665189055, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3805287679036458, "G_std": 0.17785135904947916, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.492756915092468, "val_ppl": 89.3674851366993, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3808339436848958, "G_std": 0.17828877766927084, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.376149654388428, "grad_norm": 0.9306918978691101, "tokens_seen": 73900032, "tokens_per_sec": 22629.37817155384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3788604736328125, "G_std": 0.17921956380208334, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.5242228507995605, "grad_norm": 0.8986143469810486, "tokens_seen": 74063872, "tokens_per_sec": 22629.116866954948, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.378875732421875, "G_std": 0.17777506510416666, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.400991678237915, "grad_norm": 0.9471709132194519, "tokens_seen": 74227712, "tokens_per_sec": 22629.263652904174, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3785756429036458, "G_std": 0.1797027587890625, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.304706335067749, "grad_norm": 0.916666567325592, "tokens_seen": 74391552, "tokens_per_sec": 22629.284191790888, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3785349527994792, "G_std": 0.17725372314453125, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.436150312423706, "grad_norm": 0.9374586343765259, "tokens_seen": 74555392, "tokens_per_sec": 22629.540607224422, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3842315673828125, "G_std": 0.17969258626302084, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.365990400314331, "grad_norm": 0.9248061776161194, "tokens_seen": 74719232, "tokens_per_sec": 22629.544961918313, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3827972412109375, "G_std": 0.17876688639322916, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.3052520751953125, "grad_norm": 0.9255452156066895, "tokens_seen": 74883072, "tokens_per_sec": 22629.55244759361, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722381591796875, "G_std": 0.17865498860677084, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.4296464920043945, "grad_norm": 0.9944236874580383, "tokens_seen": 75046912, "tokens_per_sec": 22629.47711472038, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3784230550130208, "G_std": 0.17908732096354166, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.354278087615967, "grad_norm": 0.9503786563873291, "tokens_seen": 75210752, "tokens_per_sec": 22629.682212805976, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3857930501302083, "G_std": 0.17754109700520834, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.473792314529419, "grad_norm": 0.9673650860786438, "tokens_seen": 75374592, "tokens_per_sec": 22629.911321983625, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3816121419270833, "G_std": 0.17687479654947916, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.32656717300415, "grad_norm": 0.9790077805519104, "tokens_seen": 75538432, "tokens_per_sec": 22629.600826169328, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3791707356770833, "G_std": 0.17494710286458334, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.358503103256226, "grad_norm": 0.900486171245575, "tokens_seen": 75702272, "tokens_per_sec": 22629.825081173738, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3759206136067708, "G_std": 0.1785888671875, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.350633144378662, "grad_norm": 0.9176325798034668, "tokens_seen": 75866112, "tokens_per_sec": 22629.87324679724, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373138427734375, "G_std": 0.1771418253580729, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.227962493896484, "grad_norm": 0.940402626991272, "tokens_seen": 76029952, "tokens_per_sec": 22630.127823427243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3821868896484375, "G_std": 0.1784083048502604, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.4802281856536865, "grad_norm": 0.9673203229904175, "tokens_seen": 76193792, "tokens_per_sec": 22630.424657638243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853505452473958, "G_std": 0.17957560221354166, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.346080541610718, "grad_norm": 0.9505698680877686, "tokens_seen": 76357632, "tokens_per_sec": 22630.624617238456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3818206787109375, "G_std": 0.17736307779947916, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.358467102050781, "grad_norm": 0.8848789930343628, "tokens_seen": 76521472, "tokens_per_sec": 22630.616889954847, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3750356038411458, "G_std": 0.17896016438802084, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.12713623046875, "grad_norm": 0.8840476870536804, "tokens_seen": 76685312, "tokens_per_sec": 22630.788596515737, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3682403564453125, "G_std": 0.175750732421875, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.495109558105469, "grad_norm": 0.982325553894043, "tokens_seen": 76849152, "tokens_per_sec": 22630.973205026894, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3883056640625, "G_std": 0.17650858561197916, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.412344217300415, "grad_norm": 0.8864902853965759, "tokens_seen": 77012992, "tokens_per_sec": 22631.011270712435, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3844655354817708, "G_std": 0.17858123779296875, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.373137950897217, "grad_norm": 0.959548830986023, "tokens_seen": 77176832, "tokens_per_sec": 22631.02080127285, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3857981363932292, "G_std": 0.17852274576822916, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.277211666107178, "grad_norm": 0.8771275877952576, "tokens_seen": 77340672, "tokens_per_sec": 22631.18293094973, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3730214436848958, "G_std": 0.17770894368489584, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.436314105987549, "grad_norm": 0.8823539018630981, "tokens_seen": 77504512, "tokens_per_sec": 22631.28204549888, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.377685546875, "G_std": 0.17698160807291666, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.251105070114136, "grad_norm": 1.0180989503860474, "tokens_seen": 77668352, "tokens_per_sec": 22631.41851738743, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3754425048828125, "G_std": 0.176300048828125, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.410628080368042, "grad_norm": 0.9318060874938965, "tokens_seen": 77832192, "tokens_per_sec": 22631.374928122306, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.381561279296875, "G_std": 0.1782989501953125, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.281808853149414, "grad_norm": 0.9058754444122314, "tokens_seen": 77996032, "tokens_per_sec": 22631.25903393894, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3802490234375, "G_std": 0.17875162760416666, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.2376861572265625, "grad_norm": 0.9252782464027405, "tokens_seen": 78159872, "tokens_per_sec": 22631.312573949184, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3779703776041667, "G_std": 0.1778717041015625, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.305477857589722, "grad_norm": 0.8957890272140503, "tokens_seen": 78323712, "tokens_per_sec": 22631.24497642409, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3780008951822917, "G_std": 0.17664082845052084, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.221777439117432, "grad_norm": 0.8870733380317688, "tokens_seen": 78487552, "tokens_per_sec": 22631.229258166484, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3693491617838542, "G_std": 0.17642720540364584, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.3734283447265625, "grad_norm": 0.9830359816551208, "tokens_seen": 78651392, "tokens_per_sec": 22631.31346518834, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3765970865885417, "G_std": 0.1794153849283854, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.47037935256958, "grad_norm": 0.8849622011184692, "tokens_seen": 78815232, "tokens_per_sec": 22631.426224556904, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.377899169921875, "G_std": 0.17705281575520834, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.387117862701416, "grad_norm": 0.9013707637786865, "tokens_seen": 78979072, "tokens_per_sec": 22631.76363199697, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3731689453125, "G_std": 0.17753092447916666, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.231690406799316, "grad_norm": 0.8894761800765991, "tokens_seen": 79142912, "tokens_per_sec": 22632.04410769925, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.374786376953125, "G_std": 0.1764348347981771, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.201007127761841, "grad_norm": 0.9305024743080139, "tokens_seen": 79306752, "tokens_per_sec": 22632.035279558047, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3736979166666667, "G_std": 0.178375244140625, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.244863510131836, "grad_norm": 0.9359287023544312, "tokens_seen": 79470592, "tokens_per_sec": 22632.137841606866, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732859293619792, "G_std": 0.17732747395833334, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.240721225738525, "grad_norm": 0.9146021604537964, "tokens_seen": 79634432, "tokens_per_sec": 22631.995835349247, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3774973551432292, "G_std": 0.17670694986979166, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.342054128646851, "grad_norm": 0.9656504988670349, "tokens_seen": 79798272, "tokens_per_sec": 22631.68593745715, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720804850260417, "G_std": 0.17890421549479166, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.2040698528289795, "grad_norm": 0.8779354095458984, "tokens_seen": 79962112, "tokens_per_sec": 22631.403520578395, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3719228108723958, "G_std": 0.1759668986002604, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.352217197418213, "grad_norm": 0.9231594800949097, "tokens_seen": 80125952, "tokens_per_sec": 22631.416918712595, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690897623697917, "G_std": 0.1772944132486979, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.392326831817627, "grad_norm": 0.9255568385124207, "tokens_seen": 80289792, "tokens_per_sec": 22631.359346892714, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722635904947917, "G_std": 0.1770172119140625, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.428140163421631, "grad_norm": 0.9492714405059814, "tokens_seen": 80453632, "tokens_per_sec": 22631.468920115076, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753712972005208, "G_std": 0.1770299275716146, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.171308994293213, "grad_norm": 0.9635439515113831, "tokens_seen": 80617472, "tokens_per_sec": 22631.587216280648, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3708038330078125, "G_std": 0.17748006184895834, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.431966781616211, "grad_norm": 0.9533605575561523, "tokens_seen": 80781312, "tokens_per_sec": 22631.903390908177, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753306070963542, "G_std": 0.17872111002604166, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.498028516769409, "grad_norm": 0.9183662533760071, "tokens_seen": 80945152, "tokens_per_sec": 22632.169735404725, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.377899169921875, "G_std": 0.1782073974609375, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.331226825714111, "grad_norm": 0.9875441193580627, "tokens_seen": 81108992, "tokens_per_sec": 22632.51656573558, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3768157958984375, "G_std": 0.17807769775390625, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.400402784347534, "grad_norm": 0.9252442121505737, "tokens_seen": 81272832, "tokens_per_sec": 22632.49738884649, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3730417887369792, "G_std": 0.17817179361979166, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.337204694747925, "grad_norm": 0.8931041955947876, "tokens_seen": 81436672, "tokens_per_sec": 22632.515570031435, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732503255208333, "G_std": 0.178802490234375, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.176830053329468, "grad_norm": 0.8684373497962952, "tokens_seen": 81600512, "tokens_per_sec": 22632.69266753065, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3719126383463542, "G_std": 0.17652384440104166, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.317150592803955, "grad_norm": 0.927945077419281, "tokens_seen": 81764352, "tokens_per_sec": 22632.97063120371, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3733418782552083, "G_std": 0.17534891764322916, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.213780164718628, "grad_norm": 0.9045813679695129, "tokens_seen": 81928192, "tokens_per_sec": 22633.17502402946, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727671305338542, "G_std": 0.17601521809895834, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.404737520217895, "val_ppl": 81.83765929281302, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3777211507161458, "G_std": 0.1781768798828125, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.493099212646484, "grad_norm": 0.9525387287139893, "tokens_seen": 82092032, "tokens_per_sec": 22625.07050417425, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.375335693359375, "G_std": 0.17618815104166666, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.528882741928101, "grad_norm": 0.9881271719932556, "tokens_seen": 82255872, "tokens_per_sec": 22624.93456879248, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3736521402994792, "G_std": 0.17737833658854166, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.511868476867676, "grad_norm": 0.9432035684585571, "tokens_seen": 82419712, "tokens_per_sec": 22625.136818588722, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796844482421875, "G_std": 0.17721048990885416, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.369120359420776, "grad_norm": 0.9360615015029907, "tokens_seen": 82583552, "tokens_per_sec": 22625.04104828545, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3678690592447917, "G_std": 0.1763331095377604, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.486907720565796, "grad_norm": 0.919406533241272, "tokens_seen": 82747392, "tokens_per_sec": 22625.12294908647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781178792317708, "G_std": 0.17690022786458334, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.280473709106445, "grad_norm": 0.9975787401199341, "tokens_seen": 82911232, "tokens_per_sec": 22625.22890924607, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3739827473958333, "G_std": 0.17779032389322916, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.279403924942017, "grad_norm": 0.897365391254425, "tokens_seen": 83075072, "tokens_per_sec": 22625.27816591627, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3681538899739583, "G_std": 0.17484029134114584, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.377606391906738, "grad_norm": 0.9334447979927063, "tokens_seen": 83238912, "tokens_per_sec": 22625.124761314964, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3758087158203125, "G_std": 0.17779032389322916, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.388896226882935, "grad_norm": 0.9374670386314392, "tokens_seen": 83402752, "tokens_per_sec": 22625.240162728347, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707529703776042, "G_std": 0.17631022135416666, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.28952169418335, "grad_norm": 0.9604432582855225, "tokens_seen": 83566592, "tokens_per_sec": 22625.08517827013, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3681691487630208, "G_std": 0.17626190185546875, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.375317335128784, "grad_norm": 0.9074344635009766, "tokens_seen": 83730432, "tokens_per_sec": 22624.971161356927, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725179036458333, "G_std": 0.1758880615234375, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.301833152770996, "grad_norm": 0.9329607486724854, "tokens_seen": 83894272, "tokens_per_sec": 22625.05543024551, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3730112711588542, "G_std": 0.176910400390625, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.314261198043823, "grad_norm": 0.931871771812439, "tokens_seen": 84058112, "tokens_per_sec": 22625.16482781625, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3791656494140625, "G_std": 0.17584737141927084, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.453997373580933, "grad_norm": 0.9222097396850586, "tokens_seen": 84221952, "tokens_per_sec": 22625.19372451494, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3731435139973958, "G_std": 0.17673746744791666, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.293858528137207, "grad_norm": 0.9318246841430664, "tokens_seen": 84385792, "tokens_per_sec": 22625.409426722013, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3811492919921875, "G_std": 0.17753092447916666, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.479961633682251, "grad_norm": 1.0204854011535645, "tokens_seen": 84549632, "tokens_per_sec": 22625.493387280403, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787587483723958, "G_std": 0.17681884765625, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.280810117721558, "grad_norm": 0.9194492697715759, "tokens_seen": 84713472, "tokens_per_sec": 22625.53729811496, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3708597819010417, "G_std": 0.1761500040690104, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.3308539390563965, "grad_norm": 0.9334615468978882, "tokens_seen": 84877312, "tokens_per_sec": 22625.61068901036, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372314453125, "G_std": 0.17674763997395834, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.438125133514404, "grad_norm": 1.0251619815826416, "tokens_seen": 85041152, "tokens_per_sec": 22625.722446331223, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3773701985677083, "G_std": 0.17571512858072916, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.425053119659424, "grad_norm": 0.9688757061958313, "tokens_seen": 85204992, "tokens_per_sec": 22625.82115780416, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853047688802083, "G_std": 0.17724609375, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.2636635303497314, "grad_norm": 0.9818354845046997, "tokens_seen": 85368832, "tokens_per_sec": 22626.003931252555, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3735758463541667, "G_std": 0.17592366536458334, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.211516618728638, "grad_norm": 0.9375207424163818, "tokens_seen": 85532672, "tokens_per_sec": 22626.160703491343, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3761240641276042, "G_std": 0.17631022135416666, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.360644340515137, "grad_norm": 0.9838488101959229, "tokens_seen": 85696512, "tokens_per_sec": 22626.335823206082, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809000651041667, "G_std": 0.1769866943359375, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.383069753646851, "grad_norm": 0.9939836263656616, "tokens_seen": 85860352, "tokens_per_sec": 22626.141167579077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.375732421875, "G_std": 0.17452494303385416, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.272790431976318, "grad_norm": 0.9023348093032837, "tokens_seen": 86024192, "tokens_per_sec": 22626.231570071286, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720448811848958, "G_std": 0.17565155029296875, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.233469009399414, "grad_norm": 0.891014575958252, "tokens_seen": 86188032, "tokens_per_sec": 22626.52590810774, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734792073567708, "G_std": 0.17519632975260416, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.101641893386841, "grad_norm": 1.001593828201294, "tokens_seen": 86351872, "tokens_per_sec": 22626.7442832337, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3683420817057292, "G_std": 0.17543538411458334, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.3457350730896, "grad_norm": 0.9165781736373901, "tokens_seen": 86515712, "tokens_per_sec": 22626.752013332956, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3748931884765625, "G_std": 0.17718505859375, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.392492771148682, "grad_norm": 0.9848149418830872, "tokens_seen": 86679552, "tokens_per_sec": 22626.744374567526, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796590169270833, "G_std": 0.17462666829427084, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.437107801437378, "grad_norm": 1.0177202224731445, "tokens_seen": 86843392, "tokens_per_sec": 22626.83275726605, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3749237060546875, "G_std": 0.1769866943359375, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.380805253982544, "grad_norm": 0.9691741466522217, "tokens_seen": 87007232, "tokens_per_sec": 22627.035904662058, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3728586832682292, "G_std": 0.17620340983072916, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.368393659591675, "grad_norm": 0.9876694083213806, "tokens_seen": 87171072, "tokens_per_sec": 22627.40760457404, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3742624918619792, "G_std": 0.17654927571614584, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.340296030044556, "grad_norm": 0.9860618114471436, "tokens_seen": 87334912, "tokens_per_sec": 22627.510228273517, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373870849609375, "G_std": 0.17666117350260416, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.423408031463623, "grad_norm": 0.9506084322929382, "tokens_seen": 87498752, "tokens_per_sec": 22627.738562820563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3711649576822917, "G_std": 0.1753997802734375, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.443308353424072, "grad_norm": 0.9388347268104553, "tokens_seen": 87662592, "tokens_per_sec": 22628.128473324497, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3749440511067708, "G_std": 0.1770477294921875, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.328280448913574, "grad_norm": 0.8832979798316956, "tokens_seen": 87826432, "tokens_per_sec": 22627.88841337295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3688812255859375, "G_std": 0.17591349283854166, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.205516338348389, "grad_norm": 0.8914402723312378, "tokens_seen": 87990272, "tokens_per_sec": 22628.071147642757, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3717600504557292, "G_std": 0.1752166748046875, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.318749904632568, "grad_norm": 0.9425916075706482, "tokens_seen": 88154112, "tokens_per_sec": 22628.482957034077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3761342366536458, "G_std": 0.17642720540364584, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.431927680969238, "grad_norm": 0.9541940093040466, "tokens_seen": 88317952, "tokens_per_sec": 22628.571955820134, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3846893310546875, "G_std": 0.17636617024739584, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.388386964797974, "grad_norm": 1.0482151508331299, "tokens_seen": 88481792, "tokens_per_sec": 22628.623225966294, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3733927408854167, "G_std": 0.178192138671875, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.305787086486816, "grad_norm": 0.9469630718231201, "tokens_seen": 88645632, "tokens_per_sec": 22628.50058518538, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.378936767578125, "G_std": 0.17693074544270834, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.308433294296265, "grad_norm": 0.9848297238349915, "tokens_seen": 88809472, "tokens_per_sec": 22628.685067156817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3789825439453125, "G_std": 0.17455037434895834, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.3007652759552, "grad_norm": 0.9737198948860168, "tokens_seen": 88973312, "tokens_per_sec": 22628.805522799194, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3772939046223958, "G_std": 0.17673238118489584, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.431079864501953, "grad_norm": 0.9671052694320679, "tokens_seen": 89137152, "tokens_per_sec": 22628.833132560187, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372955322265625, "G_std": 0.1768798828125, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.485020637512207, "grad_norm": 1.0472619533538818, "tokens_seen": 89300992, "tokens_per_sec": 22629.10974685078, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3794504801432292, "G_std": 0.17610677083333334, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.522578001022339, "grad_norm": 1.2614117860794067, "tokens_seen": 89464832, "tokens_per_sec": 22629.369479320343, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373138427734375, "G_std": 0.1756439208984375, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.51955509185791, "grad_norm": 0.9495947957038879, "tokens_seen": 89628672, "tokens_per_sec": 22629.39593671204, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3744303385416667, "G_std": 0.1765289306640625, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.289098024368286, "grad_norm": 0.9678544402122498, "tokens_seen": 89792512, "tokens_per_sec": 22629.18540700246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3699849446614583, "G_std": 0.1757354736328125, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.163210391998291, "grad_norm": 0.9050093293190002, "tokens_seen": 89956352, "tokens_per_sec": 22629.244823873003, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722279866536458, "G_std": 0.17446136474609375, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.397014379501343, "grad_norm": 0.9858291149139404, "tokens_seen": 90120192, "tokens_per_sec": 22629.391454387165, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775583902994792, "G_std": 0.17547607421875, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.4030338525772095, "val_ppl": 81.69835381969405, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3743184407552083, "G_std": 0.17530314127604166, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.338894844055176, "grad_norm": 1.0397536754608154, "tokens_seen": 90284032, "tokens_per_sec": 22622.04653088452, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37542724609375, "G_std": 0.17665608723958334, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.2768754959106445, "grad_norm": 0.9748044610023499, "tokens_seen": 90447872, "tokens_per_sec": 22621.68971487412, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3743336995442708, "G_std": 0.1767120361328125, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.386432647705078, "grad_norm": 1.035977840423584, "tokens_seen": 90611712, "tokens_per_sec": 22621.749549348384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834177652994792, "G_std": 0.17487080891927084, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.2964277267456055, "grad_norm": 1.0432024002075195, "tokens_seen": 90775552, "tokens_per_sec": 22621.559366488418, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3700358072916667, "G_std": 0.17682902018229166, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.428375720977783, "grad_norm": 0.9619847536087036, "tokens_seen": 90939392, "tokens_per_sec": 22621.872991891665, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770650227864583, "G_std": 0.1768951416015625, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.1675121784210205, "grad_norm": 0.9345172047615051, "tokens_seen": 91103232, "tokens_per_sec": 22622.167101390383, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3697662353515625, "G_std": 0.17496236165364584, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.330950736999512, "grad_norm": 0.9612212181091309, "tokens_seen": 91267072, "tokens_per_sec": 22621.730810262274, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3728383382161458, "G_std": 0.1737060546875, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.3456034660339355, "grad_norm": 0.963239312171936, "tokens_seen": 91430912, "tokens_per_sec": 22621.66014899251, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.371063232421875, "G_std": 0.17429606119791666, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.332990884780884, "grad_norm": 0.940981388092041, "tokens_seen": 91594752, "tokens_per_sec": 22621.73949202094, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3712056477864583, "G_std": 0.17476908365885416, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.179333925247192, "grad_norm": 0.999381422996521, "tokens_seen": 91758592, "tokens_per_sec": 22621.735781075124, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695220947265625, "G_std": 0.17272186279296875, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.119887828826904, "grad_norm": 0.9683344960212708, "tokens_seen": 91922432, "tokens_per_sec": 22621.618073856764, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3672993977864583, "G_std": 0.1741002400716146, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.332365036010742, "grad_norm": 1.0297191143035889, "tokens_seen": 92086272, "tokens_per_sec": 22621.89750900136, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3761240641276042, "G_std": 0.17639668782552084, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.362602233886719, "grad_norm": 1.0093997716903687, "tokens_seen": 92250112, "tokens_per_sec": 22621.970571364756, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727060953776042, "G_std": 0.17400105794270834, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.320306301116943, "grad_norm": 0.9983144402503967, "tokens_seen": 92413952, "tokens_per_sec": 22622.04388344217, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3746999104817708, "G_std": 0.1747589111328125, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.360746383666992, "grad_norm": 1.0467673540115356, "tokens_seen": 92577792, "tokens_per_sec": 22622.17173446578, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670857747395833, "G_std": 0.17499796549479166, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.229398965835571, "grad_norm": 1.0904122591018677, "tokens_seen": 92741632, "tokens_per_sec": 22622.153403412787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720855712890625, "G_std": 0.17675272623697916, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.405058860778809, "grad_norm": 1.0165410041809082, "tokens_seen": 92905472, "tokens_per_sec": 22621.728389795167, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3773651123046875, "G_std": 0.17493184407552084, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.327856779098511, "grad_norm": 1.009774923324585, "tokens_seen": 93069312, "tokens_per_sec": 22621.321979958924, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3742472330729167, "G_std": 0.17547607421875, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.333826780319214, "grad_norm": 0.9993050694465637, "tokens_seen": 93233152, "tokens_per_sec": 22621.035714789443, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37176513671875, "G_std": 0.1749420166015625, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.355352401733398, "grad_norm": 1.1643474102020264, "tokens_seen": 93396992, "tokens_per_sec": 22620.89543718028, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716227213541667, "G_std": 0.17454020182291666, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.3282904624938965, "grad_norm": 0.9983437657356262, "tokens_seen": 93560832, "tokens_per_sec": 22620.947687939453, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3755900065104167, "G_std": 0.17523193359375, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.19970965385437, "grad_norm": 0.9611848592758179, "tokens_seen": 93724672, "tokens_per_sec": 22621.12241448975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3776295979817708, "G_std": 0.1772003173828125, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.258538484573364, "grad_norm": 1.0388070344924927, "tokens_seen": 93888512, "tokens_per_sec": 22620.983437467163, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3709971110026042, "G_std": 0.17526753743489584, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.2338361740112305, "grad_norm": 1.0361735820770264, "tokens_seen": 94052352, "tokens_per_sec": 22621.2160353988, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670196533203125, "G_std": 0.17632039388020834, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.354948282241821, "grad_norm": 0.9628511071205139, "tokens_seen": 94216192, "tokens_per_sec": 22619.549213776583, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767038981119792, "G_std": 0.17660013834635416, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.448553085327148, "grad_norm": 0.9859587550163269, "tokens_seen": 94380032, "tokens_per_sec": 22615.870265379475, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3751271565755208, "G_std": 0.17617289225260416, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.315653562545776, "grad_norm": 1.0107920169830322, "tokens_seen": 94543872, "tokens_per_sec": 22610.191694408066, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37774658203125, "G_std": 0.17669169108072916, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.287634611129761, "grad_norm": 1.052086353302002, "tokens_seen": 94707712, "tokens_per_sec": 22601.75811239301, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3789265950520833, "G_std": 0.1746368408203125, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.478869438171387, "grad_norm": 1.1822283267974854, "tokens_seen": 94871552, "tokens_per_sec": 22597.031257054783, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3744099934895833, "G_std": 0.1749725341796875, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.196228265762329, "grad_norm": 0.9895741939544678, "tokens_seen": 95035392, "tokens_per_sec": 22597.03206011758, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3798319498697917, "G_std": 0.17439778645833334, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.317128658294678, "grad_norm": 1.0220997333526611, "tokens_seen": 95199232, "tokens_per_sec": 22597.077034788497, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3704681396484375, "G_std": 0.17550150553385416, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.2415690422058105, "grad_norm": 0.8928270936012268, "tokens_seen": 95363072, "tokens_per_sec": 22597.01967001348, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3709004720052083, "G_std": 0.17577616373697916, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.288992166519165, "grad_norm": 1.0489102602005005, "tokens_seen": 95526912, "tokens_per_sec": 22597.152649476782, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3713785807291667, "G_std": 0.17624409993489584, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.3082897663116455, "grad_norm": 0.9407575726509094, "tokens_seen": 95690752, "tokens_per_sec": 22597.227577891226, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3693288167317708, "G_std": 0.17567952473958334, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.34342360496521, "grad_norm": 0.9706370234489441, "tokens_seen": 95854592, "tokens_per_sec": 22597.357501722712, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3730926513671875, "G_std": 0.17549641927083334, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.288817644119263, "grad_norm": 1.022262454032898, "tokens_seen": 96018432, "tokens_per_sec": 22597.500850078526, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3698527018229167, "G_std": 0.174835205078125, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.349072694778442, "grad_norm": 1.0298840999603271, "tokens_seen": 96182272, "tokens_per_sec": 22597.469522692816, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3814493815104167, "G_std": 0.17680867513020834, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.213337421417236, "grad_norm": 0.9932962656021118, "tokens_seen": 96346112, "tokens_per_sec": 22597.62359140035, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657735188802083, "G_std": 0.17502339680989584, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.425016403198242, "grad_norm": 1.0719645023345947, "tokens_seen": 96509952, "tokens_per_sec": 22597.779850128856, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37506103515625, "G_std": 0.1751861572265625, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.320381164550781, "grad_norm": 1.1098095178604126, "tokens_seen": 96673792, "tokens_per_sec": 22597.865665152283, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3726603190104167, "G_std": 0.1757532755533854, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.485676050186157, "grad_norm": 0.9330282807350159, "tokens_seen": 96837632, "tokens_per_sec": 22598.127971261063, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721923828125, "G_std": 0.17523193359375, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.2489471435546875, "grad_norm": 0.971542477607727, "tokens_seen": 97001472, "tokens_per_sec": 22598.23961022934, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3699798583984375, "G_std": 0.17626444498697916, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.2437520027160645, "grad_norm": 1.0737046003341675, "tokens_seen": 97165312, "tokens_per_sec": 22598.35686733628, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3698832194010417, "G_std": 0.17504374186197916, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.34101676940918, "grad_norm": 1.0822736024856567, "tokens_seen": 97329152, "tokens_per_sec": 22598.035663645787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3696339925130208, "G_std": 0.17453511555989584, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.294729948043823, "grad_norm": 0.997937023639679, "tokens_seen": 97492992, "tokens_per_sec": 22598.079589598434, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3719329833984375, "G_std": 0.17499796549479166, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.215216398239136, "grad_norm": 1.059629201889038, "tokens_seen": 97656832, "tokens_per_sec": 22597.92271311634, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3739369710286458, "G_std": 0.17401123046875, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.179952621459961, "grad_norm": 1.004245400428772, "tokens_seen": 97820672, "tokens_per_sec": 22598.123944874787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367034912109375, "G_std": 0.17559814453125, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.305667161941528, "grad_norm": 0.997154712677002, "tokens_seen": 97984512, "tokens_per_sec": 22598.309897873187, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.369171142578125, "G_std": 0.17547098795572916, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.272729396820068, "grad_norm": 1.048843264579773, "tokens_seen": 98148352, "tokens_per_sec": 22598.352530434637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676300048828125, "G_std": 0.1741943359375, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.398592329025268, "val_ppl": 81.33629330240751, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3769683837890625, "G_std": 0.17468770345052084, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4351.29425239563, "nan_failures": 0} diff --git a/g_only_s3/config.json b/g_only_s3/config.json new file mode 100644 index 0000000000000000000000000000000000000000..b7035ca817d075b2ac20b00242635e1d8069a905 --- /dev/null +++ b/g_only_s3/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 3, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s3", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 3001346, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/g_only_s3/final.pt b/g_only_s3/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..cb4d41f94a62a24c212f8bf497173363d57ce24c --- /dev/null +++ b/g_only_s3/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d37bfec867b629274a6cd556f2936d559f77746549fe946cd61929dffff2bbdc +size 253629735 diff --git a/g_only_s3/metrics.jsonl b/g_only_s3/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..d67da57c187e0c9ff08e8a6476c8960595201b83 --- /dev/null +++ b/g_only_s3/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 3, "config": {"variant": "g_only", "seed": 3, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s3", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 3001346, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.875686645507812, "grad_norm": 9.675968170166016, "tokens_seen": 8192, "tokens_per_sec": 8740.889758122781, "vram_gb": 4.987702272, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4998372395833333, "G_std": 0.08642578125, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.837923049926758, "grad_norm": 1.8326470851898193, "tokens_seen": 172032, "tokens_per_sec": 21289.22548059636, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4657389322916667, "G_std": 0.06557210286458333, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.113607406616211, "grad_norm": 1.7549465894699097, "tokens_seen": 335872, "tokens_per_sec": 21997.348279202182, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.452880859375, "G_std": 0.068756103515625, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.187580108642578, "grad_norm": 1.5798726081848145, "tokens_seen": 499712, "tokens_per_sec": 22323.086789004283, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4429524739583333, "G_std": 0.07898966471354167, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.529348373413086, "grad_norm": 0.7903477549552917, "tokens_seen": 663552, "tokens_per_sec": 22487.983651219023, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4395345052083333, "G_std": 0.0963897705078125, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.112022876739502, "grad_norm": 0.7631856203079224, "tokens_seen": 827392, "tokens_per_sec": 22599.643534125975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.430908203125, "G_std": 0.110931396484375, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.191473484039307, "grad_norm": 0.7491865754127502, "tokens_seen": 991232, "tokens_per_sec": 22664.142138456344, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4300130208333333, "G_std": 0.1173095703125, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.894202470779419, "grad_norm": 0.6052424907684326, "tokens_seen": 1155072, "tokens_per_sec": 22712.93550845397, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.421875, "G_std": 0.12904866536458334, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.862511873245239, "grad_norm": 0.7154775261878967, "tokens_seen": 1318912, "tokens_per_sec": 22745.013200025074, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4200846354166667, "G_std": 0.13419596354166666, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.740646839141846, "grad_norm": 0.507909893989563, "tokens_seen": 1482752, "tokens_per_sec": 22763.54581351843, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4044596354166667, "G_std": 0.13966878255208334, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.674563407897949, "grad_norm": 0.5868122577667236, "tokens_seen": 1646592, "tokens_per_sec": 22781.92776712594, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42236328125, "G_std": 0.14719645182291666, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.599548816680908, "grad_norm": 0.5274554491043091, "tokens_seen": 1810432, "tokens_per_sec": 22792.049890739865, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3978678385416667, "G_std": 0.14906819661458334, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.613696813583374, "grad_norm": 0.4775760769844055, "tokens_seen": 1974272, "tokens_per_sec": 22803.090229179415, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3998209635416667, "G_std": 0.15677897135416666, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.612010717391968, "grad_norm": 0.4846149682998657, "tokens_seen": 2138112, "tokens_per_sec": 22813.08400005639, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40771484375, "G_std": 0.17201741536458334, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.440081357955933, "grad_norm": 0.5442310571670532, "tokens_seen": 2301952, "tokens_per_sec": 22817.47577772469, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4283854166666667, "G_std": 0.17574055989583334, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.331152677536011, "grad_norm": 0.4906560480594635, "tokens_seen": 2465792, "tokens_per_sec": 22823.551157125334, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4000651041666667, "G_std": 0.16764322916666666, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.425274610519409, "grad_norm": 0.48192834854125977, "tokens_seen": 2629632, "tokens_per_sec": 22816.19913928409, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3999837239583333, "G_std": 0.17366536458333334, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.475764989852905, "grad_norm": 0.5045626163482666, "tokens_seen": 2793472, "tokens_per_sec": 22822.82701869954, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4183756510416667, "G_std": 0.17948404947916666, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.2547478675842285, "grad_norm": 0.45880231261253357, "tokens_seen": 2957312, "tokens_per_sec": 22831.32259277916, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3988444010416667, "G_std": 0.18400065104166666, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.301198482513428, "grad_norm": 0.4343378245830536, "tokens_seen": 3121152, "tokens_per_sec": 22835.392821323436, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4153645833333333, "G_std": 0.18778483072916666, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.3642988204956055, "grad_norm": 0.49753883481025696, "tokens_seen": 3284992, "tokens_per_sec": 22841.253107573448, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4493001302083333, "G_std": 0.19533284505208334, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.283217191696167, "grad_norm": 0.43026334047317505, "tokens_seen": 3448832, "tokens_per_sec": 22843.815255226462, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4436848958333333, "G_std": 0.19691975911458334, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.220736742019653, "grad_norm": 0.5050866007804871, "tokens_seen": 3612672, "tokens_per_sec": 22844.360350895797, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4371744791666667, "G_std": 0.199462890625, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.350944995880127, "grad_norm": 0.49878308176994324, "tokens_seen": 3776512, "tokens_per_sec": 22851.220282103972, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4539388020833333, "G_std": 0.20499674479166666, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.2145490646362305, "grad_norm": 0.485794335603714, "tokens_seen": 3940352, "tokens_per_sec": 22855.766034651348, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4609375, "G_std": 0.20914713541666666, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.240497827529907, "grad_norm": 0.4851994812488556, "tokens_seen": 4104192, "tokens_per_sec": 22856.66459485765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.489501953125, "G_std": 0.21091715494791666, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.255227565765381, "grad_norm": 0.5197260975837708, "tokens_seen": 4268032, "tokens_per_sec": 22858.03117182465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5037434895833334, "G_std": 0.21793619791666666, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.160506963729858, "grad_norm": 0.4681406319141388, "tokens_seen": 4431872, "tokens_per_sec": 22860.912431388075, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5077311197916666, "G_std": 0.21734619140625, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.205121278762817, "grad_norm": 0.5471371412277222, "tokens_seen": 4595712, "tokens_per_sec": 22863.114925450067, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5309244791666666, "G_std": 0.21952311197916666, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.192448616027832, "grad_norm": 0.46229779720306396, "tokens_seen": 4759552, "tokens_per_sec": 22865.814996827692, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5105794270833334, "G_std": 0.21683756510416666, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.082369089126587, "grad_norm": 0.43431800603866577, "tokens_seen": 4923392, "tokens_per_sec": 22865.855397147927, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.53466796875, "G_std": 0.21893310546875, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 5.995082855224609, "grad_norm": 0.4881434142589569, "tokens_seen": 5087232, "tokens_per_sec": 22862.023413175175, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5511067708333334, "G_std": 0.21647135416666666, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.096728086471558, "grad_norm": 0.48686182498931885, "tokens_seen": 5251072, "tokens_per_sec": 22861.774709111098, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5537109375, "G_std": 0.20957438151041666, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.886072635650635, "grad_norm": 0.47224029898643494, "tokens_seen": 5414912, "tokens_per_sec": 22860.69249071622, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5729166666666666, "G_std": 0.208740234375, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.0085368156433105, "grad_norm": 0.4729112386703491, "tokens_seen": 5578752, "tokens_per_sec": 22861.021327026134, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5673828125, "G_std": 0.21004231770833334, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 6.048378229141235, "grad_norm": 0.4949342906475067, "tokens_seen": 5742592, "tokens_per_sec": 22864.77634699346, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5575358072916666, "G_std": 0.20953369140625, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.860652923583984, "grad_norm": 0.4617063105106354, "tokens_seen": 5906432, "tokens_per_sec": 22862.4999544478, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.571044921875, "G_std": 0.20955403645833334, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.025896072387695, "grad_norm": 0.47264912724494934, "tokens_seen": 6070272, "tokens_per_sec": 22865.488304710525, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5669759114583334, "G_std": 0.20292154947916666, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.918686866760254, "grad_norm": 0.4732891023159027, "tokens_seen": 6234112, "tokens_per_sec": 22866.353651728637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5955403645833334, "G_std": 0.19854736328125, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.993107795715332, "grad_norm": 0.535722017288208, "tokens_seen": 6397952, "tokens_per_sec": 22867.02078099144, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6043294270833334, "G_std": 0.19925944010416666, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.936357021331787, "grad_norm": 0.46369996666908264, "tokens_seen": 6561792, "tokens_per_sec": 22869.704968697693, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6045735677083334, "G_std": 0.19305419921875, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.846997261047363, "grad_norm": 0.5007457733154297, "tokens_seen": 6725632, "tokens_per_sec": 22872.445658113465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5884602864583334, "G_std": 0.20025634765625, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.871208190917969, "grad_norm": 0.5064743161201477, "tokens_seen": 6889472, "tokens_per_sec": 22872.880515685418, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6038411458333334, "G_std": 0.19346110026041666, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.839071750640869, "grad_norm": 0.49301013350486755, "tokens_seen": 7053312, "tokens_per_sec": 22873.29702545375, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6110026041666666, "G_std": 0.19289143880208334, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.967350482940674, "grad_norm": 0.5181683301925659, "tokens_seen": 7217152, "tokens_per_sec": 22872.99596730559, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6027018229166666, "G_std": 0.19073486328125, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.772544860839844, "grad_norm": 0.5059057474136353, "tokens_seen": 7380992, "tokens_per_sec": 22874.39426568266, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.596435546875, "G_std": 0.19071451822916666, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.803341388702393, "grad_norm": 0.5141953229904175, "tokens_seen": 7544832, "tokens_per_sec": 22876.02487411433, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6217447916666666, "G_std": 0.18007405598958334, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.64075493812561, "grad_norm": 0.4954575002193451, "tokens_seen": 7708672, "tokens_per_sec": 22845.695326250243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5756022135416666, "G_std": 0.18682861328125, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.8058717250823975, "grad_norm": 0.497445672750473, "tokens_seen": 7872512, "tokens_per_sec": 22846.83174223592, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5938313802083334, "G_std": 0.18581136067708334, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.718003034591675, "grad_norm": 0.46163874864578247, "tokens_seen": 8036352, "tokens_per_sec": 22846.991908923414, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.579345703125, "G_std": 0.18572998046875, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.88881516456604, "grad_norm": 0.5411009192466736, "tokens_seen": 8200192, "tokens_per_sec": 22849.00929666032, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5968831380208334, "G_std": 0.18806966145833334, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.837244820594788, "val_ppl": 342.83347052493656, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5880126953125, "G_std": 0.18729654947916666, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.736943244934082, "grad_norm": 0.48673826456069946, "tokens_seen": 8364032, "tokens_per_sec": 22771.04523602025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5802001953125, "G_std": 0.188720703125, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.648163318634033, "grad_norm": 0.5421730875968933, "tokens_seen": 8527872, "tokens_per_sec": 22770.689805398895, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.60302734375, "G_std": 0.17906697591145834, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.628333568572998, "grad_norm": 0.5229904651641846, "tokens_seen": 8691712, "tokens_per_sec": 22774.40698125367, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5662434895833334, "G_std": 0.18479410807291666, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.694540500640869, "grad_norm": 0.4669584035873413, "tokens_seen": 8855552, "tokens_per_sec": 22776.8116290299, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5757242838541666, "G_std": 0.18003336588541666, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.658460378646851, "grad_norm": 0.5667722821235657, "tokens_seen": 9019392, "tokens_per_sec": 22778.189482952628, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.569091796875, "G_std": 0.18245442708333334, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.7427661418914795, "grad_norm": 0.5248715877532959, "tokens_seen": 9183232, "tokens_per_sec": 22780.603393062243, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5704752604166666, "G_std": 0.18062337239583334, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.628190279006958, "grad_norm": 0.5046263933181763, "tokens_seen": 9347072, "tokens_per_sec": 22782.52278821575, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5753580729166666, "G_std": 0.180419921875, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.691596508026123, "grad_norm": 0.5231279134750366, "tokens_seen": 9510912, "tokens_per_sec": 22785.078041722667, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5681966145833334, "G_std": 0.18558756510416666, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.6201677322387695, "grad_norm": 0.5331099629402161, "tokens_seen": 9674752, "tokens_per_sec": 22788.055088429286, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5658365885416666, "G_std": 0.18129475911458334, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.687170028686523, "grad_norm": 0.5638405084609985, "tokens_seen": 9838592, "tokens_per_sec": 22790.84927483181, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5716145833333334, "G_std": 0.17962646484375, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.4847047328948975, "grad_norm": 0.506129801273346, "tokens_seen": 10002432, "tokens_per_sec": 22793.600241445067, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5725911458333334, "G_std": 0.178466796875, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.546180963516235, "grad_norm": 0.53159099817276, "tokens_seen": 10166272, "tokens_per_sec": 22795.573832405295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5689697265625, "G_std": 0.17665608723958334, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.698956489562988, "grad_norm": 0.5087539553642273, "tokens_seen": 10330112, "tokens_per_sec": 22797.894537978937, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5628662109375, "G_std": 0.17783610026041666, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.562479257583618, "grad_norm": 0.5763164162635803, "tokens_seen": 10493952, "tokens_per_sec": 22799.78357457366, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5513509114583334, "G_std": 0.1827392578125, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.654716730117798, "grad_norm": 0.5060703158378601, "tokens_seen": 10657792, "tokens_per_sec": 22800.86238695963, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5518798828125, "G_std": 0.18292236328125, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.569320440292358, "grad_norm": 0.5856291651725769, "tokens_seen": 10821632, "tokens_per_sec": 22802.85178220569, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5572102864583334, "G_std": 0.1812744140625, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.609747886657715, "grad_norm": 0.5133132338523865, "tokens_seen": 10985472, "tokens_per_sec": 22804.004214294597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.55322265625, "G_std": 0.18194580078125, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.463903903961182, "grad_norm": 0.532732367515564, "tokens_seen": 11149312, "tokens_per_sec": 22803.577883219543, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5423583984375, "G_std": 0.18170166015625, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.413821220397949, "grad_norm": 0.5075457096099854, "tokens_seen": 11313152, "tokens_per_sec": 22805.707005830547, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5546875, "G_std": 0.18050130208333334, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.608089208602905, "grad_norm": 0.5554553270339966, "tokens_seen": 11476992, "tokens_per_sec": 22806.547053947397, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5486246744791666, "G_std": 0.17987060546875, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.469797372817993, "grad_norm": 0.5766416788101196, "tokens_seen": 11640832, "tokens_per_sec": 22808.53406974516, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5527750651041666, "G_std": 0.17608642578125, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.3568079471588135, "grad_norm": 0.495700478553772, "tokens_seen": 11804672, "tokens_per_sec": 22811.303519902216, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5410970052083334, "G_std": 0.178955078125, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.534135341644287, "grad_norm": 0.5720467567443848, "tokens_seen": 11968512, "tokens_per_sec": 22811.979150175546, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5361328125, "G_std": 0.18064371744791666, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.379556894302368, "grad_norm": 0.5554018616676331, "tokens_seen": 12132352, "tokens_per_sec": 22813.144933160176, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.53466796875, "G_std": 0.17368570963541666, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.5036139488220215, "grad_norm": 0.5845478773117065, "tokens_seen": 12296192, "tokens_per_sec": 22815.315657266838, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5312296549479166, "G_std": 0.1787109375, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.480775356292725, "grad_norm": 0.5491233468055725, "tokens_seen": 12460032, "tokens_per_sec": 22814.8938813836, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5228678385416666, "G_std": 0.179443359375, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.341266632080078, "grad_norm": 0.49672991037368774, "tokens_seen": 12623872, "tokens_per_sec": 22816.07433895412, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5287068684895834, "G_std": 0.17561848958333334, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.408099174499512, "grad_norm": 0.5219045281410217, "tokens_seen": 12787712, "tokens_per_sec": 22815.262948798423, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5293375651041666, "G_std": 0.17962646484375, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.483325958251953, "grad_norm": 0.5178242921829224, "tokens_seen": 12951552, "tokens_per_sec": 22817.64293196875, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5349934895833334, "G_std": 0.17885335286458334, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.259401082992554, "grad_norm": 0.5235769152641296, "tokens_seen": 13115392, "tokens_per_sec": 22819.31258905074, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5268351236979166, "G_std": 0.17445882161458334, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.4751670360565186, "grad_norm": 0.562646210193634, "tokens_seen": 13279232, "tokens_per_sec": 22821.079248135986, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5283406575520834, "G_std": 0.174835205078125, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.345257759094238, "grad_norm": 0.5523222088813782, "tokens_seen": 13443072, "tokens_per_sec": 22823.14419041646, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5275675455729166, "G_std": 0.17728678385416666, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.349180459976196, "grad_norm": 0.5568654537200928, "tokens_seen": 13606912, "tokens_per_sec": 22824.178893885637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5144246419270834, "G_std": 0.181640625, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.374053478240967, "grad_norm": 0.5808308124542236, "tokens_seen": 13770752, "tokens_per_sec": 22825.74847977601, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5149739583333334, "G_std": 0.17781575520833334, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.527472972869873, "grad_norm": 0.5600183010101318, "tokens_seen": 13934592, "tokens_per_sec": 22826.4080181862, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5272013346354166, "G_std": 0.176177978515625, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.390435457229614, "grad_norm": 0.5476920008659363, "tokens_seen": 14098432, "tokens_per_sec": 22827.15022637735, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5162353515625, "G_std": 0.180694580078125, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.3222432136535645, "grad_norm": 0.5422391891479492, "tokens_seen": 14262272, "tokens_per_sec": 22827.75263507446, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5182698567708334, "G_std": 0.17657470703125, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.265296697616577, "grad_norm": 0.5615730881690979, "tokens_seen": 14426112, "tokens_per_sec": 22827.447685872805, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5232950846354166, "G_std": 0.177947998046875, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.413811445236206, "grad_norm": 0.5612704157829285, "tokens_seen": 14589952, "tokens_per_sec": 22827.75588450199, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5093790690104166, "G_std": 0.17554728190104166, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.324901103973389, "grad_norm": 0.5814974308013916, "tokens_seen": 14753792, "tokens_per_sec": 22829.228872996704, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.52410888671875, "G_std": 0.173492431640625, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.264135122299194, "grad_norm": 0.5482663512229919, "tokens_seen": 14917632, "tokens_per_sec": 22829.63596452654, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.51654052734375, "G_std": 0.176422119140625, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.431742429733276, "grad_norm": 0.603756844997406, "tokens_seen": 15081472, "tokens_per_sec": 22830.406688693016, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5089925130208334, "G_std": 0.17967732747395834, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.39985728263855, "grad_norm": 0.5765458345413208, "tokens_seen": 15245312, "tokens_per_sec": 22831.06043364012, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5112711588541666, "G_std": 0.17551676432291666, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.2872092723846436, "grad_norm": 0.5540627837181091, "tokens_seen": 15409152, "tokens_per_sec": 22832.16082874872, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5074259440104166, "G_std": 0.17815144856770834, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.187593698501587, "grad_norm": 0.5500850677490234, "tokens_seen": 15572992, "tokens_per_sec": 22832.704239844767, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5054931640625, "G_std": 0.17580159505208334, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.300659656524658, "grad_norm": 0.5496270656585693, "tokens_seen": 15736832, "tokens_per_sec": 22834.073420829962, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4974365234375, "G_std": 0.17610677083333334, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.402259349822998, "grad_norm": 0.5441314578056335, "tokens_seen": 15900672, "tokens_per_sec": 22834.928927073917, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4929606119791667, "G_std": 0.1788330078125, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.354232311248779, "grad_norm": 0.5718883275985718, "tokens_seen": 16064512, "tokens_per_sec": 22836.440116362075, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4899698893229167, "G_std": 0.18158976236979166, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.102896213531494, "grad_norm": 0.5643874406814575, "tokens_seen": 16228352, "tokens_per_sec": 22837.346891942285, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4876505533854167, "G_std": 0.1787109375, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.345609664916992, "grad_norm": 0.5548269152641296, "tokens_seen": 16392192, "tokens_per_sec": 22837.299890753915, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4908854166666667, "G_std": 0.179046630859375, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.350653719902039, "val_ppl": 210.7460217140297, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4868367513020833, "G_std": 0.17656453450520834, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.357493877410889, "grad_norm": 0.5549105405807495, "tokens_seen": 16556032, "tokens_per_sec": 22799.427794516025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4874267578125, "G_std": 0.17829386393229166, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.389228343963623, "grad_norm": 0.5507209897041321, "tokens_seen": 16719872, "tokens_per_sec": 22800.05636905886, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4888102213541667, "G_std": 0.17835489908854166, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.283127307891846, "grad_norm": 0.610722005367279, "tokens_seen": 16883712, "tokens_per_sec": 22800.61298387444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4858194986979167, "G_std": 0.176666259765625, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.19317364692688, "grad_norm": 0.5574914216995239, "tokens_seen": 17047552, "tokens_per_sec": 22801.191500061188, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4765625, "G_std": 0.17497762044270834, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.269284725189209, "grad_norm": 0.5549355745315552, "tokens_seen": 17211392, "tokens_per_sec": 22801.901820934785, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4853922526041667, "G_std": 0.176910400390625, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.323820352554321, "grad_norm": 0.5817962288856506, "tokens_seen": 17375232, "tokens_per_sec": 22804.552747859016, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4911092122395833, "G_std": 0.17711385091145834, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.117583513259888, "grad_norm": 0.5649645328521729, "tokens_seen": 17539072, "tokens_per_sec": 22805.465630358172, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4828084309895833, "G_std": 0.17729695638020834, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.094247102737427, "grad_norm": 0.5841973423957825, "tokens_seen": 17702912, "tokens_per_sec": 22806.371710663814, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4670613606770833, "G_std": 0.1759033203125, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.137040615081787, "grad_norm": 0.6193577647209167, "tokens_seen": 17866752, "tokens_per_sec": 22808.006242562293, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48150634765625, "G_std": 0.17661539713541666, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.331856727600098, "grad_norm": 0.5899665951728821, "tokens_seen": 18030592, "tokens_per_sec": 22808.76401809289, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4729817708333333, "G_std": 0.17375691731770834, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.276381254196167, "grad_norm": 0.6044086217880249, "tokens_seen": 18194432, "tokens_per_sec": 22808.25762402296, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4620157877604167, "G_std": 0.1767578125, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.258044719696045, "grad_norm": 0.6534725427627563, "tokens_seen": 18358272, "tokens_per_sec": 22809.523711436705, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4705403645833333, "G_std": 0.17403157552083334, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.120561838150024, "grad_norm": 0.6433582901954651, "tokens_seen": 18522112, "tokens_per_sec": 22810.34229093559, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.476348876953125, "G_std": 0.17470296223958334, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.236770868301392, "grad_norm": 0.7942308187484741, "tokens_seen": 18685952, "tokens_per_sec": 22811.223322907274, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4753926595052083, "G_std": 0.173797607421875, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.243417978286743, "grad_norm": 0.5824620723724365, "tokens_seen": 18849792, "tokens_per_sec": 22811.482786336095, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4723103841145833, "G_std": 0.17389933268229166, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.1854095458984375, "grad_norm": 0.5950741767883301, "tokens_seen": 19013632, "tokens_per_sec": 22812.615505115282, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4684041341145833, "G_std": 0.17332967122395834, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 4.966316223144531, "grad_norm": 0.6119787693023682, "tokens_seen": 19177472, "tokens_per_sec": 22813.069383668637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4742533365885417, "G_std": 0.17352294921875, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.2012927532196045, "grad_norm": 0.6087759733200073, "tokens_seen": 19341312, "tokens_per_sec": 22811.70314209643, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4731953938802083, "G_std": 0.17212931315104166, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.128464460372925, "grad_norm": 0.6723151206970215, "tokens_seen": 19505152, "tokens_per_sec": 22812.855441396106, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4737548828125, "G_std": 0.173553466796875, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.0776402950286865, "grad_norm": 0.6977710127830505, "tokens_seen": 19668992, "tokens_per_sec": 22812.716985409017, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4788411458333333, "G_std": 0.17078653971354166, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.108170032501221, "grad_norm": 0.6842029094696045, "tokens_seen": 19832832, "tokens_per_sec": 22813.515901055733, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4533894856770833, "G_std": 0.17438761393229166, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.130347728729248, "grad_norm": 0.6266114711761475, "tokens_seen": 19996672, "tokens_per_sec": 22814.51092276384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4674072265625, "G_std": 0.171600341796875, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.078304290771484, "grad_norm": 0.5935364365577698, "tokens_seen": 20160512, "tokens_per_sec": 22813.46472068106, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4534912109375, "G_std": 0.171722412109375, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.093810796737671, "grad_norm": 0.6163204312324524, "tokens_seen": 20324352, "tokens_per_sec": 22813.745862672058, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4597981770833333, "G_std": 0.17020670572916666, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.123593330383301, "grad_norm": 0.6341149806976318, "tokens_seen": 20488192, "tokens_per_sec": 22814.539622353015, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4583841959635417, "G_std": 0.17204793294270834, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.202310562133789, "grad_norm": 0.62943434715271, "tokens_seen": 20652032, "tokens_per_sec": 22815.074897903665, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4536234537760417, "G_std": 0.17375691731770834, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.058737516403198, "grad_norm": 0.6046798825263977, "tokens_seen": 20815872, "tokens_per_sec": 22816.456377375864, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4603068033854167, "G_std": 0.169342041015625, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.169283390045166, "grad_norm": 0.6355167627334595, "tokens_seen": 20979712, "tokens_per_sec": 22817.54216010314, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.466522216796875, "G_std": 0.170684814453125, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.034101247787476, "grad_norm": 0.6388559937477112, "tokens_seen": 21143552, "tokens_per_sec": 22818.006650091345, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4557596842447917, "G_std": 0.16986083984375, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.08925199508667, "grad_norm": 0.6586858034133911, "tokens_seen": 21307392, "tokens_per_sec": 22818.62544193851, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.463836669921875, "G_std": 0.171478271484375, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.0052924156188965, "grad_norm": 0.6375748515129089, "tokens_seen": 21471232, "tokens_per_sec": 22819.717140920886, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4479268391927083, "G_std": 0.17310587565104166, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 4.932558059692383, "grad_norm": 0.6780567169189453, "tokens_seen": 21635072, "tokens_per_sec": 22820.160327022568, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4512736002604167, "G_std": 0.17467244466145834, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.06007981300354, "grad_norm": 0.5976543426513672, "tokens_seen": 21798912, "tokens_per_sec": 22820.625791616298, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4517110188802083, "G_std": 0.1697998046875, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.037003517150879, "grad_norm": 0.5832242369651794, "tokens_seen": 21962752, "tokens_per_sec": 22821.51369601275, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4580586751302083, "G_std": 0.17084248860677084, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.0348076820373535, "grad_norm": 0.6161026358604431, "tokens_seen": 22126592, "tokens_per_sec": 22822.09581393246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4495340983072917, "G_std": 0.17349751790364584, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.106776237487793, "grad_norm": 0.7287216782569885, "tokens_seen": 22290432, "tokens_per_sec": 22823.046261799664, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4566243489583333, "G_std": 0.17139689127604166, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.077770471572876, "grad_norm": 0.623251736164093, "tokens_seen": 22454272, "tokens_per_sec": 22823.67076469657, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4525248209635417, "G_std": 0.17020670572916666, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.094495534896851, "grad_norm": 0.6832799315452576, "tokens_seen": 22618112, "tokens_per_sec": 22823.967079491213, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.451171875, "G_std": 0.16683451334635416, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.014079809188843, "grad_norm": 0.6595131754875183, "tokens_seen": 22781952, "tokens_per_sec": 22825.584426525416, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4556172688802083, "G_std": 0.17005411783854166, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.978665828704834, "grad_norm": 0.6033042073249817, "tokens_seen": 22945792, "tokens_per_sec": 22826.51404572719, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4503275553385417, "G_std": 0.16713968912760416, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.1223602294921875, "grad_norm": 0.6445069909095764, "tokens_seen": 23109632, "tokens_per_sec": 22827.416424080675, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.447479248046875, "G_std": 0.1711273193359375, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.093594074249268, "grad_norm": 0.6342979669570923, "tokens_seen": 23273472, "tokens_per_sec": 22828.617218852356, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4444783528645833, "G_std": 0.16931660970052084, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.939738988876343, "grad_norm": 0.6687210202217102, "tokens_seen": 23437312, "tokens_per_sec": 22828.99425981672, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.454376220703125, "G_std": 0.1670379638671875, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.000877857208252, "grad_norm": 0.6367366313934326, "tokens_seen": 23601152, "tokens_per_sec": 22829.034264464273, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4485066731770833, "G_std": 0.168853759765625, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.8393261432647705, "grad_norm": 0.6065342426300049, "tokens_seen": 23764992, "tokens_per_sec": 22829.754371353352, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4518534342447917, "G_std": 0.16781107584635416, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.949954509735107, "grad_norm": 0.638183057308197, "tokens_seen": 23928832, "tokens_per_sec": 22830.503497625807, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4477640787760417, "G_std": 0.16584269205729166, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.001371383666992, "grad_norm": 0.6126130819320679, "tokens_seen": 24092672, "tokens_per_sec": 22831.32316600756, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44293212890625, "G_std": 0.17189534505208334, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.848941326141357, "grad_norm": 0.6780807375907898, "tokens_seen": 24256512, "tokens_per_sec": 22831.764591790943, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.452545166015625, "G_std": 0.1674041748046875, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.765554666519165, "grad_norm": 0.9862808585166931, "tokens_seen": 24420352, "tokens_per_sec": 22831.57188846173, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4368489583333333, "G_std": 0.16910298665364584, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.0422890186309814, "grad_norm": 0.7057256698608398, "tokens_seen": 24584192, "tokens_per_sec": 22832.383532435808, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44708251953125, "G_std": 0.16956583658854166, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.007157731056213, "val_ppl": 149.47927150186646, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.440460205078125, "G_std": 0.16880289713541666, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.106330156326294, "grad_norm": 0.7093400955200195, "tokens_seen": 24748032, "tokens_per_sec": 22806.56731525546, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4429728190104167, "G_std": 0.16755167643229166, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.831097841262817, "grad_norm": 0.6279119849205017, "tokens_seen": 24911872, "tokens_per_sec": 22806.630651879328, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.436279296875, "G_std": 0.16692606608072916, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.909238576889038, "grad_norm": 0.6423178911209106, "tokens_seen": 25075712, "tokens_per_sec": 22806.899193175617, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4434000651041667, "G_std": 0.165618896484375, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.851612567901611, "grad_norm": 0.6636707782745361, "tokens_seen": 25239552, "tokens_per_sec": 22807.65935091991, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4447123209635417, "G_std": 0.16628519694010416, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.01123046875, "grad_norm": 0.7403873205184937, "tokens_seen": 25403392, "tokens_per_sec": 22808.04327684576, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4425862630208333, "G_std": 0.16737874348958334, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.761071443557739, "grad_norm": 0.7207083702087402, "tokens_seen": 25567232, "tokens_per_sec": 22808.68589620088, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4352518717447917, "G_std": 0.16495768229166666, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.898327112197876, "grad_norm": 0.6634817123413086, "tokens_seen": 25731072, "tokens_per_sec": 22809.622463213793, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.445343017578125, "G_std": 0.16994730631510416, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.832717418670654, "grad_norm": 0.7060012221336365, "tokens_seen": 25894912, "tokens_per_sec": 22810.921009893336, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4369913736979167, "G_std": 0.16759745279947916, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.870481014251709, "grad_norm": 0.6876097321510315, "tokens_seen": 26058752, "tokens_per_sec": 22811.596196388433, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4369303385416667, "G_std": 0.17035420735677084, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.849500894546509, "grad_norm": 0.6498600840568542, "tokens_seen": 26222592, "tokens_per_sec": 22812.46314913978, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4281005859375, "G_std": 0.16880289713541666, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.944379568099976, "grad_norm": 0.679940938949585, "tokens_seen": 26386432, "tokens_per_sec": 22812.554358180125, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4395039876302083, "G_std": 0.167724609375, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.93098258972168, "grad_norm": 0.6583362221717834, "tokens_seen": 26550272, "tokens_per_sec": 22812.69999373527, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4372151692708333, "G_std": 0.17014058430989584, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.987542390823364, "grad_norm": 0.6829982399940491, "tokens_seen": 26714112, "tokens_per_sec": 22813.490907749558, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.438079833984375, "G_std": 0.16834004720052084, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.800548553466797, "grad_norm": 0.6275656819343567, "tokens_seen": 26877952, "tokens_per_sec": 22814.0890840975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4372660319010417, "G_std": 0.167724609375, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.069140434265137, "grad_norm": 0.6785210371017456, "tokens_seen": 27041792, "tokens_per_sec": 22813.090165007143, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4263916015625, "G_std": 0.16918436686197916, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.912757873535156, "grad_norm": 0.6797897219657898, "tokens_seen": 27205632, "tokens_per_sec": 22813.26119942668, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4330647786458333, "G_std": 0.16748046875, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.875095844268799, "grad_norm": 0.6850050687789917, "tokens_seen": 27369472, "tokens_per_sec": 22813.883275136355, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4308980305989583, "G_std": 0.16704813639322916, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.907244443893433, "grad_norm": 0.6510031819343567, "tokens_seen": 27533312, "tokens_per_sec": 22814.862426892105, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.431793212890625, "G_std": 0.16676839192708334, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.036442041397095, "grad_norm": 0.7013916969299316, "tokens_seen": 27697152, "tokens_per_sec": 22815.983028955456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4320576985677083, "G_std": 0.16582743326822916, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.977211236953735, "grad_norm": 0.6692748069763184, "tokens_seen": 27860992, "tokens_per_sec": 22816.57317478549, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4368184407552083, "G_std": 0.16526285807291666, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.911931753158569, "grad_norm": 0.6899727582931519, "tokens_seen": 28024832, "tokens_per_sec": 22817.45828742877, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4283955891927083, "G_std": 0.166778564453125, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.712474584579468, "grad_norm": 0.6473119854927063, "tokens_seen": 28188672, "tokens_per_sec": 22818.201895879436, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4224650065104167, "G_std": 0.167388916015625, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.940701723098755, "grad_norm": 0.6846835613250732, "tokens_seen": 28352512, "tokens_per_sec": 22817.586940567333, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4336140950520833, "G_std": 0.16611735026041666, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.947508335113525, "grad_norm": 0.7365009188652039, "tokens_seen": 28516352, "tokens_per_sec": 22818.00406489974, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4278564453125, "G_std": 0.16468302408854166, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 5.05382513999939, "grad_norm": 0.6789611577987671, "tokens_seen": 28680192, "tokens_per_sec": 22818.474621172252, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4419148763020833, "G_std": 0.16460164388020834, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.816776990890503, "grad_norm": 0.6751706004142761, "tokens_seen": 28844032, "tokens_per_sec": 22819.131334305206, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4314676920572917, "G_std": 0.1686248779296875, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.8347320556640625, "grad_norm": 0.6628151535987854, "tokens_seen": 29007872, "tokens_per_sec": 22820.01352380207, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.418548583984375, "G_std": 0.16607157389322916, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.8867881298065186, "grad_norm": 0.7082439661026001, "tokens_seen": 29171712, "tokens_per_sec": 22820.539796195597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.422271728515625, "G_std": 0.16595458984375, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.9077558517456055, "grad_norm": 0.7131461501121521, "tokens_seen": 29335552, "tokens_per_sec": 22820.85301249118, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4254964192708333, "G_std": 0.16752115885416666, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.889211416244507, "grad_norm": 0.6866649985313416, "tokens_seen": 29499392, "tokens_per_sec": 22820.395281417892, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4209391276041667, "G_std": 0.16601053873697916, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.893506288528442, "grad_norm": 0.6834511756896973, "tokens_seen": 29663232, "tokens_per_sec": 22820.700022059627, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.422454833984375, "G_std": 0.165771484375, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.789041042327881, "grad_norm": 0.6470667123794556, "tokens_seen": 29827072, "tokens_per_sec": 22820.53498047562, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42218017578125, "G_std": 0.1626739501953125, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 5.004823207855225, "grad_norm": 0.7034148573875427, "tokens_seen": 29990912, "tokens_per_sec": 22821.456924406186, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42950439453125, "G_std": 0.1635589599609375, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.832180976867676, "grad_norm": 0.8216344714164734, "tokens_seen": 30154752, "tokens_per_sec": 22822.02737024025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.422943115234375, "G_std": 0.16304524739583334, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.9836266040802, "grad_norm": 0.7173963189125061, "tokens_seen": 30318592, "tokens_per_sec": 22822.368890826063, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.427398681640625, "G_std": 0.16604105631510416, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.945801258087158, "grad_norm": 0.7271575927734375, "tokens_seen": 30482432, "tokens_per_sec": 22823.163470904332, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4256490071614583, "G_std": 0.16342671712239584, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.872276544570923, "grad_norm": 0.7364190816879272, "tokens_seen": 30646272, "tokens_per_sec": 22823.229828802207, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.424468994140625, "G_std": 0.1641387939453125, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.80099630355835, "grad_norm": 0.6994839906692505, "tokens_seen": 30810112, "tokens_per_sec": 22823.89150782962, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4154510498046875, "G_std": 0.1676483154296875, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.763928651809692, "grad_norm": 0.7442662715911865, "tokens_seen": 30973952, "tokens_per_sec": 22824.449867936044, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4186299641927083, "G_std": 0.16724141438802084, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.893190860748291, "grad_norm": 0.692230224609375, "tokens_seen": 31137792, "tokens_per_sec": 22823.582857505622, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4244283040364583, "G_std": 0.1637115478515625, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 5.003762722015381, "grad_norm": 0.7365623712539673, "tokens_seen": 31301632, "tokens_per_sec": 22824.731349653648, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4277852376302083, "G_std": 0.16282145182291666, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.904173851013184, "grad_norm": 0.7149618864059448, "tokens_seen": 31465472, "tokens_per_sec": 22824.986707690674, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4226481119791667, "G_std": 0.163543701171875, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.8622846603393555, "grad_norm": 0.6719006299972534, "tokens_seen": 31629312, "tokens_per_sec": 22824.328073238645, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.413299560546875, "G_std": 0.16511027018229166, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.881468296051025, "grad_norm": 0.701644241809845, "tokens_seen": 31793152, "tokens_per_sec": 22822.319024368666, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4314982096354167, "G_std": 0.16680908203125, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.900235652923584, "grad_norm": 0.6875312924385071, "tokens_seen": 31956992, "tokens_per_sec": 22821.005604644106, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4247538248697917, "G_std": 0.166168212890625, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.862747669219971, "grad_norm": 0.6602141857147217, "tokens_seen": 32120832, "tokens_per_sec": 22819.48597419128, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4154103597005208, "G_std": 0.16510009765625, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.745626449584961, "grad_norm": 0.7077448964118958, "tokens_seen": 32284672, "tokens_per_sec": 22817.391629851933, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4094696044921875, "G_std": 0.1622314453125, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.737863302230835, "grad_norm": 0.7550477385520935, "tokens_seen": 32448512, "tokens_per_sec": 22815.055340251663, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4175059000651042, "G_std": 0.16539510091145834, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.823720932006836, "grad_norm": 0.7020337581634521, "tokens_seen": 32612352, "tokens_per_sec": 22812.595643686618, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4167734781901042, "G_std": 0.165924072265625, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.750531911849976, "grad_norm": 0.719837486743927, "tokens_seen": 32776192, "tokens_per_sec": 22810.743576195564, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.423309326171875, "G_std": 0.16449483235677084, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.882469940185547, "val_ppl": 131.9561855962991, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4209086100260417, "G_std": 0.1634979248046875, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.712092876434326, "grad_norm": 0.7037155032157898, "tokens_seen": 32940032, "tokens_per_sec": 22789.186074693713, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40753173828125, "G_std": 0.16619364420572916, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 5.091538190841675, "grad_norm": 0.7212201952934265, "tokens_seen": 33103872, "tokens_per_sec": 22787.81876645471, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4153238932291667, "G_std": 0.16530863444010416, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.740626811981201, "grad_norm": 0.720624566078186, "tokens_seen": 33267712, "tokens_per_sec": 22785.105286104186, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4163818359375, "G_std": 0.16510009765625, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.886517524719238, "grad_norm": 0.7521200180053711, "tokens_seen": 33431552, "tokens_per_sec": 22782.870066242038, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.417144775390625, "G_std": 0.16536966959635416, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.818441867828369, "grad_norm": 0.7016645073890686, "tokens_seen": 33595392, "tokens_per_sec": 22781.858428978838, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4085489908854167, "G_std": 0.1674652099609375, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.782992362976074, "grad_norm": 0.7196379899978638, "tokens_seen": 33759232, "tokens_per_sec": 22780.109520701935, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4178873697916667, "G_std": 0.16257222493489584, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.809381008148193, "grad_norm": 0.6417549252510071, "tokens_seen": 33923072, "tokens_per_sec": 22779.703164360664, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41571044921875, "G_std": 0.1636962890625, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.757610082626343, "grad_norm": 0.7354642152786255, "tokens_seen": 34086912, "tokens_per_sec": 22778.502952113988, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.410675048828125, "G_std": 0.16522725423177084, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.727074384689331, "grad_norm": 0.6785760521888733, "tokens_seen": 34250752, "tokens_per_sec": 22777.57485081086, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4163411458333333, "G_std": 0.16455078125, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.8280558586120605, "grad_norm": 0.6940272450447083, "tokens_seen": 34414592, "tokens_per_sec": 22776.378581801328, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4108734130859375, "G_std": 0.16422526041666666, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.661916971206665, "grad_norm": 0.7068778276443481, "tokens_seen": 34578432, "tokens_per_sec": 22774.32539469956, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4121449788411458, "G_std": 0.16406758626302084, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.844691276550293, "grad_norm": 0.7232838869094849, "tokens_seen": 34742272, "tokens_per_sec": 22771.9800083164, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4267069498697917, "G_std": 0.1619415283203125, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.879985809326172, "grad_norm": 0.7385551929473877, "tokens_seen": 34906112, "tokens_per_sec": 22771.461862263335, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.410614013671875, "G_std": 0.163848876953125, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.795638084411621, "grad_norm": 0.7246646881103516, "tokens_seen": 35069952, "tokens_per_sec": 22771.36952369665, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4134063720703125, "G_std": 0.1639404296875, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.712125778198242, "grad_norm": 0.6592586636543274, "tokens_seen": 35233792, "tokens_per_sec": 22769.418015995492, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.413177490234375, "G_std": 0.163543701171875, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.750871181488037, "grad_norm": 0.7194413542747498, "tokens_seen": 35397632, "tokens_per_sec": 22768.955001730516, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.407379150390625, "G_std": 0.16209920247395834, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.863519668579102, "grad_norm": 0.7364917993545532, "tokens_seen": 35561472, "tokens_per_sec": 22768.86602608387, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4117991129557292, "G_std": 0.164276123046875, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.905673265457153, "grad_norm": 0.7395825982093811, "tokens_seen": 35725312, "tokens_per_sec": 22769.226670129665, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4074147542317708, "G_std": 0.16411336263020834, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.779814004898071, "grad_norm": 0.7258769273757935, "tokens_seen": 35889152, "tokens_per_sec": 22769.247288841863, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4029998779296875, "G_std": 0.16589864095052084, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.6841514110565186, "grad_norm": 0.7617036700248718, "tokens_seen": 36052992, "tokens_per_sec": 22768.265740160237, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4013519287109375, "G_std": 0.16162109375, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 5.005581617355347, "grad_norm": 0.7274962067604065, "tokens_seen": 36216832, "tokens_per_sec": 22768.325650720002, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4063975016276042, "G_std": 0.16490681966145834, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.883593559265137, "grad_norm": 0.792150616645813, "tokens_seen": 36380672, "tokens_per_sec": 22766.7559941504, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.405426025390625, "G_std": 0.16425069173177084, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.774924278259277, "grad_norm": 0.7087907195091248, "tokens_seen": 36544512, "tokens_per_sec": 22765.237851789538, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4107157389322917, "G_std": 0.16375732421875, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.965415954589844, "grad_norm": 0.7052822113037109, "tokens_seen": 36708352, "tokens_per_sec": 22765.304199544404, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4049072265625, "G_std": 0.16338094075520834, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.908033132553101, "grad_norm": 0.7700512409210205, "tokens_seen": 36872192, "tokens_per_sec": 22765.032514871647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4048868815104167, "G_std": 0.16475423177083334, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.743353843688965, "grad_norm": 0.7622653841972351, "tokens_seen": 37036032, "tokens_per_sec": 22765.193758356505, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4059295654296875, "G_std": 0.16338094075520834, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.796634912490845, "grad_norm": 0.7187082171440125, "tokens_seen": 37199872, "tokens_per_sec": 22765.018794564556, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4032847086588542, "G_std": 0.163909912109375, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.941740989685059, "grad_norm": 0.7627132534980774, "tokens_seen": 37363712, "tokens_per_sec": 22764.827037794774, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4101206461588542, "G_std": 0.1650543212890625, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.800657033920288, "grad_norm": 0.7223362922668457, "tokens_seen": 37527552, "tokens_per_sec": 22764.869495898805, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3996378580729167, "G_std": 0.1646270751953125, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.80526876449585, "grad_norm": 0.6925631165504456, "tokens_seen": 37691392, "tokens_per_sec": 22763.42380185319, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4051259358723958, "G_std": 0.16379292805989584, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.74315071105957, "grad_norm": 0.8159737586975098, "tokens_seen": 37855232, "tokens_per_sec": 22763.31518000064, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3959910074869792, "G_std": 0.16140238444010416, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.723592758178711, "grad_norm": 0.7145493030548096, "tokens_seen": 38019072, "tokens_per_sec": 22763.201249986407, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4019775390625, "G_std": 0.1641998291015625, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.939239978790283, "grad_norm": 0.7428827881813049, "tokens_seen": 38182912, "tokens_per_sec": 22763.00410974282, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4112650553385417, "G_std": 0.163116455078125, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.595138311386108, "grad_norm": 0.7207263708114624, "tokens_seen": 38346752, "tokens_per_sec": 22762.743727564248, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39990234375, "G_std": 0.16385904947916666, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.630107402801514, "grad_norm": 0.7377076148986816, "tokens_seen": 38510592, "tokens_per_sec": 22762.909166491972, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3994547526041667, "G_std": 0.16255696614583334, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.873649835586548, "grad_norm": 0.7206186652183533, "tokens_seen": 38674432, "tokens_per_sec": 22762.6557389194, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4100596110026042, "G_std": 0.16311136881510416, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.639462947845459, "grad_norm": 0.7558109760284424, "tokens_seen": 38838272, "tokens_per_sec": 22762.59470752353, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40460205078125, "G_std": 0.16218058268229166, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.79897928237915, "grad_norm": 0.6850783228874207, "tokens_seen": 39002112, "tokens_per_sec": 22763.035208358404, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4061330159505208, "G_std": 0.16281636555989584, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.741949558258057, "grad_norm": 0.776498556137085, "tokens_seen": 39165952, "tokens_per_sec": 22763.171278315676, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4083251953125, "G_std": 0.16402689615885416, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.758931875228882, "grad_norm": 0.7536342144012451, "tokens_seen": 39329792, "tokens_per_sec": 22763.238562241404, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4020233154296875, "G_std": 0.16421000162760416, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.609412670135498, "grad_norm": 0.7370294332504272, "tokens_seen": 39493632, "tokens_per_sec": 22763.78201780527, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4012705485026042, "G_std": 0.16290283203125, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.7299675941467285, "grad_norm": 0.7098134756088257, "tokens_seen": 39657472, "tokens_per_sec": 22763.804203250464, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3994700113932292, "G_std": 0.1642888387044271, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.766841411590576, "grad_norm": 0.7420174479484558, "tokens_seen": 39821312, "tokens_per_sec": 22763.499530675766, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4029388427734375, "G_std": 0.1641693115234375, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.7863054275512695, "grad_norm": 0.7582847476005554, "tokens_seen": 39985152, "tokens_per_sec": 22763.086949033786, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4005940755208333, "G_std": 0.16329447428385416, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.571653366088867, "grad_norm": 0.7562537789344788, "tokens_seen": 40148992, "tokens_per_sec": 22762.961063934617, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3990275065104167, "G_std": 0.1612701416015625, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.794485092163086, "grad_norm": 0.7669829726219177, "tokens_seen": 40312832, "tokens_per_sec": 22762.890646231008, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3951314290364583, "G_std": 0.16234842936197916, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.656063079833984, "grad_norm": 0.7462642788887024, "tokens_seen": 40476672, "tokens_per_sec": 22763.163396543776, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3956502278645833, "G_std": 0.161224365234375, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.726163625717163, "grad_norm": 0.7286989092826843, "tokens_seen": 40640512, "tokens_per_sec": 22763.229432850472, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3982798258463542, "G_std": 0.16423797607421875, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.683706283569336, "grad_norm": 0.7476567625999451, "tokens_seen": 40804352, "tokens_per_sec": 22763.49834075557, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3992360432942708, "G_std": 0.16190592447916666, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.558263301849365, "grad_norm": 0.7119042277336121, "tokens_seen": 40968192, "tokens_per_sec": 22763.938185857456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3993784586588542, "G_std": 0.16459147135416666, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.758196091651916, "val_ppl": 116.53551678509191, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3961893717447917, "G_std": 0.16360982259114584, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.74195408821106, "grad_norm": 0.7130697965621948, "tokens_seen": 41132032, "tokens_per_sec": 22747.46478996398, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3994496663411458, "G_std": 0.16222127278645834, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.606295108795166, "grad_norm": 0.7120617032051086, "tokens_seen": 41295872, "tokens_per_sec": 22747.02804901179, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3942159016927083, "G_std": 0.16260274251302084, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.656107425689697, "grad_norm": 0.8557688593864441, "tokens_seen": 41459712, "tokens_per_sec": 22746.6871364765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4005839029947917, "G_std": 0.16082255045572916, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.72005558013916, "grad_norm": 0.7693690657615662, "tokens_seen": 41623552, "tokens_per_sec": 22745.77802691435, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3964996337890625, "G_std": 0.16325887044270834, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.613642930984497, "grad_norm": 0.7438009977340698, "tokens_seen": 41787392, "tokens_per_sec": 22745.04682981735, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4021250406901042, "G_std": 0.16117350260416666, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.73008918762207, "grad_norm": 0.7389272451400757, "tokens_seen": 41951232, "tokens_per_sec": 22744.758388633712, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3931172688802083, "G_std": 0.1625213623046875, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.6593523025512695, "grad_norm": 0.7081330418586731, "tokens_seen": 42115072, "tokens_per_sec": 22744.76203058341, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3948262532552083, "G_std": 0.16309102376302084, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.7813637256622314, "grad_norm": 0.76569002866745, "tokens_seen": 42278912, "tokens_per_sec": 22744.001606132188, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3958943684895833, "G_std": 0.16518656412760416, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.508790969848633, "grad_norm": 0.7176042795181274, "tokens_seen": 42442752, "tokens_per_sec": 22743.7509225336, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3923288981119792, "G_std": 0.16161092122395834, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.529616117477417, "grad_norm": 0.8119246363639832, "tokens_seen": 42606592, "tokens_per_sec": 22744.105422781467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3952128092447917, "G_std": 0.16182708740234375, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.731082916259766, "grad_norm": 0.734362781047821, "tokens_seen": 42770432, "tokens_per_sec": 22744.27479763347, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4017283121744792, "G_std": 0.16164398193359375, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.549100399017334, "grad_norm": 0.7865417003631592, "tokens_seen": 42934272, "tokens_per_sec": 22744.166376251465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3964589436848958, "G_std": 0.1617253621419271, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.695314407348633, "grad_norm": 0.7825378775596619, "tokens_seen": 43098112, "tokens_per_sec": 22743.439863290354, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3982747395833333, "G_std": 0.16223653157552084, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.558470964431763, "grad_norm": 0.7872017621994019, "tokens_seen": 43261952, "tokens_per_sec": 22743.341333733748, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3896280924479167, "G_std": 0.1636505126953125, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.744646072387695, "grad_norm": 0.741649866104126, "tokens_seen": 43425792, "tokens_per_sec": 22742.824665292672, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3952687581380208, "G_std": 0.16265869140625, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.580264091491699, "grad_norm": 0.7445791363716125, "tokens_seen": 43589632, "tokens_per_sec": 22742.254484891975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3903961181640625, "G_std": 0.16046142578125, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.568654537200928, "grad_norm": 0.7086814045906067, "tokens_seen": 43753472, "tokens_per_sec": 22742.432419940116, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3938547770182292, "G_std": 0.1622772216796875, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.595623254776001, "grad_norm": 0.7577221393585205, "tokens_seen": 43917312, "tokens_per_sec": 22742.565807866868, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3990020751953125, "G_std": 0.16136932373046875, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.720735311508179, "grad_norm": 0.7865819931030273, "tokens_seen": 44081152, "tokens_per_sec": 22742.892239716515, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3990325927734375, "G_std": 0.16185760498046875, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.660937309265137, "grad_norm": 0.7591977119445801, "tokens_seen": 44244992, "tokens_per_sec": 22743.099730706093, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3939107259114583, "G_std": 0.16168212890625, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.791429758071899, "grad_norm": 0.821290910243988, "tokens_seen": 44408832, "tokens_per_sec": 22743.39304292781, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3936055501302083, "G_std": 0.16324869791666666, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.532271862030029, "grad_norm": 0.7583420872688293, "tokens_seen": 44572672, "tokens_per_sec": 22743.80293174215, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3960723876953125, "G_std": 0.1648406982421875, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.607854604721069, "grad_norm": 0.7447287440299988, "tokens_seen": 44736512, "tokens_per_sec": 22744.122853274814, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3955739339192708, "G_std": 0.1614354451497396, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.572662353515625, "grad_norm": 0.7542729377746582, "tokens_seen": 44900352, "tokens_per_sec": 22744.295290601607, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3958841959635417, "G_std": 0.1631800333658854, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.778303384780884, "grad_norm": 0.7581915855407715, "tokens_seen": 45064192, "tokens_per_sec": 22744.729438965318, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3992258707682292, "G_std": 0.16208902994791666, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.789998292922974, "grad_norm": 0.7751585245132446, "tokens_seen": 45228032, "tokens_per_sec": 22744.406227560714, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3965098063151042, "G_std": 0.1623992919921875, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.667697906494141, "grad_norm": 0.7715129852294922, "tokens_seen": 45391872, "tokens_per_sec": 22744.578800552892, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3919016520182292, "G_std": 0.16186014811197916, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.685269117355347, "grad_norm": 0.8284736275672913, "tokens_seen": 45555712, "tokens_per_sec": 22745.11080467101, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3936665852864583, "G_std": 0.1632258097330729, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.503671407699585, "grad_norm": 0.7420321702957153, "tokens_seen": 45719552, "tokens_per_sec": 22744.467532371338, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3902028401692708, "G_std": 0.1624755859375, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.716025114059448, "grad_norm": 0.803184986114502, "tokens_seen": 45883392, "tokens_per_sec": 22744.746175401393, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3973846435546875, "G_std": 0.16312408447265625, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.595407724380493, "grad_norm": 0.7794885635375977, "tokens_seen": 46047232, "tokens_per_sec": 22744.178361829265, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3925882975260417, "G_std": 0.16322835286458334, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.600069999694824, "grad_norm": 0.7839471697807312, "tokens_seen": 46211072, "tokens_per_sec": 22744.216255323678, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3956756591796875, "G_std": 0.15961201985677084, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.643089771270752, "grad_norm": 0.7999619841575623, "tokens_seen": 46374912, "tokens_per_sec": 22744.598773351787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39141845703125, "G_std": 0.16039021809895834, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.736012697219849, "grad_norm": 0.7935280799865723, "tokens_seen": 46538752, "tokens_per_sec": 22744.567080990164, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3893585205078125, "G_std": 0.15952555338541666, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.615527629852295, "grad_norm": 0.7788272500038147, "tokens_seen": 46702592, "tokens_per_sec": 22744.452319789638, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3962758382161458, "G_std": 0.162353515625, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.62798547744751, "grad_norm": 0.7386086583137512, "tokens_seen": 46866432, "tokens_per_sec": 22744.03611554338, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3903859456380208, "G_std": 0.1619237263997396, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.642121076583862, "grad_norm": 0.7796318531036377, "tokens_seen": 47030272, "tokens_per_sec": 22743.970839238125, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3833669026692708, "G_std": 0.16196695963541666, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.557812213897705, "grad_norm": 0.7282493710517883, "tokens_seen": 47194112, "tokens_per_sec": 22743.907610620492, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831431070963542, "G_std": 0.1611175537109375, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.653724431991577, "grad_norm": 0.7709184288978577, "tokens_seen": 47357952, "tokens_per_sec": 22744.253088183952, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3888905843098958, "G_std": 0.1606114705403646, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.634355068206787, "grad_norm": 0.798969566822052, "tokens_seen": 47521792, "tokens_per_sec": 22743.667941480573, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3884175618489583, "G_std": 0.1605351765950521, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.4703450202941895, "grad_norm": 0.8034460544586182, "tokens_seen": 47685632, "tokens_per_sec": 22743.84053777813, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.391265869140625, "G_std": 0.16143290201822916, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.558192729949951, "grad_norm": 0.8278152346611023, "tokens_seen": 47849472, "tokens_per_sec": 22744.03620633963, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3889923095703125, "G_std": 0.16086324055989584, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.469270467758179, "grad_norm": 0.7608954906463623, "tokens_seen": 48013312, "tokens_per_sec": 22744.1202235001, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809254964192708, "G_std": 0.1613591512044271, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.498936176300049, "grad_norm": 0.7520054578781128, "tokens_seen": 48177152, "tokens_per_sec": 22744.45328820141, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3909505208333333, "G_std": 0.1609980265299479, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.541987419128418, "grad_norm": 0.7638197541236877, "tokens_seen": 48340992, "tokens_per_sec": 22744.717560532314, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3935089111328125, "G_std": 0.1612091064453125, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.517196178436279, "grad_norm": 0.7671023607254028, "tokens_seen": 48504832, "tokens_per_sec": 22744.670131382445, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3875630696614583, "G_std": 0.1607945760091146, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.688551902770996, "grad_norm": 0.7636718153953552, "tokens_seen": 48668672, "tokens_per_sec": 22742.045949120788, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3858286539713542, "G_std": 0.16158040364583334, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.609457731246948, "grad_norm": 0.7789603471755981, "tokens_seen": 48832512, "tokens_per_sec": 22742.20318756323, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3858795166015625, "G_std": 0.15970865885416666, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.446531534194946, "grad_norm": 0.780012845993042, "tokens_seen": 48996352, "tokens_per_sec": 22741.494470989186, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3843536376953125, "G_std": 0.16075897216796875, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.688552618026733, "grad_norm": 0.8090319633483887, "tokens_seen": 49160192, "tokens_per_sec": 22741.631964215554, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3870442708333333, "G_std": 0.16200764973958334, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.716137623786926, "val_ppl": 111.73585226090988, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3938140869140625, "G_std": 0.16169230143229166, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.7726829051971436, "grad_norm": 0.7867392301559448, "tokens_seen": 49324032, "tokens_per_sec": 22728.16990569647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3819630940755208, "G_std": 0.1611328125, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.5588109493255615, "grad_norm": 0.7298011779785156, "tokens_seen": 49487872, "tokens_per_sec": 22727.389552959532, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3833211263020833, "G_std": 0.16083272298177084, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.490030288696289, "grad_norm": 0.7890421748161316, "tokens_seen": 49651712, "tokens_per_sec": 22727.539533834355, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3892873128255208, "G_std": 0.1612701416015625, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.52440881729126, "grad_norm": 0.7836453318595886, "tokens_seen": 49815552, "tokens_per_sec": 22727.802786433087, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38330078125, "G_std": 0.16499837239583334, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.639634132385254, "grad_norm": 0.7445854544639587, "tokens_seen": 49979392, "tokens_per_sec": 22727.662769343795, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3903299967447917, "G_std": 0.1629155476888021, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.675338983535767, "grad_norm": 0.8139757513999939, "tokens_seen": 50143232, "tokens_per_sec": 22727.57187583276, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3945414225260417, "G_std": 0.1615447998046875, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.646991729736328, "grad_norm": 0.8253708481788635, "tokens_seen": 50307072, "tokens_per_sec": 22727.97181980595, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3904673258463542, "G_std": 0.16118876139322916, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.508465528488159, "grad_norm": 0.7944724559783936, "tokens_seen": 50470912, "tokens_per_sec": 22727.791560205093, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3872528076171875, "G_std": 0.16059112548828125, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.607658386230469, "grad_norm": 0.806622326374054, "tokens_seen": 50634752, "tokens_per_sec": 22728.178780189883, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845926920572917, "G_std": 0.16024271647135416, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.53732442855835, "grad_norm": 0.9138648509979248, "tokens_seen": 50798592, "tokens_per_sec": 22728.21638480563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3866729736328125, "G_std": 0.16080220540364584, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.5756659507751465, "grad_norm": 0.8405846953392029, "tokens_seen": 50962432, "tokens_per_sec": 22728.08971325444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3878377278645833, "G_std": 0.1586761474609375, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.666730880737305, "grad_norm": 0.7922633290290833, "tokens_seen": 51126272, "tokens_per_sec": 22728.507091994474, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3870086669921875, "G_std": 0.16053263346354166, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.545642614364624, "grad_norm": 0.782463788986206, "tokens_seen": 51290112, "tokens_per_sec": 22728.089923972966, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3788045247395833, "G_std": 0.16028340657552084, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.535852670669556, "grad_norm": 0.8046404719352722, "tokens_seen": 51453952, "tokens_per_sec": 22728.19014248086, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.383392333984375, "G_std": 0.16066487630208334, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.669864654541016, "grad_norm": 0.7716073393821716, "tokens_seen": 51617792, "tokens_per_sec": 22728.673444807515, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3852488199869792, "G_std": 0.1599884033203125, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.513406753540039, "grad_norm": 0.7928930521011353, "tokens_seen": 51781632, "tokens_per_sec": 22728.72859962678, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3807576497395833, "G_std": 0.15894063313802084, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.624105453491211, "grad_norm": 0.8150955438613892, "tokens_seen": 51945472, "tokens_per_sec": 22728.607531495534, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834330240885417, "G_std": 0.15990193684895834, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.7471764087677, "grad_norm": 0.8214708566665649, "tokens_seen": 52109312, "tokens_per_sec": 22728.874858044986, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3865814208984375, "G_std": 0.16065470377604166, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.312466859817505, "grad_norm": 0.7773107886314392, "tokens_seen": 52273152, "tokens_per_sec": 22728.47796646643, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3859049479166667, "G_std": 0.16104634602864584, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.533450603485107, "grad_norm": 0.7584893107414246, "tokens_seen": 52436992, "tokens_per_sec": 22728.29220593796, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3792317708333333, "G_std": 0.16046142578125, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.47394323348999, "grad_norm": 0.8300696015357971, "tokens_seen": 52600832, "tokens_per_sec": 22728.581623192767, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3814900716145833, "G_std": 0.1624603271484375, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.635951995849609, "grad_norm": 0.7692331671714783, "tokens_seen": 52764672, "tokens_per_sec": 22728.60987543573, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899943033854167, "G_std": 0.1591033935546875, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.509275913238525, "grad_norm": 0.7972248196601868, "tokens_seen": 52928512, "tokens_per_sec": 22728.647861228557, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787129720052083, "G_std": 0.1583989461263021, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.528252601623535, "grad_norm": 0.8184810280799866, "tokens_seen": 53092352, "tokens_per_sec": 22728.892070460493, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3933817545572917, "G_std": 0.16095479329427084, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.482345819473267, "grad_norm": 0.7753190398216248, "tokens_seen": 53256192, "tokens_per_sec": 22728.564100039177, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.379913330078125, "G_std": 0.1591339111328125, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.60255241394043, "grad_norm": 0.8105435371398926, "tokens_seen": 53420032, "tokens_per_sec": 22728.664572547765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3870798746744792, "G_std": 0.1607818603515625, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.5234315395355225, "grad_norm": 0.80916428565979, "tokens_seen": 53583872, "tokens_per_sec": 22728.449529549598, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3813069661458333, "G_std": 0.15883382161458334, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.661675453186035, "grad_norm": 0.8327955603599548, "tokens_seen": 53747712, "tokens_per_sec": 22727.9417107678, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834940592447917, "G_std": 0.1602503458658854, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.606772422790527, "grad_norm": 0.8029652833938599, "tokens_seen": 53911552, "tokens_per_sec": 22727.904247879265, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.383575439453125, "G_std": 0.16054280598958334, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.483970403671265, "grad_norm": 0.7779605388641357, "tokens_seen": 54075392, "tokens_per_sec": 22728.07903767035, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834228515625, "G_std": 0.1588465372721354, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.516463756561279, "grad_norm": 0.8537507653236389, "tokens_seen": 54239232, "tokens_per_sec": 22728.011701640415, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3857828776041667, "G_std": 0.15961710611979166, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.395956516265869, "grad_norm": 0.7955538630485535, "tokens_seen": 54403072, "tokens_per_sec": 22728.140736381094, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.384735107421875, "G_std": 0.1615422566731771, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.4615797996521, "grad_norm": 0.7949687838554382, "tokens_seen": 54566912, "tokens_per_sec": 22728.15517916706, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3804728190104167, "G_std": 0.158905029296875, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.466993808746338, "grad_norm": 0.8421257734298706, "tokens_seen": 54730752, "tokens_per_sec": 22728.080892604055, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.384002685546875, "G_std": 0.1599553426106771, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.7761640548706055, "grad_norm": 0.8741059303283691, "tokens_seen": 54894592, "tokens_per_sec": 22728.117540207466, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3881632486979167, "G_std": 0.16115315755208334, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.434309482574463, "grad_norm": 0.7681745886802673, "tokens_seen": 55058432, "tokens_per_sec": 22728.111600969947, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38323974609375, "G_std": 0.16172027587890625, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.478512525558472, "grad_norm": 0.8210609555244446, "tokens_seen": 55222272, "tokens_per_sec": 22728.24279088016, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3837025960286458, "G_std": 0.158782958984375, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.579423189163208, "grad_norm": 0.8321190476417542, "tokens_seen": 55386112, "tokens_per_sec": 22728.530805193564, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3863118489583333, "G_std": 0.16060384114583334, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.533040523529053, "grad_norm": 0.7959355711936951, "tokens_seen": 55549952, "tokens_per_sec": 22728.860168677045, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3832804361979167, "G_std": 0.15775299072265625, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.6785807609558105, "grad_norm": 0.8006656765937805, "tokens_seen": 55713792, "tokens_per_sec": 22729.006876992444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3823801676432292, "G_std": 0.1598485310872396, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.586423397064209, "grad_norm": 0.8217044472694397, "tokens_seen": 55877632, "tokens_per_sec": 22729.170680802643, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3745168050130208, "G_std": 0.15999603271484375, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.558466911315918, "grad_norm": 0.8083808422088623, "tokens_seen": 56041472, "tokens_per_sec": 22728.555640373106, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3865966796875, "G_std": 0.1585210164388021, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.564872741699219, "grad_norm": 0.8278918862342834, "tokens_seen": 56205312, "tokens_per_sec": 22728.845142074475, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3837127685546875, "G_std": 0.15916951497395834, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.559970140457153, "grad_norm": 0.8103763461112976, "tokens_seen": 56369152, "tokens_per_sec": 22728.67822813683, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737894694010417, "G_std": 0.1585566202799479, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.551755905151367, "grad_norm": 0.8688275814056396, "tokens_seen": 56532992, "tokens_per_sec": 22728.68886426174, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3813629150390625, "G_std": 0.1581573486328125, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.4856040477752686, "grad_norm": 0.8628210425376892, "tokens_seen": 56696832, "tokens_per_sec": 22728.50540597757, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.376678466796875, "G_std": 0.15799967447916666, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.367289304733276, "grad_norm": 0.7805656790733337, "tokens_seen": 56860672, "tokens_per_sec": 22728.699232654, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3726857503255208, "G_std": 0.16020965576171875, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.420158386230469, "grad_norm": 0.8146108388900757, "tokens_seen": 57024512, "tokens_per_sec": 22728.707306354278, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3791402180989583, "G_std": 0.16013336181640625, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.613049030303955, "grad_norm": 0.8133561015129089, "tokens_seen": 57188352, "tokens_per_sec": 22729.042483341786, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3798065185546875, "G_std": 0.1587092081705729, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.398998260498047, "grad_norm": 0.8094770312309265, "tokens_seen": 57352192, "tokens_per_sec": 22728.6678039005, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3794403076171875, "G_std": 0.1596501668294271, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.607291913032531, "val_ppl": 100.21239795000159, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3825124104817708, "G_std": 0.1597773234049479, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.498292922973633, "grad_norm": 0.7800793647766113, "tokens_seen": 57516032, "tokens_per_sec": 22717.02386885231, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3795674641927083, "G_std": 0.15805816650390625, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.468382120132446, "grad_norm": 0.8358118534088135, "tokens_seen": 57679872, "tokens_per_sec": 22716.761649848915, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767598470052083, "G_std": 0.1582514444986979, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.52826189994812, "grad_norm": 0.8168433308601379, "tokens_seen": 57843712, "tokens_per_sec": 22717.101395715592, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3750864664713542, "G_std": 0.1593297322591146, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.478214263916016, "grad_norm": 0.8263164758682251, "tokens_seen": 58007552, "tokens_per_sec": 22717.187671525884, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853556315104167, "G_std": 0.1568145751953125, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.48780369758606, "grad_norm": 0.8432542085647583, "tokens_seen": 58171392, "tokens_per_sec": 22717.267996423583, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734842936197917, "G_std": 0.15909830729166666, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.598917245864868, "grad_norm": 0.8602515459060669, "tokens_seen": 58335232, "tokens_per_sec": 22717.334074204045, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37896728515625, "G_std": 0.15935770670572916, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.523911714553833, "grad_norm": 0.8491951823234558, "tokens_seen": 58499072, "tokens_per_sec": 22716.914155799182, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.380523681640625, "G_std": 0.15771484375, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.680044412612915, "grad_norm": 0.8976936340332031, "tokens_seen": 58662912, "tokens_per_sec": 22717.30253711478, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810068766276042, "G_std": 0.157196044921875, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.4990434646606445, "grad_norm": 0.8359178900718689, "tokens_seen": 58826752, "tokens_per_sec": 22717.58429178878, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803761800130208, "G_std": 0.1589635213216146, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.406959772109985, "grad_norm": 0.8070235848426819, "tokens_seen": 58990592, "tokens_per_sec": 22717.090038748545, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3753407796223958, "G_std": 0.1585540771484375, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.423988580703735, "grad_norm": 0.8285340070724487, "tokens_seen": 59154432, "tokens_per_sec": 22717.347987703757, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3765411376953125, "G_std": 0.15706888834635416, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.575611114501953, "grad_norm": 0.7901179194450378, "tokens_seen": 59318272, "tokens_per_sec": 22717.03511523966, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3759613037109375, "G_std": 0.1592585245768229, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.400481700897217, "grad_norm": 0.8830401301383972, "tokens_seen": 59482112, "tokens_per_sec": 22717.1141699541, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831634521484375, "G_std": 0.1592839558919271, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.363938331604004, "grad_norm": 0.8307150602340698, "tokens_seen": 59645952, "tokens_per_sec": 22717.223698687772, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3759206136067708, "G_std": 0.1571528116861979, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.623166561126709, "grad_norm": 0.8330941200256348, "tokens_seen": 59809792, "tokens_per_sec": 22717.524694903288, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3779500325520833, "G_std": 0.1585235595703125, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.63723087310791, "grad_norm": 0.8660702705383301, "tokens_seen": 59973632, "tokens_per_sec": 22717.27241169527, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831278483072917, "G_std": 0.15904998779296875, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.433578968048096, "grad_norm": 0.8378767371177673, "tokens_seen": 60137472, "tokens_per_sec": 22716.738731532965, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787027994791667, "G_std": 0.15804036458333334, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.413294792175293, "grad_norm": 0.8434409499168396, "tokens_seen": 60301312, "tokens_per_sec": 22715.680184103134, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3726806640625, "G_std": 0.15738932291666666, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.458039999008179, "grad_norm": 0.8189897537231445, "tokens_seen": 60465152, "tokens_per_sec": 22713.762473871455, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3747100830078125, "G_std": 0.1594365437825521, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.570470094680786, "grad_norm": 0.8244464993476868, "tokens_seen": 60628992, "tokens_per_sec": 22711.757073894165, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762919108072917, "G_std": 0.15828959147135416, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.451646566390991, "grad_norm": 0.8075804114341736, "tokens_seen": 60792832, "tokens_per_sec": 22710.14687662319, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722279866536458, "G_std": 0.1575597127278646, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.516301155090332, "grad_norm": 0.8567190170288086, "tokens_seen": 60956672, "tokens_per_sec": 22708.40581936502, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3838348388671875, "G_std": 0.1594670613606771, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.593491554260254, "grad_norm": 0.8456085920333862, "tokens_seen": 61120512, "tokens_per_sec": 22705.902270865303, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3823293050130208, "G_std": 0.15747833251953125, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.500975131988525, "grad_norm": 0.8565638065338135, "tokens_seen": 61284352, "tokens_per_sec": 22703.744220225824, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.383392333984375, "G_std": 0.1582056681315104, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.581341743469238, "grad_norm": 0.8421503305435181, "tokens_seen": 61448192, "tokens_per_sec": 22701.444486070057, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3749542236328125, "G_std": 0.1589813232421875, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.38735556602478, "grad_norm": 0.8264755010604858, "tokens_seen": 61612032, "tokens_per_sec": 22698.003766235062, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734079996744792, "G_std": 0.1593043009440104, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.498680591583252, "grad_norm": 0.7856140732765198, "tokens_seen": 61775872, "tokens_per_sec": 22694.927938156354, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767496744791667, "G_std": 0.1595128377278646, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.528041362762451, "grad_norm": 0.8456675410270691, "tokens_seen": 61939712, "tokens_per_sec": 22691.047076583625, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3740081787109375, "G_std": 0.1597468058268229, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.441704511642456, "grad_norm": 0.8688793182373047, "tokens_seen": 62103552, "tokens_per_sec": 22687.513363905196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3764597574869792, "G_std": 0.1586329142252604, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.391641139984131, "grad_norm": 0.8183008432388306, "tokens_seen": 62267392, "tokens_per_sec": 22685.827496746355, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762664794921875, "G_std": 0.1585973103841146, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.3073906898498535, "grad_norm": 0.8494665026664734, "tokens_seen": 62431232, "tokens_per_sec": 22682.60886774884, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3821767171223958, "G_std": 0.160888671875, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.504243612289429, "grad_norm": 0.8159683346748352, "tokens_seen": 62595072, "tokens_per_sec": 22680.63617091491, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3756510416666667, "G_std": 0.15965779622395834, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.413009166717529, "grad_norm": 0.8393968939781189, "tokens_seen": 62758912, "tokens_per_sec": 22678.932271130023, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.381439208984375, "G_std": 0.15804036458333334, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.520811557769775, "grad_norm": 0.8745747208595276, "tokens_seen": 62922752, "tokens_per_sec": 22677.85609213968, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3848012288411458, "G_std": 0.1594060262044271, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.445680618286133, "grad_norm": 0.8824371099472046, "tokens_seen": 63086592, "tokens_per_sec": 22677.23274285888, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770955403645833, "G_std": 0.15842437744140625, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.328264474868774, "grad_norm": 0.8631773591041565, "tokens_seen": 63250432, "tokens_per_sec": 22677.385867291065, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810017903645833, "G_std": 0.1588312784830729, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.327312707901001, "grad_norm": 0.8445019125938416, "tokens_seen": 63414272, "tokens_per_sec": 22677.306070450224, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372467041015625, "G_std": 0.15790557861328125, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.433654308319092, "grad_norm": 0.8281475305557251, "tokens_seen": 63578112, "tokens_per_sec": 22677.58241808327, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781026204427083, "G_std": 0.1593475341796875, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.4373509883880615, "grad_norm": 0.799483060836792, "tokens_seen": 63741952, "tokens_per_sec": 22677.79213939423, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3799540201822917, "G_std": 0.1590118408203125, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.455743789672852, "grad_norm": 0.9598082900047302, "tokens_seen": 63905792, "tokens_per_sec": 22678.00031335825, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725433349609375, "G_std": 0.1578496297200521, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.542063474655151, "grad_norm": 0.8820227384567261, "tokens_seen": 64069632, "tokens_per_sec": 22678.283019472135, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810933430989583, "G_std": 0.15972391764322916, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.484214782714844, "grad_norm": 0.8775460124015808, "tokens_seen": 64233472, "tokens_per_sec": 22678.19255358542, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3730316162109375, "G_std": 0.157318115234375, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.232133626937866, "grad_norm": 0.7945072054862976, "tokens_seen": 64397312, "tokens_per_sec": 22678.38295883841, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3708546956380208, "G_std": 0.15970865885416666, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.530149936676025, "grad_norm": 0.8589596152305603, "tokens_seen": 64561152, "tokens_per_sec": 22678.58011024283, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3728993733723958, "G_std": 0.1591618855794271, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.534311771392822, "grad_norm": 0.9203032851219177, "tokens_seen": 64724992, "tokens_per_sec": 22678.91312098001, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3773651123046875, "G_std": 0.15915934244791666, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.31470513343811, "grad_norm": 0.8200869560241699, "tokens_seen": 64888832, "tokens_per_sec": 22679.029440354054, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729705810546875, "G_std": 0.15868631998697916, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.4699156284332275, "grad_norm": 0.8422879576683044, "tokens_seen": 65052672, "tokens_per_sec": 22679.20057932865, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3780059814453125, "G_std": 0.15619913736979166, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.531338453292847, "grad_norm": 0.8556520938873291, "tokens_seen": 65216512, "tokens_per_sec": 22679.39184786782, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3740488688151042, "G_std": 0.15775553385416666, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.397576332092285, "grad_norm": 0.8596634268760681, "tokens_seen": 65380352, "tokens_per_sec": 22679.385381333337, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3771514892578125, "G_std": 0.15842183430989584, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.471167802810669, "grad_norm": 0.8389639854431152, "tokens_seen": 65544192, "tokens_per_sec": 22679.620067973614, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3735148111979167, "G_std": 0.15740203857421875, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.490499329566956, "val_ppl": 89.16595796381097, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.380035400390625, "G_std": 0.15863545735677084, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.51519775390625, "grad_norm": 0.8762485980987549, "tokens_seen": 65708032, "tokens_per_sec": 22670.05578272014, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3769989013671875, "G_std": 0.1578648885091146, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.528287410736084, "grad_norm": 0.8398143649101257, "tokens_seen": 65871872, "tokens_per_sec": 22670.30223668489, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3826243082682292, "G_std": 0.157928466796875, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.475634336471558, "grad_norm": 0.8522191643714905, "tokens_seen": 66035712, "tokens_per_sec": 22670.675627311346, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3795928955078125, "G_std": 0.15770721435546875, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.403980255126953, "grad_norm": 0.8532475233078003, "tokens_seen": 66199552, "tokens_per_sec": 22670.900324554845, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3771769205729167, "G_std": 0.15636444091796875, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.277471303939819, "grad_norm": 0.8606414198875427, "tokens_seen": 66363392, "tokens_per_sec": 22670.568167547295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3647206624348958, "G_std": 0.15771993001302084, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.719423055648804, "grad_norm": 0.8814894556999207, "tokens_seen": 66527232, "tokens_per_sec": 22670.993537285212, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3743998209635417, "G_std": 0.1583404541015625, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.405720949172974, "grad_norm": 0.8787447214126587, "tokens_seen": 66691072, "tokens_per_sec": 22671.28531751305, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664296468098958, "G_std": 0.1575164794921875, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.393552780151367, "grad_norm": 0.8865290284156799, "tokens_seen": 66854912, "tokens_per_sec": 22671.50626384437, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.374359130859375, "G_std": 0.15702056884765625, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.49945330619812, "grad_norm": 0.8559765219688416, "tokens_seen": 67018752, "tokens_per_sec": 22671.70072613042, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762563069661458, "G_std": 0.1585362752278646, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.450342893600464, "grad_norm": 0.914595365524292, "tokens_seen": 67182592, "tokens_per_sec": 22671.432521848456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770650227864583, "G_std": 0.15635172526041666, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.461479663848877, "grad_norm": 0.8529609441757202, "tokens_seen": 67346432, "tokens_per_sec": 22671.43278398122, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372772216796875, "G_std": 0.1566340128580729, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.587791442871094, "grad_norm": 0.8654700517654419, "tokens_seen": 67510272, "tokens_per_sec": 22671.639605536824, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3772837320963542, "G_std": 0.1588414510091146, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.431720495223999, "grad_norm": 0.8527190685272217, "tokens_seen": 67674112, "tokens_per_sec": 22672.05036386767, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3746185302734375, "G_std": 0.1569188435872396, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.367236375808716, "grad_norm": 0.8469570279121399, "tokens_seen": 67837952, "tokens_per_sec": 22672.35717556885, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727315266927083, "G_std": 0.15613301595052084, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.41992712020874, "grad_norm": 0.8835727572441101, "tokens_seen": 68001792, "tokens_per_sec": 22672.732413725942, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763427734375, "G_std": 0.15850830078125, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.379971742630005, "grad_norm": 0.8860751390457153, "tokens_seen": 68165632, "tokens_per_sec": 22672.765811493355, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3743235270182292, "G_std": 0.15668487548828125, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.4641969203948975, "grad_norm": 0.8738299012184143, "tokens_seen": 68329472, "tokens_per_sec": 22673.00080647111, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3761494954427083, "G_std": 0.15732574462890625, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.316003799438477, "grad_norm": 0.8274929523468018, "tokens_seen": 68493312, "tokens_per_sec": 22673.222184549348, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3705088297526042, "G_std": 0.1575749715169271, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.42394757270813, "grad_norm": 0.8514351844787598, "tokens_seen": 68657152, "tokens_per_sec": 22673.480722826735, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3683929443359375, "G_std": 0.15741475423177084, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.4760072231292725, "grad_norm": 0.8860082030296326, "tokens_seen": 68820992, "tokens_per_sec": 22673.67144612721, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.375885009765625, "G_std": 0.1588465372721354, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.4821271896362305, "grad_norm": 0.9355820417404175, "tokens_seen": 68984832, "tokens_per_sec": 22673.997231435496, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737284342447917, "G_std": 0.1567535400390625, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.41044282913208, "grad_norm": 0.9245895147323608, "tokens_seen": 69148672, "tokens_per_sec": 22674.262782585218, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.377044677734375, "G_std": 0.1556727091471354, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.491873025894165, "grad_norm": 0.9024343490600586, "tokens_seen": 69312512, "tokens_per_sec": 22674.48715361941, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3751627604166667, "G_std": 0.158447265625, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.451792478561401, "grad_norm": 0.9068785905838013, "tokens_seen": 69476352, "tokens_per_sec": 22674.629794109384, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.374542236328125, "G_std": 0.1573638916015625, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.412322521209717, "grad_norm": 0.9331679940223694, "tokens_seen": 69640192, "tokens_per_sec": 22674.97289984577, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3782806396484375, "G_std": 0.15793863932291666, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.431736469268799, "grad_norm": 0.9198706746101379, "tokens_seen": 69804032, "tokens_per_sec": 22675.393436634124, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721516927083333, "G_std": 0.15792338053385416, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.4098961353302, "grad_norm": 0.8487981557846069, "tokens_seen": 69967872, "tokens_per_sec": 22675.557957687393, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3791249593098958, "G_std": 0.1578852335611979, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.381639719009399, "grad_norm": 0.8953742384910583, "tokens_seen": 70131712, "tokens_per_sec": 22675.69102760403, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3693084716796875, "G_std": 0.1586151123046875, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.492290019989014, "grad_norm": 0.8859279751777649, "tokens_seen": 70295552, "tokens_per_sec": 22675.95026242878, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680928548177083, "G_std": 0.1568883260091146, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.330855369567871, "grad_norm": 0.9038776159286499, "tokens_seen": 70459392, "tokens_per_sec": 22676.008569732563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3773651123046875, "G_std": 0.156890869140625, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.42045783996582, "grad_norm": 0.8817788362503052, "tokens_seen": 70623232, "tokens_per_sec": 22676.192117474122, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3772989908854167, "G_std": 0.15813700358072916, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.444501876831055, "grad_norm": 0.886688768863678, "tokens_seen": 70787072, "tokens_per_sec": 22676.68065764928, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3663686116536458, "G_std": 0.15715535481770834, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.442643880844116, "grad_norm": 0.9342748522758484, "tokens_seen": 70950912, "tokens_per_sec": 22677.058556361513, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.371124267578125, "G_std": 0.15744781494140625, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.465124607086182, "grad_norm": 0.8520236611366272, "tokens_seen": 71114752, "tokens_per_sec": 22677.476003618005, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3765716552734375, "G_std": 0.15534210205078125, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.431615591049194, "grad_norm": 0.8532736301422119, "tokens_seen": 71278592, "tokens_per_sec": 22677.913612463064, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372650146484375, "G_std": 0.1568450927734375, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.453768014907837, "grad_norm": 0.8948313593864441, "tokens_seen": 71442432, "tokens_per_sec": 22678.004070628438, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3636932373046875, "G_std": 0.15602620442708334, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.357452392578125, "grad_norm": 0.9035700559616089, "tokens_seen": 71606272, "tokens_per_sec": 22678.412932413914, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3772328694661458, "G_std": 0.1568603515625, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.4229137897491455, "grad_norm": 0.8508950471878052, "tokens_seen": 71770112, "tokens_per_sec": 22678.968991768397, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3719482421875, "G_std": 0.15786234537760416, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.292294502258301, "grad_norm": 0.8681171536445618, "tokens_seen": 71933952, "tokens_per_sec": 22679.347731012946, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763580322265625, "G_std": 0.1574427286783854, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.465691328048706, "grad_norm": 0.9170167446136475, "tokens_seen": 72097792, "tokens_per_sec": 22679.6020805147, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725484212239583, "G_std": 0.15621185302734375, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.505867958068848, "grad_norm": 0.8897050023078918, "tokens_seen": 72261632, "tokens_per_sec": 22679.951356745707, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707377115885417, "G_std": 0.1567509969075521, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.429397344589233, "grad_norm": 0.9142192602157593, "tokens_seen": 72425472, "tokens_per_sec": 22680.204701959803, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3718312581380208, "G_std": 0.1563542683919271, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.358785390853882, "grad_norm": 0.8689371943473816, "tokens_seen": 72589312, "tokens_per_sec": 22680.479718100134, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722585042317708, "G_std": 0.15804036458333334, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.561785936355591, "grad_norm": 0.9403071999549866, "tokens_seen": 72753152, "tokens_per_sec": 22680.99014224835, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762766520182292, "G_std": 0.15746561686197916, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.418927192687988, "grad_norm": 0.8593190908432007, "tokens_seen": 72916992, "tokens_per_sec": 22680.924423906254, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3704427083333333, "G_std": 0.15702056884765625, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.422455787658691, "grad_norm": 0.923844575881958, "tokens_seen": 73080832, "tokens_per_sec": 22681.21500412213, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3715871175130208, "G_std": 0.15676116943359375, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.28607964515686, "grad_norm": 0.881141185760498, "tokens_seen": 73244672, "tokens_per_sec": 22680.995267176022, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650410970052083, "G_std": 0.15714518229166666, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.443721771240234, "grad_norm": 0.8463413119316101, "tokens_seen": 73408512, "tokens_per_sec": 22681.286469130257, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3709004720052083, "G_std": 0.1556981404622396, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.434381723403931, "grad_norm": 0.9112522602081299, "tokens_seen": 73572352, "tokens_per_sec": 22681.213657813685, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3726247151692708, "G_std": 0.1566492716471354, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.53608775138855, "grad_norm": 0.9516154527664185, "tokens_seen": 73736192, "tokens_per_sec": 22681.619008943173, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3698170979817708, "G_std": 0.1564458211263021, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.470087933540344, "val_ppl": 87.36440493711285, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3694051106770833, "G_std": 0.15796661376953125, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.3451738357543945, "grad_norm": 0.9113641381263733, "tokens_seen": 73900032, "tokens_per_sec": 22672.6749379439, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695831298828125, "G_std": 0.1560211181640625, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.461600303649902, "grad_norm": 0.9227479696273804, "tokens_seen": 74063872, "tokens_per_sec": 22672.886336014664, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3685302734375, "G_std": 0.1581446329752604, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.230769634246826, "grad_norm": 0.8455044031143188, "tokens_seen": 74227712, "tokens_per_sec": 22673.020832366263, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3624216715494792, "G_std": 0.15642293294270834, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.413096904754639, "grad_norm": 0.9118974208831787, "tokens_seen": 74391552, "tokens_per_sec": 22673.200301146517, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3714701334635417, "G_std": 0.15531412760416666, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.502638339996338, "grad_norm": 0.9243237972259521, "tokens_seen": 74555392, "tokens_per_sec": 22673.141554631962, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695170084635417, "G_std": 0.15671539306640625, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.474030017852783, "grad_norm": 0.9052010178565979, "tokens_seen": 74719232, "tokens_per_sec": 22673.45501166742, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664449055989583, "G_std": 0.15472412109375, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.262221097946167, "grad_norm": 0.8757396936416626, "tokens_seen": 74883072, "tokens_per_sec": 22673.743189949615, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365478515625, "G_std": 0.1552556355794271, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.540650844573975, "grad_norm": 1.0116950273513794, "tokens_seen": 75046912, "tokens_per_sec": 22674.049465258762, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3798268636067708, "G_std": 0.1553624471028646, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.3448052406311035, "grad_norm": 0.9207939505577087, "tokens_seen": 75210752, "tokens_per_sec": 22674.113168204396, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727010091145833, "G_std": 0.1542816162109375, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.439589500427246, "grad_norm": 0.8831727504730225, "tokens_seen": 75374592, "tokens_per_sec": 22674.374926325152, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3686981201171875, "G_std": 0.156402587890625, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.377445459365845, "grad_norm": 0.9664310216903687, "tokens_seen": 75538432, "tokens_per_sec": 22674.62533161616, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670501708984375, "G_std": 0.15639495849609375, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.41172456741333, "grad_norm": 0.9926671981811523, "tokens_seen": 75702272, "tokens_per_sec": 22674.529126399997, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.371063232421875, "G_std": 0.15650177001953125, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.432583808898926, "grad_norm": 0.9403281807899475, "tokens_seen": 75866112, "tokens_per_sec": 22674.803355558935, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3677571614583333, "G_std": 0.15534464518229166, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.397729396820068, "grad_norm": 0.9020082354545593, "tokens_seen": 76029952, "tokens_per_sec": 22674.950008584914, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720855712890625, "G_std": 0.15513102213541666, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.29628586769104, "grad_norm": 0.956723153591156, "tokens_seen": 76193792, "tokens_per_sec": 22675.18537968874, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3665262858072917, "G_std": 0.15516153971354166, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.533302068710327, "grad_norm": 0.8930766582489014, "tokens_seen": 76357632, "tokens_per_sec": 22675.473294073912, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734537760416667, "G_std": 0.1551081339518229, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.538057327270508, "grad_norm": 0.9265267848968506, "tokens_seen": 76521472, "tokens_per_sec": 22675.62171092136, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3754119873046875, "G_std": 0.15535481770833334, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.360287189483643, "grad_norm": 0.9355230927467346, "tokens_seen": 76685312, "tokens_per_sec": 22675.85488424211, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3715260823567708, "G_std": 0.1551971435546875, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.427824258804321, "grad_norm": 0.9119800329208374, "tokens_seen": 76849152, "tokens_per_sec": 22676.107950341357, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3667551676432292, "G_std": 0.1550267537434896, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.407333850860596, "grad_norm": 0.9481016993522644, "tokens_seen": 77012992, "tokens_per_sec": 22676.491714519037, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707173665364583, "G_std": 0.1568018595377604, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.373780250549316, "grad_norm": 0.9349316954612732, "tokens_seen": 77176832, "tokens_per_sec": 22676.58951106692, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3679707845052083, "G_std": 0.15596771240234375, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.272114992141724, "grad_norm": 0.881401777267456, "tokens_seen": 77340672, "tokens_per_sec": 22676.92580612829, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3691965738932292, "G_std": 0.15630340576171875, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.492976665496826, "grad_norm": 0.8679569363594055, "tokens_seen": 77504512, "tokens_per_sec": 22677.3840571829, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3658447265625, "G_std": 0.15610249837239584, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.543605089187622, "grad_norm": 0.9467224478721619, "tokens_seen": 77668352, "tokens_per_sec": 22677.469674998025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3715362548828125, "G_std": 0.15506744384765625, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.298327445983887, "grad_norm": 0.858759343624115, "tokens_seen": 77832192, "tokens_per_sec": 22677.250341083043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653106689453125, "G_std": 0.15619405110677084, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.410407543182373, "grad_norm": 0.9592738151550293, "tokens_seen": 77996032, "tokens_per_sec": 22677.281047962515, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721415201822917, "G_std": 0.15308380126953125, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.364223003387451, "grad_norm": 0.921933650970459, "tokens_seen": 78159872, "tokens_per_sec": 22677.61835475409, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3672637939453125, "G_std": 0.155487060546875, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.294305324554443, "grad_norm": 0.928135335445404, "tokens_seen": 78323712, "tokens_per_sec": 22677.666700986316, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3706003824869792, "G_std": 0.1556421915690104, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.390153408050537, "grad_norm": 0.921274721622467, "tokens_seen": 78487552, "tokens_per_sec": 22677.578308165546, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36669921875, "G_std": 0.15743255615234375, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.595729112625122, "grad_norm": 0.9176888465881348, "tokens_seen": 78651392, "tokens_per_sec": 22677.182602988647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3691965738932292, "G_std": 0.15662129720052084, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.424494981765747, "grad_norm": 0.942328929901123, "tokens_seen": 78815232, "tokens_per_sec": 22677.28419166877, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36663818359375, "G_std": 0.1558049519856771, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.252566576004028, "grad_norm": 0.9345576763153076, "tokens_seen": 78979072, "tokens_per_sec": 22674.18522003995, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3705952962239583, "G_std": 0.1548894246419271, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.4963531494140625, "grad_norm": 0.9261813163757324, "tokens_seen": 79142912, "tokens_per_sec": 22673.40721902469, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722279866536458, "G_std": 0.1559270222981771, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.404249906539917, "grad_norm": 0.9719075560569763, "tokens_seen": 79306752, "tokens_per_sec": 22671.98046282852, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695831298828125, "G_std": 0.1547724405924479, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.263753890991211, "grad_norm": 0.8668637275695801, "tokens_seen": 79470592, "tokens_per_sec": 22671.197589537653, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3623148600260417, "G_std": 0.15486653645833334, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.346168518066406, "grad_norm": 0.9613667130470276, "tokens_seen": 79634432, "tokens_per_sec": 22669.147302257825, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690592447916667, "G_std": 0.15582784016927084, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.309275150299072, "grad_norm": 0.9177640080451965, "tokens_seen": 79798272, "tokens_per_sec": 22668.819637093064, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3722381591796875, "G_std": 0.15483601888020834, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.3991053104400635, "grad_norm": 0.9064663052558899, "tokens_seen": 79962112, "tokens_per_sec": 22669.10767294827, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3691253662109375, "G_std": 0.1554285685221354, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.321499347686768, "grad_norm": 0.90157151222229, "tokens_seen": 80125952, "tokens_per_sec": 22669.0044433491, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3692067464192708, "G_std": 0.1559117635091146, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.33294677734375, "grad_norm": 0.9350176453590393, "tokens_seen": 80289792, "tokens_per_sec": 22668.25802319499, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3688151041666667, "G_std": 0.1555353800455729, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.306052923202515, "grad_norm": 0.9754520058631897, "tokens_seen": 80453632, "tokens_per_sec": 22667.640174899254, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36859130859375, "G_std": 0.15456644694010416, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.328870534896851, "grad_norm": 0.9057527184486389, "tokens_seen": 80617472, "tokens_per_sec": 22666.830237610906, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3630167643229167, "G_std": 0.15270233154296875, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.325999021530151, "grad_norm": 0.9636898636817932, "tokens_seen": 80781312, "tokens_per_sec": 22666.596830403727, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3692677815755208, "G_std": 0.15610249837239584, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.240824222564697, "grad_norm": 0.9281819462776184, "tokens_seen": 80945152, "tokens_per_sec": 22665.438310394817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3681488037109375, "G_std": 0.15402730305989584, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.33661675453186, "grad_norm": 0.9756329655647278, "tokens_seen": 81108992, "tokens_per_sec": 22665.37573484936, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3659413655598958, "G_std": 0.1542994181315104, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.332391977310181, "grad_norm": 0.9257839918136597, "tokens_seen": 81272832, "tokens_per_sec": 22664.887506793362, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649800618489583, "G_std": 0.1557490030924479, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.296534299850464, "grad_norm": 0.9898708462715149, "tokens_seen": 81436672, "tokens_per_sec": 22664.47457023695, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3681742350260417, "G_std": 0.15651448567708334, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.485050201416016, "grad_norm": 1.0057638883590698, "tokens_seen": 81600512, "tokens_per_sec": 22664.259724224008, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3689829508463542, "G_std": 0.1545562744140625, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.249511003494263, "grad_norm": 0.937399685382843, "tokens_seen": 81764352, "tokens_per_sec": 22664.691277770053, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3687082926432292, "G_std": 0.15555826822916666, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.388416528701782, "grad_norm": 0.9615511894226074, "tokens_seen": 81928192, "tokens_per_sec": 22664.133178929085, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716837565104167, "G_std": 0.15501912434895834, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.462025570869446, "val_ppl": 86.66287322159867, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3693898518880208, "G_std": 0.1542485555013021, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.248687028884888, "grad_norm": 0.9936890602111816, "tokens_seen": 82092032, "tokens_per_sec": 22655.757766021783, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3633982340494792, "G_std": 0.15525054931640625, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.356619119644165, "grad_norm": 0.9441788792610168, "tokens_seen": 82255872, "tokens_per_sec": 22655.7198481342, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3728841145833333, "G_std": 0.1550013224283854, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.2573511600494385, "grad_norm": 0.9311093091964722, "tokens_seen": 82419712, "tokens_per_sec": 22656.029745669668, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3645477294921875, "G_std": 0.15558878580729166, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.185736417770386, "grad_norm": 0.8865094780921936, "tokens_seen": 82583552, "tokens_per_sec": 22656.02478302014, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3615875244140625, "G_std": 0.15369415283203125, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.497983932495117, "grad_norm": 0.9441937208175659, "tokens_seen": 82747392, "tokens_per_sec": 22655.641555011098, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3669230143229167, "G_std": 0.15644327799479166, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.4120049476623535, "grad_norm": 0.9756147265434265, "tokens_seen": 82911232, "tokens_per_sec": 22655.936936948685, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3633371988932292, "G_std": 0.1560236612955729, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.273238658905029, "grad_norm": 0.922401487827301, "tokens_seen": 83075072, "tokens_per_sec": 22655.552508056204, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.366363525390625, "G_std": 0.1550165812174479, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.423253774642944, "grad_norm": 0.8941987156867981, "tokens_seen": 83238912, "tokens_per_sec": 22655.28395964311, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649698893229167, "G_std": 0.15641276041666666, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.518935441970825, "grad_norm": 0.931053638458252, "tokens_seen": 83402752, "tokens_per_sec": 22655.088361142938, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680775960286458, "G_std": 0.15552520751953125, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.49580979347229, "grad_norm": 0.9498298764228821, "tokens_seen": 83566592, "tokens_per_sec": 22655.31875014968, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3674774169921875, "G_std": 0.15692647298177084, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.397475481033325, "grad_norm": 0.9621925354003906, "tokens_seen": 83730432, "tokens_per_sec": 22655.235578311658, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3677927652994792, "G_std": 0.1558074951171875, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.2587666511535645, "grad_norm": 0.9189404249191284, "tokens_seen": 83894272, "tokens_per_sec": 22655.16570344245, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3619232177734375, "G_std": 0.15436553955078125, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.382299423217773, "grad_norm": 0.9687818288803101, "tokens_seen": 84058112, "tokens_per_sec": 22655.266922541403, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626047770182292, "G_std": 0.15549214680989584, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.414525985717773, "grad_norm": 1.0206196308135986, "tokens_seen": 84221952, "tokens_per_sec": 22655.316689981126, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36920166015625, "G_std": 0.1555201212565104, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.297852277755737, "grad_norm": 0.9236459732055664, "tokens_seen": 84385792, "tokens_per_sec": 22655.25622173397, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662160237630208, "G_std": 0.15398661295572916, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.415698766708374, "grad_norm": 0.9120808839797974, "tokens_seen": 84549632, "tokens_per_sec": 22654.6197950471, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.369171142578125, "G_std": 0.15424346923828125, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.268591403961182, "grad_norm": 0.9707980751991272, "tokens_seen": 84713472, "tokens_per_sec": 22655.166542235373, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3629608154296875, "G_std": 0.15383402506510416, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.300748825073242, "grad_norm": 0.9245581030845642, "tokens_seen": 84877312, "tokens_per_sec": 22655.33662520295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646341959635417, "G_std": 0.15475972493489584, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.445767879486084, "grad_norm": 0.9413375854492188, "tokens_seen": 85041152, "tokens_per_sec": 22655.221434863015, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.368072509765625, "G_std": 0.1544189453125, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.395963907241821, "grad_norm": 1.0052181482315063, "tokens_seen": 85204992, "tokens_per_sec": 22655.48643831728, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365753173828125, "G_std": 0.15520477294921875, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.243811845779419, "grad_norm": 0.9235466122627258, "tokens_seen": 85368832, "tokens_per_sec": 22655.16707725772, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626556396484375, "G_std": 0.1545562744140625, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.24623441696167, "grad_norm": 1.0102041959762573, "tokens_seen": 85532672, "tokens_per_sec": 22655.259529762763, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632863362630208, "G_std": 0.1551971435546875, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.445122241973877, "grad_norm": 0.9731006026268005, "tokens_seen": 85696512, "tokens_per_sec": 22655.223358695523, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657480875651042, "G_std": 0.15522003173828125, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.562913656234741, "grad_norm": 0.9744393825531006, "tokens_seen": 85860352, "tokens_per_sec": 22655.36542587271, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670603434244792, "G_std": 0.1552098592122396, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.407247066497803, "grad_norm": 1.0437030792236328, "tokens_seen": 86024192, "tokens_per_sec": 22655.189462373677, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3679097493489583, "G_std": 0.1558812459309896, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.302065372467041, "grad_norm": 0.920810341835022, "tokens_seen": 86188032, "tokens_per_sec": 22655.372285143392, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3633321126302083, "G_std": 0.15575154622395834, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.2649266719818115, "grad_norm": 0.9300722479820251, "tokens_seen": 86351872, "tokens_per_sec": 22655.38734552206, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676656087239583, "G_std": 0.15363057454427084, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.229992389678955, "grad_norm": 0.9620591402053833, "tokens_seen": 86515712, "tokens_per_sec": 22655.3541274338, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632303873697917, "G_std": 0.1554107666015625, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.445080757141113, "grad_norm": 0.9865149855613708, "tokens_seen": 86679552, "tokens_per_sec": 22655.57719224597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671671549479167, "G_std": 0.1539586385091146, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.239706993103027, "grad_norm": 0.9055526852607727, "tokens_seen": 86843392, "tokens_per_sec": 22654.761694166158, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3660634358723958, "G_std": 0.15523783365885416, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.411089658737183, "grad_norm": 0.9865909814834595, "tokens_seen": 87007232, "tokens_per_sec": 22654.790444714858, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3642374674479167, "G_std": 0.1547063191731771, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.2494401931762695, "grad_norm": 0.9473641514778137, "tokens_seen": 87171072, "tokens_per_sec": 22654.918922202873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695322672526042, "G_std": 0.15471394856770834, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.3230884075164795, "grad_norm": 0.9391321539878845, "tokens_seen": 87334912, "tokens_per_sec": 22655.150997797624, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609568277994792, "G_std": 0.1543757120768229, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.388303518295288, "grad_norm": 1.0033190250396729, "tokens_seen": 87498752, "tokens_per_sec": 22654.842179221003, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3651784261067708, "G_std": 0.1550750732421875, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.397046804428101, "grad_norm": 0.9095959067344666, "tokens_seen": 87662592, "tokens_per_sec": 22654.71712183451, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610331217447917, "G_std": 0.1555658976236979, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.303421497344971, "grad_norm": 0.9621278047561646, "tokens_seen": 87826432, "tokens_per_sec": 22654.796205701427, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3668263753255208, "G_std": 0.15564982096354166, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.256210565567017, "grad_norm": 0.966141402721405, "tokens_seen": 87990272, "tokens_per_sec": 22655.278567069992, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3658498128255208, "G_std": 0.1544926961263021, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.357419490814209, "grad_norm": 0.9481697678565979, "tokens_seen": 88154112, "tokens_per_sec": 22655.38312552643, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3659515380859375, "G_std": 0.1550267537434896, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.339052200317383, "grad_norm": 0.9889065027236938, "tokens_seen": 88317952, "tokens_per_sec": 22655.613973437343, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632303873697917, "G_std": 0.15398661295572916, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.374632835388184, "grad_norm": 0.9601806998252869, "tokens_seen": 88481792, "tokens_per_sec": 22656.075235181805, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606821695963542, "G_std": 0.15662384033203125, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.3087077140808105, "grad_norm": 0.984874427318573, "tokens_seen": 88645632, "tokens_per_sec": 22655.959066947467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.365509033203125, "G_std": 0.1552886962890625, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.4657814502716064, "grad_norm": 1.0348137617111206, "tokens_seen": 88809472, "tokens_per_sec": 22656.290005102597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3691660563151042, "G_std": 0.15409088134765625, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.415013790130615, "grad_norm": 1.0032424926757812, "tokens_seen": 88973312, "tokens_per_sec": 22656.52183377285, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3668874104817708, "G_std": 0.1534423828125, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.4078686237335205, "grad_norm": 0.9772177934646606, "tokens_seen": 89137152, "tokens_per_sec": 22656.00058591456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3654327392578125, "G_std": 0.15512847900390625, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.279658555984497, "grad_norm": 0.9620935916900635, "tokens_seen": 89300992, "tokens_per_sec": 22654.884852434905, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603871663411458, "G_std": 0.1538060506184896, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.149922132492065, "grad_norm": 0.9507750868797302, "tokens_seen": 89464832, "tokens_per_sec": 22655.01779581956, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616689046223958, "G_std": 0.1545282999674479, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.440572261810303, "grad_norm": 0.9945494532585144, "tokens_seen": 89628672, "tokens_per_sec": 22655.038866172974, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367767333984375, "G_std": 0.15397135416666666, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.368077993392944, "grad_norm": 1.0022836923599243, "tokens_seen": 89792512, "tokens_per_sec": 22655.159364557592, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626556396484375, "G_std": 0.15427652994791666, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.327308893203735, "grad_norm": 0.9944494366645813, "tokens_seen": 89956352, "tokens_per_sec": 22654.805967875058, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3602142333984375, "G_std": 0.15478515625, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.317770004272461, "grad_norm": 0.9648866057395935, "tokens_seen": 90120192, "tokens_per_sec": 22654.70940306912, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3674163818359375, "G_std": 0.1544825236002604, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.374735808372497, "val_ppl": 79.41885498385716, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.36712646484375, "G_std": 0.15450032552083334, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.272021770477295, "grad_norm": 0.9687469005584717, "tokens_seen": 90284032, "tokens_per_sec": 22647.6353991816, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3669281005859375, "G_std": 0.15418497721354166, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.2830586433410645, "grad_norm": 0.9027559161186218, "tokens_seen": 90447872, "tokens_per_sec": 22648.04054943292, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3624369303385417, "G_std": 0.1552276611328125, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.410010099411011, "grad_norm": 0.9537557363510132, "tokens_seen": 90611712, "tokens_per_sec": 22648.152835406476, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.366363525390625, "G_std": 0.1555760701497396, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.1369287967681885, "grad_norm": 0.9732974171638489, "tokens_seen": 90775552, "tokens_per_sec": 22648.1119762286, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610585530598958, "G_std": 0.1548487345377604, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.384801387786865, "grad_norm": 1.0552197694778442, "tokens_seen": 90939392, "tokens_per_sec": 22648.346604279504, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36492919921875, "G_std": 0.1547698974609375, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.302054405212402, "grad_norm": 0.9574278593063354, "tokens_seen": 91103232, "tokens_per_sec": 22648.43823594025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3612467447916667, "G_std": 0.15555826822916666, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.340721607208252, "grad_norm": 0.9725937843322754, "tokens_seen": 91267072, "tokens_per_sec": 22648.604238102303, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3577321370442708, "G_std": 0.1549530029296875, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.353675365447998, "grad_norm": 0.9530546069145203, "tokens_seen": 91430912, "tokens_per_sec": 22648.98851703418, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3684844970703125, "G_std": 0.1557591756184896, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.307360410690308, "grad_norm": 0.9897816181182861, "tokens_seen": 91594752, "tokens_per_sec": 22649.132605980994, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610483805338542, "G_std": 0.1543909708658854, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.230745553970337, "grad_norm": 0.985200822353363, "tokens_seen": 91758592, "tokens_per_sec": 22649.426915735126, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3655192057291667, "G_std": 0.15361531575520834, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.290066957473755, "grad_norm": 1.0181477069854736, "tokens_seen": 91922432, "tokens_per_sec": 22649.412498502934, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.363433837890625, "G_std": 0.15351104736328125, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.335294723510742, "grad_norm": 0.9710813760757446, "tokens_seen": 92086272, "tokens_per_sec": 22649.146935768724, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656717936197917, "G_std": 0.1551233927408854, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.3703718185424805, "grad_norm": 1.0217798948287964, "tokens_seen": 92250112, "tokens_per_sec": 22649.555965388663, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710072835286458, "G_std": 0.15308634440104166, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.287708759307861, "grad_norm": 0.9649983644485474, "tokens_seen": 92413952, "tokens_per_sec": 22649.916064185327, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3667144775390625, "G_std": 0.15425618489583334, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.273507356643677, "grad_norm": 1.0174627304077148, "tokens_seen": 92577792, "tokens_per_sec": 22649.49149238298, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613688151041667, "G_std": 0.15343221028645834, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.326305866241455, "grad_norm": 0.9499135613441467, "tokens_seen": 92741632, "tokens_per_sec": 22649.37878741797, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657989501953125, "G_std": 0.1546808878580729, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.364255428314209, "grad_norm": 0.9708417057991028, "tokens_seen": 92905472, "tokens_per_sec": 22649.644239806057, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616790771484375, "G_std": 0.1533075968424479, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.4749627113342285, "grad_norm": 1.0101518630981445, "tokens_seen": 93069312, "tokens_per_sec": 22649.79662891246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610127766927083, "G_std": 0.15355936686197916, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.309540271759033, "grad_norm": 0.9717541933059692, "tokens_seen": 93233152, "tokens_per_sec": 22650.239015993488, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3640187581380208, "G_std": 0.15433247884114584, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.396707057952881, "grad_norm": 0.9866755604743958, "tokens_seen": 93396992, "tokens_per_sec": 22650.67747404629, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656209309895833, "G_std": 0.1536687215169271, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.308848857879639, "grad_norm": 0.9618123769760132, "tokens_seen": 93560832, "tokens_per_sec": 22650.774069570718, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613637288411458, "G_std": 0.15337371826171875, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.32674503326416, "grad_norm": 0.9967939257621765, "tokens_seen": 93724672, "tokens_per_sec": 22651.290603017565, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671468098958333, "G_std": 0.1537907918294271, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.367588043212891, "grad_norm": 1.0056805610656738, "tokens_seen": 93888512, "tokens_per_sec": 22651.44993689144, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670501708984375, "G_std": 0.15187581380208334, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.367663860321045, "grad_norm": 0.9351657629013062, "tokens_seen": 94052352, "tokens_per_sec": 22651.702063745655, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3619791666666667, "G_std": 0.15392557779947916, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.261144161224365, "grad_norm": 0.9863548278808594, "tokens_seen": 94216192, "tokens_per_sec": 22651.68900302107, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.363677978515625, "G_std": 0.1539586385091146, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.176218152046204, "grad_norm": 0.9965502023696899, "tokens_seen": 94380032, "tokens_per_sec": 22651.900590841262, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3643951416015625, "G_std": 0.15312703450520834, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.417725324630737, "grad_norm": 0.973662257194519, "tokens_seen": 94543872, "tokens_per_sec": 22652.116652303866, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3654836018880208, "G_std": 0.15360514322916666, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.297895431518555, "grad_norm": 1.0528217554092407, "tokens_seen": 94707712, "tokens_per_sec": 22651.967040831216, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3654530843098958, "G_std": 0.15375264485677084, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.173421382904053, "grad_norm": 0.8996829390525818, "tokens_seen": 94871552, "tokens_per_sec": 22652.0702402492, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36224365234375, "G_std": 0.15306854248046875, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.329948663711548, "grad_norm": 0.9501296281814575, "tokens_seen": 95035392, "tokens_per_sec": 22652.386406523026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3593292236328125, "G_std": 0.15430196126302084, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.279405355453491, "grad_norm": 1.0137370824813843, "tokens_seen": 95199232, "tokens_per_sec": 22652.644275674003, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3654581705729167, "G_std": 0.1540959676106771, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.341832160949707, "grad_norm": 0.9487008452415466, "tokens_seen": 95363072, "tokens_per_sec": 22652.571831306937, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3652598063151042, "G_std": 0.15493011474609375, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.427398920059204, "grad_norm": 1.0777387619018555, "tokens_seen": 95526912, "tokens_per_sec": 22652.76243427152, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3618672688802083, "G_std": 0.15306599934895834, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.210412979125977, "grad_norm": 1.0309721231460571, "tokens_seen": 95690752, "tokens_per_sec": 22652.908443848988, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650054931640625, "G_std": 0.15253448486328125, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.350689888000488, "grad_norm": 1.0386223793029785, "tokens_seen": 95854592, "tokens_per_sec": 22652.951193833844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597768147786458, "G_std": 0.154052734375, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.273837089538574, "grad_norm": 0.9799023270606995, "tokens_seen": 96018432, "tokens_per_sec": 22653.270792865493, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3616129557291667, "G_std": 0.1545740763346354, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.323108434677124, "grad_norm": 1.0251246690750122, "tokens_seen": 96182272, "tokens_per_sec": 22653.27393895349, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3645477294921875, "G_std": 0.1542231241861979, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.305305242538452, "grad_norm": 0.9806689620018005, "tokens_seen": 96346112, "tokens_per_sec": 22653.349348134034, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3596343994140625, "G_std": 0.1542485555013021, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.358369827270508, "grad_norm": 0.9655414819717407, "tokens_seen": 96509952, "tokens_per_sec": 22653.4654047354, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3619791666666667, "G_std": 0.1536534627278646, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.307697296142578, "grad_norm": 0.9626760482788086, "tokens_seen": 96673792, "tokens_per_sec": 22653.692008050388, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606363932291667, "G_std": 0.1542383829752604, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.363387584686279, "grad_norm": 1.0106608867645264, "tokens_seen": 96837632, "tokens_per_sec": 22653.93502920617, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610890706380208, "G_std": 0.15311686197916666, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.236125707626343, "grad_norm": 0.9760842323303223, "tokens_seen": 97001472, "tokens_per_sec": 22654.178120115754, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582661946614583, "G_std": 0.1528294881184896, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.274119138717651, "grad_norm": 0.9700624942779541, "tokens_seen": 97165312, "tokens_per_sec": 22654.232251116566, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3607889811197917, "G_std": 0.1524988810221354, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.449675559997559, "grad_norm": 1.0276602506637573, "tokens_seen": 97329152, "tokens_per_sec": 22654.169384701545, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3670857747395833, "G_std": 0.15264383951822916, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.273083209991455, "grad_norm": 0.9884598851203918, "tokens_seen": 97492992, "tokens_per_sec": 22654.32957480719, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35687255859375, "G_std": 0.1539484659830729, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.258102655410767, "grad_norm": 0.986893355846405, "tokens_seen": 97656832, "tokens_per_sec": 22654.42352079106, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3612060546875, "G_std": 0.1531346638997396, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.398390293121338, "grad_norm": 1.033732295036316, "tokens_seen": 97820672, "tokens_per_sec": 22654.80254821865, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.359466552734375, "G_std": 0.15270233154296875, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.254742622375488, "grad_norm": 1.0058592557907104, "tokens_seen": 97984512, "tokens_per_sec": 22655.018105981424, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3640034993489583, "G_std": 0.1534271240234375, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.286156415939331, "grad_norm": 0.9749442934989929, "tokens_seen": 98148352, "tokens_per_sec": 22654.94649000648, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.362548828125, "G_std": 0.15349324544270834, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.356495881080628, "val_ppl": 77.98339201086276, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3648274739583333, "G_std": 0.1544977823893229, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4340.425580263138, "nan_failures": 0} diff --git a/g_only_s4/config.json b/g_only_s4/config.json new file mode 100644 index 0000000000000000000000000000000000000000..e20be630dd55584368e4830367e79d3acd2945cf --- /dev/null +++ b/g_only_s4/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 4, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s4", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 4001349, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/g_only_s4/final.pt b/g_only_s4/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..2f3996f143dc685e68b0975c281b6880ec05836e --- /dev/null +++ b/g_only_s4/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:281e77bb5b5fc59ae5b6ad72e2cd31c370b9fccd0b610606f7ac724f586cdc35 +size 253629735 diff --git a/g_only_s4/metrics.jsonl b/g_only_s4/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..07c24598a083f5007a6332a2432d01d03919888e --- /dev/null +++ b/g_only_s4/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 4, "config": {"variant": "g_only", "seed": 4, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s4", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 4001349, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.884689331054688, "grad_norm": 10.44764232635498, "tokens_seen": 8192, "tokens_per_sec": 9139.545745157946, "vram_gb": 4.987702272, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5079752604166666, "G_std": 0.08650716145833333, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.81784439086914, "grad_norm": 1.8468997478485107, "tokens_seen": 172032, "tokens_per_sec": 21377.534975345698, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4644368489583333, "G_std": 0.07026163736979167, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.064069747924805, "grad_norm": 1.7622883319854736, "tokens_seen": 335872, "tokens_per_sec": 22109.27250243681, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.451904296875, "G_std": 0.0758819580078125, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.196197509765625, "grad_norm": 1.4814921617507935, "tokens_seen": 499712, "tokens_per_sec": 22371.845003420818, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4474283854166667, "G_std": 0.07887776692708333, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.539172172546387, "grad_norm": 0.752085268497467, "tokens_seen": 663552, "tokens_per_sec": 22511.446740852487, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.453125, "G_std": 0.09462483723958333, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 6.9923882484436035, "grad_norm": 0.8708130717277527, "tokens_seen": 827392, "tokens_per_sec": 22555.713237198208, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4402669270833333, "G_std": 0.11366780598958333, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.161527395248413, "grad_norm": 0.5656476616859436, "tokens_seen": 991232, "tokens_per_sec": 22609.88395325995, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4327799479166667, "G_std": 0.119140625, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.808979511260986, "grad_norm": 0.5383737683296204, "tokens_seen": 1155072, "tokens_per_sec": 22656.52637842258, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41162109375, "G_std": 0.1282958984375, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.725713014602661, "grad_norm": 0.5747106671333313, "tokens_seen": 1318912, "tokens_per_sec": 22678.18034551975, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.415283203125, "G_std": 0.141845703125, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.749485492706299, "grad_norm": 0.5469267964363098, "tokens_seen": 1482752, "tokens_per_sec": 22703.114898952837, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3943684895833333, "G_std": 0.14436848958333334, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.703210115432739, "grad_norm": 0.6226401925086975, "tokens_seen": 1646592, "tokens_per_sec": 22726.04967223427, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4014485677083333, "G_std": 0.15311686197916666, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.7376813888549805, "grad_norm": 0.5671080946922302, "tokens_seen": 1810432, "tokens_per_sec": 22711.632467790332, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4039713541666667, "G_std": 0.16668701171875, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.528273582458496, "grad_norm": 0.6698873043060303, "tokens_seen": 1974272, "tokens_per_sec": 22720.958180391008, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.389404296875, "G_std": 0.16717529296875, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.609171152114868, "grad_norm": 0.4693068563938141, "tokens_seen": 2138112, "tokens_per_sec": 22729.278879595637, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.406494140625, "G_std": 0.176025390625, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.67070460319519, "grad_norm": 0.5127808451652527, "tokens_seen": 2301952, "tokens_per_sec": 22735.18722138416, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4229329427083333, "G_std": 0.17803955078125, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.3502442836761475, "grad_norm": 0.4809332489967346, "tokens_seen": 2465792, "tokens_per_sec": 22743.358192039188, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.397705078125, "G_std": 0.17881266276041666, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.397493839263916, "grad_norm": 0.5434941649436951, "tokens_seen": 2629632, "tokens_per_sec": 22745.12481977731, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961588541666667, "G_std": 0.17862955729166666, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.432194709777832, "grad_norm": 0.44608065485954285, "tokens_seen": 2793472, "tokens_per_sec": 22754.492192998296, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4229329427083333, "G_std": 0.19287109375, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.4653356075286865, "grad_norm": 0.45479467511177063, "tokens_seen": 2957312, "tokens_per_sec": 22763.305421209134, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4234212239583333, "G_std": 0.19110107421875, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.404038906097412, "grad_norm": 0.5028749108314514, "tokens_seen": 3121152, "tokens_per_sec": 22765.926096134248, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.423095703125, "G_std": 0.19655354817708334, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.297331094741821, "grad_norm": 0.5006024241447449, "tokens_seen": 3284992, "tokens_per_sec": 22771.434545254582, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4342447916666667, "G_std": 0.2030029296875, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.171747446060181, "grad_norm": 0.4804859459400177, "tokens_seen": 3448832, "tokens_per_sec": 22780.79219894528, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4220377604166667, "G_std": 0.2001953125, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.269907236099243, "grad_norm": 0.4850390553474426, "tokens_seen": 3612672, "tokens_per_sec": 22782.40333083331, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4390462239583333, "G_std": 0.20735677083333334, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.298153877258301, "grad_norm": 0.46973544359207153, "tokens_seen": 3776512, "tokens_per_sec": 22788.10741763856, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4405924479166667, "G_std": 0.212158203125, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.280271768569946, "grad_norm": 0.47668272256851196, "tokens_seen": 3940352, "tokens_per_sec": 22795.358223373587, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4650065104166667, "G_std": 0.219482421875, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.217195510864258, "grad_norm": 0.47659778594970703, "tokens_seen": 4104192, "tokens_per_sec": 22793.63496826228, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.482666015625, "G_std": 0.22111002604166666, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.250023126602173, "grad_norm": 0.48558199405670166, "tokens_seen": 4268032, "tokens_per_sec": 22798.622205344433, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.484619140625, "G_std": 0.22298177083333334, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.074127197265625, "grad_norm": 0.5241894721984863, "tokens_seen": 4431872, "tokens_per_sec": 22803.478549625077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5409342447916666, "G_std": 0.22635904947916666, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.198239803314209, "grad_norm": 0.5119740962982178, "tokens_seen": 4595712, "tokens_per_sec": 22803.818020820825, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5267740885416666, "G_std": 0.21915690104166666, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.064839601516724, "grad_norm": 0.5090782046318054, "tokens_seen": 4759552, "tokens_per_sec": 22808.77023498242, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5101725260416666, "G_std": 0.22151692708333334, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.019765138626099, "grad_norm": 0.4807100296020508, "tokens_seen": 4923392, "tokens_per_sec": 22812.23777242782, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5432942708333334, "G_std": 0.2203369140625, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.1227521896362305, "grad_norm": 0.4655858278274536, "tokens_seen": 5087232, "tokens_per_sec": 22814.346239751387, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5410970052083334, "G_std": 0.22554524739583334, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.225636720657349, "grad_norm": 0.48794153332710266, "tokens_seen": 5251072, "tokens_per_sec": 22818.413406903677, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5555013020833334, "G_std": 0.216064453125, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.04355788230896, "grad_norm": 0.4941423535346985, "tokens_seen": 5414912, "tokens_per_sec": 22822.690043657847, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5693359375, "G_std": 0.218017578125, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.052117347717285, "grad_norm": 0.49683740735054016, "tokens_seen": 5578752, "tokens_per_sec": 22827.032963713802, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.549560546875, "G_std": 0.21146647135416666, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.9548680782318115, "grad_norm": 0.4648008942604065, "tokens_seen": 5742592, "tokens_per_sec": 22832.035252130958, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.571533203125, "G_std": 0.20857747395833334, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.922504663467407, "grad_norm": 0.5271976590156555, "tokens_seen": 5906432, "tokens_per_sec": 22835.448346726524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5694173177083334, "G_std": 0.20756022135416666, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.02661657333374, "grad_norm": 0.49609068036079407, "tokens_seen": 6070272, "tokens_per_sec": 22835.624048446647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.554443359375, "G_std": 0.2042236328125, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.982093095779419, "grad_norm": 0.46560999751091003, "tokens_seen": 6234112, "tokens_per_sec": 22835.802842336223, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.56298828125, "G_std": 0.2086181640625, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.110185623168945, "grad_norm": 0.510204017162323, "tokens_seen": 6397952, "tokens_per_sec": 22837.531322262534, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.573486328125, "G_std": 0.20709228515625, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.826995134353638, "grad_norm": 0.4540964961051941, "tokens_seen": 6561792, "tokens_per_sec": 22840.485239039906, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5668131510416666, "G_std": 0.20149739583333334, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.897908926010132, "grad_norm": 0.506950318813324, "tokens_seen": 6725632, "tokens_per_sec": 22840.899580528436, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5858561197916666, "G_std": 0.20045979817708334, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.838265895843506, "grad_norm": 0.5047606825828552, "tokens_seen": 6889472, "tokens_per_sec": 22842.71336157874, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.572265625, "G_std": 0.20430501302083334, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.916490077972412, "grad_norm": 0.529339611530304, "tokens_seen": 7053312, "tokens_per_sec": 22843.975298210582, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.581787109375, "G_std": 0.19472249348958334, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.75197958946228, "grad_norm": 0.47252950072288513, "tokens_seen": 7217152, "tokens_per_sec": 22846.184386665078, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5745442708333334, "G_std": 0.19779459635416666, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.7682390213012695, "grad_norm": 0.5488845109939575, "tokens_seen": 7380992, "tokens_per_sec": 22844.972787382503, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.6003824869791666, "G_std": 0.19527180989583334, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.828403949737549, "grad_norm": 0.529646098613739, "tokens_seen": 7544832, "tokens_per_sec": 22844.649010661466, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5851643880208334, "G_std": 0.19691975911458334, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.742126941680908, "grad_norm": 0.5317406058311462, "tokens_seen": 7708672, "tokens_per_sec": 22845.235550269797, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5876871744791666, "G_std": 0.19234212239583334, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.7924981117248535, "grad_norm": 0.5673325657844543, "tokens_seen": 7872512, "tokens_per_sec": 22844.70673517428, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5704752604166666, "G_std": 0.19905598958333334, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.796052694320679, "grad_norm": 0.533198893070221, "tokens_seen": 8036352, "tokens_per_sec": 22846.09463897843, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5562744140625, "G_std": 0.19217936197916666, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.699042320251465, "grad_norm": 0.5515590906143188, "tokens_seen": 8200192, "tokens_per_sec": 22846.567198331264, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5625, "G_std": 0.19228108723958334, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.7604457378387455, "val_ppl": 317.4898146065318, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.5691324869791666, "G_std": 0.19468180338541666, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.743226528167725, "grad_norm": 0.48493075370788574, "tokens_seen": 8364032, "tokens_per_sec": 22767.60128482626, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5682373046875, "G_std": 0.19285074869791666, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.710746765136719, "grad_norm": 0.5101163983345032, "tokens_seen": 8527872, "tokens_per_sec": 22769.519187866477, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5523274739583334, "G_std": 0.18760172526041666, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.602169752120972, "grad_norm": 0.5071059465408325, "tokens_seen": 8691712, "tokens_per_sec": 22771.8752779972, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5525309244791666, "G_std": 0.19449869791666666, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.647240400314331, "grad_norm": 0.5257239937782288, "tokens_seen": 8855552, "tokens_per_sec": 22771.368252160933, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.550537109375, "G_std": 0.192138671875, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.803179025650024, "grad_norm": 0.5236802697181702, "tokens_seen": 9019392, "tokens_per_sec": 22773.464859722306, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.564697265625, "G_std": 0.19024658203125, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.769007921218872, "grad_norm": 0.49891048669815063, "tokens_seen": 9183232, "tokens_per_sec": 22775.757649134564, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5551350911458334, "G_std": 0.19366455078125, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.565859079360962, "grad_norm": 0.5145788192749023, "tokens_seen": 9347072, "tokens_per_sec": 22776.22043540359, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5474039713541666, "G_std": 0.19020589192708334, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.6104700565338135, "grad_norm": 0.48260176181793213, "tokens_seen": 9510912, "tokens_per_sec": 22778.141479316408, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.54052734375, "G_std": 0.19047037760416666, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.697815418243408, "grad_norm": 0.5490573048591614, "tokens_seen": 9674752, "tokens_per_sec": 22780.756575999643, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5296223958333334, "G_std": 0.19256591796875, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.680644512176514, "grad_norm": 0.5253198742866516, "tokens_seen": 9838592, "tokens_per_sec": 22783.079786304377, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5393880208333334, "G_std": 0.19126383463541666, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.889878034591675, "grad_norm": 0.5342018604278564, "tokens_seen": 10002432, "tokens_per_sec": 22785.605705563336, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5517578125, "G_std": 0.19144694010416666, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.54028844833374, "grad_norm": 0.5398147702217102, "tokens_seen": 10166272, "tokens_per_sec": 22788.334260876763, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5680745442708334, "G_std": 0.19134521484375, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.554474115371704, "grad_norm": 0.5231049060821533, "tokens_seen": 10330112, "tokens_per_sec": 22790.368456238153, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5355224609375, "G_std": 0.1917724609375, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.434835433959961, "grad_norm": 0.4833219647407532, "tokens_seen": 10493952, "tokens_per_sec": 22792.552258798816, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5404052734375, "G_std": 0.18471272786458334, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.764480113983154, "grad_norm": 0.5514639019966125, "tokens_seen": 10657792, "tokens_per_sec": 22795.5490240749, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5497233072916666, "G_std": 0.18894449869791666, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.505972623825073, "grad_norm": 0.5484737753868103, "tokens_seen": 10821632, "tokens_per_sec": 22797.714747117654, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5470377604166666, "G_std": 0.18587239583333334, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.487522125244141, "grad_norm": 0.5535959005355835, "tokens_seen": 10985472, "tokens_per_sec": 22800.84726549201, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5228678385416666, "G_std": 0.19081624348958334, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.6085546016693115, "grad_norm": 0.5321841239929199, "tokens_seen": 11149312, "tokens_per_sec": 22803.260750176913, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5298258463541666, "G_std": 0.19374593098958334, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.548914194107056, "grad_norm": 0.5422427654266357, "tokens_seen": 11313152, "tokens_per_sec": 22804.921733041992, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5417073567708334, "G_std": 0.1920166015625, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.419216632843018, "grad_norm": 0.5443565845489502, "tokens_seen": 11476992, "tokens_per_sec": 22805.19752644745, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5318603515625, "G_std": 0.19109090169270834, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.542633533477783, "grad_norm": 0.5200701355934143, "tokens_seen": 11640832, "tokens_per_sec": 22805.48077404772, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5221761067708334, "G_std": 0.189544677734375, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.481154680252075, "grad_norm": 0.5987697243690491, "tokens_seen": 11804672, "tokens_per_sec": 22806.674936399824, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5205078125, "G_std": 0.191192626953125, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.409116268157959, "grad_norm": 0.5201418995857239, "tokens_seen": 11968512, "tokens_per_sec": 22807.301640961174, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5219523111979166, "G_std": 0.186767578125, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.445337295532227, "grad_norm": 0.5544677376747131, "tokens_seen": 12132352, "tokens_per_sec": 22808.44960876631, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5081380208333334, "G_std": 0.19608561197916666, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.48925256729126, "grad_norm": 0.5367277264595032, "tokens_seen": 12296192, "tokens_per_sec": 22809.907609717553, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.52447509765625, "G_std": 0.18921915690104166, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.312759160995483, "grad_norm": 0.5279242396354675, "tokens_seen": 12460032, "tokens_per_sec": 22811.479478760273, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5252482096354166, "G_std": 0.19159952799479166, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.416084289550781, "grad_norm": 0.5879223942756653, "tokens_seen": 12623872, "tokens_per_sec": 22812.39046848757, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.51239013671875, "G_std": 0.19035847981770834, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.493841171264648, "grad_norm": 0.5520743727684021, "tokens_seen": 12787712, "tokens_per_sec": 22812.85931702651, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5161946614583334, "G_std": 0.19091796875, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.557351589202881, "grad_norm": 0.542131245136261, "tokens_seen": 12951552, "tokens_per_sec": 22814.100726676617, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5107828776041666, "G_std": 0.19007364908854166, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.258935213088989, "grad_norm": 0.5662404298782349, "tokens_seen": 13115392, "tokens_per_sec": 22814.584903691626, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5063680013020834, "G_std": 0.19024658203125, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.387142181396484, "grad_norm": 0.5585089921951294, "tokens_seen": 13279232, "tokens_per_sec": 22816.278500564895, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5117390950520834, "G_std": 0.1934814453125, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.410473346710205, "grad_norm": 0.5693742036819458, "tokens_seen": 13443072, "tokens_per_sec": 22817.1089486056, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.50390625, "G_std": 0.18980916341145834, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.36545467376709, "grad_norm": 0.5976687669754028, "tokens_seen": 13606912, "tokens_per_sec": 22817.264040728496, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4964396158854167, "G_std": 0.19170125325520834, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.233030557632446, "grad_norm": 0.5346047282218933, "tokens_seen": 13770752, "tokens_per_sec": 22817.63392672155, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4943644205729167, "G_std": 0.19429524739583334, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.298547744750977, "grad_norm": 0.5581521391868591, "tokens_seen": 13934592, "tokens_per_sec": 22818.666288676206, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.486572265625, "G_std": 0.19154866536458334, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.390165090560913, "grad_norm": 0.5487644672393799, "tokens_seen": 14098432, "tokens_per_sec": 22819.160810912646, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.50238037109375, "G_std": 0.19249471028645834, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.466074466705322, "grad_norm": 0.5840561389923096, "tokens_seen": 14262272, "tokens_per_sec": 22821.21146228752, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5093180338541666, "G_std": 0.19373575846354166, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.29571270942688, "grad_norm": 0.5479799509048462, "tokens_seen": 14426112, "tokens_per_sec": 22821.425909429636, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.49871826171875, "G_std": 0.1923828125, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.453123092651367, "grad_norm": 0.5519861578941345, "tokens_seen": 14589952, "tokens_per_sec": 22820.946719943582, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4818929036458333, "G_std": 0.19443766276041666, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.204478740692139, "grad_norm": 0.564896821975708, "tokens_seen": 14753792, "tokens_per_sec": 22821.24045413865, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4891764322916667, "G_std": 0.19029744466145834, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.2908172607421875, "grad_norm": 0.5261496305465698, "tokens_seen": 14917632, "tokens_per_sec": 22821.626699082182, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.49139404296875, "G_std": 0.19429524739583334, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.3348448276519775, "grad_norm": 0.5976815223693848, "tokens_seen": 15081472, "tokens_per_sec": 22821.996808307136, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5031941731770834, "G_std": 0.18902587890625, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.527095794677734, "grad_norm": 0.5418450236320496, "tokens_seen": 15245312, "tokens_per_sec": 22822.969440453486, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.49322509765625, "G_std": 0.19196573893229166, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.40529727935791, "grad_norm": 0.5608721375465393, "tokens_seen": 15409152, "tokens_per_sec": 22823.275887790285, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.5029500325520834, "G_std": 0.19415283203125, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.307191371917725, "grad_norm": 0.6606405377388, "tokens_seen": 15572992, "tokens_per_sec": 22824.165739649077, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4894205729166667, "G_std": 0.18990071614583334, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.3274500370025635, "grad_norm": 0.5734745264053345, "tokens_seen": 15736832, "tokens_per_sec": 22825.252067939844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4819742838541667, "G_std": 0.192962646484375, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.326656103134155, "grad_norm": 0.5772509574890137, "tokens_seen": 15900672, "tokens_per_sec": 22826.14498331157, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4756876627604167, "G_std": 0.191162109375, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.431132793426514, "grad_norm": 0.5575348138809204, "tokens_seen": 16064512, "tokens_per_sec": 22826.89814574297, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.48016357421875, "G_std": 0.19352213541666666, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.208921194076538, "grad_norm": 0.5540492534637451, "tokens_seen": 16228352, "tokens_per_sec": 22827.856307093836, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4812418619791667, "G_std": 0.19005330403645834, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.268468141555786, "grad_norm": 0.6749595999717712, "tokens_seen": 16392192, "tokens_per_sec": 22828.558391522685, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4866129557291667, "G_std": 0.18983968098958334, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.313734102249145, "val_ppl": 203.1072373258614, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.48162841796875, "G_std": 0.18879191080729166, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.204967260360718, "grad_norm": 0.538646399974823, "tokens_seen": 16556032, "tokens_per_sec": 22790.579515792368, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47265625, "G_std": 0.188385009765625, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.406493902206421, "grad_norm": 0.718085527420044, "tokens_seen": 16719872, "tokens_per_sec": 22792.289711452624, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.46331787109375, "G_std": 0.18816121419270834, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.309306383132935, "grad_norm": 0.6173539757728577, "tokens_seen": 16883712, "tokens_per_sec": 22793.33193800754, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.47100830078125, "G_std": 0.190093994140625, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.214132070541382, "grad_norm": 0.5652042627334595, "tokens_seen": 17047552, "tokens_per_sec": 22794.510457440614, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4754638671875, "G_std": 0.18839518229166666, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.1379876136779785, "grad_norm": 0.6480914354324341, "tokens_seen": 17211392, "tokens_per_sec": 22794.968820566595, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4699910481770833, "G_std": 0.18864949544270834, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.237387657165527, "grad_norm": 0.5872015953063965, "tokens_seen": 17375232, "tokens_per_sec": 22795.717133233622, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4601949055989583, "G_std": 0.19069417317708334, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.206942319869995, "grad_norm": 0.5681350231170654, "tokens_seen": 17539072, "tokens_per_sec": 22797.035157716226, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.471527099609375, "G_std": 0.18901570638020834, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.166405200958252, "grad_norm": 0.5618085861206055, "tokens_seen": 17702912, "tokens_per_sec": 22798.261950525277, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4743855794270833, "G_std": 0.187713623046875, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.306381702423096, "grad_norm": 0.5553244948387146, "tokens_seen": 17866752, "tokens_per_sec": 22799.262779113018, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4678955078125, "G_std": 0.19110107421875, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.209230899810791, "grad_norm": 0.6063753962516785, "tokens_seen": 18030592, "tokens_per_sec": 22800.18805875504, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.475799560546875, "G_std": 0.19173177083333334, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.146823167800903, "grad_norm": 0.6075841784477234, "tokens_seen": 18194432, "tokens_per_sec": 22801.53997021469, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.472320556640625, "G_std": 0.18955485026041666, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.078459978103638, "grad_norm": 0.5940520763397217, "tokens_seen": 18358272, "tokens_per_sec": 22802.031252803386, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.468048095703125, "G_std": 0.18709309895833334, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.256550073623657, "grad_norm": 0.6037601828575134, "tokens_seen": 18522112, "tokens_per_sec": 22803.03648581922, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4583638509114583, "G_std": 0.19115193684895834, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.142352819442749, "grad_norm": 0.5904724597930908, "tokens_seen": 18685952, "tokens_per_sec": 22803.594071231746, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4744364420572917, "G_std": 0.18730672200520834, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.132841110229492, "grad_norm": 0.5813602209091187, "tokens_seen": 18849792, "tokens_per_sec": 22804.148416504646, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4678955078125, "G_std": 0.188995361328125, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.01361083984375, "grad_norm": 0.6125832200050354, "tokens_seen": 19013632, "tokens_per_sec": 22805.39482631729, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4549153645833333, "G_std": 0.188690185546875, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.173132419586182, "grad_norm": 0.5911220908164978, "tokens_seen": 19177472, "tokens_per_sec": 22805.597502317487, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.468963623046875, "G_std": 0.18806966145833334, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 4.922669887542725, "grad_norm": 0.6410222053527832, "tokens_seen": 19341312, "tokens_per_sec": 22807.077479370477, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4698282877604167, "G_std": 0.19425455729166666, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.10712194442749, "grad_norm": 0.6309055089950562, "tokens_seen": 19505152, "tokens_per_sec": 22807.901843885542, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.465484619140625, "G_std": 0.18503824869791666, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.0854880809783936, "grad_norm": 0.6812185049057007, "tokens_seen": 19668992, "tokens_per_sec": 22808.148411094226, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4626566569010417, "G_std": 0.18919881184895834, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.050454616546631, "grad_norm": 0.6010156273841858, "tokens_seen": 19832832, "tokens_per_sec": 22807.27746699245, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.46380615234375, "G_std": 0.19099934895833334, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.096140146255493, "grad_norm": 0.6425764560699463, "tokens_seen": 19996672, "tokens_per_sec": 22807.04172721854, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4608256022135417, "G_std": 0.18677775065104166, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.042188405990601, "grad_norm": 0.5904761552810669, "tokens_seen": 20160512, "tokens_per_sec": 22806.736648059166, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4639078776041667, "G_std": 0.18804931640625, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.195148229598999, "grad_norm": 0.6766517162322998, "tokens_seen": 20324352, "tokens_per_sec": 22806.868131519972, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.462371826171875, "G_std": 0.18642171223958334, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.112274408340454, "grad_norm": 0.6092290282249451, "tokens_seen": 20488192, "tokens_per_sec": 22807.002561792982, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4543253580729167, "G_std": 0.1873016357421875, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.009714841842651, "grad_norm": 0.6744822263717651, "tokens_seen": 20652032, "tokens_per_sec": 22805.446243203518, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.447296142578125, "G_std": 0.18816121419270834, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.035884141921997, "grad_norm": 0.5982872843742371, "tokens_seen": 20815872, "tokens_per_sec": 22803.988028970365, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4569905598958333, "G_std": 0.189544677734375, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.210753917694092, "grad_norm": 0.6411903500556946, "tokens_seen": 20979712, "tokens_per_sec": 22801.468645140434, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4485371907552083, "G_std": 0.18701171875, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.03915810585022, "grad_norm": 0.6379624009132385, "tokens_seen": 21143552, "tokens_per_sec": 22800.729745604545, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4401346842447917, "G_std": 0.18856302897135416, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.0223047733306885, "grad_norm": 0.6249898076057434, "tokens_seen": 21307392, "tokens_per_sec": 22800.474404839584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.45465087890625, "G_std": 0.18918355305989584, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.0772764682769775, "grad_norm": 0.5855122804641724, "tokens_seen": 21471232, "tokens_per_sec": 22800.239227554168, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4455668131510417, "G_std": 0.1878204345703125, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.066811561584473, "grad_norm": 0.630768358707428, "tokens_seen": 21635072, "tokens_per_sec": 22800.21096266787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.44287109375, "G_std": 0.18695576985677084, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.1947197914123535, "grad_norm": 0.611667811870575, "tokens_seen": 21798912, "tokens_per_sec": 22800.18451571642, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4523213704427083, "G_std": 0.1864013671875, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 4.906185865402222, "grad_norm": 0.6497766971588135, "tokens_seen": 21962752, "tokens_per_sec": 22800.68631110438, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4415079752604167, "G_std": 0.18587239583333334, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.145959138870239, "grad_norm": 0.6033578515052795, "tokens_seen": 22126592, "tokens_per_sec": 22800.83867971659, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.441131591796875, "G_std": 0.186737060546875, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.1221020221710205, "grad_norm": 0.6386371850967407, "tokens_seen": 22290432, "tokens_per_sec": 22800.798013503787, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4600016276041667, "G_std": 0.187744140625, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.091942310333252, "grad_norm": 0.6425278186798096, "tokens_seen": 22454272, "tokens_per_sec": 22800.610894346606, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4520772298177083, "G_std": 0.1886444091796875, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.031080007553101, "grad_norm": 0.6512969732284546, "tokens_seen": 22618112, "tokens_per_sec": 22800.382244757344, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4421590169270833, "G_std": 0.1881561279296875, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.08138370513916, "grad_norm": 0.682161808013916, "tokens_seen": 22781952, "tokens_per_sec": 22798.35401481993, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4552408854166667, "G_std": 0.19016011555989584, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.006202936172485, "grad_norm": 0.6244011521339417, "tokens_seen": 22945792, "tokens_per_sec": 22797.991462680297, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4362284342447917, "G_std": 0.1876373291015625, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.930246114730835, "grad_norm": 0.6648834347724915, "tokens_seen": 23109632, "tokens_per_sec": 22797.56534343279, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.441253662109375, "G_std": 0.186676025390625, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 4.915432691574097, "grad_norm": 0.6418677568435669, "tokens_seen": 23273472, "tokens_per_sec": 22797.757534305798, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4335530598958333, "G_std": 0.18852742513020834, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.041138172149658, "grad_norm": 0.6254377961158752, "tokens_seen": 23437312, "tokens_per_sec": 22797.569289408922, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4391072591145833, "G_std": 0.186279296875, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 4.997570037841797, "grad_norm": 0.6460919380187988, "tokens_seen": 23601152, "tokens_per_sec": 22797.33320147736, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4346822102864583, "G_std": 0.1869354248046875, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.011280298233032, "grad_norm": 0.656056821346283, "tokens_seen": 23764992, "tokens_per_sec": 22797.76042392435, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.437591552734375, "G_std": 0.18644205729166666, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.060715198516846, "grad_norm": 0.6214038133621216, "tokens_seen": 23928832, "tokens_per_sec": 22797.372480172577, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4399515787760417, "G_std": 0.18582661946614584, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 4.979902267456055, "grad_norm": 0.6343204975128174, "tokens_seen": 24092672, "tokens_per_sec": 22797.697220904138, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4353129069010417, "G_std": 0.18728129069010416, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.968580722808838, "grad_norm": 0.6518802046775818, "tokens_seen": 24256512, "tokens_per_sec": 22797.45201438364, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4320576985677083, "G_std": 0.18558247884114584, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.978867292404175, "grad_norm": 0.6798506379127502, "tokens_seen": 24420352, "tokens_per_sec": 22796.939818894465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4451497395833333, "G_std": 0.1842041015625, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.961907863616943, "grad_norm": 0.6673662066459656, "tokens_seen": 24584192, "tokens_per_sec": 22797.174292185362, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4250284830729167, "G_std": 0.1848907470703125, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 4.975148487091064, "val_ppl": 144.7703200759823, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4380086263020833, "G_std": 0.18488057454427084, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.882665634155273, "grad_norm": 0.6639230847358704, "tokens_seen": 24748032, "tokens_per_sec": 22770.707533981178, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4378255208333333, "G_std": 0.18743387858072916, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.843778133392334, "grad_norm": 0.6694226861000061, "tokens_seen": 24911872, "tokens_per_sec": 22770.32939944567, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.433349609375, "G_std": 0.1856536865234375, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.106774806976318, "grad_norm": 0.6683529019355774, "tokens_seen": 25075712, "tokens_per_sec": 22769.263069617, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4381612141927083, "G_std": 0.186798095703125, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.98003077507019, "grad_norm": 0.6616819500923157, "tokens_seen": 25239552, "tokens_per_sec": 22768.04861197632, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4365336100260417, "G_std": 0.18328348795572916, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.013434648513794, "grad_norm": 0.7424658536911011, "tokens_seen": 25403392, "tokens_per_sec": 22766.83207879783, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4365336100260417, "G_std": 0.187347412109375, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.889365196228027, "grad_norm": 0.6570137143135071, "tokens_seen": 25567232, "tokens_per_sec": 22766.645251562357, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4345703125, "G_std": 0.18549092610677084, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.9042558670043945, "grad_norm": 0.6794571280479431, "tokens_seen": 25731072, "tokens_per_sec": 22767.422584842236, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4269612630208333, "G_std": 0.1829376220703125, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.995380163192749, "grad_norm": 0.6529914140701294, "tokens_seen": 25894912, "tokens_per_sec": 22767.19593349178, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4348246256510417, "G_std": 0.18660481770833334, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.900910139083862, "grad_norm": 0.7545576095581055, "tokens_seen": 26058752, "tokens_per_sec": 22767.54362289574, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42681884765625, "G_std": 0.185760498046875, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.9202516078948975, "grad_norm": 0.6721084713935852, "tokens_seen": 26222592, "tokens_per_sec": 22767.899906099487, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4274698893229167, "G_std": 0.18419392903645834, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.073464393615723, "grad_norm": 0.6848506331443787, "tokens_seen": 26386432, "tokens_per_sec": 22766.506793299275, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.431427001953125, "G_std": 0.18616739908854166, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.930992603302002, "grad_norm": 0.6407506465911865, "tokens_seen": 26550272, "tokens_per_sec": 22766.28282733688, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4303995768229167, "G_std": 0.1858367919921875, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.9422149658203125, "grad_norm": 0.6393763422966003, "tokens_seen": 26714112, "tokens_per_sec": 22766.540149099274, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4198811848958333, "G_std": 0.18295796712239584, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.836318016052246, "grad_norm": 0.667823851108551, "tokens_seen": 26877952, "tokens_per_sec": 22766.91236494835, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42889404296875, "G_std": 0.18416341145833334, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.949014663696289, "grad_norm": 0.6751590967178345, "tokens_seen": 27041792, "tokens_per_sec": 22767.82519567376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4282124837239583, "G_std": 0.18533833821614584, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.8292155265808105, "grad_norm": 0.6662209630012512, "tokens_seen": 27205632, "tokens_per_sec": 22768.40078165361, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.422271728515625, "G_std": 0.18679300944010416, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.92832088470459, "grad_norm": 0.681847095489502, "tokens_seen": 27369472, "tokens_per_sec": 22768.873742224874, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4191080729166667, "G_std": 0.18721516927083334, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.895290851593018, "grad_norm": 0.7003385424613953, "tokens_seen": 27533312, "tokens_per_sec": 22769.68631535438, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.42913818359375, "G_std": 0.18337504069010416, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.803550720214844, "grad_norm": 0.6563961505889893, "tokens_seen": 27697152, "tokens_per_sec": 22769.82324949026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4199117024739583, "G_std": 0.18436686197916666, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.818454265594482, "grad_norm": 0.6778668165206909, "tokens_seen": 27860992, "tokens_per_sec": 22770.068696167193, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4147847493489583, "G_std": 0.18427022298177084, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.827872276306152, "grad_norm": 0.6366321444511414, "tokens_seen": 28024832, "tokens_per_sec": 22769.727720564773, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4180094401041667, "G_std": 0.1853179931640625, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.75632905960083, "grad_norm": 0.7011008858680725, "tokens_seen": 28188672, "tokens_per_sec": 22769.42308869639, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.424835205078125, "G_std": 0.18648783365885416, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.7665534019470215, "grad_norm": 0.6730996370315552, "tokens_seen": 28352512, "tokens_per_sec": 22769.27313634902, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4338175455729167, "G_std": 0.18598429361979166, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 5.015573263168335, "grad_norm": 0.6738974452018738, "tokens_seen": 28516352, "tokens_per_sec": 22769.36499074495, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4249216715494792, "G_std": 0.1842498779296875, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.921430349349976, "grad_norm": 0.6504217386245728, "tokens_seen": 28680192, "tokens_per_sec": 22769.60686668222, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4190775553385417, "G_std": 0.18666585286458334, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.77341890335083, "grad_norm": 0.682460606098175, "tokens_seen": 28844032, "tokens_per_sec": 22768.948114349456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4139302571614583, "G_std": 0.18304951985677084, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 5.033503532409668, "grad_norm": 0.729605495929718, "tokens_seen": 29007872, "tokens_per_sec": 22769.284131909415, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4175923665364583, "G_std": 0.18308003743489584, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.904363393783569, "grad_norm": 0.6774214506149292, "tokens_seen": 29171712, "tokens_per_sec": 22769.23710581737, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41876220703125, "G_std": 0.18149312337239584, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 5.13550591468811, "grad_norm": 0.6882465481758118, "tokens_seen": 29335552, "tokens_per_sec": 22769.585457330893, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4257710774739583, "G_std": 0.18497212727864584, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.892268419265747, "grad_norm": 0.6623278856277466, "tokens_seen": 29499392, "tokens_per_sec": 22769.685973437277, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4251861572265625, "G_std": 0.18361409505208334, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.735612392425537, "grad_norm": 0.7016228437423706, "tokens_seen": 29663232, "tokens_per_sec": 22769.483857145908, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4156545003255208, "G_std": 0.18279012044270834, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.857436418533325, "grad_norm": 0.6905819773674011, "tokens_seen": 29827072, "tokens_per_sec": 22769.227887267993, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4140675862630208, "G_std": 0.18209330240885416, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.792129039764404, "grad_norm": 0.8280881643295288, "tokens_seen": 29990912, "tokens_per_sec": 22768.855898331734, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4297688802083333, "G_std": 0.18250528971354166, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.888483762741089, "grad_norm": 0.7138917446136475, "tokens_seen": 30154752, "tokens_per_sec": 22769.349334665043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.412139892578125, "G_std": 0.18230183919270834, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.873267412185669, "grad_norm": 0.7212814092636108, "tokens_seen": 30318592, "tokens_per_sec": 22769.65963788565, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4247080485026042, "G_std": 0.1836700439453125, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.781928062438965, "grad_norm": 0.7054493427276611, "tokens_seen": 30482432, "tokens_per_sec": 22769.539740055414, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4207865397135417, "G_std": 0.18195088704427084, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.6355767250061035, "grad_norm": 0.6940087676048279, "tokens_seen": 30646272, "tokens_per_sec": 22770.079004790266, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4164225260416667, "G_std": 0.18426513671875, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.77205753326416, "grad_norm": 0.7017620205879211, "tokens_seen": 30810112, "tokens_per_sec": 22770.100834719397, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4212239583333333, "G_std": 0.18158976236979166, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.758403301239014, "grad_norm": 0.7448235750198364, "tokens_seen": 30973952, "tokens_per_sec": 22770.30368730584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4118448893229167, "G_std": 0.18317159016927084, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.837531805038452, "grad_norm": 0.6990503668785095, "tokens_seen": 31137792, "tokens_per_sec": 22770.97302098166, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4119313557942708, "G_std": 0.1823883056640625, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.800737142562866, "grad_norm": 0.7051395773887634, "tokens_seen": 31301632, "tokens_per_sec": 22770.96217609418, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4187672932942708, "G_std": 0.1847686767578125, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.610753059387207, "grad_norm": 0.663740336894989, "tokens_seen": 31465472, "tokens_per_sec": 22771.0405867463, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4101664225260417, "G_std": 0.18182373046875, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.906745910644531, "grad_norm": 0.6784107089042664, "tokens_seen": 31629312, "tokens_per_sec": 22771.499685080817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4245402018229167, "G_std": 0.18248494466145834, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.839893341064453, "grad_norm": 0.6730288863182068, "tokens_seen": 31793152, "tokens_per_sec": 22771.626379621073, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4127451578776042, "G_std": 0.18221537272135416, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.8566882610321045, "grad_norm": 0.7028924822807312, "tokens_seen": 31956992, "tokens_per_sec": 22771.41057997222, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.41290283203125, "G_std": 0.18443806966145834, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.854246139526367, "grad_norm": 0.7081459760665894, "tokens_seen": 32120832, "tokens_per_sec": 22771.110145746265, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4188639322916667, "G_std": 0.18239847819010416, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.764474630355835, "grad_norm": 0.7532795071601868, "tokens_seen": 32284672, "tokens_per_sec": 22771.3000174723, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4054819742838542, "G_std": 0.18111673990885416, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.936530828475952, "grad_norm": 0.7288445234298706, "tokens_seen": 32448512, "tokens_per_sec": 22771.9415715439, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4201456705729167, "G_std": 0.1852264404296875, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.778748273849487, "grad_norm": 0.7154732346534729, "tokens_seen": 32612352, "tokens_per_sec": 22772.38245690733, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4018809000651042, "G_std": 0.182525634765625, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.728479862213135, "grad_norm": 0.646082878112793, "tokens_seen": 32776192, "tokens_per_sec": 22772.816201272977, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.394866943359375, "G_std": 0.17977396647135416, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.886761283874511, "val_ppl": 132.52367170866677, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.4130910237630208, "G_std": 0.1837921142578125, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.659307479858398, "grad_norm": 0.6852912306785583, "tokens_seen": 32940032, "tokens_per_sec": 22752.675958627635, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4092661539713542, "G_std": 0.18224589029947916, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.763942003250122, "grad_norm": 0.6595437526702881, "tokens_seen": 33103872, "tokens_per_sec": 22752.806409050867, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3988037109375, "G_std": 0.18439229329427084, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.764780521392822, "grad_norm": 0.7036289572715759, "tokens_seen": 33267712, "tokens_per_sec": 22752.594647597238, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4037933349609375, "G_std": 0.1796112060546875, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.790308475494385, "grad_norm": 0.8920735120773315, "tokens_seen": 33431552, "tokens_per_sec": 22752.40469457358, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4068044026692708, "G_std": 0.18048095703125, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.854727268218994, "grad_norm": 0.6858335733413696, "tokens_seen": 33595392, "tokens_per_sec": 22752.65869224578, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4125874837239583, "G_std": 0.18133544921875, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.808092355728149, "grad_norm": 0.7725369334220886, "tokens_seen": 33759232, "tokens_per_sec": 22752.708812119607, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4035085042317708, "G_std": 0.1816558837890625, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.801795482635498, "grad_norm": 0.7077882885932922, "tokens_seen": 33923072, "tokens_per_sec": 22753.20176201576, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.399688720703125, "G_std": 0.1822967529296875, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.892246246337891, "grad_norm": 0.773230254650116, "tokens_seen": 34086912, "tokens_per_sec": 22753.010374812806, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4095509847005208, "G_std": 0.18244425455729166, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.804261684417725, "grad_norm": 0.6875062584877014, "tokens_seen": 34250752, "tokens_per_sec": 22752.281828467978, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4121449788411458, "G_std": 0.18207804361979166, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.8073344230651855, "grad_norm": 0.7491021156311035, "tokens_seen": 34414592, "tokens_per_sec": 22752.74065900377, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4057973225911458, "G_std": 0.17966206868489584, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.763682842254639, "grad_norm": 0.6966115236282349, "tokens_seen": 34578432, "tokens_per_sec": 22753.141365960735, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3969523111979167, "G_std": 0.182403564453125, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.661432981491089, "grad_norm": 0.6551949381828308, "tokens_seen": 34742272, "tokens_per_sec": 22752.61793014839, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4042816162109375, "G_std": 0.18213399251302084, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.7580084800720215, "grad_norm": 0.7204416990280151, "tokens_seen": 34906112, "tokens_per_sec": 22752.413414301795, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4057566324869792, "G_std": 0.18234761555989584, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.876194953918457, "grad_norm": 0.7603632807731628, "tokens_seen": 35069952, "tokens_per_sec": 22752.355128465322, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4034322102864583, "G_std": 0.18231709798177084, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.890466690063477, "grad_norm": 0.742621123790741, "tokens_seen": 35233792, "tokens_per_sec": 22752.302033377597, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.395599365234375, "G_std": 0.18119303385416666, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.691739797592163, "grad_norm": 0.7646871209144592, "tokens_seen": 35397632, "tokens_per_sec": 22752.681925028202, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3969624837239583, "G_std": 0.17857869466145834, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.927332878112793, "grad_norm": 0.7348985075950623, "tokens_seen": 35561472, "tokens_per_sec": 22752.337091733792, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4102834065755208, "G_std": 0.18157958984375, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.7230072021484375, "grad_norm": 0.7797888517379761, "tokens_seen": 35725312, "tokens_per_sec": 22752.128142477584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4126536051432292, "G_std": 0.18208821614583334, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.442463397979736, "grad_norm": 0.7131375670433044, "tokens_seen": 35889152, "tokens_per_sec": 22752.083010362523, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3967692057291667, "G_std": 0.17830403645833334, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.910670280456543, "grad_norm": 0.7311794757843018, "tokens_seen": 36052992, "tokens_per_sec": 22751.746620664115, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4088999430338542, "G_std": 0.1798858642578125, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.747463703155518, "grad_norm": 0.7201254367828369, "tokens_seen": 36216832, "tokens_per_sec": 22751.92376189448, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39776611328125, "G_std": 0.17998758951822916, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.898735284805298, "grad_norm": 0.7557728290557861, "tokens_seen": 36380672, "tokens_per_sec": 22752.100592681345, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.407562255859375, "G_std": 0.181121826171875, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.620288848876953, "grad_norm": 0.7866494059562683, "tokens_seen": 36544512, "tokens_per_sec": 22752.351718307968, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4121042887369792, "G_std": 0.1801910400390625, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.763003826141357, "grad_norm": 0.7336161136627197, "tokens_seen": 36708352, "tokens_per_sec": 22751.688053819344, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39703369140625, "G_std": 0.180633544921875, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.840174198150635, "grad_norm": 0.7165125012397766, "tokens_seen": 36872192, "tokens_per_sec": 22746.492818516814, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.39208984375, "G_std": 0.1810328165690104, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.5172789096832275, "grad_norm": 0.73215651512146, "tokens_seen": 37036032, "tokens_per_sec": 22746.497151137155, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4025166829427083, "G_std": 0.18017069498697916, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.657405614852905, "grad_norm": 0.7066168785095215, "tokens_seen": 37199872, "tokens_per_sec": 22746.867502710025, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3954010009765625, "G_std": 0.179779052734375, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.641260623931885, "grad_norm": 0.7138396501541138, "tokens_seen": 37363712, "tokens_per_sec": 22746.402053625596, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4017283121744792, "G_std": 0.18014017740885416, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.710746765136719, "grad_norm": 0.7234963178634644, "tokens_seen": 37527552, "tokens_per_sec": 22746.573012447843, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3923848470052083, "G_std": 0.18026224772135416, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.605560779571533, "grad_norm": 0.7436040639877319, "tokens_seen": 37691392, "tokens_per_sec": 22747.12406487252, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.40411376953125, "G_std": 0.179443359375, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.68670654296875, "grad_norm": 0.7192152142524719, "tokens_seen": 37855232, "tokens_per_sec": 22747.883936490733, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3933054606119792, "G_std": 0.17911783854166666, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.739245653152466, "grad_norm": 0.7861876487731934, "tokens_seen": 38019072, "tokens_per_sec": 22748.199979838177, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4031219482421875, "G_std": 0.17862955729166666, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.663691520690918, "grad_norm": 0.7140848636627197, "tokens_seen": 38182912, "tokens_per_sec": 22748.664222262432, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3994242350260417, "G_std": 0.1807734171549479, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.618999481201172, "grad_norm": 0.7030860185623169, "tokens_seen": 38346752, "tokens_per_sec": 22749.298911943202, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4063822428385417, "G_std": 0.179931640625, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.674562454223633, "grad_norm": 0.7336957454681396, "tokens_seen": 38510592, "tokens_per_sec": 22749.425273956665, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3968556722005208, "G_std": 0.17931620279947916, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.67861533164978, "grad_norm": 0.7634555101394653, "tokens_seen": 38674432, "tokens_per_sec": 22749.726318000892, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3924306233723958, "G_std": 0.1794102986653646, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.728050470352173, "grad_norm": 0.7481067776679993, "tokens_seen": 38838272, "tokens_per_sec": 22749.547558511145, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3916269938151042, "G_std": 0.1801300048828125, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.769251108169556, "grad_norm": 0.716325044631958, "tokens_seen": 39002112, "tokens_per_sec": 22749.967562677004, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899739583333333, "G_std": 0.17783355712890625, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.654233932495117, "grad_norm": 0.7863517999649048, "tokens_seen": 39165952, "tokens_per_sec": 22750.380615032136, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4073486328125, "G_std": 0.1786371866861979, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.834278583526611, "grad_norm": 0.7276110053062439, "tokens_seen": 39329792, "tokens_per_sec": 22750.311312366877, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3903147379557292, "G_std": 0.1805445353190104, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.629570722579956, "grad_norm": 0.7183221578598022, "tokens_seen": 39493632, "tokens_per_sec": 22750.224191015608, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3924407958984375, "G_std": 0.17887115478515625, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.5485148429870605, "grad_norm": 0.7596896290779114, "tokens_seen": 39657472, "tokens_per_sec": 22750.041751196728, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3968607584635417, "G_std": 0.1783269246419271, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.64787483215332, "grad_norm": 0.7258752584457397, "tokens_seen": 39821312, "tokens_per_sec": 22749.631720329562, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3960215250651042, "G_std": 0.17843119303385416, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.6168293952941895, "grad_norm": 0.7154649496078491, "tokens_seen": 39985152, "tokens_per_sec": 22750.006289010104, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3918355305989583, "G_std": 0.1793390909830729, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.681709289550781, "grad_norm": 0.7370897531509399, "tokens_seen": 40148992, "tokens_per_sec": 22750.29108014028, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3927663167317708, "G_std": 0.1797358194986979, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.657932281494141, "grad_norm": 0.8675095438957214, "tokens_seen": 40312832, "tokens_per_sec": 22750.019785165612, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3901824951171875, "G_std": 0.17831929524739584, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.631618976593018, "grad_norm": 0.7392502427101135, "tokens_seen": 40476672, "tokens_per_sec": 22749.00371189781, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3929901123046875, "G_std": 0.1795374552408854, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.686983585357666, "grad_norm": 0.7092121243476868, "tokens_seen": 40640512, "tokens_per_sec": 22749.575441443936, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3910776774088542, "G_std": 0.17867024739583334, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.63603663444519, "grad_norm": 0.8096112012863159, "tokens_seen": 40804352, "tokens_per_sec": 22749.233881331817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3902587890625, "G_std": 0.17885335286458334, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.775631666183472, "grad_norm": 0.7308331727981567, "tokens_seen": 40968192, "tokens_per_sec": 22749.11369182417, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3961995442708333, "G_std": 0.1778386433919271, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.75543315410614, "val_ppl": 116.21398082678148, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3960113525390625, "G_std": 0.17876434326171875, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.738928318023682, "grad_norm": 0.7435871362686157, "tokens_seen": 41132032, "tokens_per_sec": 22733.414368957667, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38897705078125, "G_std": 0.17906697591145834, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.806763410568237, "grad_norm": 0.7709891200065613, "tokens_seen": 41295872, "tokens_per_sec": 22733.492764565424, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3918100992838542, "G_std": 0.1780853271484375, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.660881280899048, "grad_norm": 0.771450400352478, "tokens_seen": 41459712, "tokens_per_sec": 22733.081160960814, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3990071614583333, "G_std": 0.17801920572916666, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.72913932800293, "grad_norm": 0.7495477199554443, "tokens_seen": 41623552, "tokens_per_sec": 22732.341422631416, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3879038492838542, "G_std": 0.17771402994791666, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.687295913696289, "grad_norm": 0.7918254733085632, "tokens_seen": 41787392, "tokens_per_sec": 22731.10105114844, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3881429036458333, "G_std": 0.1780548095703125, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.721603631973267, "grad_norm": 0.7490725517272949, "tokens_seen": 41951232, "tokens_per_sec": 22731.263849498988, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3898213704427083, "G_std": 0.1745020548502604, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.647319078445435, "grad_norm": 0.7089464068412781, "tokens_seen": 42115072, "tokens_per_sec": 22731.85508625715, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.386627197265625, "G_std": 0.17857869466145834, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.733291864395142, "grad_norm": 0.786590039730072, "tokens_seen": 42278912, "tokens_per_sec": 22732.086098701082, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3922220865885417, "G_std": 0.1795806884765625, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.567891836166382, "grad_norm": 0.737129807472229, "tokens_seen": 42442752, "tokens_per_sec": 22731.574320941905, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3943532307942708, "G_std": 0.1779327392578125, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.52513313293457, "grad_norm": 0.7467988133430481, "tokens_seen": 42606592, "tokens_per_sec": 22731.859888404008, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3850148518880208, "G_std": 0.17774200439453125, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.69079327583313, "grad_norm": 0.7608184814453125, "tokens_seen": 42770432, "tokens_per_sec": 22731.420890575086, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3876546223958333, "G_std": 0.1797510782877604, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.739513397216797, "grad_norm": 0.7964015603065491, "tokens_seen": 42934272, "tokens_per_sec": 22731.543666626694, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.4030100504557292, "G_std": 0.1760075887044271, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.620012998580933, "grad_norm": 0.7179175615310669, "tokens_seen": 43098112, "tokens_per_sec": 22731.731391484744, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3902842203776042, "G_std": 0.17748514811197916, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.600877285003662, "grad_norm": 0.7889708876609802, "tokens_seen": 43261952, "tokens_per_sec": 22732.051116703027, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3784535725911458, "G_std": 0.17531585693359375, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.6135172843933105, "grad_norm": 0.7470240592956543, "tokens_seen": 43425792, "tokens_per_sec": 22732.64077350326, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3802541097005208, "G_std": 0.1773351033528646, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.628833770751953, "grad_norm": 0.7372109293937683, "tokens_seen": 43589632, "tokens_per_sec": 22733.108005468446, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3906351725260417, "G_std": 0.17817179361979166, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.726673126220703, "grad_norm": 0.7669997215270996, "tokens_seen": 43753472, "tokens_per_sec": 22733.487420897876, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845062255859375, "G_std": 0.1762237548828125, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.646277189254761, "grad_norm": 0.7127547860145569, "tokens_seen": 43917312, "tokens_per_sec": 22734.015454332857, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3881479899088542, "G_std": 0.1757634480794271, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.593831539154053, "grad_norm": 0.7425090670585632, "tokens_seen": 44081152, "tokens_per_sec": 22734.40426030455, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803660074869792, "G_std": 0.17441813151041666, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.7048420906066895, "grad_norm": 0.7668972611427307, "tokens_seen": 44244992, "tokens_per_sec": 22734.66638410075, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3941294352213542, "G_std": 0.1786829630533854, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.572599411010742, "grad_norm": 0.8638679385185242, "tokens_seen": 44408832, "tokens_per_sec": 22734.935603935624, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3951517740885417, "G_std": 0.1774012247721354, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.749000549316406, "grad_norm": 0.7692195773124695, "tokens_seen": 44572672, "tokens_per_sec": 22735.233449344018, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3949330647786458, "G_std": 0.1756871541341146, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.553843975067139, "grad_norm": 0.759100079536438, "tokens_seen": 44736512, "tokens_per_sec": 22735.15934071909, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3893839518229167, "G_std": 0.1746826171875, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.6783201694488525, "grad_norm": 0.780767560005188, "tokens_seen": 44900352, "tokens_per_sec": 22735.517064257547, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796895345052083, "G_std": 0.17711385091145834, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.721264362335205, "grad_norm": 0.7462127804756165, "tokens_seen": 45064192, "tokens_per_sec": 22735.77658274318, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3897196451822917, "G_std": 0.1765289306640625, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.812126874923706, "grad_norm": 0.7680205702781677, "tokens_seen": 45228032, "tokens_per_sec": 22735.720014834802, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3932545979817708, "G_std": 0.1759796142578125, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.714275360107422, "grad_norm": 0.7814955711364746, "tokens_seen": 45391872, "tokens_per_sec": 22736.15865370096, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3931986490885417, "G_std": 0.1760101318359375, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.522135496139526, "grad_norm": 0.7746780514717102, "tokens_seen": 45555712, "tokens_per_sec": 22736.075607613035, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3875223795572917, "G_std": 0.1759185791015625, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.556609630584717, "grad_norm": 0.7755431532859802, "tokens_seen": 45719552, "tokens_per_sec": 22736.33184308504, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3899637858072917, "G_std": 0.1750513712565104, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.67831826210022, "grad_norm": 0.7910717725753784, "tokens_seen": 45883392, "tokens_per_sec": 22736.97577053524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3900858561197917, "G_std": 0.17718505859375, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.403783798217773, "grad_norm": 0.7258089780807495, "tokens_seen": 46047232, "tokens_per_sec": 22736.78572344097, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845062255859375, "G_std": 0.1759033203125, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.680089712142944, "grad_norm": 0.7488067150115967, "tokens_seen": 46211072, "tokens_per_sec": 22736.76562861821, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.382659912109375, "G_std": 0.17409261067708334, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.702242374420166, "grad_norm": 0.7486933469772339, "tokens_seen": 46374912, "tokens_per_sec": 22736.91321078048, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3870951334635417, "G_std": 0.1762542724609375, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.516723394393921, "grad_norm": 0.7757247090339661, "tokens_seen": 46538752, "tokens_per_sec": 22736.936936534064, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3829294840494792, "G_std": 0.1758880615234375, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.483517169952393, "grad_norm": 0.7849562764167786, "tokens_seen": 46702592, "tokens_per_sec": 22737.36733853809, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3800862630208333, "G_std": 0.1737823486328125, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.614622354507446, "grad_norm": 0.7730791568756104, "tokens_seen": 46866432, "tokens_per_sec": 22737.642590941352, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3812611897786458, "G_std": 0.17514546712239584, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.6698291301727295, "grad_norm": 0.8340988159179688, "tokens_seen": 47030272, "tokens_per_sec": 22737.60240157696, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3901214599609375, "G_std": 0.17697906494140625, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.569226264953613, "grad_norm": 0.81275874376297, "tokens_seen": 47194112, "tokens_per_sec": 22738.055759004295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.38909912109375, "G_std": 0.17730712890625, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.53596305847168, "grad_norm": 0.8228566646575928, "tokens_seen": 47357952, "tokens_per_sec": 22737.91883210241, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3803660074869792, "G_std": 0.17578887939453125, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.492552757263184, "grad_norm": 0.7829546928405762, "tokens_seen": 47521792, "tokens_per_sec": 22738.087123105037, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3903656005859375, "G_std": 0.17606353759765625, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.534112453460693, "grad_norm": 0.8730286359786987, "tokens_seen": 47685632, "tokens_per_sec": 22738.644036131107, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3813120524088542, "G_std": 0.1745783487955729, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.7282915115356445, "grad_norm": 0.8012731671333313, "tokens_seen": 47849472, "tokens_per_sec": 22738.80554863572, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3820597330729167, "G_std": 0.177032470703125, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.5422892570495605, "grad_norm": 0.7490097284317017, "tokens_seen": 48013312, "tokens_per_sec": 22738.876338383205, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3785807291666667, "G_std": 0.1764678955078125, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.6535725593566895, "grad_norm": 0.7806025743484497, "tokens_seen": 48177152, "tokens_per_sec": 22739.10310974184, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3838399251302083, "G_std": 0.17498270670572916, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.480575084686279, "grad_norm": 0.8006040453910828, "tokens_seen": 48340992, "tokens_per_sec": 22739.029472717066, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787689208984375, "G_std": 0.17523956298828125, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.442547082901001, "grad_norm": 0.8143429160118103, "tokens_seen": 48504832, "tokens_per_sec": 22739.04015122943, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3802388509114583, "G_std": 0.17497762044270834, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.603480577468872, "grad_norm": 0.786353349685669, "tokens_seen": 48668672, "tokens_per_sec": 22738.79827117647, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3855489095052083, "G_std": 0.1755523681640625, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.531986474990845, "grad_norm": 0.8006551861763, "tokens_seen": 48832512, "tokens_per_sec": 22738.563418978072, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3875071207682292, "G_std": 0.1747767130533854, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.645469903945923, "grad_norm": 0.8303367495536804, "tokens_seen": 48996352, "tokens_per_sec": 22738.89657866909, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3857981363932292, "G_std": 0.1733423868815104, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.687795162200928, "grad_norm": 0.8224771022796631, "tokens_seen": 49160192, "tokens_per_sec": 22738.885018932564, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3807729085286458, "G_std": 0.17630513509114584, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.643598484992981, "val_ppl": 103.91762157179384, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3729909261067708, "G_std": 0.1752802530924479, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.670934200286865, "grad_norm": 0.8060156106948853, "tokens_seen": 49324032, "tokens_per_sec": 22725.716978537694, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3762715657552083, "G_std": 0.17623392740885416, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.474414348602295, "grad_norm": 0.821953296661377, "tokens_seen": 49487872, "tokens_per_sec": 22725.953416644457, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3778025309244792, "G_std": 0.1745885213216146, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.552260875701904, "grad_norm": 0.7931053042411804, "tokens_seen": 49651712, "tokens_per_sec": 22726.365291301816, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732706705729167, "G_std": 0.1744664510091146, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.731844663619995, "grad_norm": 0.7595917582511902, "tokens_seen": 49815552, "tokens_per_sec": 22726.248612324813, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853658040364583, "G_std": 0.176422119140625, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.390408277511597, "grad_norm": 0.7618564963340759, "tokens_seen": 49979392, "tokens_per_sec": 22726.281660041215, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3747660319010417, "G_std": 0.1727879842122396, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.6573100090026855, "grad_norm": 0.7635703086853027, "tokens_seen": 50143232, "tokens_per_sec": 22726.628919381612, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3853556315104167, "G_std": 0.17580668131510416, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.4794020652771, "grad_norm": 0.7603148818016052, "tokens_seen": 50307072, "tokens_per_sec": 22726.901533257966, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3821919759114583, "G_std": 0.176605224609375, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.564353942871094, "grad_norm": 0.7675836682319641, "tokens_seen": 50470912, "tokens_per_sec": 22726.518829060158, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3832753499348958, "G_std": 0.17425537109375, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.482041835784912, "grad_norm": 0.7605775594711304, "tokens_seen": 50634752, "tokens_per_sec": 22726.81464197767, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3716786702473958, "G_std": 0.1738764444986979, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.623391628265381, "grad_norm": 0.7899247407913208, "tokens_seen": 50798592, "tokens_per_sec": 22726.888003393196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3826548258463542, "G_std": 0.17510223388671875, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.665791273117065, "grad_norm": 0.7875866293907166, "tokens_seen": 50962432, "tokens_per_sec": 22726.894947209967, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3864237467447917, "G_std": 0.17534891764322916, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.6098692417144775, "grad_norm": 0.7680973410606384, "tokens_seen": 51126272, "tokens_per_sec": 22727.06953903642, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831227620442708, "G_std": 0.1755701700846354, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.493938446044922, "grad_norm": 0.7709775567054749, "tokens_seen": 51290112, "tokens_per_sec": 22726.845597184456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737589518229167, "G_std": 0.17523193359375, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.462294816970825, "grad_norm": 0.7624689340591431, "tokens_seen": 51453952, "tokens_per_sec": 22726.874826843145, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37420654296875, "G_std": 0.17508443196614584, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.59368371963501, "grad_norm": 0.8341550230979919, "tokens_seen": 51617792, "tokens_per_sec": 22726.45090395206, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3807118733723958, "G_std": 0.17586263020833334, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.343931674957275, "grad_norm": 0.7457745671272278, "tokens_seen": 51781632, "tokens_per_sec": 22726.23620799001, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3752187093098958, "G_std": 0.17543538411458334, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.598952770233154, "grad_norm": 0.7685834765434265, "tokens_seen": 51945472, "tokens_per_sec": 22726.439175080995, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3767191569010417, "G_std": 0.17450459798177084, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.461414098739624, "grad_norm": 0.7944334149360657, "tokens_seen": 52109312, "tokens_per_sec": 22726.453291882455, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3798878987630208, "G_std": 0.17438252766927084, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.611702919006348, "grad_norm": 0.7890737652778625, "tokens_seen": 52273152, "tokens_per_sec": 22726.316267176982, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.383209228515625, "G_std": 0.17388407389322916, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.569426536560059, "grad_norm": 0.8294888138771057, "tokens_seen": 52436992, "tokens_per_sec": 22726.251458663715, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3851064046223958, "G_std": 0.1731236775716146, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.613162517547607, "grad_norm": 0.8110024333000183, "tokens_seen": 52600832, "tokens_per_sec": 22726.14179586331, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3783009847005208, "G_std": 0.17447916666666666, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.645836353302002, "grad_norm": 0.8191481232643127, "tokens_seen": 52764672, "tokens_per_sec": 22725.989613859765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3756205240885417, "G_std": 0.1713129679361979, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.535562992095947, "grad_norm": 0.7982776761054993, "tokens_seen": 52928512, "tokens_per_sec": 22726.008706979577, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3831787109375, "G_std": 0.1747919718424479, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.7135233879089355, "grad_norm": 0.8384478092193604, "tokens_seen": 53092352, "tokens_per_sec": 22725.724077273717, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.386627197265625, "G_std": 0.1732966105143229, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.447329759597778, "grad_norm": 0.8104409575462341, "tokens_seen": 53256192, "tokens_per_sec": 22725.213250693392, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3734385172526042, "G_std": 0.17405192057291666, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.522411346435547, "grad_norm": 0.7600160241127014, "tokens_seen": 53420032, "tokens_per_sec": 22725.344407018376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3695271809895833, "G_std": 0.17415873209635416, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.591115951538086, "grad_norm": 0.818462610244751, "tokens_seen": 53583872, "tokens_per_sec": 22725.593470843552, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3845011393229167, "G_std": 0.17357889811197916, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.372602462768555, "grad_norm": 0.78221195936203, "tokens_seen": 53747712, "tokens_per_sec": 22724.948154416797, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796742757161458, "G_std": 0.17436726888020834, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.692523956298828, "grad_norm": 0.8405027985572815, "tokens_seen": 53911552, "tokens_per_sec": 22725.13082749411, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787384033203125, "G_std": 0.1735102335611979, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.496654987335205, "grad_norm": 0.811914324760437, "tokens_seen": 54075392, "tokens_per_sec": 22725.167802090196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.382049560546875, "G_std": 0.17255147298177084, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.758195638656616, "grad_norm": 0.9459537267684937, "tokens_seen": 54239232, "tokens_per_sec": 22725.523838593188, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834431966145833, "G_std": 0.17537943522135416, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.446686267852783, "grad_norm": 0.8434168696403503, "tokens_seen": 54403072, "tokens_per_sec": 22725.75990152765, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3771718343098958, "G_std": 0.172271728515625, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.448424577713013, "grad_norm": 0.7988737225532532, "tokens_seen": 54566912, "tokens_per_sec": 22725.952001180664, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725941975911458, "G_std": 0.17382049560546875, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.444029092788696, "grad_norm": 0.8236095309257507, "tokens_seen": 54730752, "tokens_per_sec": 22726.083233160196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3738759358723958, "G_std": 0.17391204833984375, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.59927773475647, "grad_norm": 0.8508237600326538, "tokens_seen": 54894592, "tokens_per_sec": 22726.221639857118, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763224283854167, "G_std": 0.17394002278645834, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.490990400314331, "grad_norm": 0.804397463798523, "tokens_seen": 55058432, "tokens_per_sec": 22725.96682423467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3796234130859375, "G_std": 0.17409769694010416, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.4922943115234375, "grad_norm": 0.7794414758682251, "tokens_seen": 55222272, "tokens_per_sec": 22726.27632855874, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810475667317708, "G_std": 0.17539469401041666, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.584060192108154, "grad_norm": 0.8114550113677979, "tokens_seen": 55386112, "tokens_per_sec": 22726.02759521102, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3834635416666667, "G_std": 0.17562357584635416, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.451319217681885, "grad_norm": 0.8022083640098572, "tokens_seen": 55549952, "tokens_per_sec": 22726.04049939, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3809407552083333, "G_std": 0.17341105143229166, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.379695177078247, "grad_norm": 0.7996254563331604, "tokens_seen": 55713792, "tokens_per_sec": 22726.172962537854, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373016357421875, "G_std": 0.1739959716796875, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.604206323623657, "grad_norm": 0.8266644477844238, "tokens_seen": 55877632, "tokens_per_sec": 22726.13004748063, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.377349853515625, "G_std": 0.17315165201822916, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.408324241638184, "grad_norm": 0.8098465204238892, "tokens_seen": 56041472, "tokens_per_sec": 22726.13093752899, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37164306640625, "G_std": 0.17333984375, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.622768402099609, "grad_norm": 0.8021777272224426, "tokens_seen": 56205312, "tokens_per_sec": 22726.186352917794, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3792724609375, "G_std": 0.17414093017578125, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.516326904296875, "grad_norm": 0.793551504611969, "tokens_seen": 56369152, "tokens_per_sec": 22725.94318405599, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3789469401041667, "G_std": 0.1736729939778646, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.455028533935547, "grad_norm": 0.8259894847869873, "tokens_seen": 56532992, "tokens_per_sec": 22725.879286239142, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3814748128255208, "G_std": 0.17359415690104166, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.48229455947876, "grad_norm": 0.8334675431251526, "tokens_seen": 56696832, "tokens_per_sec": 22726.049059805875, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3781534830729167, "G_std": 0.1723658243815104, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.547537565231323, "grad_norm": 0.8389289379119873, "tokens_seen": 56860672, "tokens_per_sec": 22726.10129139244, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3810068766276042, "G_std": 0.1746190388997396, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.523510932922363, "grad_norm": 0.854002833366394, "tokens_seen": 57024512, "tokens_per_sec": 22726.17170096379, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.376129150390625, "G_std": 0.1709162394205729, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.474287986755371, "grad_norm": 0.7768053412437439, "tokens_seen": 57188352, "tokens_per_sec": 22726.56048282916, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729960123697917, "G_std": 0.1727116902669271, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.336347579956055, "grad_norm": 0.8200768828392029, "tokens_seen": 57352192, "tokens_per_sec": 22725.992565851873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721466064453125, "G_std": 0.1728668212890625, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.612994289398193, "val_ppl": 100.78547916910708, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.381378173828125, "G_std": 0.17232513427734375, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.684235095977783, "grad_norm": 0.8575955629348755, "tokens_seen": 57516032, "tokens_per_sec": 22714.695231854013, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3775533040364583, "G_std": 0.17371368408203125, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.672255516052246, "grad_norm": 0.8618173003196716, "tokens_seen": 57679872, "tokens_per_sec": 22714.9621940183, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3763275146484375, "G_std": 0.1712214152018229, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.554540157318115, "grad_norm": 0.785793125629425, "tokens_seen": 57843712, "tokens_per_sec": 22715.127429171356, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3770802815755208, "G_std": 0.17278035481770834, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.517644166946411, "grad_norm": 0.8401362895965576, "tokens_seen": 58007552, "tokens_per_sec": 22714.93752682758, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3772481282552083, "G_std": 0.1725616455078125, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.534260272979736, "grad_norm": 0.8056399822235107, "tokens_seen": 58171392, "tokens_per_sec": 22715.16030577011, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.374725341796875, "G_std": 0.172454833984375, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.73876953125, "grad_norm": 0.8926053643226624, "tokens_seen": 58335232, "tokens_per_sec": 22715.192741317132, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3842417399088542, "G_std": 0.1732635498046875, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.442449569702148, "grad_norm": 0.8441016674041748, "tokens_seen": 58499072, "tokens_per_sec": 22714.82534009803, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3679555257161458, "G_std": 0.17314656575520834, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.480886697769165, "grad_norm": 0.8265759348869324, "tokens_seen": 58662912, "tokens_per_sec": 22714.716442009012, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.376617431640625, "G_std": 0.1729151407877604, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.531209945678711, "grad_norm": 0.8770467042922974, "tokens_seen": 58826752, "tokens_per_sec": 22714.815898398043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3787027994791667, "G_std": 0.17252095540364584, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.535391807556152, "grad_norm": 0.8576106429100037, "tokens_seen": 58990592, "tokens_per_sec": 22715.012760982707, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3795878092447917, "G_std": 0.17124430338541666, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.341658592224121, "grad_norm": 0.8214116096496582, "tokens_seen": 59154432, "tokens_per_sec": 22715.096650779687, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3737436930338542, "G_std": 0.1720097859700521, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.416807174682617, "grad_norm": 0.8132200837135315, "tokens_seen": 59318272, "tokens_per_sec": 22714.854872899592, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3740030924479167, "G_std": 0.17166646321614584, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.536416530609131, "grad_norm": 0.792972981929779, "tokens_seen": 59482112, "tokens_per_sec": 22715.084951691984, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680521647135417, "G_std": 0.17272186279296875, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.425282955169678, "grad_norm": 0.8290470242500305, "tokens_seen": 59645952, "tokens_per_sec": 22715.1285589941, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690032958984375, "G_std": 0.1721979777018229, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.5671916007995605, "grad_norm": 0.8048394918441772, "tokens_seen": 59809792, "tokens_per_sec": 22715.345063118493, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3738911946614583, "G_std": 0.17230987548828125, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.402732849121094, "grad_norm": 0.7849382758140564, "tokens_seen": 59973632, "tokens_per_sec": 22715.737483649318, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3745574951171875, "G_std": 0.17093404134114584, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.489201784133911, "grad_norm": 0.8491334319114685, "tokens_seen": 60137472, "tokens_per_sec": 22715.832250812007, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3709869384765625, "G_std": 0.17121378580729166, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.555924892425537, "grad_norm": 0.8125899434089661, "tokens_seen": 60301312, "tokens_per_sec": 22716.150126965462, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3720804850260417, "G_std": 0.17078399658203125, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.535666465759277, "grad_norm": 0.8583818674087524, "tokens_seen": 60465152, "tokens_per_sec": 22716.24361986821, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3714650472005208, "G_std": 0.1726201375325521, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.396115064620972, "grad_norm": 0.8032708764076233, "tokens_seen": 60628992, "tokens_per_sec": 22716.315950452066, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3733469645182292, "G_std": 0.1710535685221354, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.474930763244629, "grad_norm": 0.8569002151489258, "tokens_seen": 60792832, "tokens_per_sec": 22716.223236948903, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.373870849609375, "G_std": 0.1733856201171875, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.392188310623169, "grad_norm": 0.8181526064872742, "tokens_seen": 60956672, "tokens_per_sec": 22716.472602241236, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3675486246744792, "G_std": 0.1718928019205729, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.558997869491577, "grad_norm": 0.8283645510673523, "tokens_seen": 61120512, "tokens_per_sec": 22716.420525462698, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3675181070963542, "G_std": 0.17207845052083334, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.406140327453613, "grad_norm": 0.8603248000144958, "tokens_seen": 61284352, "tokens_per_sec": 22716.670088054987, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3682454427083333, "G_std": 0.1717809041341146, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.453248977661133, "grad_norm": 0.858015239238739, "tokens_seen": 61448192, "tokens_per_sec": 22717.045664445566, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680165608723958, "G_std": 0.17286173502604166, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.502720832824707, "grad_norm": 0.8345274925231934, "tokens_seen": 61612032, "tokens_per_sec": 22717.12813502457, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729960123697917, "G_std": 0.1719233194986979, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.386187791824341, "grad_norm": 0.8342280983924866, "tokens_seen": 61775872, "tokens_per_sec": 22717.29891077353, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3739725748697917, "G_std": 0.1728185017903646, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.332123279571533, "grad_norm": 0.908276379108429, "tokens_seen": 61939712, "tokens_per_sec": 22717.415326764396, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3660481770833333, "G_std": 0.1720123291015625, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.59987211227417, "grad_norm": 0.8898764848709106, "tokens_seen": 62103552, "tokens_per_sec": 22716.96116134965, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3724873860677083, "G_std": 0.17119598388671875, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.338215112686157, "grad_norm": 0.8225619792938232, "tokens_seen": 62267392, "tokens_per_sec": 22716.150380683266, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3663787841796875, "G_std": 0.17115275065104166, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.420476675033569, "grad_norm": 0.8289906978607178, "tokens_seen": 62431232, "tokens_per_sec": 22716.226197984182, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3703053792317708, "G_std": 0.17223103841145834, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.412550687789917, "grad_norm": 0.8384657502174377, "tokens_seen": 62595072, "tokens_per_sec": 22715.85103668026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3750864664713542, "G_std": 0.17171223958333334, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.434375047683716, "grad_norm": 0.8314024806022644, "tokens_seen": 62758912, "tokens_per_sec": 22716.128826313125, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707784016927083, "G_std": 0.1711400349934896, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.616982698440552, "grad_norm": 0.9734861254692078, "tokens_seen": 62922752, "tokens_per_sec": 22716.080310908073, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3702341715494792, "G_std": 0.17287190755208334, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.56450891494751, "grad_norm": 0.87856525182724, "tokens_seen": 63086592, "tokens_per_sec": 22716.070403315112, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.368896484375, "G_std": 0.17093404134114584, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.559325218200684, "grad_norm": 0.8673940896987915, "tokens_seen": 63250432, "tokens_per_sec": 22715.857873010285, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3693084716796875, "G_std": 0.17136128743489584, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.474352836608887, "grad_norm": 0.8765813112258911, "tokens_seen": 63414272, "tokens_per_sec": 22715.615220488526, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3727823893229167, "G_std": 0.17212931315104166, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.476953029632568, "grad_norm": 0.8318116068840027, "tokens_seen": 63578112, "tokens_per_sec": 22715.86743530604, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690897623697917, "G_std": 0.17276509602864584, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.557422161102295, "grad_norm": 0.8660502433776855, "tokens_seen": 63741952, "tokens_per_sec": 22715.8532108547, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710276285807292, "G_std": 0.1727447509765625, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.473018407821655, "grad_norm": 0.864423930644989, "tokens_seen": 63905792, "tokens_per_sec": 22715.89221970089, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3678639729817708, "G_std": 0.172088623046875, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.339956283569336, "grad_norm": 0.8356070518493652, "tokens_seen": 64069632, "tokens_per_sec": 22716.121300631094, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3628743489583333, "G_std": 0.17189788818359375, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.3662590980529785, "grad_norm": 0.8557109832763672, "tokens_seen": 64233472, "tokens_per_sec": 22716.29294559903, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3733774820963542, "G_std": 0.1728693644205729, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.387852191925049, "grad_norm": 0.876551628112793, "tokens_seen": 64397312, "tokens_per_sec": 22716.32806147513, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.372894287109375, "G_std": 0.17192586263020834, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.3819804191589355, "grad_norm": 0.8441809415817261, "tokens_seen": 64561152, "tokens_per_sec": 22716.396094950505, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690541585286458, "G_std": 0.17107137044270834, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.358683347702026, "grad_norm": 0.850175142288208, "tokens_seen": 64724992, "tokens_per_sec": 22716.374503379127, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3694610595703125, "G_std": 0.1705780029296875, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.500793695449829, "grad_norm": 0.9053347110748291, "tokens_seen": 64888832, "tokens_per_sec": 22716.5081748465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.366485595703125, "G_std": 0.17169189453125, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.496995210647583, "grad_norm": 0.8265226483345032, "tokens_seen": 65052672, "tokens_per_sec": 22716.797634419865, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3647308349609375, "G_std": 0.17140960693359375, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.490614414215088, "grad_norm": 0.8363847136497498, "tokens_seen": 65216512, "tokens_per_sec": 22716.980995796133, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662261962890625, "G_std": 0.17243448893229166, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.4932050704956055, "grad_norm": 0.8505926728248596, "tokens_seen": 65380352, "tokens_per_sec": 22716.836546995113, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3687845865885417, "G_std": 0.17029317220052084, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.54436182975769, "grad_norm": 0.9302070140838623, "tokens_seen": 65544192, "tokens_per_sec": 22716.935574313113, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3592071533203125, "G_std": 0.1699244181315104, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.540160131454468, "val_ppl": 93.7058041648451, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3690948486328125, "G_std": 0.17206827799479166, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.5220677852630615, "grad_norm": 0.8624861240386963, "tokens_seen": 65708032, "tokens_per_sec": 22707.344072324817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657633463541667, "G_std": 0.1710052490234375, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.407389163970947, "grad_norm": 0.8667652010917664, "tokens_seen": 65871872, "tokens_per_sec": 22707.345599480155, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673909505208333, "G_std": 0.1697514851888021, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.443568229675293, "grad_norm": 0.8217335939407349, "tokens_seen": 66035712, "tokens_per_sec": 22707.121186099303, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3671213785807292, "G_std": 0.1719690958658854, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.53204607963562, "grad_norm": 0.8648771643638611, "tokens_seen": 66199552, "tokens_per_sec": 22706.862267945235, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36712646484375, "G_std": 0.170745849609375, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.516592264175415, "grad_norm": 0.868898332118988, "tokens_seen": 66363392, "tokens_per_sec": 22706.67826219186, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3729502360026042, "G_std": 0.17258199055989584, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.650005340576172, "grad_norm": 0.874454140663147, "tokens_seen": 66527232, "tokens_per_sec": 22706.666145775816, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721415201822917, "G_std": 0.1713841756184896, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.314448118209839, "grad_norm": 0.8308342099189758, "tokens_seen": 66691072, "tokens_per_sec": 22706.933696892655, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609517415364583, "G_std": 0.1709747314453125, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.390969753265381, "grad_norm": 0.8478720188140869, "tokens_seen": 66854912, "tokens_per_sec": 22706.87795561187, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3732961018880208, "G_std": 0.1695709228515625, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.497514486312866, "grad_norm": 0.8953856229782104, "tokens_seen": 67018752, "tokens_per_sec": 22706.932775110563, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3723398844401042, "G_std": 0.17122395833333334, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.352961540222168, "grad_norm": 0.9004115462303162, "tokens_seen": 67182592, "tokens_per_sec": 22707.181375223376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676300048828125, "G_std": 0.171051025390625, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.434463977813721, "grad_norm": 0.8528105020523071, "tokens_seen": 67346432, "tokens_per_sec": 22706.950615798753, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3614603678385417, "G_std": 0.17075093587239584, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.361809730529785, "grad_norm": 0.8787255883216858, "tokens_seen": 67510272, "tokens_per_sec": 22707.03823216141, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3629506429036458, "G_std": 0.16907501220703125, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.337425708770752, "grad_norm": 0.8883303999900818, "tokens_seen": 67674112, "tokens_per_sec": 22706.945733713175, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649495442708333, "G_std": 0.16899617513020834, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.3172056674957275, "grad_norm": 0.8177887201309204, "tokens_seen": 67837952, "tokens_per_sec": 22706.684465077393, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662567138671875, "G_std": 0.1708958943684896, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.523303747177124, "grad_norm": 0.8636707067489624, "tokens_seen": 68001792, "tokens_per_sec": 22706.62930749699, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710683186848958, "G_std": 0.16970062255859375, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.365171432495117, "grad_norm": 0.8540323376655579, "tokens_seen": 68165632, "tokens_per_sec": 22706.1156275869, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3672587076822917, "G_std": 0.1704279581705729, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.436448335647583, "grad_norm": 0.8546950221061707, "tokens_seen": 68329472, "tokens_per_sec": 22706.16842958304, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680318196614583, "G_std": 0.17063395182291666, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.531183481216431, "grad_norm": 0.9083428978919983, "tokens_seen": 68493312, "tokens_per_sec": 22706.426151728607, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3692118326822917, "G_std": 0.17057291666666666, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.368419885635376, "grad_norm": 0.8718618154525757, "tokens_seen": 68657152, "tokens_per_sec": 22706.375072771465, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664957682291667, "G_std": 0.1699676513671875, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.455631732940674, "grad_norm": 0.863776445388794, "tokens_seen": 68820992, "tokens_per_sec": 22706.36016171736, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3697713216145833, "G_std": 0.1719945271809896, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.332261800765991, "grad_norm": 0.8622826337814331, "tokens_seen": 68984832, "tokens_per_sec": 22706.331265968744, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3669230143229167, "G_std": 0.1693878173828125, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.294569969177246, "grad_norm": 0.8205406665802002, "tokens_seen": 69148672, "tokens_per_sec": 22706.415241748833, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3648529052734375, "G_std": 0.16905721028645834, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.353424072265625, "grad_norm": 0.9547763466835022, "tokens_seen": 69312512, "tokens_per_sec": 22706.738388508045, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707427978515625, "G_std": 0.1701329549153646, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.363018751144409, "grad_norm": 0.8441389203071594, "tokens_seen": 69476352, "tokens_per_sec": 22706.773928588773, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358734130859375, "G_std": 0.1699956258138021, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.380381107330322, "grad_norm": 0.8755869269371033, "tokens_seen": 69640192, "tokens_per_sec": 22706.781069079785, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653361002604167, "G_std": 0.1683476765950521, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.4826178550720215, "grad_norm": 0.8823776245117188, "tokens_seen": 69804032, "tokens_per_sec": 22706.842550280246, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3706715901692708, "G_std": 0.16916656494140625, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.557932615280151, "grad_norm": 0.882984459400177, "tokens_seen": 69967872, "tokens_per_sec": 22707.01637491867, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.367431640625, "G_std": 0.16980743408203125, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.348479747772217, "grad_norm": 0.8482206463813782, "tokens_seen": 70131712, "tokens_per_sec": 22707.033330359456, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.360992431640625, "G_std": 0.1699066162109375, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.470395088195801, "grad_norm": 0.885158360004425, "tokens_seen": 70295552, "tokens_per_sec": 22707.110940123286, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3608245849609375, "G_std": 0.17018636067708334, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.282986879348755, "grad_norm": 0.9063722491264343, "tokens_seen": 70459392, "tokens_per_sec": 22707.34135676272, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662923177083333, "G_std": 0.16925557454427084, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.419222354888916, "grad_norm": 0.884830892086029, "tokens_seen": 70623232, "tokens_per_sec": 22707.18149142741, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.364959716796875, "G_std": 0.16973368326822916, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.482027292251587, "grad_norm": 0.9024568200111389, "tokens_seen": 70787072, "tokens_per_sec": 22707.52636950951, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656565348307292, "G_std": 0.1701990763346354, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.278512001037598, "grad_norm": 0.9519372582435608, "tokens_seen": 70950912, "tokens_per_sec": 22707.766272727316, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3629252115885417, "G_std": 0.1697845458984375, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.385930061340332, "grad_norm": 0.842400312423706, "tokens_seen": 71114752, "tokens_per_sec": 22707.745442524472, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3680369059244792, "G_std": 0.16992696126302084, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.262077331542969, "grad_norm": 0.9007447361946106, "tokens_seen": 71278592, "tokens_per_sec": 22707.915419687524, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3664499918619792, "G_std": 0.1695098876953125, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.4997334480285645, "grad_norm": 0.9040096998214722, "tokens_seen": 71442432, "tokens_per_sec": 22707.6852536453, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3633677164713542, "G_std": 0.17090861002604166, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.4034199714660645, "grad_norm": 0.8964843153953552, "tokens_seen": 71606272, "tokens_per_sec": 22707.942080871715, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3659820556640625, "G_std": 0.17083740234375, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.3298118114471436, "grad_norm": 0.8758347630500793, "tokens_seen": 71770112, "tokens_per_sec": 22708.112685134165, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653615315755208, "G_std": 0.16915384928385416, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.39911413192749, "grad_norm": 0.8950247168540955, "tokens_seen": 71933952, "tokens_per_sec": 22707.96075537376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725077311197917, "G_std": 0.1691462198893229, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.334354400634766, "grad_norm": 0.9051274061203003, "tokens_seen": 72097792, "tokens_per_sec": 22708.23630555495, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3666534423828125, "G_std": 0.1691741943359375, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.274702548980713, "grad_norm": 0.8848286867141724, "tokens_seen": 72261632, "tokens_per_sec": 22708.535331007544, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3689931233723958, "G_std": 0.16986846923828125, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.492239713668823, "grad_norm": 0.9411482214927673, "tokens_seen": 72425472, "tokens_per_sec": 22708.720888065815, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676096598307292, "G_std": 0.16924031575520834, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.351914644241333, "grad_norm": 0.8898032307624817, "tokens_seen": 72589312, "tokens_per_sec": 22709.009272678526, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3635152180989583, "G_std": 0.1701838175455729, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.369543075561523, "grad_norm": 0.8429890871047974, "tokens_seen": 72753152, "tokens_per_sec": 22709.24560053224, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646392822265625, "G_std": 0.16912078857421875, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.178383111953735, "grad_norm": 0.9248517155647278, "tokens_seen": 72916992, "tokens_per_sec": 22709.213979139084, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657633463541667, "G_std": 0.16967264811197916, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.376397132873535, "grad_norm": 0.8934305310249329, "tokens_seen": 73080832, "tokens_per_sec": 22709.362247894165, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3710072835286458, "G_std": 0.1693700154622396, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.386098384857178, "grad_norm": 0.8628537058830261, "tokens_seen": 73244672, "tokens_per_sec": 22709.61489860195, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3586985270182292, "G_std": 0.16779327392578125, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.3783118724823, "grad_norm": 0.8774500489234924, "tokens_seen": 73408512, "tokens_per_sec": 22709.508013000475, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609975179036458, "G_std": 0.1674474080403646, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.465879440307617, "grad_norm": 0.8895463943481445, "tokens_seen": 73572352, "tokens_per_sec": 22709.8592372026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3641357421875, "G_std": 0.1688232421875, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.3748321533203125, "grad_norm": 0.8690779805183411, "tokens_seen": 73736192, "tokens_per_sec": 22709.976049865734, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3645121256510417, "G_std": 0.16961669921875, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.449000692367553, "val_ppl": 85.54141908348532, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3610076904296875, "G_std": 0.1704279581705729, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.469362735748291, "grad_norm": 0.9332758188247681, "tokens_seen": 73900032, "tokens_per_sec": 22700.872618034373, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3705190022786458, "G_std": 0.1691131591796875, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.39422869682312, "grad_norm": 0.9016246199607849, "tokens_seen": 74063872, "tokens_per_sec": 22700.599180021003, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3659922281901042, "G_std": 0.17022196451822916, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.135119438171387, "grad_norm": 0.8796554803848267, "tokens_seen": 74227712, "tokens_per_sec": 22700.763280675008, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676961263020833, "G_std": 0.1696192423502604, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.489082336425781, "grad_norm": 0.9072863459587097, "tokens_seen": 74391552, "tokens_per_sec": 22700.741429343463, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3704884847005208, "G_std": 0.168975830078125, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.418206214904785, "grad_norm": 0.8728377223014832, "tokens_seen": 74555392, "tokens_per_sec": 22700.728551526594, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3602396647135417, "G_std": 0.1687774658203125, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.296327352523804, "grad_norm": 0.9397804141044617, "tokens_seen": 74719232, "tokens_per_sec": 22700.859158713283, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657582600911458, "G_std": 0.16927591959635416, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.2114667892456055, "grad_norm": 0.8437927961349487, "tokens_seen": 74883072, "tokens_per_sec": 22700.993652185232, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3647308349609375, "G_std": 0.16831207275390625, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.4602556228637695, "grad_norm": 0.9223365187644958, "tokens_seen": 75046912, "tokens_per_sec": 22701.027930175595, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3637034098307292, "G_std": 0.16930389404296875, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.4517858028411865, "grad_norm": 0.8719024658203125, "tokens_seen": 75210752, "tokens_per_sec": 22700.994650834873, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3634847005208333, "G_std": 0.16927337646484375, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.249480247497559, "grad_norm": 0.9000194668769836, "tokens_seen": 75374592, "tokens_per_sec": 22700.937031161295, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3669077555338542, "G_std": 0.16852315266927084, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.439760208129883, "grad_norm": 0.8994457721710205, "tokens_seen": 75538432, "tokens_per_sec": 22701.03936364376, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649749755859375, "G_std": 0.1682891845703125, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.481149435043335, "grad_norm": 0.9175962805747986, "tokens_seen": 75702272, "tokens_per_sec": 22701.05493519051, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35992431640625, "G_std": 0.16929117838541666, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.354656219482422, "grad_norm": 0.9044580459594727, "tokens_seen": 75866112, "tokens_per_sec": 22700.860040558175, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3672078450520833, "G_std": 0.16875457763671875, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.35158109664917, "grad_norm": 0.9783331155776978, "tokens_seen": 76029952, "tokens_per_sec": 22700.98664045389, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3675435384114583, "G_std": 0.17070770263671875, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.4972333908081055, "grad_norm": 0.9755143523216248, "tokens_seen": 76193792, "tokens_per_sec": 22700.85296265242, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3683624267578125, "G_std": 0.16766866048177084, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.287558555603027, "grad_norm": 0.9487568140029907, "tokens_seen": 76357632, "tokens_per_sec": 22700.823448951687, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3663889567057292, "G_std": 0.16851806640625, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.486910820007324, "grad_norm": 0.924112856388092, "tokens_seen": 76521472, "tokens_per_sec": 22701.07332652459, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3638254801432292, "G_std": 0.16822052001953125, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.299346446990967, "grad_norm": 0.9201599955558777, "tokens_seen": 76685312, "tokens_per_sec": 22701.18079383489, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3569997151692708, "G_std": 0.16904195149739584, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.437903881072998, "grad_norm": 0.8665485978126526, "tokens_seen": 76849152, "tokens_per_sec": 22701.37242066135, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606313069661458, "G_std": 0.16953531901041666, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.20976448059082, "grad_norm": 0.916463315486908, "tokens_seen": 77012992, "tokens_per_sec": 22701.498016073045, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646748860677083, "G_std": 0.1688664754231771, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.253358364105225, "grad_norm": 0.9004866480827332, "tokens_seen": 77176832, "tokens_per_sec": 22701.482577153394, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.360198974609375, "G_std": 0.1693293253580729, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.2975757122039795, "grad_norm": 1.0166352987289429, "tokens_seen": 77340672, "tokens_per_sec": 22701.549321921182, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594716389973958, "G_std": 0.1681950887044271, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.376450300216675, "grad_norm": 0.8968499302864075, "tokens_seen": 77504512, "tokens_per_sec": 22700.891406786664, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3649495442708333, "G_std": 0.16872660319010416, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.378598213195801, "grad_norm": 0.920678436756134, "tokens_seen": 77668352, "tokens_per_sec": 22699.126367158562, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3634389241536458, "G_std": 0.169158935546875, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.530623912811279, "grad_norm": 1.0354002714157104, "tokens_seen": 77832192, "tokens_per_sec": 22699.2272711301, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36468505859375, "G_std": 0.168792724609375, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.214595079421997, "grad_norm": 0.9497108459472656, "tokens_seen": 77996032, "tokens_per_sec": 22699.45559464887, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3654683430989583, "G_std": 0.1688359578450521, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.373206853866577, "grad_norm": 0.8633197546005249, "tokens_seen": 78159872, "tokens_per_sec": 22699.374976494855, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3653411865234375, "G_std": 0.16820526123046875, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.548502206802368, "grad_norm": 0.9303144812583923, "tokens_seen": 78323712, "tokens_per_sec": 22699.400802542877, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3615163167317708, "G_std": 0.1683476765950521, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.280268430709839, "grad_norm": 0.9374925494194031, "tokens_seen": 78487552, "tokens_per_sec": 22699.727524775666, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3634389241536458, "G_std": 0.168853759765625, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.417036056518555, "grad_norm": 0.8945485353469849, "tokens_seen": 78651392, "tokens_per_sec": 22699.59208585979, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656056722005208, "G_std": 0.1679509480794271, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.352720022201538, "grad_norm": 0.9848591089248657, "tokens_seen": 78815232, "tokens_per_sec": 22699.753872413567, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3614501953125, "G_std": 0.16742960611979166, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.52186918258667, "grad_norm": 0.9694873094558716, "tokens_seen": 78979072, "tokens_per_sec": 22699.878415054114, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3778279622395833, "G_std": 0.16865030924479166, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.324957847595215, "grad_norm": 0.8978450894355774, "tokens_seen": 79142912, "tokens_per_sec": 22699.854879736748, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3622792561848958, "G_std": 0.16784159342447916, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.524587869644165, "grad_norm": 0.9180188775062561, "tokens_seen": 79306752, "tokens_per_sec": 22700.2062737565, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36639404296875, "G_std": 0.1705907185872396, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.276386976242065, "grad_norm": 0.899053156375885, "tokens_seen": 79470592, "tokens_per_sec": 22700.17338721479, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3625132242838542, "G_std": 0.16777547200520834, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.317055702209473, "grad_norm": 0.9322189092636108, "tokens_seen": 79634432, "tokens_per_sec": 22700.222639860614, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3661651611328125, "G_std": 0.16744486490885416, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.35967230796814, "grad_norm": 0.8858231902122498, "tokens_seen": 79798272, "tokens_per_sec": 22700.033342429888, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650665283203125, "G_std": 0.1698481241861979, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.397899389266968, "grad_norm": 0.9111723899841309, "tokens_seen": 79962112, "tokens_per_sec": 22700.20891277429, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3622894287109375, "G_std": 0.16961669921875, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.111938714981079, "grad_norm": 0.9006540179252625, "tokens_seen": 80125952, "tokens_per_sec": 22700.396986586587, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589731852213542, "G_std": 0.16796366373697916, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.2747743129730225, "grad_norm": 0.941178560256958, "tokens_seen": 80289792, "tokens_per_sec": 22700.58321100057, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613942464192708, "G_std": 0.16866302490234375, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.172783851623535, "grad_norm": 0.9419688582420349, "tokens_seen": 80453632, "tokens_per_sec": 22700.487290195953, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.355865478515625, "G_std": 0.1663996378580729, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.495100498199463, "grad_norm": 0.8952797651290894, "tokens_seen": 80617472, "tokens_per_sec": 22700.50718190281, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3721364339192708, "G_std": 0.16853841145833334, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.306765794754028, "grad_norm": 0.9220017790794373, "tokens_seen": 80781312, "tokens_per_sec": 22700.346165298426, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3672231038411458, "G_std": 0.1679407755533854, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.321565628051758, "grad_norm": 0.917202889919281, "tokens_seen": 80945152, "tokens_per_sec": 22700.487135684245, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610585530598958, "G_std": 0.1687291463216146, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.267274618148804, "grad_norm": 0.9565364122390747, "tokens_seen": 81108992, "tokens_per_sec": 22700.58399781296, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3667093912760417, "G_std": 0.17020670572916666, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.307460308074951, "grad_norm": 0.9272598624229431, "tokens_seen": 81272832, "tokens_per_sec": 22700.17675991584, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3636372884114583, "G_std": 0.16884613037109375, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.505145311355591, "grad_norm": 0.9161402583122253, "tokens_seen": 81436672, "tokens_per_sec": 22700.128055145375, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.362945556640625, "G_std": 0.1692937215169271, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.402723789215088, "grad_norm": 0.9256883859634399, "tokens_seen": 81600512, "tokens_per_sec": 22700.156832033488, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3688201904296875, "G_std": 0.1675237019856771, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.359560489654541, "grad_norm": 0.8873986005783081, "tokens_seen": 81764352, "tokens_per_sec": 22700.456813091678, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.360443115234375, "G_std": 0.1684748331705729, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.339418172836304, "grad_norm": 0.8964932560920715, "tokens_seen": 81928192, "tokens_per_sec": 22700.611516119323, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3642171223958333, "G_std": 0.16828409830729166, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.4181747198104855, "val_ppl": 82.94474969445154, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3657684326171875, "G_std": 0.16766866048177084, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.495157241821289, "grad_norm": 0.9281230568885803, "tokens_seen": 82092032, "tokens_per_sec": 22692.58821914882, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3707834879557292, "G_std": 0.16779581705729166, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.311462640762329, "grad_norm": 0.9134242534637451, "tokens_seen": 82255872, "tokens_per_sec": 22692.71531542058, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3625895182291667, "G_std": 0.1677093505859375, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.305040597915649, "grad_norm": 0.9151542782783508, "tokens_seen": 82419712, "tokens_per_sec": 22692.98427091108, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580474853515625, "G_std": 0.1688359578450521, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.342167854309082, "grad_norm": 0.9559622406959534, "tokens_seen": 82583552, "tokens_per_sec": 22692.50803682268, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3640238444010417, "G_std": 0.1677881876627604, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.371118783950806, "grad_norm": 0.9117138385772705, "tokens_seen": 82747392, "tokens_per_sec": 22692.53719911952, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3601531982421875, "G_std": 0.1684544881184896, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.366031885147095, "grad_norm": 0.9182577729225159, "tokens_seen": 82911232, "tokens_per_sec": 22692.37877068007, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3637237548828125, "G_std": 0.1674067179361979, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.149143218994141, "grad_norm": 0.9170728325843811, "tokens_seen": 83075072, "tokens_per_sec": 22692.18386266363, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3648325602213542, "G_std": 0.1671574910481771, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.277498245239258, "grad_norm": 0.9699214696884155, "tokens_seen": 83238912, "tokens_per_sec": 22692.150645457135, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3674112955729167, "G_std": 0.1680933634440104, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.449490308761597, "grad_norm": 0.9067417979240417, "tokens_seen": 83402752, "tokens_per_sec": 22692.01127164161, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3628743489583333, "G_std": 0.16902669270833334, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.320975303649902, "grad_norm": 0.9260513782501221, "tokens_seen": 83566592, "tokens_per_sec": 22691.912621283216, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.37005615234375, "G_std": 0.16799418131510416, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.399608135223389, "grad_norm": 1.0271047353744507, "tokens_seen": 83730432, "tokens_per_sec": 22692.0575279057, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358154296875, "G_std": 0.1670049031575521, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.22560453414917, "grad_norm": 0.9761094450950623, "tokens_seen": 83894272, "tokens_per_sec": 22691.843112015704, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36346435546875, "G_std": 0.16832987467447916, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.397109031677246, "grad_norm": 0.9753445982933044, "tokens_seen": 84058112, "tokens_per_sec": 22692.09775248878, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657277425130208, "G_std": 0.16928609212239584, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.399495363235474, "grad_norm": 0.9345627427101135, "tokens_seen": 84221952, "tokens_per_sec": 22692.11051805204, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3659871419270833, "G_std": 0.16844940185546875, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.277798652648926, "grad_norm": 0.9175277948379517, "tokens_seen": 84385792, "tokens_per_sec": 22692.091800410224, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3607126871744792, "G_std": 0.16840108235677084, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.44081974029541, "grad_norm": 0.9274055361747742, "tokens_seen": 84549632, "tokens_per_sec": 22692.147406784068, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3635711669921875, "G_std": 0.16780853271484375, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.335970878601074, "grad_norm": 0.9696254730224609, "tokens_seen": 84713472, "tokens_per_sec": 22691.96441067017, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3646138509114583, "G_std": 0.1679407755533854, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.424907445907593, "grad_norm": 0.9074596762657166, "tokens_seen": 84877312, "tokens_per_sec": 22692.029983120046, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.362640380859375, "G_std": 0.1682866414388021, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.406120777130127, "grad_norm": 0.9722196459770203, "tokens_seen": 85041152, "tokens_per_sec": 22692.210206391206, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3676350911458333, "G_std": 0.1666590372721354, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.439797878265381, "grad_norm": 0.9614447951316833, "tokens_seen": 85204992, "tokens_per_sec": 22692.002616469992, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3673960367838542, "G_std": 0.16915384928385416, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.409392356872559, "grad_norm": 0.9420799016952515, "tokens_seen": 85368832, "tokens_per_sec": 22692.122807546017, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3663533528645833, "G_std": 0.16852823893229166, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.271472454071045, "grad_norm": 0.912104070186615, "tokens_seen": 85532672, "tokens_per_sec": 22692.362628591305, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3657073974609375, "G_std": 0.16764068603515625, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.403433322906494, "grad_norm": 1.0043857097625732, "tokens_seen": 85696512, "tokens_per_sec": 22692.43689237043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590799967447917, "G_std": 0.16641998291015625, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.255246162414551, "grad_norm": 0.9007797837257385, "tokens_seen": 85860352, "tokens_per_sec": 22692.562553591255, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.363922119140625, "G_std": 0.1684697469075521, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.355767726898193, "grad_norm": 0.9720332622528076, "tokens_seen": 86024192, "tokens_per_sec": 22692.636641691577, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3645172119140625, "G_std": 0.16626485188802084, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.417717218399048, "grad_norm": 0.9943029284477234, "tokens_seen": 86188032, "tokens_per_sec": 22692.626191919026, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3625081380208333, "G_std": 0.1694208780924479, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.3254194259643555, "grad_norm": 1.0655771493911743, "tokens_seen": 86351872, "tokens_per_sec": 22692.796165151176, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3564707438151042, "G_std": 0.1673126220703125, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.324447154998779, "grad_norm": 0.9336347579956055, "tokens_seen": 86515712, "tokens_per_sec": 22692.68607038573, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3623199462890625, "G_std": 0.16799163818359375, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.374187469482422, "grad_norm": 0.943249523639679, "tokens_seen": 86679552, "tokens_per_sec": 22692.794994830398, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582356770833333, "G_std": 0.16732025146484375, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.234880208969116, "grad_norm": 0.9462572336196899, "tokens_seen": 86843392, "tokens_per_sec": 22692.935684288586, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626963297526042, "G_std": 0.1665827433268229, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.262358665466309, "grad_norm": 0.9086964726448059, "tokens_seen": 87007232, "tokens_per_sec": 22693.086561863536, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3582000732421875, "G_std": 0.16634368896484375, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.255984783172607, "grad_norm": 0.9087048768997192, "tokens_seen": 87171072, "tokens_per_sec": 22693.56114376424, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3604838053385417, "G_std": 0.1675898234049479, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.421839714050293, "grad_norm": 0.9226240515708923, "tokens_seen": 87334912, "tokens_per_sec": 22693.49903868467, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.364501953125, "G_std": 0.1668574015299479, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.323816299438477, "grad_norm": 0.9377562403678894, "tokens_seen": 87498752, "tokens_per_sec": 22693.731271620054, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3589019775390625, "G_std": 0.16714986165364584, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.308427810668945, "grad_norm": 0.9698933362960815, "tokens_seen": 87662592, "tokens_per_sec": 22693.90273463893, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3656005859375, "G_std": 0.16690826416015625, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.422605752944946, "grad_norm": 1.019178032875061, "tokens_seen": 87826432, "tokens_per_sec": 22694.052017127953, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3690439860026042, "G_std": 0.16658528645833334, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.414920330047607, "grad_norm": 0.9769250750541687, "tokens_seen": 87990272, "tokens_per_sec": 22693.961926297794, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3612721761067708, "G_std": 0.1682306925455729, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.190639019012451, "grad_norm": 0.9946147203445435, "tokens_seen": 88154112, "tokens_per_sec": 22694.1900008329, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3662261962890625, "G_std": 0.1686833699544271, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.25807523727417, "grad_norm": 1.1870286464691162, "tokens_seen": 88317952, "tokens_per_sec": 22694.355551682864, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3622233072916667, "G_std": 0.16695149739583334, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.3299949169158936, "grad_norm": 0.9901502728462219, "tokens_seen": 88481792, "tokens_per_sec": 22694.39329890737, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3630421956380208, "G_std": 0.1666386922200521, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.301011085510254, "grad_norm": 0.9368744492530823, "tokens_seen": 88645632, "tokens_per_sec": 22694.427367465836, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3620452880859375, "G_std": 0.16761271158854166, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.288751602172852, "grad_norm": 1.0418813228607178, "tokens_seen": 88809472, "tokens_per_sec": 22694.64681042881, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580576578776042, "G_std": 0.16628519694010416, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.273224830627441, "grad_norm": 0.9816871285438538, "tokens_seen": 88973312, "tokens_per_sec": 22693.664904478097, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3591969807942708, "G_std": 0.1678924560546875, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.245591163635254, "grad_norm": 1.008870005607605, "tokens_seen": 89137152, "tokens_per_sec": 22693.247070720987, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580474853515625, "G_std": 0.16565958658854166, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.175727605819702, "grad_norm": 1.0070295333862305, "tokens_seen": 89300992, "tokens_per_sec": 22692.760503774338, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35662841796875, "G_std": 0.1666844685872396, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.337958335876465, "grad_norm": 0.9447870850563049, "tokens_seen": 89464832, "tokens_per_sec": 22692.297029337627, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3638356526692708, "G_std": 0.1677729288736979, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.319709777832031, "grad_norm": 1.0019100904464722, "tokens_seen": 89628672, "tokens_per_sec": 22690.788302487763, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3725331624348958, "G_std": 0.16634368896484375, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.311490297317505, "grad_norm": 0.983665943145752, "tokens_seen": 89792512, "tokens_per_sec": 22690.303416084444, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3660736083984375, "G_std": 0.16657257080078125, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.470181465148926, "grad_norm": 1.046170711517334, "tokens_seen": 89956352, "tokens_per_sec": 22689.29213749052, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626861572265625, "G_std": 0.1680755615234375, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.397889614105225, "grad_norm": 0.9263256192207336, "tokens_seen": 90120192, "tokens_per_sec": 22688.253157939642, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3596293131510417, "G_std": 0.166717529296875, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.3692220687866214, "val_ppl": 78.98216510273853, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.3597920735677083, "G_std": 0.1672515869140625, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.184070587158203, "grad_norm": 0.9608316421508789, "tokens_seen": 90284032, "tokens_per_sec": 22679.772573194874, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3645273844401042, "G_std": 0.1669286092122396, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.286098003387451, "grad_norm": 0.9867356419563293, "tokens_seen": 90447872, "tokens_per_sec": 22678.148739254466, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3636016845703125, "G_std": 0.16646575927734375, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.467132091522217, "grad_norm": 0.9906895756721497, "tokens_seen": 90611712, "tokens_per_sec": 22676.110925007983, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3629557291666667, "G_std": 0.16729227701822916, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.3124072551727295, "grad_norm": 0.9586393237113953, "tokens_seen": 90775552, "tokens_per_sec": 22674.256521412673, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3620808919270833, "G_std": 0.1669464111328125, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.339928865432739, "grad_norm": 1.0549155473709106, "tokens_seen": 90939392, "tokens_per_sec": 22672.785731394903, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3698221842447917, "G_std": 0.16597747802734375, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.401381015777588, "grad_norm": 0.9482362866401672, "tokens_seen": 91103232, "tokens_per_sec": 22671.508417685047, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3639068603515625, "G_std": 0.16765594482421875, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.361533164978027, "grad_norm": 1.058694839477539, "tokens_seen": 91267072, "tokens_per_sec": 22669.850820459207, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3565622965494792, "G_std": 0.16644287109375, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.2398905754089355, "grad_norm": 0.9432865977287292, "tokens_seen": 91430912, "tokens_per_sec": 22668.48018208801, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3595937093098958, "G_std": 0.1665369669596354, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.3750832080841064, "grad_norm": 1.0358381271362305, "tokens_seen": 91594752, "tokens_per_sec": 22666.734376265147, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3611195882161458, "G_std": 0.16597747802734375, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.349583625793457, "grad_norm": 0.9412621259689331, "tokens_seen": 91758592, "tokens_per_sec": 22665.218962604315, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3609771728515625, "G_std": 0.16681925455729166, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.210165977478027, "grad_norm": 1.019284725189209, "tokens_seen": 91922432, "tokens_per_sec": 22664.758899097982, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3597920735677083, "G_std": 0.16791788736979166, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.415677785873413, "grad_norm": 1.0727267265319824, "tokens_seen": 92086272, "tokens_per_sec": 22664.831786777457, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3590342203776042, "G_std": 0.16649627685546875, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.3924500942230225, "grad_norm": 0.9473420977592468, "tokens_seen": 92250112, "tokens_per_sec": 22664.697666875585, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.35809326171875, "G_std": 0.1670989990234375, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.41212272644043, "grad_norm": 1.0459680557250977, "tokens_seen": 92413952, "tokens_per_sec": 22664.42503953839, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606516520182292, "G_std": 0.16617838541666666, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.269590854644775, "grad_norm": 1.0631442070007324, "tokens_seen": 92577792, "tokens_per_sec": 22664.61090898108, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358917236328125, "G_std": 0.1670049031575521, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.352066278457642, "grad_norm": 0.8972607254981995, "tokens_seen": 92741632, "tokens_per_sec": 22664.66814678559, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3580830891927083, "G_std": 0.1663691202799479, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.58634090423584, "grad_norm": 1.0100029706954956, "tokens_seen": 92905472, "tokens_per_sec": 22664.685562887324, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603515625, "G_std": 0.1667302449544271, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.271239995956421, "grad_norm": 0.9370018839836121, "tokens_seen": 93069312, "tokens_per_sec": 22664.869133049422, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3560078938802083, "G_std": 0.1670989990234375, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.3542516231536865, "grad_norm": 1.0098493099212646, "tokens_seen": 93233152, "tokens_per_sec": 22664.98399781041, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.356658935546875, "G_std": 0.16652933756510416, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.312385082244873, "grad_norm": 0.9473836421966553, "tokens_seen": 93396992, "tokens_per_sec": 22664.25840222389, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3566335042317708, "G_std": 0.1662012736002604, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.270079612731934, "grad_norm": 0.9856559634208679, "tokens_seen": 93560832, "tokens_per_sec": 22663.874027241458, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.36224365234375, "G_std": 0.16688028971354166, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.3454508781433105, "grad_norm": 1.0005974769592285, "tokens_seen": 93724672, "tokens_per_sec": 22663.94731786043, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3650614420572917, "G_std": 0.16575113932291666, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.276952743530273, "grad_norm": 0.9589465260505676, "tokens_seen": 93888512, "tokens_per_sec": 22663.96066524379, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3593597412109375, "G_std": 0.16607666015625, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.50894021987915, "grad_norm": 0.9879507422447205, "tokens_seen": 94052352, "tokens_per_sec": 22664.149622676432, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3605550130208333, "G_std": 0.16573079427083334, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.255289316177368, "grad_norm": 1.0147145986557007, "tokens_seen": 94216192, "tokens_per_sec": 22664.23002548799, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3587239583333333, "G_std": 0.165985107421875, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.446711540222168, "grad_norm": 1.0105708837509155, "tokens_seen": 94380032, "tokens_per_sec": 22664.293385872752, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.363128662109375, "G_std": 0.16684977213541666, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.369149684906006, "grad_norm": 0.9812448620796204, "tokens_seen": 94543872, "tokens_per_sec": 22664.401109811268, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3604583740234375, "G_std": 0.16733296712239584, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.337387323379517, "grad_norm": 0.9765744805335999, "tokens_seen": 94707712, "tokens_per_sec": 22664.320722172928, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3594207763671875, "G_std": 0.16579437255859375, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.444333791732788, "grad_norm": 0.972500205039978, "tokens_seen": 94871552, "tokens_per_sec": 22664.033978046697, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.356658935546875, "G_std": 0.16726175944010416, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.305108308792114, "grad_norm": 0.9941396713256836, "tokens_seen": 95035392, "tokens_per_sec": 22663.71520562801, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3613942464192708, "G_std": 0.16640726725260416, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.199220657348633, "grad_norm": 1.016911506652832, "tokens_seen": 95199232, "tokens_per_sec": 22663.150633714158, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3608194986979167, "G_std": 0.16707611083984375, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.309818267822266, "grad_norm": 0.9513329863548279, "tokens_seen": 95363072, "tokens_per_sec": 22662.96318775705, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3599955240885417, "G_std": 0.16608174641927084, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.199582099914551, "grad_norm": 0.989449679851532, "tokens_seen": 95526912, "tokens_per_sec": 22662.869553834073, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3600107828776042, "G_std": 0.16638946533203125, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.344703674316406, "grad_norm": 1.0601714849472046, "tokens_seen": 95690752, "tokens_per_sec": 22662.409381328278, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3603363037109375, "G_std": 0.1661402384440104, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.232050657272339, "grad_norm": 0.9739459753036499, "tokens_seen": 95854592, "tokens_per_sec": 22662.06283411011, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358123779296875, "G_std": 0.166656494140625, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.296408653259277, "grad_norm": 1.0060503482818604, "tokens_seen": 96018432, "tokens_per_sec": 22661.741198443196, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3606821695963542, "G_std": 0.1667912801106771, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.257741451263428, "grad_norm": 0.9928119778633118, "tokens_seen": 96182272, "tokens_per_sec": 22661.657183463092, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3583221435546875, "G_std": 0.16751607259114584, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.32025146484375, "grad_norm": 0.9584746360778809, "tokens_seen": 96346112, "tokens_per_sec": 22660.982777699748, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3598836263020833, "G_std": 0.1662114461263021, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.570691823959351, "grad_norm": 1.1011199951171875, "tokens_seen": 96509952, "tokens_per_sec": 22660.573713509817, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.358154296875, "G_std": 0.16623942057291666, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.42535924911499, "grad_norm": 1.052901268005371, "tokens_seen": 96673792, "tokens_per_sec": 22660.175644508592, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3610127766927083, "G_std": 0.1668980916341146, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.435664892196655, "grad_norm": 1.0184862613677979, "tokens_seen": 96837632, "tokens_per_sec": 22660.27055194946, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3675587972005208, "G_std": 0.16593170166015625, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.426026344299316, "grad_norm": 0.9876941442489624, "tokens_seen": 97001472, "tokens_per_sec": 22659.966592674242, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3652852376302083, "G_std": 0.16622161865234375, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.272016525268555, "grad_norm": 1.021315336227417, "tokens_seen": 97165312, "tokens_per_sec": 22659.265434596557, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.359405517578125, "G_std": 0.1663335164388021, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.3054187297821045, "grad_norm": 1.0181493759155273, "tokens_seen": 97329152, "tokens_per_sec": 22659.169789148833, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3626556396484375, "G_std": 0.1662139892578125, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.313184976577759, "grad_norm": 1.0071958303451538, "tokens_seen": 97492992, "tokens_per_sec": 22659.19122851274, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3600616455078125, "G_std": 0.16483561197916666, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.294190883636475, "grad_norm": 0.9843533635139465, "tokens_seen": 97656832, "tokens_per_sec": 22659.61359023958, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3632761637369792, "G_std": 0.16631062825520834, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.475567579269409, "grad_norm": 1.0466736555099487, "tokens_seen": 97820672, "tokens_per_sec": 22659.698057189747, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3619028727213542, "G_std": 0.1668370564778646, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.36839747428894, "grad_norm": 1.055371642112732, "tokens_seen": 97984512, "tokens_per_sec": 22659.22927324211, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3579050699869792, "G_std": 0.16670735677083334, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.296967267990112, "grad_norm": 1.0594738721847534, "tokens_seen": 98148352, "tokens_per_sec": 22659.15681206053, "vram_gb": 5.496054784, "lambda": null, "lambda_min": null, "lambda_max": null, "S_mean": null, "G_mean": 0.3562774658203125, "G_std": 0.16449737548828125, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.34392523765564, "val_ppl": 77.00922636947199, "lambda": null, "lambda_min": null, "lambda_max": null, "lambda_per_layer": null, "S_mean": null, "G_mean": 0.35980224609375, "G_std": 0.16666666666666666, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4339.748939752579, "nan_failures": 0} diff --git a/runs.log b/runs.log new file mode 100644 index 0000000000000000000000000000000000000000..f819274163eac9b470eabc9f1b0f7a038904f10e --- /dev/null +++ b/runs.log @@ -0,0 +1,900 @@ +[init] variant=baseline seed=0 params=63.38M +[init] use_S=False use_G=False +[baseline s0] step 0/4000 loss=10.8790 gn=15.69 lr=3.00e-06 tps=8974 mem=5.5G lam=None Gm=None +[baseline s0] step 20/4000 loss=9.8580 gn=1.93 lr=6.30e-05 tps=18098 mem=6.0G lam=None Gm=None +[baseline s0] step 40/4000 loss=9.1089 gn=1.75 lr=1.23e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s0] step 60/4000 loss=8.1093 gn=1.57 lr=1.83e-04 tps=18726 mem=6.0G lam=None Gm=None +[baseline s0] step 80/4000 loss=7.4265 gn=0.73 lr=2.43e-04 tps=18779 mem=6.0G lam=None Gm=None +[baseline s0] step 100/4000 loss=7.2358 gn=0.90 lr=3.03e-04 tps=18805 mem=6.0G lam=None Gm=None +[baseline s0] step 120/4000 loss=7.0669 gn=0.67 lr=3.63e-04 tps=18838 mem=6.0G lam=None Gm=None +[baseline s0] step 140/4000 loss=6.8653 gn=0.92 lr=4.23e-04 tps=18849 mem=6.0G lam=None Gm=None +[baseline s0] step 160/4000 loss=6.9054 gn=0.56 lr=4.83e-04 tps=18856 mem=6.0G lam=None Gm=None +[baseline s0] step 180/4000 loss=6.8181 gn=0.80 lr=5.43e-04 tps=18835 mem=6.0G lam=None Gm=None +[baseline s0] step 200/4000 loss=6.6996 gn=0.67 lr=6.00e-04 tps=18841 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 200: val_loss=6.7583 val_ppl=861.17 +[baseline s0] step 220/4000 loss=6.5854 gn=0.53 lr=6.00e-04 tps=18244 mem=6.0G lam=None Gm=None +[baseline s0] step 240/4000 loss=6.6331 gn=0.57 lr=6.00e-04 tps=18291 mem=6.0G lam=None Gm=None +[baseline s0] step 260/4000 loss=6.6194 gn=0.57 lr=6.00e-04 tps=18333 mem=6.0G lam=None Gm=None +[baseline s0] step 280/4000 loss=6.5077 gn=0.54 lr=5.99e-04 tps=18370 mem=6.0G lam=None Gm=None +[baseline s0] step 300/4000 loss=6.3495 gn=0.50 lr=5.99e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s0] step 320/4000 loss=6.4110 gn=0.53 lr=5.99e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s0] step 340/4000 loss=6.3930 gn=0.50 lr=5.98e-04 tps=18462 mem=6.0G lam=None Gm=None +[baseline s0] step 360/4000 loss=6.2602 gn=0.54 lr=5.98e-04 tps=18487 mem=6.0G lam=None Gm=None +[baseline s0] step 380/4000 loss=6.2289 gn=0.55 lr=5.97e-04 tps=18512 mem=6.0G lam=None Gm=None +[baseline s0] step 400/4000 loss=6.2561 gn=0.53 lr=5.96e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 400: val_loss=6.4037 val_ppl=604.09 +[baseline s0] step 420/4000 loss=6.4232 gn=0.80 lr=5.96e-04 tps=18241 mem=6.0G lam=None Gm=None +[baseline s0] step 440/4000 loss=6.3506 gn=0.59 lr=5.95e-04 tps=18273 mem=6.0G lam=None Gm=None +[baseline s0] step 460/4000 loss=6.3034 gn=0.55 lr=5.94e-04 tps=18295 mem=6.0G lam=None Gm=None +[baseline s0] step 480/4000 loss=6.3906 gn=0.58 lr=5.93e-04 tps=18308 mem=6.0G lam=None Gm=None +[baseline s0] step 500/4000 loss=6.3182 gn=0.56 lr=5.92e-04 tps=18328 mem=6.0G lam=None Gm=None +[baseline s0] step 520/4000 loss=6.1685 gn=0.55 lr=5.91e-04 tps=18349 mem=6.0G lam=None Gm=None +[baseline s0] step 540/4000 loss=6.1850 gn=0.49 lr=5.89e-04 tps=18368 mem=6.0G lam=None Gm=None +[baseline s0] step 560/4000 loss=6.0446 gn=0.56 lr=5.88e-04 tps=18388 mem=6.0G lam=None Gm=None +[baseline s0] step 580/4000 loss=6.2096 gn=0.51 lr=5.87e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s0] step 600/4000 loss=5.9779 gn=0.55 lr=5.85e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 600: val_loss=6.1749 val_ppl=480.52 +[baseline s0] step 620/4000 loss=6.0322 gn=0.50 lr=5.84e-04 tps=18235 mem=6.0G lam=None Gm=None +[baseline s0] step 640/4000 loss=6.1268 gn=0.65 lr=5.82e-04 tps=18254 mem=6.0G lam=None Gm=None +[baseline s0] step 660/4000 loss=5.9413 gn=0.54 lr=5.81e-04 tps=18275 mem=6.0G lam=None Gm=None +[baseline s0] step 680/4000 loss=5.9597 gn=0.56 lr=5.79e-04 tps=18276 mem=6.0G lam=None Gm=None +[baseline s0] step 700/4000 loss=6.0550 gn=0.57 lr=5.77e-04 tps=18288 mem=6.0G lam=None Gm=None +[baseline s0] step 720/4000 loss=6.0272 gn=0.54 lr=5.75e-04 tps=18306 mem=6.0G lam=None Gm=None +[baseline s0] step 740/4000 loss=6.0446 gn=0.52 lr=5.74e-04 tps=18320 mem=6.0G lam=None Gm=None +[baseline s0] step 760/4000 loss=5.9250 gn=0.57 lr=5.72e-04 tps=18335 mem=6.0G lam=None Gm=None +[baseline s0] step 780/4000 loss=5.9373 gn=0.54 lr=5.70e-04 tps=18348 mem=6.0G lam=None Gm=None +[baseline s0] step 800/4000 loss=5.9007 gn=0.55 lr=5.67e-04 tps=18362 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 800: val_loss=5.9891 val_ppl=399.06 +[baseline s0] step 820/4000 loss=5.9677 gn=0.51 lr=5.65e-04 tps=18212 mem=6.0G lam=None Gm=None +[baseline s0] step 840/4000 loss=6.1449 gn=0.53 lr=5.63e-04 tps=18229 mem=6.0G lam=None Gm=None +[baseline s0] step 860/4000 loss=5.9614 gn=0.54 lr=5.61e-04 tps=18243 mem=6.0G lam=None Gm=None +[baseline s0] step 880/4000 loss=5.9419 gn=0.64 lr=5.58e-04 tps=18255 mem=6.0G lam=None Gm=None +[baseline s0] step 900/4000 loss=5.8523 gn=0.52 lr=5.56e-04 tps=18269 mem=6.0G lam=None Gm=None +[baseline s0] step 920/4000 loss=5.7571 gn=0.52 lr=5.54e-04 tps=18283 mem=6.0G lam=None Gm=None +[baseline s0] step 940/4000 loss=5.8122 gn=0.55 lr=5.51e-04 tps=18294 mem=6.0G lam=None Gm=None +[baseline s0] step 960/4000 loss=5.7968 gn=0.58 lr=5.48e-04 tps=18305 mem=6.0G lam=None Gm=None +[baseline s0] step 980/4000 loss=5.7111 gn=0.51 lr=5.46e-04 tps=18318 mem=6.0G lam=None Gm=None +[baseline s0] step 1000/4000 loss=5.8198 gn=0.57 lr=5.43e-04 tps=18330 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1000: val_loss=5.8357 val_ppl=342.32 +[baseline s0] step 1020/4000 loss=5.5483 gn=0.50 lr=5.40e-04 tps=18217 mem=6.0G lam=None Gm=None +[baseline s0] step 1040/4000 loss=5.8019 gn=0.51 lr=5.37e-04 tps=18230 mem=6.0G lam=None Gm=None +[baseline s0] step 1060/4000 loss=5.5165 gn=0.56 lr=5.35e-04 tps=18240 mem=6.0G lam=None Gm=None +[baseline s0] step 1080/4000 loss=5.7172 gn=0.57 lr=5.32e-04 tps=18253 mem=6.0G lam=None Gm=None +[baseline s0] step 1100/4000 loss=5.6952 gn=0.56 lr=5.29e-04 tps=18265 mem=6.0G lam=None Gm=None +[baseline s0] step 1120/4000 loss=5.6321 gn=0.59 lr=5.26e-04 tps=18275 mem=6.0G lam=None Gm=None +[baseline s0] step 1140/4000 loss=5.8674 gn=0.51 lr=5.22e-04 tps=18282 mem=6.0G lam=None Gm=None +[baseline s0] step 1160/4000 loss=5.6313 gn=0.54 lr=5.19e-04 tps=18287 mem=6.0G lam=None Gm=None +[baseline s0] step 1180/4000 loss=5.6920 gn=0.53 lr=5.16e-04 tps=18296 mem=6.0G lam=None Gm=None +[baseline s0] step 1200/4000 loss=5.5678 gn=0.53 lr=5.13e-04 tps=18304 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1200: val_loss=5.6887 val_ppl=295.50 +[baseline s0] step 1220/4000 loss=5.5960 gn=0.61 lr=5.10e-04 tps=18204 mem=6.0G lam=None Gm=None +[baseline s0] step 1240/4000 loss=5.4634 gn=0.54 lr=5.06e-04 tps=18208 mem=6.0G lam=None Gm=None +[baseline s0] step 1260/4000 loss=5.4685 gn=0.56 lr=5.03e-04 tps=18216 mem=6.0G lam=None Gm=None +[baseline s0] step 1280/4000 loss=5.5946 gn=0.58 lr=4.99e-04 tps=18223 mem=6.0G lam=None Gm=None +[baseline s0] step 1300/4000 loss=5.6908 gn=0.59 lr=4.96e-04 tps=18232 mem=6.0G lam=None Gm=None +[baseline s0] step 1320/4000 loss=5.6382 gn=0.59 lr=4.92e-04 tps=18238 mem=6.0G lam=None Gm=None +[baseline s0] step 1340/4000 loss=5.7493 gn=0.54 lr=4.89e-04 tps=18242 mem=6.0G lam=None Gm=None +[baseline s0] step 1360/4000 loss=5.5626 gn=0.56 lr=4.85e-04 tps=18249 mem=6.0G lam=None Gm=None +[baseline s0] step 1380/4000 loss=5.4316 gn=0.56 lr=4.81e-04 tps=18252 mem=6.0G lam=None Gm=None +[baseline s0] step 1400/4000 loss=5.4411 gn=0.59 lr=4.78e-04 tps=18257 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1400: val_loss=5.5751 val_ppl=263.76 +[baseline s0] step 1420/4000 loss=5.4237 gn=0.60 lr=4.74e-04 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 1440/4000 loss=5.4160 gn=0.62 lr=4.70e-04 tps=18177 mem=6.0G lam=None Gm=None +[baseline s0] step 1460/4000 loss=5.5336 gn=0.60 lr=4.66e-04 tps=18184 mem=6.0G lam=None Gm=None +[baseline s0] step 1480/4000 loss=5.5815 gn=0.57 lr=4.62e-04 tps=18191 mem=6.0G lam=None Gm=None +[baseline s0] step 1500/4000 loss=5.4963 gn=0.61 lr=4.59e-04 tps=18197 mem=6.0G lam=None Gm=None +[baseline s0] step 1520/4000 loss=5.4520 gn=0.57 lr=4.55e-04 tps=18206 mem=6.0G lam=None Gm=None +[baseline s0] step 1540/4000 loss=5.3651 gn=0.55 lr=4.51e-04 tps=18212 mem=6.0G lam=None Gm=None +[baseline s0] step 1560/4000 loss=5.3625 gn=0.58 lr=4.47e-04 tps=18218 mem=6.0G lam=None Gm=None +[baseline s0] step 1580/4000 loss=5.4415 gn=0.61 lr=4.43e-04 tps=18225 mem=6.0G lam=None Gm=None +[baseline s0] step 1600/4000 loss=5.4264 gn=0.57 lr=4.38e-04 tps=18233 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1600: val_loss=5.4921 val_ppl=242.78 +[baseline s0] step 1620/4000 loss=5.4024 gn=0.56 lr=4.34e-04 tps=18161 mem=6.0G lam=None Gm=None +[baseline s0] step 1640/4000 loss=5.2968 gn=0.58 lr=4.30e-04 tps=18168 mem=6.0G lam=None Gm=None +[baseline s0] step 1660/4000 loss=5.4722 gn=0.60 lr=4.26e-04 tps=18176 mem=6.0G lam=None Gm=None +[baseline s0] step 1680/4000 loss=5.6134 gn=0.56 lr=4.22e-04 tps=18183 mem=6.0G lam=None Gm=None +[baseline s0] step 1700/4000 loss=5.4681 gn=0.59 lr=4.18e-04 tps=18190 mem=6.0G lam=None Gm=None +[baseline s0] step 1720/4000 loss=5.2368 gn=0.60 lr=4.13e-04 tps=18195 mem=6.0G lam=None Gm=None +[baseline s0] step 1740/4000 loss=5.2863 gn=0.59 lr=4.09e-04 tps=18195 mem=6.0G lam=None Gm=None +[baseline s0] step 1760/4000 loss=5.3700 gn=0.57 lr=4.05e-04 tps=18198 mem=6.0G lam=None Gm=None +[baseline s0] step 1780/4000 loss=5.2317 gn=0.60 lr=4.01e-04 tps=18204 mem=6.0G lam=None Gm=None +[baseline s0] step 1800/4000 loss=5.3057 gn=0.59 lr=3.96e-04 tps=18209 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1800: val_loss=5.4254 val_ppl=227.11 +[baseline s0] step 1820/4000 loss=5.3907 gn=0.58 lr=3.92e-04 tps=18146 mem=6.0G lam=None Gm=None +[baseline s0] step 1840/4000 loss=5.2270 gn=0.61 lr=3.88e-04 tps=18154 mem=6.0G lam=None Gm=None +[baseline s0] step 1860/4000 loss=5.2737 gn=0.59 lr=3.83e-04 tps=18159 mem=6.0G lam=None Gm=None +[baseline s0] step 1880/4000 loss=5.3796 gn=0.70 lr=3.79e-04 tps=18166 mem=6.0G lam=None Gm=None +[baseline s0] step 1900/4000 loss=5.3310 gn=0.66 lr=3.74e-04 tps=18173 mem=6.0G lam=None Gm=None +[baseline s0] step 1920/4000 loss=5.2796 gn=0.59 lr=3.70e-04 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 1940/4000 loss=5.3340 gn=0.61 lr=3.66e-04 tps=18186 mem=6.0G lam=None Gm=None +[baseline s0] step 1960/4000 loss=5.5327 gn=0.63 lr=3.61e-04 tps=18193 mem=6.0G lam=None Gm=None +[baseline s0] step 1980/4000 loss=5.4405 gn=0.61 lr=3.57e-04 tps=18199 mem=6.0G lam=None Gm=None +[baseline s0] step 2000/4000 loss=5.2762 gn=0.61 lr=3.52e-04 tps=18205 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2000: val_loss=5.3791 val_ppl=216.82 +[baseline s0] step 2020/4000 loss=5.2284 gn=0.58 lr=3.48e-04 tps=18147 mem=6.0G lam=None Gm=None +[baseline s0] step 2040/4000 loss=5.3605 gn=0.67 lr=3.43e-04 tps=18154 mem=6.0G lam=None Gm=None +[baseline s0] step 2060/4000 loss=5.1268 gn=0.72 lr=3.39e-04 tps=18159 mem=6.0G lam=None Gm=None +[baseline s0] step 2080/4000 loss=5.2638 gn=0.63 lr=3.34e-04 tps=18165 mem=6.0G lam=None Gm=None +[baseline s0] step 2100/4000 loss=5.2701 gn=0.58 lr=3.30e-04 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 2120/4000 loss=5.1915 gn=0.64 lr=3.26e-04 tps=18176 mem=6.0G lam=None Gm=None +[baseline s0] step 2140/4000 loss=5.0964 gn=0.62 lr=3.21e-04 tps=18183 mem=6.0G lam=None Gm=None +[baseline s0] step 2160/4000 loss=5.1444 gn=0.60 lr=3.17e-04 tps=18187 mem=6.0G lam=None Gm=None +[baseline s0] step 2180/4000 loss=5.1955 gn=0.73 lr=3.12e-04 tps=18192 mem=6.0G lam=None Gm=None +[baseline s0] step 2200/4000 loss=5.3442 gn=0.62 lr=3.08e-04 tps=18198 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2200: val_loss=5.2990 val_ppl=200.13 +[baseline s0] step 2220/4000 loss=5.2295 gn=0.63 lr=3.03e-04 tps=18147 mem=6.0G lam=None Gm=None +[baseline s0] step 2240/4000 loss=5.1707 gn=0.62 lr=2.99e-04 tps=18152 mem=6.0G lam=None Gm=None +[baseline s0] step 2260/4000 loss=5.1685 gn=0.60 lr=2.94e-04 tps=18158 mem=6.0G lam=None Gm=None +[baseline s0] step 2280/4000 loss=5.3584 gn=0.66 lr=2.90e-04 tps=18163 mem=6.0G lam=None Gm=None +[baseline s0] step 2300/4000 loss=5.1364 gn=0.66 lr=2.86e-04 tps=18169 mem=6.0G lam=None Gm=None +[baseline s0] step 2320/4000 loss=5.2594 gn=0.67 lr=2.81e-04 tps=18174 mem=6.0G lam=None Gm=None +[baseline s0] step 2340/4000 loss=5.1838 gn=0.63 lr=2.77e-04 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 2360/4000 loss=5.2727 gn=0.66 lr=2.72e-04 tps=18185 mem=6.0G lam=None Gm=None +[baseline s0] step 2380/4000 loss=5.1474 gn=0.68 lr=2.68e-04 tps=18190 mem=6.0G lam=None Gm=None +[baseline s0] step 2400/4000 loss=5.0503 gn=0.68 lr=2.64e-04 tps=18195 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2400: val_loss=5.2233 val_ppl=185.54 +[baseline s0] step 2420/4000 loss=5.0304 gn=0.62 lr=2.59e-04 tps=18148 mem=6.0G lam=None Gm=None +[baseline s0] step 2440/4000 loss=5.1615 gn=0.66 lr=2.55e-04 tps=18154 mem=6.0G lam=None Gm=None +[baseline s0] step 2460/4000 loss=5.2470 gn=0.63 lr=2.51e-04 tps=18159 mem=6.0G lam=None Gm=None +[baseline s0] step 2480/4000 loss=5.1272 gn=0.61 lr=2.47e-04 tps=18164 mem=6.0G lam=None Gm=None +[baseline s0] step 2500/4000 loss=4.9969 gn=0.66 lr=2.42e-04 tps=18169 mem=6.0G lam=None Gm=None +[baseline s0] step 2520/4000 loss=5.0420 gn=0.62 lr=2.38e-04 tps=18174 mem=6.0G lam=None Gm=None +[baseline s0] step 2540/4000 loss=5.1197 gn=0.65 lr=2.34e-04 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 2560/4000 loss=4.9050 gn=0.67 lr=2.30e-04 tps=18185 mem=6.0G lam=None Gm=None +[baseline s0] step 2580/4000 loss=5.2086 gn=0.63 lr=2.26e-04 tps=18190 mem=6.0G lam=None Gm=None +[baseline s0] step 2600/4000 loss=4.9868 gn=0.63 lr=2.22e-04 tps=18194 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2600: val_loss=5.1919 val_ppl=179.82 +[baseline s0] step 2620/4000 loss=5.1561 gn=0.69 lr=2.17e-04 tps=18151 mem=6.0G lam=None Gm=None +[baseline s0] step 2640/4000 loss=5.1533 gn=0.64 lr=2.13e-04 tps=18156 mem=6.0G lam=None Gm=None +[baseline s0] step 2660/4000 loss=5.0194 gn=0.64 lr=2.09e-04 tps=18160 mem=6.0G lam=None Gm=None +[baseline s0] step 2680/4000 loss=4.8527 gn=0.79 lr=2.05e-04 tps=18165 mem=6.0G lam=None Gm=None +[baseline s0] step 2700/4000 loss=5.1246 gn=0.73 lr=2.01e-04 tps=18170 mem=6.0G lam=None Gm=None +[baseline s0] step 2720/4000 loss=5.1081 gn=0.66 lr=1.98e-04 tps=18175 mem=6.0G lam=None Gm=None +[baseline s0] step 2740/4000 loss=5.0874 gn=0.63 lr=1.94e-04 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 2760/4000 loss=4.9413 gn=0.66 lr=1.90e-04 tps=18184 mem=6.0G lam=None Gm=None +[baseline s0] step 2780/4000 loss=5.0563 gn=0.64 lr=1.86e-04 tps=18188 mem=6.0G lam=None Gm=None +[baseline s0] step 2800/4000 loss=5.0011 gn=0.69 lr=1.82e-04 tps=18193 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2800: val_loss=5.1333 val_ppl=169.58 +[baseline s0] step 2820/4000 loss=5.0585 gn=0.67 lr=1.79e-04 tps=18152 mem=6.0G lam=None Gm=None +[baseline s0] step 2840/4000 loss=4.8624 gn=0.73 lr=1.75e-04 tps=18157 mem=6.0G lam=None Gm=None +[baseline s0] step 2860/4000 loss=5.0362 gn=0.65 lr=1.71e-04 tps=18162 mem=6.0G lam=None Gm=None +[baseline s0] step 2880/4000 loss=5.0276 gn=0.68 lr=1.68e-04 tps=18166 mem=6.0G lam=None Gm=None +[baseline s0] step 2900/4000 loss=4.8816 gn=0.68 lr=1.64e-04 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 2920/4000 loss=5.0774 gn=0.70 lr=1.61e-04 tps=18175 mem=6.0G lam=None Gm=None +[baseline s0] step 2940/4000 loss=4.9940 gn=0.64 lr=1.57e-04 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 2960/4000 loss=4.9860 gn=0.67 lr=1.54e-04 tps=18184 mem=6.0G lam=None Gm=None +[baseline s0] step 2980/4000 loss=4.9343 gn=0.64 lr=1.50e-04 tps=18188 mem=6.0G lam=None Gm=None +[baseline s0] step 3000/4000 loss=4.8842 gn=0.68 lr=1.47e-04 tps=18192 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3000: val_loss=5.1456 val_ppl=171.68 +[baseline s0] step 3020/4000 loss=4.9521 gn=0.66 lr=1.44e-04 tps=18155 mem=6.0G lam=None Gm=None +[baseline s0] step 3040/4000 loss=5.0338 gn=0.72 lr=1.41e-04 tps=18159 mem=6.0G lam=None Gm=None +[baseline s0] step 3060/4000 loss=4.9991 gn=0.70 lr=1.38e-04 tps=18163 mem=6.0G lam=None Gm=None +[baseline s0] step 3080/4000 loss=4.8222 gn=0.65 lr=1.34e-04 tps=18167 mem=6.0G lam=None Gm=None +[baseline s0] step 3100/4000 loss=5.0607 gn=0.70 lr=1.31e-04 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 3120/4000 loss=5.0802 gn=0.81 lr=1.28e-04 tps=18173 mem=6.0G lam=None Gm=None +[baseline s0] step 3140/4000 loss=4.9207 gn=0.68 lr=1.25e-04 tps=18173 mem=6.0G lam=None Gm=None +[baseline s0] step 3160/4000 loss=5.0691 gn=0.69 lr=1.23e-04 tps=18177 mem=6.0G lam=None Gm=None +[baseline s0] step 3180/4000 loss=4.9020 gn=0.67 lr=1.20e-04 tps=18181 mem=6.0G lam=None Gm=None +[baseline s0] step 3200/4000 loss=4.9458 gn=0.73 lr=1.17e-04 tps=18185 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3200: val_loss=5.0536 val_ppl=156.58 +[baseline s0] step 3220/4000 loss=5.0413 gn=0.75 lr=1.14e-04 tps=18150 mem=6.0G lam=None Gm=None +[baseline s0] step 3240/4000 loss=5.0531 gn=0.70 lr=1.12e-04 tps=18154 mem=6.0G lam=None Gm=None +[baseline s0] step 3260/4000 loss=4.8129 gn=0.67 lr=1.09e-04 tps=18158 mem=6.0G lam=None Gm=None +[baseline s0] step 3280/4000 loss=5.0799 gn=0.77 lr=1.06e-04 tps=18162 mem=6.0G lam=None Gm=None +[baseline s0] step 3300/4000 loss=4.8907 gn=0.70 lr=1.04e-04 tps=18166 mem=6.0G lam=None Gm=None +[baseline s0] step 3320/4000 loss=5.0456 gn=0.69 lr=1.02e-04 tps=18170 mem=6.0G lam=None Gm=None +[baseline s0] step 3340/4000 loss=4.8715 gn=0.67 lr=9.92e-05 tps=18173 mem=6.0G lam=None Gm=None +[baseline s0] step 3360/4000 loss=5.0376 gn=0.74 lr=9.69e-05 tps=18177 mem=6.0G lam=None Gm=None +[baseline s0] step 3380/4000 loss=5.0292 gn=0.73 lr=9.47e-05 tps=18181 mem=6.0G lam=None Gm=None +[baseline s0] step 3400/4000 loss=4.8974 gn=0.68 lr=9.25e-05 tps=18185 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3400: val_loss=5.0562 val_ppl=156.99 +[baseline s0] step 3420/4000 loss=4.9393 gn=0.71 lr=9.04e-05 tps=18152 mem=6.0G lam=None Gm=None +[baseline s0] step 3440/4000 loss=4.9874 gn=0.75 lr=8.84e-05 tps=18156 mem=6.0G lam=None Gm=None +[baseline s0] step 3460/4000 loss=4.8650 gn=0.76 lr=8.65e-05 tps=18160 mem=6.0G lam=None Gm=None +[baseline s0] step 3480/4000 loss=4.9717 gn=0.70 lr=8.46e-05 tps=18164 mem=6.0G lam=None Gm=None +[baseline s0] step 3500/4000 loss=4.8648 gn=0.70 lr=8.27e-05 tps=18168 mem=6.0G lam=None Gm=None +[baseline s0] step 3520/4000 loss=4.7724 gn=0.78 lr=8.10e-05 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 3540/4000 loss=4.7628 gn=0.68 lr=7.93e-05 tps=18176 mem=6.0G lam=None Gm=None +[baseline s0] step 3560/4000 loss=5.0116 gn=0.68 lr=7.77e-05 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 3580/4000 loss=4.8404 gn=0.66 lr=7.61e-05 tps=18183 mem=6.0G lam=None Gm=None +[baseline s0] step 3600/4000 loss=4.8622 gn=0.77 lr=7.46e-05 tps=18187 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3600: val_loss=5.0255 val_ppl=152.25 +[baseline s0] step 3620/4000 loss=4.9256 gn=0.73 lr=7.32e-05 tps=18156 mem=6.0G lam=None Gm=None +[baseline s0] step 3640/4000 loss=4.9979 gn=0.69 lr=7.19e-05 tps=18160 mem=6.0G lam=None Gm=None +[baseline s0] step 3660/4000 loss=5.0407 gn=0.78 lr=7.06e-05 tps=18164 mem=6.0G lam=None Gm=None +[baseline s0] step 3680/4000 loss=4.9707 gn=0.70 lr=6.94e-05 tps=18168 mem=6.0G lam=None Gm=None +[baseline s0] step 3700/4000 loss=5.0190 gn=0.73 lr=6.83e-05 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 3720/4000 loss=4.7931 gn=0.69 lr=6.72e-05 tps=18175 mem=6.0G lam=None Gm=None +[baseline s0] step 3740/4000 loss=4.9578 gn=0.75 lr=6.62e-05 tps=18179 mem=6.0G lam=None Gm=None +[baseline s0] step 3760/4000 loss=4.8983 gn=0.72 lr=6.53e-05 tps=18182 mem=6.0G lam=None Gm=None +[baseline s0] step 3780/4000 loss=4.8004 gn=0.76 lr=6.45e-05 tps=18186 mem=6.0G lam=None Gm=None +[baseline s0] step 3800/4000 loss=4.9087 gn=0.75 lr=6.37e-05 tps=18190 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3800: val_loss=5.0057 val_ppl=149.26 +[baseline s0] step 3820/4000 loss=4.9865 gn=0.74 lr=6.30e-05 tps=18160 mem=6.0G lam=None Gm=None +[baseline s0] step 3840/4000 loss=4.9053 gn=0.76 lr=6.24e-05 tps=18164 mem=6.0G lam=None Gm=None +[baseline s0] step 3860/4000 loss=4.9029 gn=0.68 lr=6.18e-05 tps=18167 mem=6.0G lam=None Gm=None +[baseline s0] step 3880/4000 loss=4.9808 gn=0.79 lr=6.13e-05 tps=18171 mem=6.0G lam=None Gm=None +[baseline s0] step 3900/4000 loss=4.8809 gn=0.78 lr=6.09e-05 tps=18174 mem=6.0G lam=None Gm=None +[baseline s0] step 3920/4000 loss=4.8140 gn=0.76 lr=6.06e-05 tps=18178 mem=6.0G lam=None Gm=None +[baseline s0] step 3940/4000 loss=4.9906 gn=0.77 lr=6.03e-05 tps=18182 mem=6.0G lam=None Gm=None +[baseline s0] step 3960/4000 loss=4.9362 gn=0.75 lr=6.01e-05 tps=18185 mem=6.0G lam=None Gm=None +[baseline s0] step 3980/4000 loss=4.8850 gn=0.76 lr=6.00e-05 tps=18189 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3999: val_loss=5.0002 val_ppl=148.44 +[init] variant=s_only seed=0 params=63.97M +[init] use_S=True use_G=False +[s_only s0] step 0/4000 loss=10.9196 gn=16.24 lr=3.00e-06 tps=7866 mem=5.6G lam=0.12692801654338837 Gm=None +[s_only s0] step 20/4000 loss=9.8201 gn=1.89 lr=6.30e-05 tps=15859 mem=6.2G lam=0.1269285697489977 Gm=None +[s_only s0] step 40/4000 loss=9.0759 gn=1.82 lr=1.23e-04 tps=16244 mem=6.2G lam=0.126861909404397 Gm=None +[s_only s0] step 60/4000 loss=8.0906 gn=1.51 lr=1.83e-04 tps=16370 mem=6.2G lam=0.1267800386995077 Gm=None +[s_only s0] step 80/4000 loss=7.4281 gn=0.74 lr=2.43e-04 tps=16458 mem=6.2G lam=0.12668094101051489 Gm=None +[s_only s0] step 100/4000 loss=7.2174 gn=0.68 lr=3.03e-04 tps=16511 mem=6.2G lam=0.1265610239158074 Gm=None +[s_only s0] step 120/4000 loss=7.0768 gn=0.80 lr=3.63e-04 tps=16546 mem=6.2G lam=0.12644990781943002 Gm=None +[s_only s0] step 140/4000 loss=6.8256 gn=0.54 lr=4.23e-04 tps=16558 mem=6.2G lam=0.12640769282976785 Gm=None +[s_only s0] step 160/4000 loss=6.9099 gn=0.79 lr=4.83e-04 tps=16572 mem=6.2G lam=0.12637964511911073 Gm=None +[s_only s0] step 180/4000 loss=6.8124 gn=0.77 lr=5.43e-04 tps=16584 mem=6.2G lam=0.12634567295511565 Gm=None +[s_only s0] step 200/4000 loss=6.6985 gn=0.77 lr=6.00e-04 tps=16593 mem=6.2G lam=0.12624020067354044 Gm=None +[s_only s0] EVAL step 200: val_loss=6.7505 val_ppl=854.45 +[s_only s0] step 220/4000 loss=6.5877 gn=0.59 lr=6.00e-04 tps=16051 mem=6.2G lam=0.12625185896952948 Gm=None +[s_only s0] step 240/4000 loss=6.6281 gn=0.54 lr=6.00e-04 tps=16097 mem=6.2G lam=0.1262657322610418 Gm=None +[s_only s0] step 260/4000 loss=6.6135 gn=0.58 lr=6.00e-04 tps=16137 mem=6.2G lam=0.12628101371228695 Gm=None +[s_only s0] step 280/4000 loss=6.5128 gn=0.54 lr=5.99e-04 tps=16173 mem=6.2G lam=0.12627421288440624 Gm=None +[s_only s0] step 300/4000 loss=6.3416 gn=0.54 lr=5.99e-04 tps=16203 mem=6.2G lam=0.12628612201660872 Gm=None +[s_only s0] step 320/4000 loss=6.4067 gn=0.48 lr=5.99e-04 tps=16231 mem=6.2G lam=0.1263035830731193 Gm=None +[s_only s0] step 340/4000 loss=6.3989 gn=0.53 lr=5.98e-04 tps=16252 mem=6.2G lam=0.126261239250501 Gm=None +[s_only s0] step 360/4000 loss=6.2496 gn=0.53 lr=5.98e-04 tps=16270 mem=6.2G lam=0.12632379370431104 Gm=None +[s_only s0] step 380/4000 loss=6.2258 gn=0.55 lr=5.97e-04 tps=16290 mem=6.2G lam=0.12632035743445158 Gm=None +[s_only s0] step 400/4000 loss=6.2672 gn=0.56 lr=5.96e-04 tps=16306 mem=6.2G lam=0.12638370475421348 Gm=None +[s_only s0] EVAL step 400: val_loss=6.3981 val_ppl=600.70 +[s_only s0] step 420/4000 loss=6.3893 gn=0.77 lr=5.96e-04 tps=16042 mem=6.2G lam=0.1264245609442393 Gm=None +[s_only s0] step 440/4000 loss=6.3583 gn=0.66 lr=5.95e-04 tps=16068 mem=6.2G lam=0.12652567743013302 Gm=None +[s_only s0] step 460/4000 loss=6.2952 gn=0.53 lr=5.94e-04 tps=16091 mem=6.2G lam=0.1265407477815946 Gm=None +[s_only s0] step 480/4000 loss=6.3867 gn=0.52 lr=5.93e-04 tps=16112 mem=6.2G lam=0.12667018050948778 Gm=None +[s_only s0] step 500/4000 loss=6.3163 gn=0.54 lr=5.92e-04 tps=16132 mem=6.2G lam=0.1267355838790536 Gm=None +[s_only s0] step 520/4000 loss=6.1707 gn=0.56 lr=5.91e-04 tps=16152 mem=6.2G lam=0.12688766761372486 Gm=None +[s_only s0] step 540/4000 loss=6.1835 gn=0.53 lr=5.89e-04 tps=16170 mem=6.2G lam=0.12706252994636694 Gm=None +[s_only s0] step 560/4000 loss=6.0407 gn=0.56 lr=5.88e-04 tps=16189 mem=6.2G lam=0.12717174366116524 Gm=None +[s_only s0] step 580/4000 loss=6.2094 gn=0.59 lr=5.87e-04 tps=16205 mem=6.2G lam=0.12731845180193582 Gm=None +[s_only s0] step 600/4000 loss=5.9602 gn=0.56 lr=5.85e-04 tps=16220 mem=6.2G lam=0.12746825131277242 Gm=None +[s_only s0] EVAL step 600: val_loss=6.1718 val_ppl=479.03 +[s_only s0] step 620/4000 loss=6.0255 gn=0.50 lr=5.84e-04 tps=16046 mem=6.2G lam=0.1276232535019517 Gm=None +[s_only s0] step 640/4000 loss=6.1234 gn=0.64 lr=5.82e-04 tps=16064 mem=6.2G lam=0.12782887276262045 Gm=None +[s_only s0] step 660/4000 loss=5.9412 gn=0.59 lr=5.81e-04 tps=16079 mem=6.2G lam=0.12805543839931488 Gm=None +[s_only s0] step 680/4000 loss=5.9505 gn=0.52 lr=5.79e-04 tps=16096 mem=6.2G lam=0.1282186039413015 Gm=None +[s_only s0] step 700/4000 loss=6.0496 gn=0.58 lr=5.77e-04 tps=16110 mem=6.2G lam=0.12838227239747843 Gm=None +[s_only s0] step 720/4000 loss=6.0181 gn=0.53 lr=5.75e-04 tps=16125 mem=6.2G lam=0.12860822770744562 Gm=None +[s_only s0] step 740/4000 loss=6.0501 gn=0.58 lr=5.74e-04 tps=16138 mem=6.2G lam=0.1289047651613752 Gm=None +[s_only s0] step 760/4000 loss=5.9148 gn=0.52 lr=5.72e-04 tps=16151 mem=6.2G lam=0.12920729536563158 Gm=None +[s_only s0] step 780/4000 loss=5.9283 gn=0.52 lr=5.70e-04 tps=16163 mem=6.2G lam=0.12947439402341843 Gm=None +[s_only s0] step 800/4000 loss=5.8881 gn=0.54 lr=5.67e-04 tps=16175 mem=6.2G lam=0.12977179295072952 Gm=None +[s_only s0] EVAL step 800: val_loss=5.9815 val_ppl=396.01 +[s_only s0] step 820/4000 loss=5.9666 gn=0.53 lr=5.65e-04 tps=16042 mem=6.2G lam=0.13008342559138933 Gm=None +[s_only s0] step 840/4000 loss=6.1374 gn=0.54 lr=5.63e-04 tps=16056 mem=6.2G lam=0.1303543041770657 Gm=None +[s_only s0] step 860/4000 loss=5.9518 gn=0.56 lr=5.61e-04 tps=16070 mem=6.2G lam=0.13066580053418875 Gm=None +[s_only s0] step 880/4000 loss=5.9320 gn=0.62 lr=5.58e-04 tps=16083 mem=6.2G lam=0.13096548213313022 Gm=None +[s_only s0] step 900/4000 loss=5.8557 gn=0.52 lr=5.56e-04 tps=16094 mem=6.2G lam=0.1312484402830402 Gm=None +[s_only s0] step 920/4000 loss=5.7370 gn=0.54 lr=5.54e-04 tps=16104 mem=6.2G lam=0.1315185483545065 Gm=None +[s_only s0] step 940/4000 loss=5.7989 gn=0.55 lr=5.51e-04 tps=16115 mem=6.2G lam=0.13187734379122654 Gm=None +[s_only s0] step 960/4000 loss=5.7840 gn=0.58 lr=5.48e-04 tps=16124 mem=6.2G lam=0.13223367060224214 Gm=None +[s_only s0] step 980/4000 loss=5.7109 gn=0.57 lr=5.46e-04 tps=16134 mem=6.2G lam=0.13256636603424946 Gm=None +[s_only s0] step 1000/4000 loss=5.8045 gn=0.58 lr=5.43e-04 tps=16143 mem=6.2G lam=0.13293919060379267 Gm=None +[s_only s0] EVAL step 1000: val_loss=5.8303 val_ppl=340.47 +[s_only s0] step 1020/4000 loss=5.5516 gn=0.52 lr=5.40e-04 tps=16037 mem=6.2G lam=0.13332742483665547 Gm=None +[s_only s0] step 1040/4000 loss=5.8040 gn=0.50 lr=5.37e-04 tps=16049 mem=6.2G lam=0.1336649345854918 Gm=None +[s_only s0] step 1060/4000 loss=5.5133 gn=0.54 lr=5.35e-04 tps=16060 mem=6.2G lam=0.13397249206900597 Gm=None +[s_only s0] step 1080/4000 loss=5.7270 gn=0.65 lr=5.32e-04 tps=16071 mem=6.2G lam=0.13436876454701027 Gm=None +[s_only s0] step 1100/4000 loss=5.6774 gn=0.58 lr=5.29e-04 tps=16082 mem=6.2G lam=0.1346825072541833 Gm=None +[s_only s0] step 1120/4000 loss=5.6185 gn=0.57 lr=5.26e-04 tps=16092 mem=6.2G lam=0.13510261848568916 Gm=None +[s_only s0] step 1140/4000 loss=5.8675 gn=0.56 lr=5.22e-04 tps=16100 mem=6.2G lam=0.1354632107540965 Gm=None +[s_only s0] step 1160/4000 loss=5.6197 gn=0.54 lr=5.19e-04 tps=16109 mem=6.2G lam=0.1358909821137786 Gm=None +[s_only s0] step 1180/4000 loss=5.6770 gn=0.56 lr=5.16e-04 tps=16117 mem=6.2G lam=0.1362527391562859 Gm=None +[s_only s0] step 1200/4000 loss=5.5601 gn=0.52 lr=5.13e-04 tps=16124 mem=6.2G lam=0.13655376713722944 Gm=None +[s_only s0] EVAL step 1200: val_loss=5.6810 val_ppl=293.24 +[s_only s0] step 1220/4000 loss=5.5830 gn=0.60 lr=5.10e-04 tps=16036 mem=6.2G lam=0.13686958079536757 Gm=None +[s_only s0] step 1240/4000 loss=5.4571 gn=0.53 lr=5.06e-04 tps=16045 mem=6.2G lam=0.13723452016711235 Gm=None +[s_only s0] step 1260/4000 loss=5.4601 gn=0.56 lr=5.03e-04 tps=16053 mem=6.2G lam=0.1376482384900252 Gm=None +[s_only s0] step 1280/4000 loss=5.5864 gn=0.55 lr=4.99e-04 tps=16062 mem=6.2G lam=0.13803729942689338 Gm=None +[s_only s0] step 1300/4000 loss=5.6888 gn=0.61 lr=4.96e-04 tps=16070 mem=6.2G lam=0.1383867928137382 Gm=None +[s_only s0] step 1320/4000 loss=5.6395 gn=0.58 lr=4.92e-04 tps=16079 mem=6.2G lam=0.1387265526379148 Gm=None +[s_only s0] step 1340/4000 loss=5.7577 gn=0.57 lr=4.89e-04 tps=16087 mem=6.2G lam=0.13911849415550628 Gm=None +[s_only s0] step 1360/4000 loss=5.5489 gn=0.58 lr=4.85e-04 tps=16095 mem=6.2G lam=0.13947931056221327 Gm=None +[s_only s0] step 1380/4000 loss=5.4243 gn=0.57 lr=4.81e-04 tps=16101 mem=6.2G lam=0.13979739509522915 Gm=None +[s_only s0] step 1400/4000 loss=5.4404 gn=0.58 lr=4.78e-04 tps=16106 mem=6.2G lam=0.14012548172225556 Gm=None +[s_only s0] EVAL step 1400: val_loss=5.5705 val_ppl=262.55 +[s_only s0] step 1420/4000 loss=5.4244 gn=0.63 lr=4.74e-04 tps=16030 mem=6.2G lam=0.1404859513665239 Gm=None +[s_only s0] step 1440/4000 loss=5.4084 gn=0.62 lr=4.70e-04 tps=16038 mem=6.2G lam=0.140826303511858 Gm=None +[s_only s0] step 1460/4000 loss=5.5294 gn=0.63 lr=4.66e-04 tps=16046 mem=6.2G lam=0.14116969301054874 Gm=None +[s_only s0] step 1480/4000 loss=5.5820 gn=0.57 lr=4.62e-04 tps=16053 mem=6.2G lam=0.14155861859520277 Gm=None +[s_only s0] step 1500/4000 loss=5.4778 gn=0.63 lr=4.59e-04 tps=16061 mem=6.2G lam=0.1419260473921895 Gm=None +[s_only s0] step 1520/4000 loss=5.4672 gn=0.58 lr=4.55e-04 tps=16067 mem=6.2G lam=0.14222985816498598 Gm=None +[s_only s0] step 1540/4000 loss=5.3713 gn=0.61 lr=4.51e-04 tps=16074 mem=6.2G lam=0.14253480080515146 Gm=None +[s_only s0] step 1560/4000 loss=5.3767 gn=0.57 lr=4.47e-04 tps=16081 mem=6.2G lam=0.14288722444325686 Gm=None +[s_only s0] step 1580/4000 loss=5.4439 gn=0.65 lr=4.43e-04 tps=16088 mem=6.2G lam=0.14320783875882626 Gm=None +[s_only s0] step 1600/4000 loss=5.4300 gn=0.57 lr=4.38e-04 tps=16093 mem=6.2G lam=0.14355705057581267 Gm=None +[s_only s0] EVAL step 1600: val_loss=5.4941 val_ppl=243.26 +[s_only s0] step 1620/4000 loss=5.4115 gn=0.64 lr=4.34e-04 tps=16027 mem=6.2G lam=0.14388109495242438 Gm=None +[s_only s0] step 1640/4000 loss=5.2882 gn=0.59 lr=4.30e-04 tps=16034 mem=6.2G lam=0.1442484213039279 Gm=None +[s_only s0] step 1660/4000 loss=5.4577 gn=0.59 lr=4.26e-04 tps=16041 mem=6.2G lam=0.14454306599994501 Gm=None +[s_only s0] step 1680/4000 loss=5.6048 gn=0.59 lr=4.22e-04 tps=16047 mem=6.2G lam=0.14485854376107454 Gm=None +[s_only s0] step 1700/4000 loss=5.4706 gn=0.58 lr=4.18e-04 tps=16053 mem=6.2G lam=0.14520258642733097 Gm=None +[s_only s0] step 1720/4000 loss=5.2282 gn=0.60 lr=4.13e-04 tps=16059 mem=6.2G lam=0.14549836268027624 Gm=None +[s_only s0] step 1740/4000 loss=5.2964 gn=0.60 lr=4.09e-04 tps=16065 mem=6.2G lam=0.1458452930673957 Gm=None +[s_only s0] step 1760/4000 loss=5.3622 gn=0.61 lr=4.05e-04 tps=16072 mem=6.2G lam=0.14619173482060432 Gm=None +[s_only s0] step 1780/4000 loss=5.2387 gn=0.63 lr=4.01e-04 tps=16077 mem=6.2G lam=0.14650953530023494 Gm=None +[s_only s0] step 1800/4000 loss=5.2973 gn=0.61 lr=3.96e-04 tps=16083 mem=6.2G lam=0.14679855667054653 Gm=None +[s_only s0] EVAL step 1800: val_loss=5.4237 val_ppl=226.71 +[s_only s0] step 1820/4000 loss=5.3838 gn=0.56 lr=3.92e-04 tps=16025 mem=6.2G lam=0.14706969012816748 Gm=None +[s_only s0] step 1840/4000 loss=5.2330 gn=0.63 lr=3.88e-04 tps=16030 mem=6.2G lam=0.1473601134493947 Gm=None +[s_only s0] step 1860/4000 loss=5.2746 gn=0.59 lr=3.83e-04 tps=16036 mem=6.2G lam=0.14768042291204134 Gm=None +[s_only s0] step 1880/4000 loss=5.3684 gn=0.65 lr=3.79e-04 tps=16042 mem=6.2G lam=0.14795536641031504 Gm=None +[s_only s0] step 1900/4000 loss=5.3387 gn=0.66 lr=3.74e-04 tps=16047 mem=6.2G lam=0.14826269975552955 Gm=None +[s_only s0] step 1920/4000 loss=5.2729 gn=0.60 lr=3.70e-04 tps=16053 mem=6.2G lam=0.1485254509995381 Gm=None +[s_only s0] step 1940/4000 loss=5.3370 gn=0.62 lr=3.66e-04 tps=16058 mem=6.2G lam=0.1487580065925916 Gm=None +[s_only s0] step 1960/4000 loss=5.5228 gn=0.66 lr=3.61e-04 tps=16063 mem=6.2G lam=0.1491000996902585 Gm=None +[s_only s0] step 1980/4000 loss=5.4326 gn=0.62 lr=3.57e-04 tps=16068 mem=6.2G lam=0.1493905217697223 Gm=None +[s_only s0] step 2000/4000 loss=5.2669 gn=0.60 lr=3.52e-04 tps=16074 mem=6.2G lam=0.14968158894528946 Gm=None +[s_only s0] EVAL step 2000: val_loss=5.3785 val_ppl=216.70 +[s_only s0] step 2020/4000 loss=5.2263 gn=0.59 lr=3.48e-04 tps=16022 mem=6.2G lam=0.14995385675380626 Gm=None +[s_only s0] step 2040/4000 loss=5.3614 gn=0.67 lr=3.43e-04 tps=16027 mem=6.2G lam=0.15023998698840538 Gm=None +[s_only s0] step 2060/4000 loss=5.1188 gn=0.67 lr=3.39e-04 tps=16033 mem=6.2G lam=0.15052441755930582 Gm=None +[s_only s0] step 2080/4000 loss=5.2630 gn=0.63 lr=3.34e-04 tps=16038 mem=6.2G lam=0.15079285173366466 Gm=None +[s_only s0] step 2100/4000 loss=5.2620 gn=0.60 lr=3.30e-04 tps=16043 mem=6.2G lam=0.15108702021340528 Gm=None +[s_only s0] step 2120/4000 loss=5.1780 gn=0.61 lr=3.26e-04 tps=16047 mem=6.2G lam=0.15135678462684155 Gm=None +[s_only s0] step 2140/4000 loss=5.0995 gn=0.63 lr=3.21e-04 tps=16052 mem=6.2G lam=0.15164329887678227 Gm=None +[s_only s0] step 2160/4000 loss=5.1503 gn=0.62 lr=3.17e-04 tps=16057 mem=6.2G lam=0.15192982678612074 Gm=None +[s_only s0] step 2180/4000 loss=5.1897 gn=0.68 lr=3.12e-04 tps=16061 mem=6.2G lam=0.152193579201897 Gm=None +[s_only s0] step 2200/4000 loss=5.3355 gn=0.63 lr=3.08e-04 tps=16066 mem=6.2G lam=0.15246522768090168 Gm=None +[s_only s0] EVAL step 2200: val_loss=5.2928 val_ppl=198.90 +[s_only s0] step 2220/4000 loss=5.2258 gn=0.65 lr=3.03e-04 tps=16018 mem=6.2G lam=0.1527368761599064 Gm=None +[s_only s0] step 2240/4000 loss=5.1550 gn=0.60 lr=2.99e-04 tps=16023 mem=6.2G lam=0.15297844167798758 Gm=None +[s_only s0] step 2260/4000 loss=5.1668 gn=0.59 lr=2.94e-04 tps=16028 mem=6.2G lam=0.15325090506424507 Gm=None +[s_only s0] step 2280/4000 loss=5.3703 gn=0.73 lr=2.90e-04 tps=16032 mem=6.2G lam=0.15346690950294337 Gm=None +[s_only s0] step 2300/4000 loss=5.1344 gn=0.68 lr=2.86e-04 tps=16037 mem=6.2G lam=0.15374540848036608 Gm=None +[s_only s0] step 2320/4000 loss=5.2560 gn=0.68 lr=2.81e-04 tps=16041 mem=6.2G lam=0.15396491096665463 Gm=None +[s_only s0] step 2340/4000 loss=5.1864 gn=0.67 lr=2.77e-04 tps=16046 mem=6.2G lam=0.1541658298422893 Gm=None +[s_only s0] step 2360/4000 loss=5.2709 gn=0.64 lr=2.72e-04 tps=16050 mem=6.2G lam=0.15444632650663456 Gm=None +[s_only s0] step 2380/4000 loss=5.1435 gn=0.71 lr=2.68e-04 tps=16054 mem=6.2G lam=0.15467509596298137 Gm=None +[s_only s0] step 2400/4000 loss=5.0452 gn=0.66 lr=2.64e-04 tps=16059 mem=6.2G lam=0.1549065876752138 Gm=None +[s_only s0] EVAL step 2400: val_loss=5.2244 val_ppl=185.75 +[s_only s0] step 2420/4000 loss=5.0378 gn=0.65 lr=2.59e-04 tps=16015 mem=6.2G lam=0.1551545119533936 Gm=None +[s_only s0] step 2440/4000 loss=5.1605 gn=0.67 lr=2.55e-04 tps=16020 mem=6.2G lam=0.15540442988276482 Gm=None +[s_only s0] step 2460/4000 loss=5.2340 gn=0.67 lr=2.51e-04 tps=16024 mem=6.2G lam=0.15560725517570972 Gm=None +[s_only s0] step 2480/4000 loss=5.1356 gn=0.63 lr=2.47e-04 tps=16029 mem=6.2G lam=0.15582442035277685 Gm=None +[s_only s0] step 2500/4000 loss=4.9740 gn=0.65 lr=2.42e-04 tps=16033 mem=6.2G lam=0.15603429606805244 Gm=None +[s_only s0] step 2520/4000 loss=5.0311 gn=0.63 lr=2.38e-04 tps=16038 mem=6.2G lam=0.15623840938011804 Gm=None +[s_only s0] step 2540/4000 loss=5.1130 gn=0.64 lr=2.34e-04 tps=16042 mem=6.2G lam=0.15647442545741796 Gm=None +[s_only s0] step 2560/4000 loss=4.9190 gn=0.71 lr=2.30e-04 tps=16047 mem=6.2G lam=0.15665741016467413 Gm=None +[s_only s0] step 2580/4000 loss=5.1944 gn=0.71 lr=2.26e-04 tps=16051 mem=6.2G lam=0.15685578901320696 Gm=None +[s_only s0] step 2600/4000 loss=4.9774 gn=0.65 lr=2.22e-04 tps=16055 mem=6.2G lam=0.1570213104908665 Gm=None +[s_only s0] EVAL step 2600: val_loss=5.1874 val_ppl=179.00 +[s_only s0] step 2620/4000 loss=5.1547 gn=0.72 lr=2.17e-04 tps=16014 mem=6.2G lam=0.15720977572103342 Gm=None +[s_only s0] step 2640/4000 loss=5.1468 gn=0.77 lr=2.13e-04 tps=16018 mem=6.2G lam=0.1574209677055478 Gm=None +[s_only s0] step 2660/4000 loss=5.0151 gn=0.65 lr=2.09e-04 tps=16023 mem=6.2G lam=0.15761489141732454 Gm=None +[s_only s0] step 2680/4000 loss=4.8517 gn=0.86 lr=2.05e-04 tps=16027 mem=6.2G lam=0.15779203393807015 Gm=None +[s_only s0] step 2700/4000 loss=5.1029 gn=0.68 lr=2.01e-04 tps=16031 mem=6.2G lam=0.1579585187137127 Gm=None +[s_only s0] step 2720/4000 loss=5.0902 gn=0.68 lr=1.98e-04 tps=16035 mem=6.2G lam=0.15815662002811828 Gm=None +[s_only s0] step 2740/4000 loss=5.0854 gn=0.65 lr=1.94e-04 tps=16039 mem=6.2G lam=0.15834523302813372 Gm=None +[s_only s0] step 2760/4000 loss=4.9313 gn=0.66 lr=1.90e-04 tps=16043 mem=6.2G lam=0.15851814641306797 Gm=None +[s_only s0] step 2780/4000 loss=5.0645 gn=0.64 lr=1.86e-04 tps=16046 mem=6.2G lam=0.15868712433924279 Gm=None +[s_only s0] step 2800/4000 loss=5.0114 gn=0.72 lr=1.82e-04 tps=16050 mem=6.2G lam=0.15886648620168367 Gm=None +[s_only s0] EVAL step 2800: val_loss=5.1255 val_ppl=168.25 +[s_only s0] step 2820/4000 loss=5.0534 gn=0.65 lr=1.79e-04 tps=16013 mem=6.2G lam=0.1590363153566917 Gm=None +[s_only s0] step 2840/4000 loss=4.8508 gn=0.69 lr=1.75e-04 tps=16017 mem=6.2G lam=0.1591800656169653 Gm=None +[s_only s0] step 2860/4000 loss=5.0366 gn=0.64 lr=1.71e-04 tps=16020 mem=6.2G lam=0.15933304311086735 Gm=None +[s_only s0] step 2880/4000 loss=5.0254 gn=0.66 lr=1.68e-04 tps=16024 mem=6.2G lam=0.15949062475313744 Gm=None +[s_only s0] step 2900/4000 loss=4.8737 gn=0.72 lr=1.64e-04 tps=16028 mem=6.2G lam=0.15964605690290531 Gm=None +[s_only s0] step 2920/4000 loss=5.0762 gn=0.74 lr=1.61e-04 tps=16032 mem=6.2G lam=0.1597970174625516 Gm=None +[s_only s0] step 2940/4000 loss=4.9907 gn=0.68 lr=1.57e-04 tps=16036 mem=6.2G lam=0.15993541137625775 Gm=None +[s_only s0] step 2960/4000 loss=4.9784 gn=0.68 lr=1.54e-04 tps=16040 mem=6.2G lam=0.16006658226251602 Gm=None +[s_only s0] step 2980/4000 loss=4.9393 gn=0.64 lr=1.50e-04 tps=16044 mem=6.2G lam=0.16020564020921788 Gm=None +[s_only s0] step 3000/4000 loss=4.8949 gn=0.69 lr=1.47e-04 tps=16048 mem=6.2G lam=0.1603458309546113 Gm=None +[s_only s0] EVAL step 3000: val_loss=5.1437 val_ppl=171.34 +[s_only s0] step 3020/4000 loss=4.9444 gn=0.68 lr=1.44e-04 tps=16013 mem=6.2G lam=0.1604727047185103 Gm=None +[s_only s0] step 3040/4000 loss=5.0316 gn=0.69 lr=1.41e-04 tps=16016 mem=6.2G lam=0.16060379706323147 Gm=None +[s_only s0] step 3060/4000 loss=4.9870 gn=0.74 lr=1.38e-04 tps=16020 mem=6.2G lam=0.1607404612004757 Gm=None +[s_only s0] step 3080/4000 loss=4.8295 gn=0.66 lr=1.34e-04 tps=16023 mem=6.2G lam=0.16087450087070465 Gm=None +[s_only s0] step 3100/4000 loss=5.0511 gn=0.69 lr=1.31e-04 tps=16027 mem=6.2G lam=0.16098162159323692 Gm=None +[s_only s0] step 3120/4000 loss=5.0937 gn=1.61 lr=1.28e-04 tps=16030 mem=6.2G lam=0.1611141012981534 Gm=None +[s_only s0] step 3140/4000 loss=4.9128 gn=0.69 lr=1.25e-04 tps=16034 mem=6.2G lam=0.16125171103825173 Gm=None +[s_only s0] step 3160/4000 loss=5.0753 gn=0.70 lr=1.23e-04 tps=16037 mem=6.2G lam=0.16136677749454975 Gm=None +[s_only s0] step 3180/4000 loss=4.8985 gn=0.65 lr=1.20e-04 tps=16041 mem=6.2G lam=0.16147833845267692 Gm=None +[s_only s0] step 3200/4000 loss=4.9372 gn=0.73 lr=1.17e-04 tps=16044 mem=6.2G lam=0.16159397643059492 Gm=None +[s_only s0] EVAL step 3200: val_loss=5.0508 val_ppl=156.14 +[s_only s0] step 3220/4000 loss=5.0421 gn=0.77 lr=1.14e-04 tps=16012 mem=6.2G lam=0.16170350648462772 Gm=None +[s_only s0] step 3240/4000 loss=5.0556 gn=0.68 lr=1.12e-04 tps=16015 mem=6.2G lam=0.16180988680571318 Gm=None +[s_only s0] step 3260/4000 loss=4.8043 gn=0.69 lr=1.09e-04 tps=16018 mem=6.2G lam=0.16191981422404447 Gm=None +[s_only s0] step 3280/4000 loss=5.0738 gn=0.77 lr=1.06e-04 tps=16022 mem=6.2G lam=0.1620280851299564 Gm=None +[s_only s0] step 3300/4000 loss=4.8867 gn=0.73 lr=1.04e-04 tps=16025 mem=6.2G lam=0.16213232837617397 Gm=None +[s_only s0] step 3320/4000 loss=5.0384 gn=0.73 lr=1.02e-04 tps=16028 mem=6.2G lam=0.16222489128510156 Gm=None +[s_only s0] step 3340/4000 loss=4.8808 gn=0.68 lr=9.92e-05 tps=16031 mem=6.2G lam=0.16232431990404925 Gm=None +[s_only s0] step 3360/4000 loss=5.0324 gn=0.69 lr=9.69e-05 tps=16034 mem=6.2G lam=0.162418513558805 Gm=None +[s_only s0] step 3380/4000 loss=5.0188 gn=0.73 lr=9.47e-05 tps=16038 mem=6.2G lam=0.16249898045013347 Gm=None +[s_only s0] step 3400/4000 loss=4.8903 gn=0.68 lr=9.25e-05 tps=16041 mem=6.2G lam=0.1626018676906824 Gm=None +[s_only s0] EVAL step 3400: val_loss=5.0540 val_ppl=156.65 +[s_only s0] step 3420/4000 loss=4.9338 gn=0.71 lr=9.04e-05 tps=16010 mem=6.2G lam=0.1626981602360805 Gm=None +[s_only s0] step 3440/4000 loss=4.9736 gn=0.79 lr=8.84e-05 tps=16014 mem=6.2G lam=0.16279214837898812 Gm=None +[s_only s0] step 3460/4000 loss=4.8672 gn=0.74 lr=8.65e-05 tps=16017 mem=6.2G lam=0.1628744974732399 Gm=None +[s_only s0] step 3480/4000 loss=4.9579 gn=0.74 lr=8.46e-05 tps=16020 mem=6.2G lam=0.16295635203520456 Gm=None +[s_only s0] step 3500/4000 loss=4.8617 gn=0.71 lr=8.27e-05 tps=16023 mem=6.2G lam=0.16303630731999874 Gm=None +[s_only s0] step 3520/4000 loss=4.7774 gn=0.74 lr=8.10e-05 tps=16027 mem=6.2G lam=0.16311296044538418 Gm=None +[s_only s0] step 3540/4000 loss=4.7582 gn=0.70 lr=7.93e-05 tps=16030 mem=6.2G lam=0.16320041629175344 Gm=None +[s_only s0] step 3560/4000 loss=5.0014 gn=0.70 lr=7.77e-05 tps=16033 mem=6.2G lam=0.16328275483101606 Gm=None +[s_only s0] step 3580/4000 loss=4.8396 gn=0.69 lr=7.61e-05 tps=16036 mem=6.2G lam=0.163363772444427 Gm=None +[s_only s0] step 3600/4000 loss=4.8693 gn=0.80 lr=7.46e-05 tps=16039 mem=6.2G lam=0.16343969789644083 Gm=None +[s_only s0] EVAL step 3600: val_loss=5.0216 val_ppl=151.66 +[s_only s0] step 3620/4000 loss=4.9188 gn=0.75 lr=7.32e-05 tps=16009 mem=6.2G lam=0.16351322084665298 Gm=None +[s_only s0] step 3640/4000 loss=4.9910 gn=0.72 lr=7.19e-05 tps=16012 mem=6.2G lam=0.16358474300553402 Gm=None +[s_only s0] step 3660/4000 loss=5.0379 gn=0.80 lr=7.06e-05 tps=16016 mem=6.2G lam=0.16365407779812813 Gm=None +[s_only s0] step 3680/4000 loss=4.9527 gn=0.71 lr=6.94e-05 tps=16019 mem=6.2G lam=0.16372491512447596 Gm=None +[s_only s0] step 3700/4000 loss=5.0138 gn=0.69 lr=6.83e-05 tps=16021 mem=6.2G lam=0.16379747508714596 Gm=None +[s_only s0] step 3720/4000 loss=4.7727 gn=0.70 lr=6.72e-05 tps=16024 mem=6.2G lam=0.16386881253371635 Gm=None +[s_only s0] step 3740/4000 loss=4.9555 gn=0.71 lr=6.62e-05 tps=16027 mem=6.2G lam=0.1639383789151907 Gm=None +[s_only s0] step 3760/4000 loss=4.9015 gn=0.71 lr=6.53e-05 tps=16030 mem=6.2G lam=0.16399934142827988 Gm=None +[s_only s0] step 3780/4000 loss=4.8116 gn=0.73 lr=6.45e-05 tps=16033 mem=6.2G lam=0.16407388728111982 Gm=None +[s_only s0] step 3800/4000 loss=4.9232 gn=0.77 lr=6.37e-05 tps=16036 mem=6.2G lam=0.16412982872376838 Gm=None +[s_only s0] EVAL step 3800: val_loss=5.0013 val_ppl=148.60 +[s_only s0] step 3820/4000 loss=4.9799 gn=0.76 lr=6.30e-05 tps=16006 mem=6.2G lam=0.1641976566364368 Gm=None +[s_only s0] step 3840/4000 loss=4.8974 gn=0.75 lr=6.24e-05 tps=16008 mem=6.2G lam=0.16426111416270336 Gm=None +[s_only s0] step 3860/4000 loss=4.9083 gn=0.70 lr=6.18e-05 tps=16009 mem=6.2G lam=0.1643272421012322 Gm=None +[s_only s0] step 3880/4000 loss=4.9749 gn=0.78 lr=6.13e-05 tps=16011 mem=6.2G lam=0.1643884942556421 Gm=None +[s_only s0] step 3900/4000 loss=4.8772 gn=0.77 lr=6.09e-05 tps=16012 mem=6.2G lam=0.164447876624763 Gm=None +[s_only s0] step 3920/4000 loss=4.8052 gn=0.78 lr=6.06e-05 tps=16013 mem=6.2G lam=0.16450831635544697 Gm=None +[s_only s0] step 3940/4000 loss=4.9839 gn=0.77 lr=6.03e-05 tps=16014 mem=6.2G lam=0.1645667472233375 Gm=None +[s_only s0] step 3960/4000 loss=4.9248 gn=0.77 lr=6.01e-05 tps=16014 mem=6.2G lam=0.16462658531963825 Gm=None +[s_only s0] step 3980/4000 loss=4.8933 gn=0.82 lr=6.00e-05 tps=16014 mem=6.2G lam=0.16468850150704384 Gm=None +[s_only s0] EVAL step 3999: val_loss=4.9965 val_ppl=147.89 +[init] variant=g_only seed=0 params=63.39M +[init] use_S=False use_G=True +[g_only s0] step 0/4000 loss=10.9116 gn=10.42 lr=3.00e-06 tps=7314 mem=5.8G lam=None Gm=0.4969075520833333 +[g_only s0] step 20/4000 loss=9.8400 gn=2.09 lr=6.30e-05 tps=16617 mem=6.3G lam=None Gm=0.4551595052083333 +[g_only s0] step 40/4000 loss=9.0910 gn=1.78 lr=1.23e-04 tps=17205 mem=6.3G lam=None Gm=0.4195963541666667 +[g_only s0] step 60/4000 loss=8.0935 gn=1.55 lr=1.83e-04 tps=17418 mem=6.3G lam=None Gm=0.4182942708333333 +[g_only s0] step 80/4000 loss=7.4238 gn=0.78 lr=2.43e-04 tps=17527 mem=6.3G lam=None Gm=0.4287923177083333 +[g_only s0] step 100/4000 loss=7.2057 gn=0.72 lr=3.03e-04 tps=17595 mem=6.3G lam=None Gm=0.4186197916666667 +[g_only s0] step 120/4000 loss=7.0529 gn=0.57 lr=3.63e-04 tps=17643 mem=6.3G lam=None Gm=0.4054361979166667 +[g_only s0] step 140/4000 loss=6.8452 gn=0.84 lr=4.23e-04 tps=17671 mem=6.3G lam=None Gm=0.4053548177083333 +[g_only s0] step 160/4000 loss=6.9112 gn=0.60 lr=4.83e-04 tps=17691 mem=6.3G lam=None Gm=0.4039713541666667 +[g_only s0] step 180/4000 loss=6.7947 gn=0.49 lr=5.43e-04 tps=17707 mem=6.3G lam=None Gm=0.3907063802083333 +[g_only s0] step 200/4000 loss=6.6789 gn=0.54 lr=6.00e-04 tps=17716 mem=6.3G lam=None Gm=0.4029947916666667 +[g_only s0] EVAL step 200: val_loss=6.7469 val_ppl=851.43 +[g_only s0] step 220/4000 loss=6.5800 gn=0.53 lr=6.00e-04 tps=17149 mem=6.3G lam=None Gm=0.39599609375 +[g_only s0] step 240/4000 loss=6.6272 gn=0.50 lr=6.00e-04 tps=17204 mem=6.3G lam=None Gm=0.3841145833333333 +[g_only s0] step 260/4000 loss=6.5962 gn=0.51 lr=6.00e-04 tps=17252 mem=6.3G lam=None Gm=0.3935546875 +[g_only s0] step 280/4000 loss=6.5161 gn=0.47 lr=5.99e-04 tps=17292 mem=6.3G lam=None Gm=0.3889973958333333 +[g_only s0] step 300/4000 loss=6.3437 gn=0.49 lr=5.99e-04 tps=17325 mem=6.3G lam=None Gm=0.4014485677083333 +[g_only s0] step 320/4000 loss=6.4121 gn=0.48 lr=5.99e-04 tps=17354 mem=6.3G lam=None Gm=0.4002278645833333 +[g_only s0] step 340/4000 loss=6.3941 gn=0.46 lr=5.98e-04 tps=17380 mem=6.3G lam=None Gm=0.3993326822916667 +[g_only s0] step 360/4000 loss=6.2629 gn=0.47 lr=5.98e-04 tps=17403 mem=6.3G lam=None Gm=0.3941243489583333 +[g_only s0] step 380/4000 loss=6.2178 gn=0.50 lr=5.97e-04 tps=17425 mem=6.3G lam=None Gm=0.4176432291666667 +[g_only s0] step 400/4000 loss=6.2702 gn=0.50 lr=5.96e-04 tps=17444 mem=6.3G lam=None Gm=0.41162109375 +[g_only s0] EVAL step 400: val_loss=6.4036 val_ppl=604.04 +[g_only s0] step 420/4000 loss=6.4023 gn=0.70 lr=5.96e-04 tps=17163 mem=6.3G lam=None Gm=0.4402669270833333 +[g_only s0] step 440/4000 loss=6.3540 gn=0.56 lr=5.95e-04 tps=17192 mem=6.3G lam=None Gm=0.4210611979166667 +[g_only s0] step 460/4000 loss=6.3021 gn=0.47 lr=5.94e-04 tps=17216 mem=6.3G lam=None Gm=0.416015625 +[g_only s0] step 480/4000 loss=6.3700 gn=0.49 lr=5.93e-04 tps=17240 mem=6.3G lam=None Gm=0.4447428385416667 +[g_only s0] step 500/4000 loss=6.3026 gn=0.49 lr=5.92e-04 tps=17262 mem=6.3G lam=None Gm=0.44580078125 +[g_only s0] step 520/4000 loss=6.1557 gn=0.47 lr=5.91e-04 tps=17283 mem=6.3G lam=None Gm=0.4806315104166667 +[g_only s0] step 540/4000 loss=6.1738 gn=0.46 lr=5.89e-04 tps=17302 mem=6.3G lam=None Gm=0.4768880208333333 +[g_only s0] step 560/4000 loss=6.0412 gn=0.56 lr=5.88e-04 tps=17321 mem=6.3G lam=None Gm=0.4951985677083333 +[g_only s0] step 580/4000 loss=6.2063 gn=0.54 lr=5.87e-04 tps=17338 mem=6.3G lam=None Gm=0.5145670572916666 +[g_only s0] step 600/4000 loss=5.9606 gn=0.48 lr=5.85e-04 tps=17353 mem=6.3G lam=None Gm=0.526123046875 +[g_only s0] EVAL step 600: val_loss=6.1682 val_ppl=477.31 +[g_only s0] step 620/4000 loss=6.0180 gn=0.48 lr=5.84e-04 tps=17168 mem=6.3G lam=None Gm=0.5377604166666666 +[g_only s0] step 640/4000 loss=6.1040 gn=0.61 lr=5.82e-04 tps=17189 mem=6.3G lam=None Gm=0.5563151041666666 +[g_only s0] step 660/4000 loss=5.9488 gn=0.48 lr=5.81e-04 tps=17208 mem=6.3G lam=None Gm=0.5437825520833334 +[g_only s0] step 680/4000 loss=5.9541 gn=0.48 lr=5.79e-04 tps=17225 mem=6.3G lam=None Gm=0.5411783854166666 +[g_only s0] step 700/4000 loss=6.0356 gn=0.49 lr=5.77e-04 tps=17242 mem=6.3G lam=None Gm=0.581298828125 +[g_only s0] step 720/4000 loss=6.0191 gn=0.51 lr=5.75e-04 tps=17258 mem=6.3G lam=None Gm=0.5633138020833334 +[g_only s0] step 740/4000 loss=6.0332 gn=0.50 lr=5.74e-04 tps=17273 mem=6.3G lam=None Gm=0.5607096354166666 +[g_only s0] step 760/4000 loss=5.9036 gn=0.51 lr=5.72e-04 tps=17287 mem=6.3G lam=None Gm=0.5413411458333334 +[g_only s0] step 780/4000 loss=5.9232 gn=0.51 lr=5.70e-04 tps=17301 mem=6.3G lam=None Gm=0.5846354166666666 +[g_only s0] step 800/4000 loss=5.8875 gn=0.56 lr=5.67e-04 tps=17314 mem=6.3G lam=None Gm=0.596923828125 +[g_only s0] EVAL step 800: val_loss=5.9858 val_ppl=397.74 +[g_only s0] step 820/4000 loss=5.9567 gn=0.45 lr=5.65e-04 tps=17172 mem=6.3G lam=None Gm=0.5618489583333334 +[g_only s0] step 840/4000 loss=6.1340 gn=0.50 lr=5.63e-04 tps=17187 mem=6.3G lam=None Gm=0.5763346354166666 +[g_only s0] step 860/4000 loss=5.9471 gn=0.53 lr=5.61e-04 tps=17202 mem=6.3G lam=None Gm=0.5804036458333334 +[g_only s0] step 880/4000 loss=5.9260 gn=0.53 lr=5.58e-04 tps=17214 mem=6.3G lam=None Gm=0.5691731770833334 +[g_only s0] step 900/4000 loss=5.8427 gn=0.48 lr=5.56e-04 tps=17225 mem=6.3G lam=None Gm=0.5870768229166666 +[g_only s0] step 920/4000 loss=5.7483 gn=0.53 lr=5.54e-04 tps=17235 mem=6.3G lam=None Gm=0.591552734375 +[g_only s0] step 940/4000 loss=5.7881 gn=0.49 lr=5.51e-04 tps=17243 mem=6.3G lam=None Gm=0.5843912760416666 +[g_only s0] step 960/4000 loss=5.7854 gn=0.57 lr=5.48e-04 tps=17250 mem=6.3G lam=None Gm=0.602783203125 +[g_only s0] step 980/4000 loss=5.7003 gn=0.50 lr=5.46e-04 tps=17255 mem=6.3G lam=None Gm=0.5874837239583334 +[g_only s0] step 1000/4000 loss=5.7974 gn=0.52 lr=5.43e-04 tps=17263 mem=6.3G lam=None Gm=0.5813802083333334 +[g_only s0] EVAL step 1000: val_loss=5.8254 val_ppl=338.79 +[g_only s0] step 1020/4000 loss=5.5420 gn=0.50 lr=5.40e-04 tps=17146 mem=6.3G lam=None Gm=0.577880859375 +[g_only s0] step 1040/4000 loss=5.7812 gn=0.48 lr=5.37e-04 tps=17156 mem=6.3G lam=None Gm=0.5787353515625 +[g_only s0] step 1060/4000 loss=5.4968 gn=0.50 lr=5.35e-04 tps=17165 mem=6.3G lam=None Gm=0.5699869791666666 +[g_only s0] step 1080/4000 loss=5.7063 gn=0.53 lr=5.32e-04 tps=17175 mem=6.3G lam=None Gm=0.5565592447916666 +[g_only s0] step 1100/4000 loss=5.6721 gn=0.49 lr=5.29e-04 tps=17183 mem=6.3G lam=None Gm=0.5721435546875 +[g_only s0] step 1120/4000 loss=5.6122 gn=0.53 lr=5.26e-04 tps=17192 mem=6.3G lam=None Gm=0.5660807291666666 +[g_only s0] step 1140/4000 loss=5.8697 gn=0.54 lr=5.22e-04 tps=17200 mem=6.3G lam=None Gm=0.5578206380208334 +[g_only s0] step 1160/4000 loss=5.6152 gn=0.52 lr=5.19e-04 tps=17208 mem=6.3G lam=None Gm=0.5635579427083334 +[g_only s0] step 1180/4000 loss=5.6635 gn=0.51 lr=5.16e-04 tps=17216 mem=6.3G lam=None Gm=0.5602213541666666 +[g_only s0] step 1200/4000 loss=5.5461 gn=0.51 lr=5.13e-04 tps=17223 mem=6.3G lam=None Gm=0.5563557942708334 +[g_only s0] EVAL step 1200: val_loss=5.6726 val_ppl=290.78 +[g_only s0] step 1220/4000 loss=5.5768 gn=0.53 lr=5.10e-04 tps=17129 mem=6.3G lam=None Gm=0.5413818359375 +[g_only s0] step 1240/4000 loss=5.4647 gn=0.51 lr=5.06e-04 tps=17137 mem=6.3G lam=None Gm=0.5687255859375 +[g_only s0] step 1260/4000 loss=5.4552 gn=0.52 lr=5.03e-04 tps=17146 mem=6.3G lam=None Gm=0.5423177083333334 +[g_only s0] step 1280/4000 loss=5.5719 gn=0.58 lr=4.99e-04 tps=17154 mem=6.3G lam=None Gm=0.5581461588541666 +[g_only s0] step 1300/4000 loss=5.6728 gn=0.56 lr=4.96e-04 tps=17162 mem=6.3G lam=None Gm=0.5564371744791666 +[g_only s0] step 1320/4000 loss=5.6312 gn=0.57 lr=4.92e-04 tps=17169 mem=6.3G lam=None Gm=0.5655517578125 +[g_only s0] step 1340/4000 loss=5.7253 gn=0.54 lr=4.89e-04 tps=17176 mem=6.3G lam=None Gm=0.5574544270833334 +[g_only s0] step 1360/4000 loss=5.5400 gn=0.56 lr=4.85e-04 tps=17184 mem=6.3G lam=None Gm=0.5557861328125 +[g_only s0] step 1380/4000 loss=5.4131 gn=0.54 lr=4.81e-04 tps=17192 mem=6.3G lam=None Gm=0.5404052734375 +[g_only s0] step 1400/4000 loss=5.4202 gn=0.61 lr=4.78e-04 tps=17198 mem=6.3G lam=None Gm=0.5691731770833334 +[g_only s0] EVAL step 1400: val_loss=5.5586 val_ppl=259.45 +[g_only s0] step 1420/4000 loss=5.4152 gn=0.60 lr=4.74e-04 tps=17118 mem=6.3G lam=None Gm=0.5450846354166666 +[g_only s0] step 1440/4000 loss=5.3993 gn=0.56 lr=4.70e-04 tps=17127 mem=6.3G lam=None Gm=0.5409342447916666 +[g_only s0] step 1460/4000 loss=5.5153 gn=0.56 lr=4.66e-04 tps=17134 mem=6.3G lam=None Gm=0.5529378255208334 +[g_only s0] step 1480/4000 loss=5.5728 gn=0.57 lr=4.62e-04 tps=17141 mem=6.3G lam=None Gm=0.5250244140625 +[g_only s0] step 1500/4000 loss=5.4834 gn=0.60 lr=4.59e-04 tps=17149 mem=6.3G lam=None Gm=0.5370279947916666 +[g_only s0] step 1520/4000 loss=5.4469 gn=0.53 lr=4.55e-04 tps=17157 mem=6.3G lam=None Gm=0.5386555989583334 +[g_only s0] step 1540/4000 loss=5.3628 gn=0.52 lr=4.51e-04 tps=17164 mem=6.3G lam=None Gm=0.5360514322916666 +[g_only s0] step 1560/4000 loss=5.3563 gn=0.52 lr=4.47e-04 tps=17171 mem=6.3G lam=None Gm=0.5361328125 +[g_only s0] step 1580/4000 loss=5.4303 gn=0.58 lr=4.43e-04 tps=17178 mem=6.3G lam=None Gm=0.5266520182291666 +[g_only s0] step 1600/4000 loss=5.4288 gn=0.55 lr=4.38e-04 tps=17185 mem=6.3G lam=None Gm=0.5325520833333334 +[g_only s0] EVAL step 1600: val_loss=5.4864 val_ppl=241.38 +[g_only s0] step 1620/4000 loss=5.3893 gn=0.52 lr=4.34e-04 tps=17116 mem=6.3G lam=None Gm=0.54119873046875 +[g_only s0] step 1640/4000 loss=5.2717 gn=0.53 lr=4.30e-04 tps=17124 mem=6.3G lam=None Gm=0.52777099609375 +[g_only s0] step 1660/4000 loss=5.4406 gn=0.58 lr=4.26e-04 tps=17132 mem=6.3G lam=None Gm=0.5328776041666666 +[g_only s0] step 1680/4000 loss=5.5967 gn=0.55 lr=4.22e-04 tps=17140 mem=6.3G lam=None Gm=0.5220540364583334 +[g_only s0] step 1700/4000 loss=5.4539 gn=0.58 lr=4.18e-04 tps=17147 mem=6.3G lam=None Gm=0.5311482747395834 +[g_only s0] step 1720/4000 loss=5.2196 gn=0.56 lr=4.13e-04 tps=17154 mem=6.3G lam=None Gm=0.5096638997395834 +[g_only s0] step 1740/4000 loss=5.2746 gn=0.56 lr=4.09e-04 tps=17162 mem=6.3G lam=None Gm=0.5119222005208334 +[g_only s0] step 1760/4000 loss=5.3570 gn=0.54 lr=4.05e-04 tps=17169 mem=6.3G lam=None Gm=0.50897216796875 +[g_only s0] step 1780/4000 loss=5.2094 gn=0.54 lr=4.01e-04 tps=17175 mem=6.3G lam=None Gm=0.5138956705729166 +[g_only s0] step 1800/4000 loss=5.2770 gn=0.58 lr=3.96e-04 tps=17181 mem=6.3G lam=None Gm=0.5162760416666666 +[g_only s0] EVAL step 1800: val_loss=5.4124 val_ppl=224.17 +[g_only s0] step 1820/4000 loss=5.3734 gn=0.59 lr=3.92e-04 tps=17119 mem=6.3G lam=None Gm=0.5155843098958334 +[g_only s0] step 1840/4000 loss=5.2164 gn=0.59 lr=3.88e-04 tps=17126 mem=6.3G lam=None Gm=0.5003458658854166 +[g_only s0] step 1860/4000 loss=5.2742 gn=0.64 lr=3.83e-04 tps=17133 mem=6.3G lam=None Gm=0.5200398763020834 +[g_only s0] step 1880/4000 loss=5.3516 gn=0.59 lr=3.79e-04 tps=17139 mem=6.3G lam=None Gm=0.5229085286458334 +[g_only s0] step 1900/4000 loss=5.3081 gn=0.62 lr=3.74e-04 tps=17145 mem=6.3G lam=None Gm=0.5034586588541666 +[g_only s0] step 1920/4000 loss=5.2620 gn=0.57 lr=3.70e-04 tps=17151 mem=6.3G lam=None Gm=0.5080362955729166 +[g_only s0] step 1940/4000 loss=5.3117 gn=0.57 lr=3.66e-04 tps=17158 mem=6.3G lam=None Gm=0.5031331380208334 +[g_only s0] step 1960/4000 loss=5.5161 gn=0.62 lr=3.61e-04 tps=17163 mem=6.3G lam=None Gm=0.50909423828125 +[g_only s0] step 1980/4000 loss=5.4206 gn=0.57 lr=3.57e-04 tps=17170 mem=6.3G lam=None Gm=0.5130411783854166 +[g_only s0] step 2000/4000 loss=5.2534 gn=0.57 lr=3.52e-04 tps=17176 mem=6.3G lam=None Gm=0.5052286783854166 +[g_only s0] EVAL step 2000: val_loss=5.3724 val_ppl=215.37 +[g_only s0] step 2020/4000 loss=5.2267 gn=0.61 lr=3.48e-04 tps=17122 mem=6.3G lam=None Gm=0.5080159505208334 +[g_only s0] step 2040/4000 loss=5.3692 gn=0.66 lr=3.43e-04 tps=17129 mem=6.3G lam=None Gm=0.5000203450520834 +[g_only s0] step 2060/4000 loss=5.0994 gn=0.62 lr=3.39e-04 tps=17136 mem=6.3G lam=None Gm=0.5040283203125 +[g_only s0] step 2080/4000 loss=5.2446 gn=0.59 lr=3.34e-04 tps=17142 mem=6.3G lam=None Gm=0.5069580078125 +[g_only s0] step 2100/4000 loss=5.2432 gn=0.55 lr=3.30e-04 tps=17149 mem=6.3G lam=None Gm=0.49273681640625 +[g_only s0] step 2120/4000 loss=5.1525 gn=0.59 lr=3.26e-04 tps=17155 mem=6.3G lam=None Gm=0.4863688151041667 +[g_only s0] step 2140/4000 loss=5.0868 gn=0.60 lr=3.21e-04 tps=17161 mem=6.3G lam=None Gm=0.4986979166666667 +[g_only s0] step 2160/4000 loss=5.1276 gn=0.62 lr=3.17e-04 tps=17167 mem=6.3G lam=None Gm=0.4900105794270833 +[g_only s0] step 2180/4000 loss=5.1740 gn=0.65 lr=3.12e-04 tps=17173 mem=6.3G lam=None Gm=0.4993896484375 +[g_only s0] step 2200/4000 loss=5.3146 gn=0.60 lr=3.08e-04 tps=17179 mem=6.3G lam=None Gm=0.4974365234375 +[g_only s0] EVAL step 2200: val_loss=5.2782 val_ppl=196.02 +[g_only s0] step 2220/4000 loss=5.2071 gn=0.60 lr=3.03e-04 tps=17129 mem=6.3G lam=None Gm=0.4936726888020833 +[g_only s0] step 2240/4000 loss=5.1489 gn=0.57 lr=2.99e-04 tps=17134 mem=6.3G lam=None Gm=0.4821573893229167 +[g_only s0] step 2260/4000 loss=5.1579 gn=0.57 lr=2.94e-04 tps=17140 mem=6.3G lam=None Gm=0.4826456705729167 +[g_only s0] step 2280/4000 loss=5.3368 gn=0.64 lr=2.90e-04 tps=17146 mem=6.3G lam=None Gm=0.483154296875 +[g_only s0] step 2300/4000 loss=5.1231 gn=0.63 lr=2.86e-04 tps=17152 mem=6.3G lam=None Gm=0.4803059895833333 +[g_only s0] step 2320/4000 loss=5.2337 gn=0.63 lr=2.81e-04 tps=17157 mem=6.3G lam=None Gm=0.478759765625 +[g_only s0] step 2340/4000 loss=5.1670 gn=0.63 lr=2.77e-04 tps=17156 mem=6.3G lam=None Gm=0.483642578125 +[g_only s0] step 2360/4000 loss=5.2510 gn=0.61 lr=2.72e-04 tps=17150 mem=6.3G lam=None Gm=0.4729207356770833 +[g_only s0] step 2380/4000 loss=5.1356 gn=0.61 lr=2.68e-04 tps=17153 mem=6.3G lam=None Gm=0.48333740234375 +[g_only s0] step 2400/4000 loss=5.0308 gn=0.62 lr=2.64e-04 tps=17156 mem=6.3G lam=None Gm=0.4855143229166667 +[g_only s0] EVAL step 2400: val_loss=5.2118 val_ppl=183.43 +[g_only s0] step 2420/4000 loss=5.0347 gn=0.64 lr=2.59e-04 tps=17108 mem=6.3G lam=None Gm=0.47509765625 +[g_only s0] step 2440/4000 loss=5.1393 gn=0.62 lr=2.55e-04 tps=17111 mem=6.3G lam=None Gm=0.46600341796875 +[g_only s0] step 2460/4000 loss=5.2275 gn=0.62 lr=2.51e-04 tps=17115 mem=6.3G lam=None Gm=0.47979736328125 +[g_only s0] step 2480/4000 loss=5.1158 gn=0.61 lr=2.47e-04 tps=17119 mem=6.3G lam=None Gm=0.4676513671875 +[g_only s0] step 2500/4000 loss=4.9617 gn=0.61 lr=2.42e-04 tps=17123 mem=6.3G lam=None Gm=0.4730021158854167 +[g_only s0] step 2520/4000 loss=5.0374 gn=0.58 lr=2.38e-04 tps=17128 mem=6.3G lam=None Gm=0.46563720703125 +[g_only s0] step 2540/4000 loss=5.0929 gn=0.61 lr=2.34e-04 tps=17132 mem=6.3G lam=None Gm=0.4740397135416667 +[g_only s0] step 2560/4000 loss=4.8987 gn=0.66 lr=2.30e-04 tps=17136 mem=6.3G lam=None Gm=0.4639485677083333 +[g_only s0] step 2580/4000 loss=5.1882 gn=0.65 lr=2.26e-04 tps=17141 mem=6.3G lam=None Gm=0.4627888997395833 +[g_only s0] step 2600/4000 loss=4.9632 gn=0.61 lr=2.22e-04 tps=17146 mem=6.3G lam=None Gm=0.4612630208333333 +[g_only s0] EVAL step 2600: val_loss=5.1738 val_ppl=176.59 +[g_only s0] step 2620/4000 loss=5.1408 gn=0.67 lr=2.17e-04 tps=17103 mem=6.3G lam=None Gm=0.4739176432291667 +[g_only s0] step 2640/4000 loss=5.1478 gn=0.69 lr=2.13e-04 tps=17108 mem=6.3G lam=None Gm=0.4818522135416667 +[g_only s0] step 2660/4000 loss=5.0090 gn=0.63 lr=2.09e-04 tps=17113 mem=6.3G lam=None Gm=0.4649454752604167 +[g_only s0] step 2680/4000 loss=4.8474 gn=0.78 lr=2.05e-04 tps=17118 mem=6.3G lam=None Gm=0.4655558268229167 +[g_only s0] step 2700/4000 loss=5.0901 gn=0.69 lr=2.01e-04 tps=17122 mem=6.3G lam=None Gm=0.4690958658854167 +[g_only s0] step 2720/4000 loss=5.0884 gn=0.69 lr=1.98e-04 tps=17127 mem=6.3G lam=None Gm=0.4544677734375 +[g_only s0] step 2740/4000 loss=5.0685 gn=0.64 lr=1.94e-04 tps=17131 mem=6.3G lam=None Gm=0.4671630859375 +[g_only s0] step 2760/4000 loss=4.9182 gn=0.66 lr=1.90e-04 tps=17136 mem=6.3G lam=None Gm=0.4716796875 +[g_only s0] step 2780/4000 loss=5.0491 gn=0.64 lr=1.86e-04 tps=17140 mem=6.3G lam=None Gm=0.4642130533854167 +[g_only s0] step 2800/4000 loss=4.9962 gn=0.67 lr=1.82e-04 tps=17144 mem=6.3G lam=None Gm=0.4649658203125 +[g_only s0] EVAL step 2800: val_loss=5.1065 val_ppl=165.09 +[g_only s0] step 2820/4000 loss=5.0312 gn=0.65 lr=1.79e-04 tps=17104 mem=6.3G lam=None Gm=0.46160888671875 +[g_only s0] step 2840/4000 loss=4.8502 gn=0.68 lr=1.75e-04 tps=17108 mem=6.3G lam=None Gm=0.449462890625 +[g_only s0] step 2860/4000 loss=5.0163 gn=0.63 lr=1.71e-04 tps=17112 mem=6.3G lam=None Gm=0.4520263671875 +[g_only s0] step 2880/4000 loss=5.0066 gn=0.76 lr=1.68e-04 tps=17117 mem=6.3G lam=None Gm=0.4512125651041667 +[g_only s0] step 2900/4000 loss=4.8510 gn=0.65 lr=1.64e-04 tps=17121 mem=6.3G lam=None Gm=0.4589640299479167 +[g_only s0] step 2920/4000 loss=5.0601 gn=0.66 lr=1.61e-04 tps=17125 mem=6.3G lam=None Gm=0.4770711263020833 +[g_only s0] step 2940/4000 loss=4.9675 gn=0.62 lr=1.57e-04 tps=17129 mem=6.3G lam=None Gm=0.4576619466145833 +[g_only s0] step 2960/4000 loss=4.9508 gn=0.66 lr=1.54e-04 tps=17133 mem=6.3G lam=None Gm=0.4636637369791667 +[g_only s0] step 2980/4000 loss=4.9112 gn=0.64 lr=1.50e-04 tps=17137 mem=6.3G lam=None Gm=0.4538981119791667 +[g_only s0] step 3000/4000 loss=4.8637 gn=0.68 lr=1.47e-04 tps=17141 mem=6.3G lam=None Gm=0.4453328450520833 +[g_only s0] EVAL step 3000: val_loss=5.1212 val_ppl=167.53 +[g_only s0] step 3020/4000 loss=4.9321 gn=0.62 lr=1.44e-04 tps=17104 mem=6.3G lam=None Gm=0.4552815755208333 +[g_only s0] step 3040/4000 loss=5.0088 gn=0.71 lr=1.41e-04 tps=17109 mem=6.3G lam=None Gm=0.4568888346354167 +[g_only s0] step 3060/4000 loss=4.9715 gn=0.69 lr=1.38e-04 tps=17113 mem=6.3G lam=None Gm=0.4580485026041667 +[g_only s0] step 3080/4000 loss=4.8162 gn=0.64 lr=1.34e-04 tps=17118 mem=6.3G lam=None Gm=0.4505411783854167 +[g_only s0] step 3100/4000 loss=5.0214 gn=0.69 lr=1.31e-04 tps=17121 mem=6.3G lam=None Gm=0.4586181640625 +[g_only s0] step 3120/4000 loss=5.0558 gn=0.71 lr=1.28e-04 tps=17125 mem=6.3G lam=None Gm=0.4617106119791667 +[g_only s0] step 3140/4000 loss=4.9006 gn=0.65 lr=1.25e-04 tps=17130 mem=6.3G lam=None Gm=0.456298828125 +[g_only s0] step 3160/4000 loss=5.0574 gn=0.80 lr=1.23e-04 tps=17134 mem=6.3G lam=None Gm=0.462158203125 +[g_only s0] step 3180/4000 loss=4.8870 gn=0.65 lr=1.20e-04 tps=17138 mem=6.3G lam=None Gm=0.4506429036458333 +[g_only s0] step 3200/4000 loss=4.9075 gn=0.69 lr=1.17e-04 tps=17142 mem=6.3G lam=None Gm=0.45135498046875 +[g_only s0] EVAL step 3200: val_loss=5.0290 val_ppl=152.78 +[g_only s0] step 3220/4000 loss=5.0219 gn=0.74 lr=1.14e-04 tps=17108 mem=6.3G lam=None Gm=0.4473470052083333 +[g_only s0] step 3240/4000 loss=5.0297 gn=0.67 lr=1.12e-04 tps=17112 mem=6.3G lam=None Gm=0.4608561197916667 +[g_only s0] step 3260/4000 loss=4.7921 gn=0.67 lr=1.09e-04 tps=17116 mem=6.3G lam=None Gm=0.45697021484375 +[g_only s0] step 3280/4000 loss=5.0383 gn=0.70 lr=1.06e-04 tps=17120 mem=6.3G lam=None Gm=0.4521077473958333 +[g_only s0] step 3300/4000 loss=4.8601 gn=0.68 lr=1.04e-04 tps=17124 mem=6.3G lam=None Gm=0.4398193359375 +[g_only s0] step 3320/4000 loss=5.0289 gn=0.71 lr=1.02e-04 tps=17128 mem=6.3G lam=None Gm=0.4434407552083333 +[g_only s0] step 3340/4000 loss=4.8558 gn=0.67 lr=9.92e-05 tps=17131 mem=6.3G lam=None Gm=0.4430745442708333 +[g_only s0] step 3360/4000 loss=5.0094 gn=0.72 lr=9.69e-05 tps=17135 mem=6.3G lam=None Gm=0.445068359375 +[g_only s0] step 3380/4000 loss=4.9999 gn=0.69 lr=9.47e-05 tps=17139 mem=6.3G lam=None Gm=0.4417724609375 +[g_only s0] step 3400/4000 loss=4.8891 gn=0.68 lr=9.25e-05 tps=17142 mem=6.3G lam=None Gm=0.4418131510416667 +[g_only s0] EVAL step 3400: val_loss=5.0328 val_ppl=153.36 +[g_only s0] step 3420/4000 loss=4.9196 gn=0.69 lr=9.04e-05 tps=17110 mem=6.3G lam=None Gm=0.4532674153645833 +[g_only s0] step 3440/4000 loss=4.9692 gn=0.73 lr=8.84e-05 tps=17114 mem=6.3G lam=None Gm=0.4556070963541667 +[g_only s0] step 3460/4000 loss=4.8335 gn=0.70 lr=8.65e-05 tps=17118 mem=6.3G lam=None Gm=0.4396565755208333 +[g_only s0] step 3480/4000 loss=4.9484 gn=0.69 lr=8.46e-05 tps=17122 mem=6.3G lam=None Gm=0.4453125 +[g_only s0] step 3500/4000 loss=4.8527 gn=0.68 lr=8.27e-05 tps=17126 mem=6.3G lam=None Gm=0.4424031575520833 +[g_only s0] step 3520/4000 loss=4.7420 gn=0.74 lr=8.10e-05 tps=17130 mem=6.3G lam=None Gm=0.4512532552083333 +[g_only s0] step 3540/4000 loss=4.7381 gn=0.70 lr=7.93e-05 tps=17133 mem=6.3G lam=None Gm=0.4396565755208333 +[g_only s0] step 3560/4000 loss=4.9895 gn=0.68 lr=7.77e-05 tps=17137 mem=6.3G lam=None Gm=0.4422607421875 +[g_only s0] step 3580/4000 loss=4.8193 gn=0.66 lr=7.61e-05 tps=17141 mem=6.3G lam=None Gm=0.43359375 +[g_only s0] step 3600/4000 loss=4.8507 gn=0.76 lr=7.46e-05 tps=17145 mem=6.3G lam=None Gm=0.4462483723958333 +[g_only s0] EVAL step 3600: val_loss=4.9974 val_ppl=148.03 +[g_only s0] step 3620/4000 loss=4.9051 gn=0.72 lr=7.32e-05 tps=17115 mem=6.3G lam=None Gm=0.435791015625 +[g_only s0] step 3640/4000 loss=4.9773 gn=0.70 lr=7.19e-05 tps=17119 mem=6.3G lam=None Gm=0.4398600260416667 +[g_only s0] step 3660/4000 loss=5.0183 gn=0.80 lr=7.06e-05 tps=17123 mem=6.3G lam=None Gm=0.4494222005208333 +[g_only s0] step 3680/4000 loss=4.9392 gn=0.69 lr=6.94e-05 tps=17127 mem=6.3G lam=None Gm=0.435791015625 +[g_only s0] step 3700/4000 loss=4.9855 gn=0.68 lr=6.83e-05 tps=17130 mem=6.3G lam=None Gm=0.4390665690104167 +[g_only s0] step 3720/4000 loss=4.7607 gn=0.68 lr=6.72e-05 tps=17134 mem=6.3G lam=None Gm=0.43572998046875 +[g_only s0] step 3740/4000 loss=4.9413 gn=0.75 lr=6.62e-05 tps=17138 mem=6.3G lam=None Gm=0.43365478515625 +[g_only s0] step 3760/4000 loss=4.8730 gn=0.70 lr=6.53e-05 tps=17141 mem=6.3G lam=None Gm=0.4378662109375 +[g_only s0] step 3780/4000 loss=4.7835 gn=0.71 lr=6.45e-05 tps=17144 mem=6.3G lam=None Gm=0.4366455078125 +[g_only s0] step 3800/4000 loss=4.9004 gn=0.73 lr=6.37e-05 tps=17148 mem=6.3G lam=None Gm=0.4463297526041667 +[g_only s0] EVAL step 3800: val_loss=4.9756 val_ppl=144.84 +[g_only s0] step 3820/4000 loss=4.9450 gn=0.68 lr=6.30e-05 tps=17119 mem=6.3G lam=None Gm=0.4392903645833333 +[g_only s0] step 3840/4000 loss=4.8772 gn=0.69 lr=6.24e-05 tps=17123 mem=6.3G lam=None Gm=0.4384765625 +[g_only s0] step 3860/4000 loss=4.8904 gn=0.73 lr=6.18e-05 tps=17126 mem=6.3G lam=None Gm=0.4364217122395833 +[g_only s0] step 3880/4000 loss=4.9577 gn=0.74 lr=6.13e-05 tps=17129 mem=6.3G lam=None Gm=0.445556640625 +[g_only s0] step 3900/4000 loss=4.8484 gn=0.71 lr=6.09e-05 tps=17133 mem=6.3G lam=None Gm=0.4344685872395833 +[g_only s0] step 3920/4000 loss=4.7851 gn=0.75 lr=6.06e-05 tps=17136 mem=6.3G lam=None Gm=0.433837890625 +[g_only s0] step 3940/4000 loss=4.9613 gn=0.74 lr=6.03e-05 tps=17140 mem=6.3G lam=None Gm=0.4343668619791667 +[g_only s0] step 3960/4000 loss=4.9078 gn=0.72 lr=6.01e-05 tps=17143 mem=6.3G lam=None Gm=0.4307047526041667 +[g_only s0] step 3980/4000 loss=4.8536 gn=0.76 lr=6.00e-05 tps=17147 mem=6.3G lam=None Gm=0.4358113606770833 +[g_only s0] EVAL step 3999: val_loss=4.9765 val_ppl=144.97 +[init] variant=full seed=0 params=63.98M +[init] use_S=True use_G=True +[full s0] step 0/4000 loss=10.9002 gn=11.22 lr=3.00e-06 tps=7177 mem=5.9G lam=0.12692801654338837 Gm=0.5060585426787535 +[full s0] step 20/4000 loss=9.8395 gn=2.04 lr=6.30e-05 tps=14665 mem=6.4G lam=0.12691388465464115 Gm=0.46947087223331135 +[full s0] step 40/4000 loss=9.1119 gn=1.75 lr=1.23e-04 tps=15000 mem=6.4G lam=0.12681447404126325 Gm=0.4522867053747177 +[full s0] step 60/4000 loss=8.1301 gn=1.59 lr=1.83e-04 tps=15154 mem=6.4G lam=0.12670463137328625 Gm=0.44430506974458694 +[full s0] step 80/4000 loss=7.4261 gn=0.77 lr=2.43e-04 tps=15223 mem=6.4G lam=0.12660598816970983 Gm=0.4503638831277688 +[full s0] step 100/4000 loss=7.2137 gn=0.97 lr=3.03e-04 tps=15267 mem=6.4G lam=0.126494025811553 Gm=0.4433469759921233 +[full s0] step 120/4000 loss=7.0544 gn=0.63 lr=3.63e-04 tps=15301 mem=6.4G lam=0.12652451234559217 Gm=0.4225224206844966 +[full s0] step 140/4000 loss=6.8386 gn=0.58 lr=4.23e-04 tps=15322 mem=6.4G lam=0.12646274889508882 Gm=0.4161691988507907 +[full s0] step 160/4000 loss=6.9098 gn=0.79 lr=4.83e-04 tps=15327 mem=6.4G lam=0.12636082619428635 Gm=0.41686703140536946 +[full s0] step 180/4000 loss=6.7997 gn=0.61 lr=5.43e-04 tps=15329 mem=6.4G lam=0.12636513697604337 Gm=0.40593364586432773 +[full s0] step 200/4000 loss=6.6765 gn=0.48 lr=6.00e-04 tps=15341 mem=6.4G lam=0.12632009697457156 Gm=0.4146435211102168 +[full s0] EVAL step 200: val_loss=6.7406 val_ppl=846.05 +[full s0] step 220/4000 loss=6.5711 gn=0.49 lr=6.00e-04 tps=14849 mem=6.4G lam=0.12637140632917485 Gm=0.4111607000231743 +[full s0] step 240/4000 loss=6.6276 gn=0.55 lr=6.00e-04 tps=14896 mem=6.4G lam=0.12644176309307417 Gm=0.4052786926428477 +[full s0] step 260/4000 loss=6.5910 gn=0.50 lr=6.00e-04 tps=14935 mem=6.4G lam=0.12642955655852953 Gm=0.40804433325926465 +[full s0] step 280/4000 loss=6.5056 gn=0.49 lr=5.99e-04 tps=14969 mem=6.4G lam=0.12644678788880506 Gm=0.4035586975514889 +[full s0] step 300/4000 loss=6.3354 gn=0.47 lr=5.99e-04 tps=14999 mem=6.4G lam=0.1264594110349814 Gm=0.41425324728091556 +[full s0] step 320/4000 loss=6.4140 gn=0.53 lr=5.99e-04 tps=15024 mem=6.4G lam=0.12646780970195928 Gm=0.4137684901555379 +[full s0] step 340/4000 loss=6.3941 gn=0.53 lr=5.98e-04 tps=15047 mem=6.4G lam=0.12651423954715332 Gm=0.41517286996046704 +[full s0] step 360/4000 loss=6.2520 gn=0.48 lr=5.98e-04 tps=15071 mem=6.4G lam=0.1266111023724079 Gm=0.40464745213588077 +[full s0] step 380/4000 loss=6.2105 gn=0.51 lr=5.97e-04 tps=15091 mem=6.4G lam=0.12667184695601463 Gm=0.41207972665627796 +[full s0] step 400/4000 loss=6.2497 gn=0.48 lr=5.96e-04 tps=15109 mem=6.4G lam=0.12674504642685255 Gm=0.41418295974532765 +[full s0] EVAL step 400: val_loss=6.3913 val_ppl=596.65 +[full s0] step 420/4000 loss=6.3861 gn=0.66 lr=5.96e-04 tps=14860 mem=6.4G lam=0.1268323635061582 Gm=0.43083703393737477 +[full s0] step 440/4000 loss=6.3404 gn=0.53 lr=5.95e-04 tps=14884 mem=6.4G lam=0.1269418727606535 Gm=0.4135419428348541 +[full s0] step 460/4000 loss=6.2730 gn=0.50 lr=5.94e-04 tps=14907 mem=6.4G lam=0.12701714721818766 Gm=0.42971306045850116 +[full s0] step 480/4000 loss=6.3662 gn=0.48 lr=5.93e-04 tps=14927 mem=6.4G lam=0.12716670148074627 Gm=0.43850669637322426 +[full s0] step 500/4000 loss=6.2924 gn=0.50 lr=5.92e-04 tps=14946 mem=6.4G lam=0.12731456570327282 Gm=0.4456465107699235 +[full s0] step 520/4000 loss=6.1516 gn=0.52 lr=5.91e-04 tps=14963 mem=6.4G lam=0.12751984472076097 Gm=0.4579197404285272 +[full s0] step 540/4000 loss=6.1714 gn=0.48 lr=5.89e-04 tps=14981 mem=6.4G lam=0.12762735784053802 Gm=0.46552255749702454 +[full s0] step 560/4000 loss=6.0217 gn=0.51 lr=5.88e-04 tps=14998 mem=6.4G lam=0.127838846296072 Gm=0.4710191413760185 +[full s0] step 580/4000 loss=6.1941 gn=0.53 lr=5.87e-04 tps=15012 mem=6.4G lam=0.1280172864596049 Gm=0.4879833509524663 +[full s0] step 600/4000 loss=5.9497 gn=0.48 lr=5.85e-04 tps=15023 mem=6.4G lam=0.1282313127691547 Gm=0.5293142658968767 +[full s0] EVAL step 600: val_loss=6.1594 val_ppl=473.16 +[full s0] step 620/4000 loss=6.0152 gn=0.47 lr=5.84e-04 tps=14857 mem=6.4G lam=0.1284641269594431 Gm=0.5108194835484028 +[full s0] step 640/4000 loss=6.0979 gn=0.60 lr=5.82e-04 tps=14872 mem=6.4G lam=0.12870528859396776 Gm=0.5306381694972515 +[full s0] step 660/4000 loss=5.9328 gn=0.52 lr=5.81e-04 tps=14885 mem=6.4G lam=0.12897684735556444 Gm=0.569448359310627 +[full s0] step 680/4000 loss=5.9431 gn=0.50 lr=5.79e-04 tps=14900 mem=6.4G lam=0.12917988343785206 Gm=0.5352130594352881 +[full s0] step 700/4000 loss=6.0199 gn=0.45 lr=5.77e-04 tps=14917 mem=6.4G lam=0.12941528173784414 Gm=0.5478734436134497 +[full s0] step 720/4000 loss=6.0052 gn=0.50 lr=5.75e-04 tps=14928 mem=6.4G lam=0.12969180072347322 Gm=0.5475347439448038 +[full s0] step 740/4000 loss=6.0267 gn=0.49 lr=5.74e-04 tps=14940 mem=6.4G lam=0.13002433255314827 Gm=0.5399082625905672 +[full s0] step 760/4000 loss=5.8964 gn=0.54 lr=5.72e-04 tps=14954 mem=6.4G lam=0.13041084135572115 Gm=0.5374414511024952 +[full s0] step 780/4000 loss=5.9099 gn=0.47 lr=5.70e-04 tps=14966 mem=6.4G lam=0.13078376495589814 Gm=0.5692521668970585 +[full s0] step 800/4000 loss=5.8641 gn=0.48 lr=5.67e-04 tps=14977 mem=6.4G lam=0.13108982487271229 Gm=0.5646844009558359 +[full s0] EVAL step 800: val_loss=5.9699 val_ppl=391.45 +[full s0] step 820/4000 loss=5.9465 gn=0.50 lr=5.65e-04 tps=14853 mem=6.4G lam=0.13149288886537155 Gm=0.5422952758769194 +[full s0] step 840/4000 loss=6.1234 gn=0.49 lr=5.63e-04 tps=14864 mem=6.4G lam=0.13181905820965767 Gm=0.5669704626003901 +[full s0] step 860/4000 loss=5.9374 gn=0.54 lr=5.61e-04 tps=14873 mem=6.4G lam=0.1321754902601242 Gm=0.5561091440419356 +[full s0] step 880/4000 loss=5.9279 gn=0.59 lr=5.58e-04 tps=14883 mem=6.4G lam=0.13254042839010557 Gm=0.563983965665102 +[full s0] step 900/4000 loss=5.8279 gn=0.48 lr=5.56e-04 tps=14895 mem=6.4G lam=0.13288607758780321 Gm=0.565286461263895 +[full s0] step 920/4000 loss=5.7348 gn=0.49 lr=5.54e-04 tps=14906 mem=6.4G lam=0.13325560869028172 Gm=0.5610882913072904 +[full s0] step 940/4000 loss=5.7798 gn=0.52 lr=5.51e-04 tps=14916 mem=6.4G lam=0.13368192439277968 Gm=0.5797739451130232 +[full s0] step 960/4000 loss=5.7783 gn=0.58 lr=5.48e-04 tps=14926 mem=6.4G lam=0.1340254961202542 Gm=0.5871407128870487 +[full s0] step 980/4000 loss=5.6761 gn=0.49 lr=5.46e-04 tps=14935 mem=6.4G lam=0.13437930028885603 Gm=0.563348725438118 +[full s0] step 1000/4000 loss=5.7929 gn=0.52 lr=5.43e-04 tps=14946 mem=6.4G lam=0.13478029984980822 Gm=0.5567546946307024 +[full s0] EVAL step 1000: val_loss=5.8160 val_ppl=335.64 +[full s0] step 1020/4000 loss=5.5333 gn=0.50 lr=5.40e-04 tps=14848 mem=6.4G lam=0.13519225362688303 Gm=0.5625596257547537 +[full s0] step 1040/4000 loss=5.7766 gn=0.52 lr=5.37e-04 tps=14856 mem=6.4G lam=0.13549139443784952 Gm=0.57581627368927 +[full s0] step 1060/4000 loss=5.4969 gn=0.51 lr=5.35e-04 tps=14866 mem=6.4G lam=0.13587642088532448 Gm=0.532783313964804 +[full s0] step 1080/4000 loss=5.6966 gn=0.53 lr=5.32e-04 tps=14874 mem=6.4G lam=0.13626651465892792 Gm=0.5369457434862852 +[full s0] step 1100/4000 loss=5.6652 gn=0.60 lr=5.29e-04 tps=14883 mem=6.4G lam=0.13660530528674522 Gm=0.5775514940420786 +[full s0] step 1120/4000 loss=5.5989 gn=0.53 lr=5.26e-04 tps=14892 mem=6.4G lam=0.13700281300892433 Gm=0.5345892341186603 +[full s0] step 1140/4000 loss=5.8584 gn=0.53 lr=5.22e-04 tps=14899 mem=6.4G lam=0.1373259505877892 Gm=0.5394855334113041 +[full s0] step 1160/4000 loss=5.5913 gn=0.49 lr=5.19e-04 tps=14907 mem=6.4G lam=0.1377501521880428 Gm=0.54696986079216 +[full s0] step 1180/4000 loss=5.6599 gn=0.51 lr=5.16e-04 tps=14916 mem=6.4G lam=0.13811772099385658 Gm=0.5348725412040949 +[full s0] step 1200/4000 loss=5.5536 gn=0.50 lr=5.13e-04 tps=14924 mem=6.4G lam=0.13851364267369112 Gm=0.5457661300897598 +[full s0] EVAL step 1200: val_loss=5.6667 val_ppl=289.07 +[full s0] step 1220/4000 loss=5.5612 gn=0.54 lr=5.10e-04 tps=14842 mem=6.4G lam=0.13890980711827675 Gm=0.5422924067825079 +[full s0] step 1240/4000 loss=5.4509 gn=0.52 lr=5.06e-04 tps=14851 mem=6.4G lam=0.13927926247318587 Gm=0.5455869063735008 +[full s0] step 1260/4000 loss=5.4327 gn=0.49 lr=5.03e-04 tps=14858 mem=6.4G lam=0.13965522777289152 Gm=0.5358441267162561 +[full s0] step 1280/4000 loss=5.5771 gn=0.57 lr=4.99e-04 tps=14866 mem=6.4G lam=0.140042661378781 Gm=0.5416806563735008 +[full s0] step 1300/4000 loss=5.6711 gn=0.57 lr=4.96e-04 tps=14874 mem=6.4G lam=0.14043778398384651 Gm=0.539463714386026 +[full s0] step 1320/4000 loss=5.6225 gn=0.56 lr=4.92e-04 tps=14881 mem=6.4G lam=0.14081103230516115 Gm=0.534868306790789 +[full s0] step 1340/4000 loss=5.7384 gn=0.55 lr=4.89e-04 tps=14889 mem=6.4G lam=0.1412075301632285 Gm=0.537356911227107 +[full s0] step 1360/4000 loss=5.5310 gn=0.57 lr=4.85e-04 tps=14897 mem=6.4G lam=0.14158294132600227 Gm=0.5233028481403986 +[full s0] step 1380/4000 loss=5.4097 gn=0.53 lr=4.81e-04 tps=14904 mem=6.4G lam=0.14195760743071636 Gm=0.5073543048153321 +[full s0] step 1400/4000 loss=5.4238 gn=0.57 lr=4.78e-04 tps=14910 mem=6.4G lam=0.14229492781062922 Gm=0.524269043157498 +[full s0] EVAL step 1400: val_loss=5.5600 val_ppl=259.81 +[full s0] step 1420/4000 loss=5.3996 gn=0.56 lr=4.74e-04 tps=14840 mem=6.4G lam=0.14264872639129558 Gm=0.5051169519623121 +[full s0] step 1440/4000 loss=5.3948 gn=0.57 lr=4.70e-04 tps=14847 mem=6.4G lam=0.14301192946732044 Gm=0.5213417652994394 +[full s0] step 1460/4000 loss=5.5144 gn=0.59 lr=4.66e-04 tps=14853 mem=6.4G lam=0.1434003453080853 Gm=0.5063190547128519 +[full s0] step 1480/4000 loss=5.5660 gn=0.56 lr=4.62e-04 tps=14860 mem=6.4G lam=0.1437656677638491 Gm=0.5025668038676182 +[full s0] step 1500/4000 loss=5.4600 gn=0.57 lr=4.59e-04 tps=14866 mem=6.4G lam=0.14413854635010162 Gm=0.4969680073360602 +[full s0] step 1520/4000 loss=5.4459 gn=0.54 lr=4.55e-04 tps=14873 mem=6.4G lam=0.14453936771800122 Gm=0.5143021817008654 +[full s0] step 1540/4000 loss=5.3414 gn=0.51 lr=4.51e-04 tps=14880 mem=6.4G lam=0.1449366994202137 Gm=0.5127804862956206 +[full s0] step 1560/4000 loss=5.3644 gn=0.56 lr=4.47e-04 tps=14886 mem=6.4G lam=0.14528835751116276 Gm=0.4950882631043593 +[full s0] step 1580/4000 loss=5.4323 gn=0.62 lr=4.43e-04 tps=14892 mem=6.4G lam=0.14564090967178345 Gm=0.49802837893366814 +[full s0] step 1600/4000 loss=5.4197 gn=0.55 lr=4.38e-04 tps=14898 mem=6.4G lam=0.14599672239273787 Gm=0.5047087588657936 +[full s0] EVAL step 1600: val_loss=5.4789 val_ppl=239.59 +[full s0] step 1620/4000 loss=5.3902 gn=0.54 lr=4.34e-04 tps=14836 mem=6.4G lam=0.14638349320739508 Gm=0.5047999446590742 +[full s0] step 1640/4000 loss=5.2625 gn=0.53 lr=4.30e-04 tps=14843 mem=6.4G lam=0.14677999913692474 Gm=0.5010045052816471 +[full s0] step 1660/4000 loss=5.4421 gn=0.54 lr=4.26e-04 tps=14850 mem=6.4G lam=0.14713385639091334 Gm=0.501681626153489 +[full s0] step 1680/4000 loss=5.5906 gn=0.65 lr=4.22e-04 tps=14856 mem=6.4G lam=0.1474885701512297 Gm=0.5060627665370703 +[full s0] step 1700/4000 loss=5.4464 gn=0.54 lr=4.18e-04 tps=14862 mem=6.4G lam=0.14782526809722185 Gm=0.4984721277529995 +[full s0] step 1720/4000 loss=5.2113 gn=0.56 lr=4.13e-04 tps=14867 mem=6.4G lam=0.14819134771823883 Gm=0.4783928568164508 +[full s0] step 1740/4000 loss=5.2665 gn=0.59 lr=4.09e-04 tps=14873 mem=6.4G lam=0.1485797017812729 Gm=0.4868131143351396 +[full s0] step 1760/4000 loss=5.3345 gn=0.53 lr=4.05e-04 tps=14878 mem=6.4G lam=0.14889246287445226 Gm=0.49005475516120595 +[full s0] step 1780/4000 loss=5.2198 gn=0.64 lr=4.01e-04 tps=14883 mem=6.4G lam=0.14913748173664013 Gm=0.484699172899127 +[full s0] step 1800/4000 loss=5.2915 gn=0.62 lr=3.96e-04 tps=14888 mem=6.4G lam=0.14946761603156725 Gm=0.48724480501065653 +[full s0] EVAL step 1800: val_loss=5.4097 val_ppl=223.56 +[full s0] step 1820/4000 loss=5.3663 gn=0.58 lr=3.92e-04 tps=14834 mem=6.4G lam=0.14975205467393002 Gm=0.48330537664393586 +[full s0] step 1840/4000 loss=5.2046 gn=0.57 lr=3.88e-04 tps=14840 mem=6.4G lam=0.15008515336861214 Gm=0.47825769148766994 +[full s0] step 1860/4000 loss=5.2649 gn=0.60 lr=3.83e-04 tps=14846 mem=6.4G lam=0.15041279854873815 Gm=0.4925144451359908 +[full s0] step 1880/4000 loss=5.3589 gn=0.74 lr=3.79e-04 tps=14851 mem=6.4G lam=0.15075742484380802 Gm=0.5020403222491344 +[full s0] step 1900/4000 loss=5.3196 gn=0.62 lr=3.74e-04 tps=14858 mem=6.4G lam=0.15108068535725275 Gm=0.474533099681139 +[full s0] step 1920/4000 loss=5.2498 gn=0.61 lr=3.70e-04 tps=14862 mem=6.4G lam=0.15135481270651022 Gm=0.4734061478326718 +[full s0] step 1940/4000 loss=5.3131 gn=0.58 lr=3.66e-04 tps=14866 mem=6.4G lam=0.1516401826714476 Gm=0.4780598379050692 +[full s0] step 1960/4000 loss=5.5175 gn=0.66 lr=3.61e-04 tps=14871 mem=6.4G lam=0.151948948080341 Gm=0.478161768677334 +[full s0] step 1980/4000 loss=5.4044 gn=0.58 lr=3.57e-04 tps=14875 mem=6.4G lam=0.1522312369197607 Gm=0.4881879823903243 +[full s0] step 2000/4000 loss=5.2543 gn=0.58 lr=3.52e-04 tps=14880 mem=6.4G lam=0.15254930996646485 Gm=0.4732829745238026 +[full s0] EVAL step 2000: val_loss=5.3565 val_ppl=211.97 +[full s0] step 2020/4000 loss=5.2164 gn=0.60 lr=3.48e-04 tps=14831 mem=6.4G lam=0.1528791462381681 Gm=0.4744356938948234 +[full s0] step 2040/4000 loss=5.3486 gn=0.64 lr=3.43e-04 tps=14836 mem=6.4G lam=0.15319231462975344 Gm=0.4752217202136914 +[full s0] step 2060/4000 loss=5.0926 gn=0.66 lr=3.39e-04 tps=14841 mem=6.4G lam=0.15346764431645474 Gm=0.4769606015955408 +[full s0] step 2080/4000 loss=5.2508 gn=0.62 lr=3.34e-04 tps=14847 mem=6.4G lam=0.15376037017752728 Gm=0.47535418905317783 +[full s0] step 2100/4000 loss=5.2524 gn=0.59 lr=3.30e-04 tps=14851 mem=6.4G lam=0.1540603842586279 Gm=0.4680200172588229 +[full s0] step 2120/4000 loss=5.1454 gn=0.60 lr=3.26e-04 tps=14856 mem=6.4G lam=0.15434814927478632 Gm=0.46248059688756865 +[full s0] step 2140/4000 loss=5.0802 gn=0.57 lr=3.21e-04 tps=14860 mem=6.4G lam=0.15463932634641728 Gm=0.47031607913474244 +[full s0] step 2160/4000 loss=5.1326 gn=0.64 lr=3.17e-04 tps=14865 mem=6.4G lam=0.15491167145470777 Gm=0.46407177827010554 +[full s0] step 2180/4000 loss=5.1673 gn=0.65 lr=3.12e-04 tps=14870 mem=6.4G lam=0.15516483535369238 Gm=0.4632302640626828 +[full s0] step 2200/4000 loss=5.3162 gn=0.62 lr=3.08e-04 tps=14875 mem=6.4G lam=0.15542515770842633 Gm=0.47018811851739883 +[full s0] EVAL step 2200: val_loss=5.2720 val_ppl=194.80 +[full s0] step 2220/4000 loss=5.1938 gn=0.60 lr=3.03e-04 tps=14830 mem=6.4G lam=0.15566586796194315 Gm=0.4673734738801916 +[full s0] step 2240/4000 loss=5.1447 gn=0.58 lr=2.99e-04 tps=14835 mem=6.4G lam=0.1559303660566608 Gm=0.46014175936579704 +[full s0] step 2260/4000 loss=5.1438 gn=0.58 lr=2.94e-04 tps=14840 mem=6.4G lam=0.15621465692917505 Gm=0.4625430420661966 +[full s0] step 2280/4000 loss=5.3504 gn=0.63 lr=2.90e-04 tps=14844 mem=6.4G lam=0.15644852196176848 Gm=0.45796986234684783 +[full s0] step 2300/4000 loss=5.1145 gn=0.60 lr=2.86e-04 tps=14849 mem=6.4G lam=0.15674412685136 Gm=0.4564528313154976 +[full s0] step 2320/4000 loss=5.2212 gn=0.62 lr=2.81e-04 tps=14853 mem=6.4G lam=0.1569712401057283 Gm=0.45699896446118754 +[full s0] step 2340/4000 loss=5.1594 gn=0.66 lr=2.77e-04 tps=14857 mem=6.4G lam=0.15721193235367537 Gm=0.4585708274195592 +[full s0] step 2360/4000 loss=5.2329 gn=0.59 lr=2.72e-04 tps=14862 mem=6.4G lam=0.1574559115494291 Gm=0.4535028813406825 +[full s0] step 2380/4000 loss=5.1232 gn=0.66 lr=2.68e-04 tps=14865 mem=6.4G lam=0.1576677868142724 Gm=0.4569100613395373 +[full s0] step 2400/4000 loss=5.0122 gn=0.69 lr=2.64e-04 tps=14868 mem=6.4G lam=0.15789554237077633 Gm=0.46349408601721126 +[full s0] EVAL step 2400: val_loss=5.1964 val_ppl=180.62 +[full s0] step 2420/4000 loss=5.0019 gn=0.65 lr=2.59e-04 tps=14826 mem=6.4G lam=0.15812122200926146 Gm=0.4450850347056985 +[full s0] step 2440/4000 loss=5.1299 gn=0.64 lr=2.55e-04 tps=14831 mem=6.4G lam=0.15837650404622158 Gm=0.44899142626672983 +[full s0] step 2460/4000 loss=5.2215 gn=0.64 lr=2.51e-04 tps=14835 mem=6.4G lam=0.15859906499584517 Gm=0.4574682153761387 +[full s0] step 2480/4000 loss=5.1100 gn=0.63 lr=2.47e-04 tps=14839 mem=6.4G lam=0.15882964866856733 Gm=0.447203541174531 +[full s0] step 2500/4000 loss=4.9479 gn=0.66 lr=2.42e-04 tps=14843 mem=6.4G lam=0.15902110282331705 Gm=0.4507615032295386 +[full s0] step 2520/4000 loss=5.0244 gn=0.60 lr=2.38e-04 tps=14848 mem=6.4G lam=0.15924381371587515 Gm=0.4480586371695002 +[full s0] step 2540/4000 loss=5.0845 gn=0.64 lr=2.34e-04 tps=14852 mem=6.4G lam=0.15946097454677025 Gm=0.4542156746610999 +[full s0] step 2560/4000 loss=4.8791 gn=0.63 lr=2.30e-04 tps=14856 mem=6.4G lam=0.15967680917431912 Gm=0.4438647028679649 +[full s0] step 2580/4000 loss=5.1637 gn=0.69 lr=2.26e-04 tps=14860 mem=6.4G lam=0.15990279925366244 Gm=0.4438582655663292 +[full s0] step 2600/4000 loss=4.9393 gn=0.66 lr=2.22e-04 tps=14863 mem=6.4G lam=0.1600928260013461 Gm=0.4401321733991305 +[full s0] EVAL step 2600: val_loss=5.1592 val_ppl=174.03 +[full s0] step 2620/4000 loss=5.1345 gn=0.70 lr=2.17e-04 tps=14825 mem=6.4G lam=0.16027720520893732 Gm=0.44931645939747494 +[full s0] step 2640/4000 loss=5.1138 gn=0.69 lr=2.13e-04 tps=14829 mem=6.4G lam=0.16046080024292073 Gm=0.4500795149554809 +[full s0] step 2660/4000 loss=4.9892 gn=0.66 lr=2.09e-04 tps=14833 mem=6.4G lam=0.1606252252434691 Gm=0.44443822465837 +[full s0] step 2680/4000 loss=4.8321 gn=0.81 lr=2.05e-04 tps=14837 mem=6.4G lam=0.1608069073408842 Gm=0.4425751141582926 +[full s0] step 2700/4000 loss=5.0882 gn=0.68 lr=2.01e-04 tps=14840 mem=6.4G lam=0.16100376565009356 Gm=0.4432884758959214 +[full s0] step 2720/4000 loss=5.0655 gn=0.69 lr=1.98e-04 tps=14844 mem=6.4G lam=0.16119438720246157 Gm=0.4416095061848561 +[full s0] step 2740/4000 loss=5.0501 gn=0.65 lr=1.94e-04 tps=14848 mem=6.4G lam=0.16136763679484525 Gm=0.44492132123559713 +[full s0] step 2760/4000 loss=4.8957 gn=0.66 lr=1.90e-04 tps=14851 mem=6.4G lam=0.1615272182971239 Gm=0.44701481858889264 +[full s0] step 2780/4000 loss=5.0218 gn=0.66 lr=1.86e-04 tps=14855 mem=6.4G lam=0.16169719553242126 Gm=0.4440632878492276 +[full s0] step 2800/4000 loss=4.9772 gn=0.75 lr=1.82e-04 tps=14858 mem=6.4G lam=0.1618487909436226 Gm=0.4365478853384654 +[full s0] EVAL step 2800: val_loss=5.0906 val_ppl=162.49 +[full s0] step 2820/4000 loss=5.0181 gn=0.67 lr=1.79e-04 tps=14824 mem=6.4G lam=0.16200900978098312 Gm=0.4376520048826933 +[full s0] step 2840/4000 loss=4.8275 gn=0.67 lr=1.75e-04 tps=14827 mem=6.4G lam=0.16216511682917675 Gm=0.43590452677259844 +[full s0] step 2860/4000 loss=4.9932 gn=0.63 lr=1.71e-04 tps=14831 mem=6.4G lam=0.16232836867372194 Gm=0.4350059659530719 +[full s0] step 2880/4000 loss=4.9926 gn=0.67 lr=1.68e-04 tps=14834 mem=6.4G lam=0.16248036300142607 Gm=0.4299806446457903 +[full s0] step 2900/4000 loss=4.8354 gn=0.72 lr=1.64e-04 tps=14838 mem=6.4G lam=0.16262648720294237 Gm=0.4372340639432271 +[full s0] step 2920/4000 loss=5.0496 gn=0.68 lr=1.61e-04 tps=14842 mem=6.4G lam=0.16277169901877642 Gm=0.44400854377696913 +[full s0] step 2940/4000 loss=4.9499 gn=0.64 lr=1.57e-04 tps=14845 mem=6.4G lam=0.16289791526893774 Gm=0.43647429657479125 +[full s0] step 2960/4000 loss=4.9569 gn=0.70 lr=1.54e-04 tps=14849 mem=6.4G lam=0.16302600068350634 Gm=0.43787237629294395 +[full s0] step 2980/4000 loss=4.9083 gn=0.64 lr=1.50e-04 tps=14852 mem=6.4G lam=0.16315330937504768 Gm=0.43096340727061033 +[full s0] step 3000/4000 loss=4.8441 gn=0.66 lr=1.47e-04 tps=14855 mem=6.4G lam=0.16329338184247413 Gm=0.4300241045032938 +[full s0] EVAL step 3000: val_loss=5.1066 val_ppl=165.11 +[full s0] step 3020/4000 loss=4.9192 gn=0.64 lr=1.44e-04 tps=14822 mem=6.4G lam=0.16343002735326687 Gm=0.4366761473938823 +[full s0] step 3040/4000 loss=4.9838 gn=0.68 lr=1.41e-04 tps=14826 mem=6.4G lam=0.1635501366108656 Gm=0.4348375468204419 +[full s0] step 3060/4000 loss=4.9574 gn=0.69 lr=1.38e-04 tps=14829 mem=6.4G lam=0.163682596758008 Gm=0.43888544384390116 +[full s0] step 3080/4000 loss=4.8064 gn=0.72 lr=1.34e-04 tps=14833 mem=6.4G lam=0.16378284866611162 Gm=0.42993523366749287 +[full s0] step 3100/4000 loss=5.0109 gn=0.75 lr=1.31e-04 tps=14836 mem=6.4G lam=0.1638804872830709 Gm=0.4370205185065667 +[full s0] step 3120/4000 loss=5.0369 gn=0.79 lr=1.28e-04 tps=14839 mem=6.4G lam=0.1640077323342363 Gm=0.43461609942217666 +[full s0] step 3140/4000 loss=4.8797 gn=0.67 lr=1.25e-04 tps=14843 mem=6.4G lam=0.16411774698644876 Gm=0.43366697058081627 +[full s0] step 3160/4000 loss=5.0279 gn=0.77 lr=1.23e-04 tps=14846 mem=6.4G lam=0.1642256739238898 Gm=0.4341176695500811 +[full s0] step 3180/4000 loss=4.8814 gn=0.68 lr=1.20e-04 tps=14850 mem=6.4G lam=0.16433581709861755 Gm=0.4314587600529194 +[full s0] step 3200/4000 loss=4.8908 gn=0.74 lr=1.17e-04 tps=14852 mem=6.4G lam=0.1644469192251563 Gm=0.43080346534649533 +[full s0] EVAL step 3200: val_loss=5.0128 val_ppl=150.33 +[full s0] step 3220/4000 loss=5.0098 gn=0.75 lr=1.14e-04 tps=14822 mem=6.4G lam=0.16455033235251904 Gm=0.423968060563008 +[full s0] step 3240/4000 loss=5.0121 gn=0.68 lr=1.12e-04 tps=14825 mem=6.4G lam=0.164661834637324 Gm=0.4364341401184599 +[full s0] step 3260/4000 loss=4.7679 gn=0.68 lr=1.09e-04 tps=14827 mem=6.4G lam=0.16475263237953186 Gm=0.42899767123162746 +[full s0] step 3280/4000 loss=5.0351 gn=0.75 lr=1.06e-04 tps=14830 mem=6.4G lam=0.16484248265624046 Gm=0.42665910472472507 +[full s0] step 3300/4000 loss=4.8429 gn=0.69 lr=1.04e-04 tps=14834 mem=6.4G lam=0.1649355192979177 Gm=0.4216356985270977 +[full s0] step 3320/4000 loss=5.0145 gn=0.71 lr=1.02e-04 tps=14837 mem=6.4G lam=0.16502051676313081 Gm=0.4199893881256382 +[full s0] step 3340/4000 loss=4.8423 gn=0.67 lr=9.92e-05 tps=14840 mem=6.4G lam=0.16509454355885586 Gm=0.4251132579520345 +[full s0] step 3360/4000 loss=5.0007 gn=0.71 lr=9.69e-05 tps=14843 mem=6.4G lam=0.16518759479125342 Gm=0.42819516733288765 +[full s0] step 3380/4000 loss=4.9616 gn=0.71 lr=9.47e-05 tps=14847 mem=6.4G lam=0.16527666803449392 Gm=0.42280324424306553 +[full s0] step 3400/4000 loss=4.8588 gn=0.69 lr=9.25e-05 tps=14850 mem=6.4G lam=0.16536484193056822 Gm=0.4210895566890637 +[full s0] EVAL step 3400: val_loss=5.0103 val_ppl=149.95 +[full s0] step 3420/4000 loss=4.8995 gn=0.70 lr=9.04e-05 tps=14820 mem=6.4G lam=0.16544855292886496 Gm=0.43159068406869966 +[full s0] step 3440/4000 loss=4.9295 gn=0.72 lr=8.84e-05 tps=14823 mem=6.4G lam=0.1655213125050068 Gm=0.432841124633948 +[full s0] step 3460/4000 loss=4.8067 gn=0.73 lr=8.65e-05 tps=14826 mem=6.4G lam=0.16559547061721483 Gm=0.42248809306571883 +[full s0] step 3480/4000 loss=4.9165 gn=0.71 lr=8.46e-05 tps=14830 mem=6.4G lam=0.16566639052083096 Gm=0.4233105207482974 +[full s0] step 3500/4000 loss=4.8292 gn=0.68 lr=8.27e-05 tps=14832 mem=6.4G lam=0.16573535868277153 Gm=0.424697866042455 +[full s0] step 3520/4000 loss=4.7182 gn=0.77 lr=8.10e-05 tps=14835 mem=6.4G lam=0.16580549317101637 Gm=0.42924326844513416 +[full s0] step 3540/4000 loss=4.7145 gn=0.73 lr=7.93e-05 tps=14838 mem=6.4G lam=0.1658791173249483 Gm=0.42203957090775174 +[full s0] step 3560/4000 loss=4.9728 gn=0.67 lr=7.77e-05 tps=14841 mem=6.4G lam=0.165962186952432 Gm=0.4248941869785388 +[full s0] step 3580/4000 loss=4.7990 gn=0.66 lr=7.61e-05 tps=14844 mem=6.4G lam=0.16603116939465204 Gm=0.4190280182907979 +[full s0] step 3600/4000 loss=4.8231 gn=0.75 lr=7.46e-05 tps=14847 mem=6.4G lam=0.16610508256902298 Gm=0.4256579965973894 +[full s0] EVAL step 3600: val_loss=4.9789 val_ppl=145.31 +[full s0] step 3620/4000 loss=4.8785 gn=0.75 lr=7.32e-05 tps=14819 mem=6.4G lam=0.16617669692883888 Gm=0.4205183299879233 +[full s0] step 3640/4000 loss=4.9570 gn=0.71 lr=7.19e-05 tps=14822 mem=6.4G lam=0.1662404810388883 Gm=0.4220012916872899 +[full s0] step 3660/4000 loss=4.9918 gn=0.80 lr=7.06e-05 tps=14825 mem=6.4G lam=0.16630153047541776 Gm=0.42589141490558785 +[full s0] step 3680/4000 loss=4.9207 gn=0.68 lr=6.94e-05 tps=14828 mem=6.4G lam=0.1663592647140225 Gm=0.416047852486372 +[full s0] step 3700/4000 loss=4.9807 gn=0.70 lr=6.83e-05 tps=14831 mem=6.4G lam=0.1664255866780877 Gm=0.4216521034638087 +[full s0] step 3720/4000 loss=4.7302 gn=0.69 lr=6.72e-05 tps=14833 mem=6.4G lam=0.16649126230428615 Gm=0.4164570265760024 +[full s0] step 3740/4000 loss=4.9217 gn=0.68 lr=6.62e-05 tps=14836 mem=6.4G lam=0.16654821671545506 Gm=0.4178890536228816 +[full s0] step 3760/4000 loss=4.8702 gn=0.73 lr=6.53e-05 tps=14839 mem=6.4G lam=0.16661168665935597 Gm=0.41870569561918575 +[full s0] step 3780/4000 loss=4.7588 gn=0.72 lr=6.45e-05 tps=14842 mem=6.4G lam=0.16666329310586056 Gm=0.4184085186570883 +[full s0] step 3800/4000 loss=4.8776 gn=0.77 lr=6.37e-05 tps=14843 mem=6.4G lam=0.16672033909708261 Gm=0.42720757083346445 +[full s0] EVAL step 3800: val_loss=4.9583 val_ppl=142.36 +[full s0] step 3820/4000 loss=4.9376 gn=0.74 lr=6.30e-05 tps=14816 mem=6.4G lam=0.16677486648162207 Gm=0.42343480822940666 +[full s0] step 3840/4000 loss=4.8648 gn=0.77 lr=6.24e-05 tps=14818 mem=6.4G lam=0.16683314150820175 Gm=0.41695012773076695 +[full s0] step 3860/4000 loss=4.8737 gn=0.70 lr=6.18e-05 tps=14820 mem=6.4G lam=0.1668974378456672 Gm=0.42252734551827115 +[full s0] step 3880/4000 loss=4.9272 gn=0.79 lr=6.13e-05 tps=14822 mem=6.4G lam=0.16694826819002628 Gm=0.428834938754638 +[full s0] step 3900/4000 loss=4.8355 gn=0.79 lr=6.09e-05 tps=14825 mem=6.4G lam=0.1669996650889516 Gm=0.41820415544013184 +[full s0] step 3920/4000 loss=4.7652 gn=0.78 lr=6.06e-05 tps=14827 mem=6.4G lam=0.16705392487347126 Gm=0.4173752376809716 +[full s0] step 3940/4000 loss=4.9381 gn=0.77 lr=6.03e-05 tps=14829 mem=6.4G lam=0.16710669671495756 Gm=0.4203516334916155 +[full s0] step 3960/4000 loss=4.8886 gn=0.73 lr=6.01e-05 tps=14832 mem=6.4G lam=0.16715015222628912 Gm=0.41584287708004314 +[full s0] step 3980/4000 loss=4.8316 gn=0.78 lr=6.00e-05 tps=14833 mem=6.4G lam=0.16720068734139204 Gm=0.4214151619623105 +[full s0] EVAL step 3999: val_loss=4.9528 val_ppl=141.57 + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant baseline --seed 0 --out C:\Users\Bhorn\Desktop\SMATest\runs\baseline_s0 --max_steps 4000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 200 --data_seed 1337 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant s_only --seed 0 --out C:\Users\Bhorn\Desktop\SMATest\runs\s_only_s0 --max_steps 4000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 200 --data_seed 1337 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant g_only --seed 0 --out C:\Users\Bhorn\Desktop\SMATest\runs\g_only_s0 --max_steps 4000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 200 --data_seed 1337 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant full --seed 0 --out C:\Users\Bhorn\Desktop\SMATest\runs\full_s0 --max_steps 4000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 200 --data_seed 1337 === + diff --git a/runs_4ksteps/baseline_s0/config.json b/runs_4ksteps/baseline_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..39608d9f0e4bb190b8bfc2929302a23e6c456e81 --- /dev/null +++ b/runs_4ksteps/baseline_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "baseline", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s0", + "max_steps": 4000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 200, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/runs_4ksteps/baseline_s0/final.pt b/runs_4ksteps/baseline_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..abfa5439a98d93d10e0e1027d17ae413afb4d2b8 --- /dev/null +++ b/runs_4ksteps/baseline_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e163795bcc66f9b2858d8adcddb9db8b91db0dd5addfbd19f6817791b16c378 +size 253572039 diff --git a/runs_4ksteps/baseline_s0/metrics.jsonl b/runs_4ksteps/baseline_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..e3fb2145f7de18ae3e0c13ea03529e3edfcf5f7f --- /dev/null +++ b/runs_4ksteps/baseline_s0/metrics.jsonl @@ -0,0 +1,222 @@ +{"event": "start", "params": 63376512, "variant": "baseline", "seed": 0, "config": {"variant": "baseline", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\baseline_s0", "max_steps": 4000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 200, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.879043579101562, "grad_norm": 15.694275856018066, "tokens_seen": 8192, "tokens_per_sec": 8974.252466762888, "vram_gb": 5.453331456, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.858049392700195, "grad_norm": 1.9308255910873413, "tokens_seen": 172032, "tokens_per_sec": 18098.45951299182, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.108919143676758, "grad_norm": 1.7496861219406128, "tokens_seen": 335872, "tokens_per_sec": 18557.58863811033, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.109308242797852, "grad_norm": 1.5726726055145264, "tokens_seen": 499712, "tokens_per_sec": 18726.133452476177, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.426548004150391, "grad_norm": 0.7255967259407043, "tokens_seen": 663552, "tokens_per_sec": 18778.889795370644, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.23581075668335, "grad_norm": 0.8989492058753967, "tokens_seen": 827392, "tokens_per_sec": 18805.200829820296, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.066903352737427, "grad_norm": 0.6682687401771545, "tokens_seen": 991232, "tokens_per_sec": 18838.277310293135, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.865255355834961, "grad_norm": 0.9195041060447693, "tokens_seen": 1155072, "tokens_per_sec": 18848.800681624038, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.905359268188477, "grad_norm": 0.56206876039505, "tokens_seen": 1318912, "tokens_per_sec": 18855.682616002345, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.818137168884277, "grad_norm": 0.795799732208252, "tokens_seen": 1482752, "tokens_per_sec": 18834.501298916526, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.699586391448975, "grad_norm": 0.6688268184661865, "tokens_seen": 1646592, "tokens_per_sec": 18840.74383690803, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 200, "val_loss": 6.758291625976563, "val_ppl": 861.1697383825413, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 1646592} +{"event": "step", "step": 220, "lr": 0.0005999630923479665, "train_loss": 6.585353374481201, "grad_norm": 0.5337157845497131, "tokens_seen": 1810432, "tokens_per_sec": 18243.578487039074, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.00059985237948205, "train_loss": 6.633080959320068, "grad_norm": 0.5738077759742737, "tokens_seen": 1974272, "tokens_per_sec": 18291.48039684577, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005996678916700422, "train_loss": 6.619430780410767, "grad_norm": 0.5747467875480652, "tokens_seen": 2138112, "tokens_per_sec": 18332.99571985207, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.000599409679349069, "train_loss": 6.507688283920288, "grad_norm": 0.5374479293823242, "tokens_seen": 2301952, "tokens_per_sec": 18370.354901457038, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005990778131118008, "train_loss": 6.3495306968688965, "grad_norm": 0.5000535845756531, "tokens_seen": 2465792, "tokens_per_sec": 18406.667946071495, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.0005986723836871531, "train_loss": 6.41103982925415, "grad_norm": 0.5301920771598816, "tokens_seen": 2629632, "tokens_per_sec": 18431.147038369254, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005981935019154823, "train_loss": 6.393010854721069, "grad_norm": 0.4962553381919861, "tokens_seen": 2793472, "tokens_per_sec": 18461.83694386546, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005976412987182833, "train_loss": 6.260194301605225, "grad_norm": 0.5374064445495605, "tokens_seen": 2957312, "tokens_per_sec": 18486.905515148577, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005970159250623965, "train_loss": 6.228891372680664, "grad_norm": 0.5478425025939941, "tokens_seen": 3121152, "tokens_per_sec": 18512.47039103094, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.000596317551918735, "train_loss": 6.256117343902588, "grad_norm": 0.5280329585075378, "tokens_seen": 3284992, "tokens_per_sec": 18533.08082121937, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 400, "val_loss": 6.403730702400208, "val_ppl": 604.0945360945648, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 3284992} +{"event": "step", "step": 420, "lr": 0.0005955463702155432, "train_loss": 6.423207521438599, "grad_norm": 0.7976609468460083, "tokens_seen": 3448832, "tokens_per_sec": 18241.129911580912, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005947025907861985, "train_loss": 6.350582599639893, "grad_norm": 0.5881041288375854, "tokens_seen": 3612672, "tokens_per_sec": 18272.907061898233, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005937864443115718, "train_loss": 6.303401708602905, "grad_norm": 0.5540409088134766, "tokens_seen": 3776512, "tokens_per_sec": 18295.080754771672, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005927981812569615, "train_loss": 6.390613317489624, "grad_norm": 0.5809852480888367, "tokens_seen": 3940352, "tokens_per_sec": 18307.61416287619, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005917380718036191, "train_loss": 6.31821346282959, "grad_norm": 0.555970311164856, "tokens_seen": 4104192, "tokens_per_sec": 18327.841063288877, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005906064057748841, "train_loss": 6.1684863567352295, "grad_norm": 0.5530833601951599, "tokens_seen": 4268032, "tokens_per_sec": 18348.692613540687, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.000589403492556949, "train_loss": 6.184960842132568, "grad_norm": 0.4939723312854767, "tokens_seen": 4431872, "tokens_per_sec": 18367.96749834036, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.0005881296610142765, "train_loss": 6.044617176055908, "grad_norm": 0.5632477402687073, "tokens_seen": 4595712, "tokens_per_sec": 18387.66690299601, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005867852593996914, "train_loss": 6.209620714187622, "grad_norm": 0.514613926410675, "tokens_seen": 4759552, "tokens_per_sec": 18407.46559941583, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005853706552591712, "train_loss": 5.977893114089966, "grad_norm": 0.5459417700767517, "tokens_seen": 4923392, "tokens_per_sec": 18425.29441693462, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 600, "val_loss": 6.174860370159149, "val_ppl": 480.5159248741871, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 4923392} +{"event": "step", "step": 620, "lr": 0.0005838862353313635, "train_loss": 6.032171726226807, "grad_norm": 0.5044503211975098, "tokens_seen": 5087232, "tokens_per_sec": 18235.00349637991, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005823324054418544, "train_loss": 6.126847743988037, "grad_norm": 0.6466651558876038, "tokens_seen": 5251072, "tokens_per_sec": 18253.86411217129, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005807095903922209, "train_loss": 5.941319465637207, "grad_norm": 0.5354759097099304, "tokens_seen": 5414912, "tokens_per_sec": 18275.335348927852, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005790182338438938, "train_loss": 5.959714651107788, "grad_norm": 0.5574734807014465, "tokens_seen": 5578752, "tokens_per_sec": 18275.65796438867, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005772587981968655, "train_loss": 6.054962396621704, "grad_norm": 0.566072940826416, "tokens_seen": 5742592, "tokens_per_sec": 18288.244744881486, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005754317644632739, "train_loss": 6.027182579040527, "grad_norm": 0.5405449867248535, "tokens_seen": 5906432, "tokens_per_sec": 18305.763924719417, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005735376321358989, "train_loss": 6.0445544719696045, "grad_norm": 0.5218998193740845, "tokens_seen": 6070272, "tokens_per_sec": 18320.143653764593, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.000571576919051606, "train_loss": 5.924992322921753, "grad_norm": 0.5737737417221069, "tokens_seen": 6234112, "tokens_per_sec": 18334.959771643546, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.0005695501612497752, "train_loss": 5.937316417694092, "grad_norm": 0.540032684803009, "tokens_seen": 6397952, "tokens_per_sec": 18348.268892469863, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000567457912825752, "train_loss": 5.900677919387817, "grad_norm": 0.5537795424461365, "tokens_seen": 6561792, "tokens_per_sec": 18361.93661981959, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 800, "val_loss": 5.989102041721344, "val_ppl": 399.0561132531265, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 6561792} +{"event": "step", "step": 820, "lr": 0.0005653007457793648, "train_loss": 5.967679023742676, "grad_norm": 0.51093989610672, "tokens_seen": 6725632, "tokens_per_sec": 18212.075332547527, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.000563079249858545, "train_loss": 6.144945383071899, "grad_norm": 0.5331567525863647, "tokens_seen": 6889472, "tokens_per_sec": 18229.358105592357, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005607940323980955, "train_loss": 5.961447238922119, "grad_norm": 0.5413928031921387, "tokens_seen": 7053312, "tokens_per_sec": 18242.652898273387, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005584457181536525, "train_loss": 5.941934108734131, "grad_norm": 0.6394031047821045, "tokens_seen": 7217152, "tokens_per_sec": 18255.09267114837, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005560349491308827, "train_loss": 5.852266788482666, "grad_norm": 0.5245711207389832, "tokens_seen": 7380992, "tokens_per_sec": 18268.58082817199, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005535623844099654, "train_loss": 5.757063865661621, "grad_norm": 0.5159236192703247, "tokens_seen": 7544832, "tokens_per_sec": 18283.356521577018, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.000551028699965407, "train_loss": 5.812169075012207, "grad_norm": 0.5459425449371338, "tokens_seen": 7708672, "tokens_per_sec": 18293.60565000495, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005484345884812357, "train_loss": 5.7968010902404785, "grad_norm": 0.5779032111167908, "tokens_seen": 7872512, "tokens_per_sec": 18304.943142431137, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005457807591616296, "train_loss": 5.711064100265503, "grad_norm": 0.512640655040741, "tokens_seen": 8036352, "tokens_per_sec": 18318.391350336333, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.0005430679375370263, "train_loss": 5.8197853565216064, "grad_norm": 0.5706220865249634, "tokens_seen": 8200192, "tokens_per_sec": 18330.267106350148, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.835738861560822, "val_ppl": 342.3175659259516, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005402968652657703, "train_loss": 5.548312664031982, "grad_norm": 0.5044246912002563, "tokens_seen": 8364032, "tokens_per_sec": 18216.872321990562, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005374682999313519, "train_loss": 5.801883697509766, "grad_norm": 0.5103521943092346, "tokens_seen": 8527872, "tokens_per_sec": 18229.92628193537, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005345830148352904, "train_loss": 5.516525030136108, "grad_norm": 0.563991129398346, "tokens_seen": 8691712, "tokens_per_sec": 18240.37108057075, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005316417987857215, "train_loss": 5.7171947956085205, "grad_norm": 0.5711981654167175, "tokens_seen": 8855552, "tokens_per_sec": 18252.524529010137, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005286454558817454, "train_loss": 5.695245265960693, "grad_norm": 0.5603589415550232, "tokens_seen": 9019392, "tokens_per_sec": 18264.979003504108, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.000525594805293594, "train_loss": 5.632111310958862, "grad_norm": 0.5876843929290771, "tokens_seen": 9183232, "tokens_per_sec": 18275.04473465683, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005224906810386772, "train_loss": 5.867396831512451, "grad_norm": 0.5058356523513794, "tokens_seen": 9347072, "tokens_per_sec": 18282.48499692255, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005193339317535718, "train_loss": 5.631261348724365, "grad_norm": 0.5425214171409607, "tokens_seen": 9510912, "tokens_per_sec": 18286.959993649878, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005161254204620127, "train_loss": 5.692009925842285, "grad_norm": 0.527843177318573, "tokens_seen": 9674752, "tokens_per_sec": 18295.566566138066, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005128660243389501, "train_loss": 5.567769527435303, "grad_norm": 0.5317879915237427, "tokens_seen": 9838592, "tokens_per_sec": 18303.519527405904, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1200, "val_loss": 5.688670861721039, "val_ppl": 295.50059835113456, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 9838592} +{"event": "step", "step": 1220, "lr": 0.000509556634470739, "train_loss": 5.595981597900391, "grad_norm": 0.6077878475189209, "tokens_seen": 10002432, "tokens_per_sec": 18204.10842807112, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005061981556115254, "train_loss": 5.463430643081665, "grad_norm": 0.5411402583122253, "tokens_seen": 10166272, "tokens_per_sec": 18208.469336263184, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005027915059358954, "train_loss": 5.468475818634033, "grad_norm": 0.555474042892456, "tokens_seen": 10330112, "tokens_per_sec": 18215.688867623187, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0004993376167878549, "train_loss": 5.594573974609375, "grad_norm": 0.5843530297279358, "tokens_seen": 10493952, "tokens_per_sec": 18223.120739605798, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0004958374324262102, "train_loss": 5.690805673599243, "grad_norm": 0.5893575549125671, "tokens_seen": 10657792, "tokens_per_sec": 18231.60663855159, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0004922919097664167, "train_loss": 5.638151168823242, "grad_norm": 0.585822343826294, "tokens_seen": 10821632, "tokens_per_sec": 18238.268399514367, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0004887020181189677, "train_loss": 5.749315977096558, "grad_norm": 0.5425845980644226, "tokens_seen": 10985472, "tokens_per_sec": 18242.329374980043, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.00048506873892439467, "train_loss": 5.562595367431641, "grad_norm": 0.5632890462875366, "tokens_seen": 11149312, "tokens_per_sec": 18249.251593994937, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0004813930654849516, "train_loss": 5.431564569473267, "grad_norm": 0.5592644214630127, "tokens_seen": 11313152, "tokens_per_sec": 18251.549113662517, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0004776760026930552, "train_loss": 5.441120386123657, "grad_norm": 0.5881975889205933, "tokens_seen": 11476992, "tokens_per_sec": 18257.291870583213, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1400, "val_loss": 5.575053381919861, "val_ppl": 263.7636354703822, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 11476992} +{"event": "step", "step": 1420, "lr": 0.00047391856675655823, "train_loss": 5.423702001571655, "grad_norm": 0.5991970896720886, "tokens_seen": 11640832, "tokens_per_sec": 18170.73741671897, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.00047012178492092805, "train_loss": 5.416006088256836, "grad_norm": 0.6186622977256775, "tokens_seen": 11804672, "tokens_per_sec": 18176.53892153469, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.00046628669518840776, "train_loss": 5.533634662628174, "grad_norm": 0.5994482636451721, "tokens_seen": 11968512, "tokens_per_sec": 18184.299134897585, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.00046241434603423715, "train_loss": 5.5814995765686035, "grad_norm": 0.5704628825187683, "tokens_seen": 12132352, "tokens_per_sec": 18190.77026819325, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0004585057961200098, "train_loss": 5.496256351470947, "grad_norm": 0.6145368814468384, "tokens_seen": 12296192, "tokens_per_sec": 18197.47746423212, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.00045456211400424565, "train_loss": 5.45197319984436, "grad_norm": 0.5655416250228882, "tokens_seen": 12460032, "tokens_per_sec": 18205.910046169487, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0004505843778502577, "train_loss": 5.365142822265625, "grad_norm": 0.5457947254180908, "tokens_seen": 12623872, "tokens_per_sec": 18212.401214775073, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.000446573675131393, "train_loss": 5.3625428676605225, "grad_norm": 0.5804733037948608, "tokens_seen": 12787712, "tokens_per_sec": 18218.184232644315, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0004425311023337283, "train_loss": 5.4414825439453125, "grad_norm": 0.6063989400863647, "tokens_seen": 12951552, "tokens_per_sec": 18225.127344736713, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0004384577646563017, "train_loss": 5.426446199417114, "grad_norm": 0.5683524012565613, "tokens_seen": 13115392, "tokens_per_sec": 18232.771587823532, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1600, "val_loss": 5.4921424865722654, "val_ppl": 242.7767960783921, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 13115392} +{"event": "step", "step": 1620, "lr": 0.00043435477570896204, "train_loss": 5.402351140975952, "grad_norm": 0.564184308052063, "tokens_seen": 13279232, "tokens_per_sec": 18160.92115318257, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0004302232572079195, "train_loss": 5.296796083450317, "grad_norm": 0.5806705951690674, "tokens_seen": 13443072, "tokens_per_sec": 18167.7514291124, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0004260643386690794, "train_loss": 5.472200393676758, "grad_norm": 0.6005545854568481, "tokens_seen": 13606912, "tokens_per_sec": 18175.537857583913, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.00042187915709924356, "train_loss": 5.613367557525635, "grad_norm": 0.5630645751953125, "tokens_seen": 13770752, "tokens_per_sec": 18182.957657226158, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0004176688566852645, "train_loss": 5.468089580535889, "grad_norm": 0.5940316319465637, "tokens_seen": 13934592, "tokens_per_sec": 18189.827380776176, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.00041343458848123576, "train_loss": 5.236758232116699, "grad_norm": 0.603863537311554, "tokens_seen": 14098432, "tokens_per_sec": 18194.70352241102, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.00040917751009380545, "train_loss": 5.286284923553467, "grad_norm": 0.589327871799469, "tokens_seen": 14262272, "tokens_per_sec": 18195.366836308378, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.000404898785365698, "train_loss": 5.370003938674927, "grad_norm": 0.5687751173973083, "tokens_seen": 14426112, "tokens_per_sec": 18197.57233319017, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.00040059958405753125, "train_loss": 5.231732368469238, "grad_norm": 0.5961577296257019, "tokens_seen": 14589952, "tokens_per_sec": 18204.04805674367, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.00039628108152801574, "train_loss": 5.305721998214722, "grad_norm": 0.5853603482246399, "tokens_seen": 14753792, "tokens_per_sec": 18209.173717332487, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1800, "val_loss": 5.425427579879761, "val_ppl": 227.10843255402182, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 14753792} +{"event": "step", "step": 1820, "lr": 0.0003919444584126229, "train_loss": 5.390699863433838, "grad_norm": 0.5760704278945923, "tokens_seen": 14917632, "tokens_per_sec": 18146.17093757952, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0003875909003008116, "train_loss": 5.226965427398682, "grad_norm": 0.6145718693733215, "tokens_seen": 15081472, "tokens_per_sec": 18153.71406694133, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.00038322159741189957, "train_loss": 5.273662328720093, "grad_norm": 0.5883156061172485, "tokens_seen": 15245312, "tokens_per_sec": 18159.159885742443, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.00037883774426966964, "train_loss": 5.379571199417114, "grad_norm": 0.6980408430099487, "tokens_seen": 15409152, "tokens_per_sec": 18166.318778710083, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0003744405393757981, "train_loss": 5.331045866012573, "grad_norm": 0.6614974141120911, "tokens_seen": 15572992, "tokens_per_sec": 18173.08379876603, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0003700311848821973, "train_loss": 5.2796337604522705, "grad_norm": 0.5873194932937622, "tokens_seen": 15736832, "tokens_per_sec": 18179.21415770243, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0003656108862623586, "train_loss": 5.333995819091797, "grad_norm": 0.608514130115509, "tokens_seen": 15900672, "tokens_per_sec": 18186.138792544247, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.00036118085198178824, "train_loss": 5.532715320587158, "grad_norm": 0.627738893032074, "tokens_seen": 16064512, "tokens_per_sec": 18192.543169870274, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.00035674229316762495, "train_loss": 5.440483808517456, "grad_norm": 0.608600378036499, "tokens_seen": 16228352, "tokens_per_sec": 18198.799373593105, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0003522964232775297, "train_loss": 5.276167869567871, "grad_norm": 0.609562873840332, "tokens_seen": 16392192, "tokens_per_sec": 18204.677846362425, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.379060471057892, "val_ppl": 216.8184724704824, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0003478444577679388, "train_loss": 5.22844672203064, "grad_norm": 0.5849410891532898, "tokens_seen": 16556032, "tokens_per_sec": 18147.22256523689, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.00034338761376176993, "train_loss": 5.3605055809021, "grad_norm": 0.6659375429153442, "tokens_seen": 16719872, "tokens_per_sec": 18153.523419729612, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0003389271097156731, "train_loss": 5.126803159713745, "grad_norm": 0.7242559790611267, "tokens_seen": 16883712, "tokens_per_sec": 18159.456434641488, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.00033446416508691677, "train_loss": 5.2638373374938965, "grad_norm": 0.6333378553390503, "tokens_seen": 17047552, "tokens_per_sec": 18165.222689967104, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.00032999999999999994, "train_loss": 5.270089387893677, "grad_norm": 0.5755181312561035, "tokens_seen": 17211392, "tokens_per_sec": 18170.543445947726, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0003255358349130832, "train_loss": 5.191451549530029, "grad_norm": 0.6366748809814453, "tokens_seen": 17375232, "tokens_per_sec": 18176.298114329806, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0003210728902843267, "train_loss": 5.096358776092529, "grad_norm": 0.6194387078285217, "tokens_seen": 17539072, "tokens_per_sec": 18182.523261611284, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0003166123862382299, "train_loss": 5.144412040710449, "grad_norm": 0.602814793586731, "tokens_seen": 17702912, "tokens_per_sec": 18186.60455118253, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0003121555422320611, "train_loss": 5.19546914100647, "grad_norm": 0.7343387603759766, "tokens_seen": 17866752, "tokens_per_sec": 18192.203248612004, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.00030770357672247023, "train_loss": 5.344171047210693, "grad_norm": 0.6192429065704346, "tokens_seen": 18030592, "tokens_per_sec": 18198.078922414617, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2200, "val_loss": 5.298965728282928, "val_ppl": 200.12971439335882, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 18030592} +{"event": "step", "step": 2220, "lr": 0.00030325770683237505, "train_loss": 5.229546546936035, "grad_norm": 0.6312416791915894, "tokens_seen": 18194432, "tokens_per_sec": 18146.50365238002, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0002988191480182117, "train_loss": 5.170661449432373, "grad_norm": 0.6174649596214294, "tokens_seen": 18358272, "tokens_per_sec": 18152.39762884044, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0002943891137376413, "train_loss": 5.168545722961426, "grad_norm": 0.5986061692237854, "tokens_seen": 18522112, "tokens_per_sec": 18157.951086152298, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.0002899688151178026, "train_loss": 5.358444452285767, "grad_norm": 0.6642295718193054, "tokens_seen": 18685952, "tokens_per_sec": 18163.456156119788, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.00028555946062420177, "train_loss": 5.136403799057007, "grad_norm": 0.6614784002304077, "tokens_seen": 18849792, "tokens_per_sec": 18168.83888876449, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.00028116225573033036, "train_loss": 5.25939416885376, "grad_norm": 0.6678224205970764, "tokens_seen": 19013632, "tokens_per_sec": 18174.24127839023, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0002767784025881003, "train_loss": 5.183807134628296, "grad_norm": 0.6320900917053223, "tokens_seen": 19177472, "tokens_per_sec": 18179.484509852562, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.00027240909969918834, "train_loss": 5.2727437019348145, "grad_norm": 0.6584456562995911, "tokens_seen": 19341312, "tokens_per_sec": 18184.70756200374, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.00026805554158737706, "train_loss": 5.147434711456299, "grad_norm": 0.679394006729126, "tokens_seen": 19505152, "tokens_per_sec": 18190.204997512585, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.0002637189184719842, "train_loss": 5.0502846240997314, "grad_norm": 0.6821886897087097, "tokens_seen": 19668992, "tokens_per_sec": 18195.446920939594, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2400, "val_loss": 5.223274123668671, "val_ppl": 185.5406737792588, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 19668992} +{"event": "step", "step": 2420, "lr": 0.00025940041594246864, "train_loss": 5.030417442321777, "grad_norm": 0.6206470727920532, "tokens_seen": 19832832, "tokens_per_sec": 18148.25052553576, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.000255101214634302, "train_loss": 5.161475658416748, "grad_norm": 0.657909095287323, "tokens_seen": 19996672, "tokens_per_sec": 18153.61930205887, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.00025082248990619454, "train_loss": 5.247021198272705, "grad_norm": 0.6310542821884155, "tokens_seen": 20160512, "tokens_per_sec": 18158.76840784858, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.0002465654115187642, "train_loss": 5.1272358894348145, "grad_norm": 0.6100175976753235, "tokens_seen": 20324352, "tokens_per_sec": 18163.856308902363, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.00024233114331473544, "train_loss": 4.996895790100098, "grad_norm": 0.6594232320785522, "tokens_seen": 20488192, "tokens_per_sec": 18168.955901538513, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.00023812084290075633, "train_loss": 5.042043685913086, "grad_norm": 0.6174346804618835, "tokens_seen": 20652032, "tokens_per_sec": 18173.881356049613, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.00023393566133092048, "train_loss": 5.11972975730896, "grad_norm": 0.6548559665679932, "tokens_seen": 20815872, "tokens_per_sec": 18179.33168736976, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.00022977674279208032, "train_loss": 4.905049800872803, "grad_norm": 0.6702408194541931, "tokens_seen": 20979712, "tokens_per_sec": 18184.52509260183, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.00022564522429103793, "train_loss": 5.2085723876953125, "grad_norm": 0.6307780742645264, "tokens_seen": 21143552, "tokens_per_sec": 18189.822049968978, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00022154223534369825, "train_loss": 4.986791610717773, "grad_norm": 0.6316693425178528, "tokens_seen": 21307392, "tokens_per_sec": 18194.03828998051, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2600, "val_loss": 5.191937220096588, "val_ppl": 179.81655999358043, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 21307392} +{"event": "step", "step": 2620, "lr": 0.00021746889766627162, "train_loss": 5.156065464019775, "grad_norm": 0.6918879747390747, "tokens_seen": 21471232, "tokens_per_sec": 18150.55957989652, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0002134263248686069, "train_loss": 5.153292655944824, "grad_norm": 0.6376813650131226, "tokens_seen": 21635072, "tokens_per_sec": 18155.524229345854, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.00020941562214974222, "train_loss": 5.0194292068481445, "grad_norm": 0.6389662027359009, "tokens_seen": 21798912, "tokens_per_sec": 18160.28073235553, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0002054378859957543, "train_loss": 4.8527398109436035, "grad_norm": 0.7898289561271667, "tokens_seen": 21962752, "tokens_per_sec": 18165.020272942173, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0002014942038799901, "train_loss": 5.124648094177246, "grad_norm": 0.728542149066925, "tokens_seen": 22126592, "tokens_per_sec": 18169.74279254264, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.00019758565396576285, "train_loss": 5.1080543994903564, "grad_norm": 0.6628199219703674, "tokens_seen": 22290432, "tokens_per_sec": 18174.547324187042, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.00019371330481159223, "train_loss": 5.087373971939087, "grad_norm": 0.633826732635498, "tokens_seen": 22454272, "tokens_per_sec": 18179.177349170106, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.00018987821507907194, "train_loss": 4.941345930099487, "grad_norm": 0.6558794975280762, "tokens_seen": 22618112, "tokens_per_sec": 18183.650243100554, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.00018608143324344171, "train_loss": 5.05634880065918, "grad_norm": 0.6421486139297485, "tokens_seen": 22781952, "tokens_per_sec": 18188.159466901805, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.00018232399730694476, "train_loss": 5.001089096069336, "grad_norm": 0.6859514117240906, "tokens_seen": 22945792, "tokens_per_sec": 18192.738600906607, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2800, "val_loss": 5.133323204517365, "val_ppl": 169.57973081596052, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 22945792} +{"event": "step", "step": 2820, "lr": 0.00017860693451504837, "train_loss": 5.0584540367126465, "grad_norm": 0.6676008105278015, "tokens_seen": 23109632, "tokens_per_sec": 18152.38310226935, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.00017493126107560522, "train_loss": 4.8624114990234375, "grad_norm": 0.7296179533004761, "tokens_seen": 23273472, "tokens_per_sec": 18157.117341836925, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.00017129798188103226, "train_loss": 5.036167144775391, "grad_norm": 0.653252899646759, "tokens_seen": 23437312, "tokens_per_sec": 18161.544265270804, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.00016770809023358322, "train_loss": 5.027592897415161, "grad_norm": 0.683820903301239, "tokens_seen": 23601152, "tokens_per_sec": 18166.389635477102, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.00016416256757378968, "train_loss": 4.881633520126343, "grad_norm": 0.6839029788970947, "tokens_seen": 23764992, "tokens_per_sec": 18170.89653966359, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.000160662383212145, "train_loss": 5.077390193939209, "grad_norm": 0.7003127932548523, "tokens_seen": 23928832, "tokens_per_sec": 18174.895521496564, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.00015720849406410448, "train_loss": 4.994017839431763, "grad_norm": 0.6357890367507935, "tokens_seen": 24092672, "tokens_per_sec": 18179.318664170263, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.00015380184438847444, "train_loss": 4.985954523086548, "grad_norm": 0.6746111512184143, "tokens_seen": 24256512, "tokens_per_sec": 18183.708705130248, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.00015044336552926094, "train_loss": 4.934262752532959, "grad_norm": 0.6357934474945068, "tokens_seen": 24420352, "tokens_per_sec": 18187.984296791357, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.00014713397566104995, "train_loss": 4.884220600128174, "grad_norm": 0.6793148517608643, "tokens_seen": 24584192, "tokens_per_sec": 18192.14859320166, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.145605909824371, "val_ppl": 171.67547301980596, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.00014387457953798726, "train_loss": 4.952087163925171, "grad_norm": 0.6581339836120605, "tokens_seen": 24748032, "tokens_per_sec": 18154.817612931314, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.00014066606824642808, "train_loss": 5.0338215827941895, "grad_norm": 0.7215169072151184, "tokens_seen": 24911872, "tokens_per_sec": 18159.242181539244, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.00013750931896132278, "train_loss": 4.999096393585205, "grad_norm": 0.7020614147186279, "tokens_seen": 25075712, "tokens_per_sec": 18163.31449225733, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.00013440519470640596, "train_loss": 4.822230815887451, "grad_norm": 0.6542312502861023, "tokens_seen": 25239552, "tokens_per_sec": 18166.63102237743, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.00013135454411825447, "train_loss": 5.060709238052368, "grad_norm": 0.6968449950218201, "tokens_seen": 25403392, "tokens_per_sec": 18170.819916770983, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.00012835820121427852, "train_loss": 5.080213308334351, "grad_norm": 0.8090695142745972, "tokens_seen": 25567232, "tokens_per_sec": 18172.95076398586, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0001254169851647097, "train_loss": 4.920665740966797, "grad_norm": 0.6797633767127991, "tokens_seen": 25731072, "tokens_per_sec": 18172.859342862586, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.00012253170006864812, "train_loss": 5.069065570831299, "grad_norm": 0.6946625709533691, "tokens_seen": 25894912, "tokens_per_sec": 18176.83900514705, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.00011970313473422966, "train_loss": 4.9019787311553955, "grad_norm": 0.6690545082092285, "tokens_seen": 26058752, "tokens_per_sec": 18180.948553233982, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.00011693206246297376, "train_loss": 4.945833206176758, "grad_norm": 0.7340360879898071, "tokens_seen": 26222592, "tokens_per_sec": 18185.01747784885, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3200, "val_loss": 5.05357563495636, "val_ppl": 156.58134244071942, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 26222592} +{"event": "step", "step": 3220, "lr": 0.00011421924083837032, "train_loss": 5.041339874267578, "grad_norm": 0.7465406060218811, "tokens_seen": 26386432, "tokens_per_sec": 18149.526079890147, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.00011156541151876421, "train_loss": 5.053134202957153, "grad_norm": 0.6979294419288635, "tokens_seen": 26550272, "tokens_per_sec": 18153.83492715786, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.00010897130003459306, "train_loss": 4.812916994094849, "grad_norm": 0.6672149300575256, "tokens_seen": 26714112, "tokens_per_sec": 18158.038935565113, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.00010643761559003458, "train_loss": 5.079880714416504, "grad_norm": 0.7650197148323059, "tokens_seen": 26877952, "tokens_per_sec": 18162.05747313307, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0001039650508691173, "train_loss": 4.890748739242554, "grad_norm": 0.7002509236335754, "tokens_seen": 27041792, "tokens_per_sec": 18166.042106256904, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0001015542818463474, "train_loss": 5.045630931854248, "grad_norm": 0.6922226548194885, "tokens_seen": 27205632, "tokens_per_sec": 18169.87877114927, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 9.920596760190444e-05, "train_loss": 4.871500253677368, "grad_norm": 0.6724729537963867, "tokens_seen": 27369472, "tokens_per_sec": 18173.46477724081, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 9.692075014145504e-05, "train_loss": 5.037615060806274, "grad_norm": 0.7356539964675903, "tokens_seen": 27533312, "tokens_per_sec": 18177.37782741223, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 9.46992542206351e-05, "train_loss": 5.029249668121338, "grad_norm": 0.7290816307067871, "tokens_seen": 27697152, "tokens_per_sec": 18181.371055550866, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 9.254208717424797e-05, "train_loss": 4.897435903549194, "grad_norm": 0.6796750426292419, "tokens_seen": 27860992, "tokens_per_sec": 18185.28192425286, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3400, "val_loss": 5.056186878681183, "val_ppl": 156.99074878583568, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 27860992} +{"event": "step", "step": 3420, "lr": 9.04498387502248e-05, "train_loss": 4.939289093017578, "grad_norm": 0.7133690714836121, "tokens_seen": 28024832, "tokens_per_sec": 18152.126936181478, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 8.842308094839387e-05, "train_loss": 4.987432241439819, "grad_norm": 0.7499947547912598, "tokens_seen": 28188672, "tokens_per_sec": 18156.352386831648, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 8.646236786410109e-05, "train_loss": 4.864980220794678, "grad_norm": 0.758357048034668, "tokens_seen": 28352512, "tokens_per_sec": 18160.29517064148, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 8.456823553672612e-05, "train_loss": 4.971749544143677, "grad_norm": 0.6977288126945496, "tokens_seen": 28516352, "tokens_per_sec": 18163.825658082063, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 8.274120180313448e-05, "train_loss": 4.864761114120483, "grad_norm": 0.6976288557052612, "tokens_seen": 28680192, "tokens_per_sec": 18167.737286120544, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 8.098176615610612e-05, "train_loss": 4.772442579269409, "grad_norm": 0.7818974256515503, "tokens_seen": 28844032, "tokens_per_sec": 18171.38432916916, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 7.929040960777911e-05, "train_loss": 4.7628209590911865, "grad_norm": 0.682862401008606, "tokens_seen": 29007872, "tokens_per_sec": 18175.62293740538, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 7.76675945581456e-05, "train_loss": 5.011586904525757, "grad_norm": 0.6828359961509705, "tokens_seen": 29171712, "tokens_per_sec": 18179.474318197103, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 7.611376466863648e-05, "train_loss": 4.840420246124268, "grad_norm": 0.6581909656524658, "tokens_seen": 29335552, "tokens_per_sec": 18182.887434260894, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 7.462934474082865e-05, "train_loss": 4.862175464630127, "grad_norm": 0.765326976776123, "tokens_seen": 29499392, "tokens_per_sec": 18187.049941392423, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3600, "val_loss": 5.025522518157959, "val_ppl": 152.2497886113675, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 29499392} +{"event": "step", "step": 3620, "lr": 7.321474060030854e-05, "train_loss": 4.925624132156372, "grad_norm": 0.7254661321640015, "tokens_seen": 29663232, "tokens_per_sec": 18155.940015251494, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 7.187033898572343e-05, "train_loss": 4.997907400131226, "grad_norm": 0.6864737868309021, "tokens_seen": 29827072, "tokens_per_sec": 18159.881733124792, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 7.0596507443051e-05, "train_loss": 5.04073166847229, "grad_norm": 0.777411937713623, "tokens_seen": 29990912, "tokens_per_sec": 18163.652251289033, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 6.93935942251159e-05, "train_loss": 4.970736026763916, "grad_norm": 0.6997203230857849, "tokens_seen": 30154752, "tokens_per_sec": 18167.526713493175, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 6.82619281963808e-05, "train_loss": 5.018963575363159, "grad_norm": 0.7302367687225342, "tokens_seen": 30318592, "tokens_per_sec": 18170.901028362034, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 6.720181874303845e-05, "train_loss": 4.793079376220703, "grad_norm": 0.6881300210952759, "tokens_seen": 30482432, "tokens_per_sec": 18174.78401832493, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 6.62135556884282e-05, "train_loss": 4.957826137542725, "grad_norm": 0.7509734034538269, "tokens_seen": 30646272, "tokens_per_sec": 18178.66230622976, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 6.529740921380147e-05, "train_loss": 4.898303270339966, "grad_norm": 0.7236469388008118, "tokens_seen": 30810112, "tokens_per_sec": 18182.396374684973, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 6.445362978445676e-05, "train_loss": 4.800357341766357, "grad_norm": 0.7561899423599243, "tokens_seen": 30973952, "tokens_per_sec": 18185.872850132888, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 6.368244808126497e-05, "train_loss": 4.908676385879517, "grad_norm": 0.7471641898155212, "tokens_seen": 31137792, "tokens_per_sec": 18189.502467406604, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3800, "val_loss": 5.0056939125061035, "val_ppl": 149.26062104272722, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 31137792} +{"event": "step", "step": 3820, "lr": 6.29840749376035e-05, "train_loss": 4.986484527587891, "grad_norm": 0.737016499042511, "tokens_seen": 31301632, "tokens_per_sec": 18159.95685358954, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 6.23587012817167e-05, "train_loss": 4.905251979827881, "grad_norm": 0.7553510665893555, "tokens_seen": 31465472, "tokens_per_sec": 18163.635972145206, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 6.18064980845177e-05, "train_loss": 4.902936220169067, "grad_norm": 0.6817551851272583, "tokens_seen": 31629312, "tokens_per_sec": 18167.17573373074, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 6.132761631284691e-05, "train_loss": 4.980824947357178, "grad_norm": 0.7894168496131897, "tokens_seen": 31793152, "tokens_per_sec": 18170.784514219835, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 6.092218688819914e-05, "train_loss": 4.880906105041504, "grad_norm": 0.7770647406578064, "tokens_seen": 31956992, "tokens_per_sec": 18174.419308152195, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 6.059032065093088e-05, "train_loss": 4.813976049423218, "grad_norm": 0.7603470683097839, "tokens_seen": 32120832, "tokens_per_sec": 18178.17438162713, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 6.033210832995772e-05, "train_loss": 4.990556716918945, "grad_norm": 0.7670046091079712, "tokens_seen": 32284672, "tokens_per_sec": 18181.68601398659, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 6.014762051794997e-05, "train_loss": 4.936196327209473, "grad_norm": 0.7520145773887634, "tokens_seen": 32448512, "tokens_per_sec": 18185.2270276044, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 6.003690765203337e-05, "train_loss": 4.885009527206421, "grad_norm": 0.7599259614944458, "tokens_seen": 32612352, "tokens_per_sec": 18188.723793420642, "vram_gb": 5.96580096, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3999, "val_loss": 5.000214111804962, "val_ppl": 148.44493951411144, "lambda": null, "S_mean": null, "G_mean": null, "G_std": null, "tokens_seen": 32768000} +{"event": "done", "step": 3999, "elapsed_s": 1804.3665130138397, "nan_failures": 0} diff --git a/runs_4ksteps/full_s0/config.json b/runs_4ksteps/full_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..5b8c6a98e824181ec6a98e7f1282b5293a12dc5e --- /dev/null +++ b/runs_4ksteps/full_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "full", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s0", + "max_steps": 4000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 200, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/runs_4ksteps/full_s0/final.pt b/runs_4ksteps/full_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..e1cc17a8e7f17f21f8f996aed0ed22d5fb83624b --- /dev/null +++ b/runs_4ksteps/full_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a34a16b626b6bc9fb097e863c9e155d5d396b6968ffb36b3c522c0dfbd149912 +size 256003111 diff --git a/runs_4ksteps/full_s0/metrics.jsonl b/runs_4ksteps/full_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..a3b03f452f595327228861defa96e58c2ba6a47d --- /dev/null +++ b/runs_4ksteps/full_s0/metrics.jsonl @@ -0,0 +1,222 @@ +{"event": "start", "params": 63975624, "variant": "full", "seed": 0, "config": {"variant": "full", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\full_s0", "max_steps": 4000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 200, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.900177001953125, "grad_norm": 11.217808723449707, "tokens_seen": 8192, "tokens_per_sec": 7176.8324336811565, "vram_gb": 5.920787456, "lambda": 0.12692801654338837, "S_mean": 0.1771087646484375, "G_mean": 0.5060585426787535, "G_std": 0.08616725790003936, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.83952522277832, "grad_norm": 2.0422682762145996, "tokens_seen": 172032, "tokens_per_sec": 14664.667173540085, "vram_gb": 6.442267648, "lambda": 0.12691388465464115, "S_mean": 0.4490610758463542, "G_mean": 0.46947087223331135, "G_std": 0.0659222734781603, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.111881256103516, "grad_norm": 1.7455397844314575, "tokens_seen": 335872, "tokens_per_sec": 15000.446293061179, "vram_gb": 6.442267648, "lambda": 0.12681447404126325, "S_mean": 0.499298095703125, "G_mean": 0.4522867053747177, "G_std": 0.07119619349638621, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.130086898803711, "grad_norm": 1.5878995656967163, "tokens_seen": 499712, "tokens_per_sec": 15154.322446997336, "vram_gb": 6.442267648, "lambda": 0.12670463137328625, "S_mean": 0.4830118815104167, "G_mean": 0.44430506974458694, "G_std": 0.08355965232476592, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.426074981689453, "grad_norm": 0.7696790099143982, "tokens_seen": 663552, "tokens_per_sec": 15222.849280485496, "vram_gb": 6.442267648, "lambda": 0.12660598816970983, "S_mean": 0.3882242838541667, "G_mean": 0.4503638831277688, "G_std": 0.09322876522007088, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.213724613189697, "grad_norm": 0.9722426533699036, "tokens_seen": 827392, "tokens_per_sec": 15266.59563220955, "vram_gb": 6.442267648, "lambda": 0.126494025811553, "S_mean": 0.2703959147135417, "G_mean": 0.4433469759921233, "G_std": 0.10379659477621317, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.054359436035156, "grad_norm": 0.6330053210258484, "tokens_seen": 991232, "tokens_per_sec": 15300.929968645398, "vram_gb": 6.442267648, "lambda": 0.12652451234559217, "S_mean": 0.242279052734375, "G_mean": 0.4225224206844966, "G_std": 0.1187041209389766, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.838622093200684, "grad_norm": 0.5778141617774963, "tokens_seen": 1155072, "tokens_per_sec": 15322.297940695447, "vram_gb": 6.442267648, "lambda": 0.12646274889508882, "S_mean": 0.3059488932291667, "G_mean": 0.4161691988507907, "G_std": 0.12325356776515643, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.909845590591431, "grad_norm": 0.791572630405426, "tokens_seen": 1318912, "tokens_per_sec": 15326.947603040517, "vram_gb": 6.442267648, "lambda": 0.12636082619428635, "S_mean": 0.276123046875, "G_mean": 0.41686703140536946, "G_std": 0.1313117379322648, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.799734830856323, "grad_norm": 0.6125876307487488, "tokens_seen": 1482752, "tokens_per_sec": 15329.370919100018, "vram_gb": 6.442267648, "lambda": 0.12636513697604337, "S_mean": 0.3077392578125, "G_mean": 0.40593364586432773, "G_std": 0.1365721638624867, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.676538467407227, "grad_norm": 0.4776296317577362, "tokens_seen": 1646592, "tokens_per_sec": 15341.296128920862, "vram_gb": 6.442267648, "lambda": 0.12632009697457156, "S_mean": 0.3238321940104167, "G_mean": 0.4146435211102168, "G_std": 0.14508001475284496, "skipped": 0} +{"event": "eval", "step": 200, "val_loss": 6.740578007698059, "val_ppl": 846.0496177406719, "lambda": 0.12631632015109062, "S_mean": 0.30438232421875, "G_mean": 0.4108195664982001, "G_std": 0.1411371904735764, "tokens_seen": 1646592} +{"event": "step", "step": 220, "lr": 0.0005999630923479665, "train_loss": 6.571054697036743, "grad_norm": 0.485915869474411, "tokens_seen": 1810432, "tokens_per_sec": 14849.062126735316, "vram_gb": 6.442267648, "lambda": 0.12637140632917485, "S_mean": 0.30487060546875, "G_mean": 0.4111607000231743, "G_std": 0.15017301759993038, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.00059985237948205, "train_loss": 6.627575874328613, "grad_norm": 0.5539427399635315, "tokens_seen": 1974272, "tokens_per_sec": 14895.810205847609, "vram_gb": 6.442267648, "lambda": 0.12644176309307417, "S_mean": 0.3676961263020833, "G_mean": 0.4052786926428477, "G_std": 0.15695051973064741, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005996678916700422, "train_loss": 6.59104585647583, "grad_norm": 0.4981743097305298, "tokens_seen": 2138112, "tokens_per_sec": 14934.810827041905, "vram_gb": 6.442267648, "lambda": 0.12642955655852953, "S_mean": 0.3065388997395833, "G_mean": 0.40804433325926465, "G_std": 0.16824711362520853, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.000599409679349069, "train_loss": 6.505597114562988, "grad_norm": 0.49355703592300415, "tokens_seen": 2301952, "tokens_per_sec": 14969.398724404386, "vram_gb": 6.442267648, "lambda": 0.12644678788880506, "S_mean": 0.2553609212239583, "G_mean": 0.4035586975514889, "G_std": 0.168953076004982, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005990778131118008, "train_loss": 6.335359573364258, "grad_norm": 0.4680493175983429, "tokens_seen": 2465792, "tokens_per_sec": 14998.635597458462, "vram_gb": 6.442267648, "lambda": 0.1264594110349814, "S_mean": 0.3079427083333333, "G_mean": 0.41425324728091556, "G_std": 0.17298902198672295, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.0005986723836871531, "train_loss": 6.4139697551727295, "grad_norm": 0.5300203561782837, "tokens_seen": 2629632, "tokens_per_sec": 15024.378636251833, "vram_gb": 6.442267648, "lambda": 0.12646780970195928, "S_mean": 0.3309733072916667, "G_mean": 0.4137684901555379, "G_std": 0.1818290495624145, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005981935019154823, "train_loss": 6.394108295440674, "grad_norm": 0.5258840918540955, "tokens_seen": 2793472, "tokens_per_sec": 15046.873584805757, "vram_gb": 6.442267648, "lambda": 0.12651423954715332, "S_mean": 0.3216756184895833, "G_mean": 0.41517286996046704, "G_std": 0.19027234924336275, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005976412987182833, "train_loss": 6.2520225048065186, "grad_norm": 0.48427748680114746, "tokens_seen": 2957312, "tokens_per_sec": 15070.537752963446, "vram_gb": 6.442267648, "lambda": 0.1266111023724079, "S_mean": 0.3291219075520833, "G_mean": 0.40464745213588077, "G_std": 0.1908633985246221, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005970159250623965, "train_loss": 6.210533380508423, "grad_norm": 0.5072532892227173, "tokens_seen": 3121152, "tokens_per_sec": 15090.675090508359, "vram_gb": 6.442267648, "lambda": 0.12667184695601463, "S_mean": 0.3456827799479167, "G_mean": 0.41207972665627796, "G_std": 0.1978065405661861, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.000596317551918735, "train_loss": 6.249718189239502, "grad_norm": 0.4806963801383972, "tokens_seen": 3284992, "tokens_per_sec": 15108.855286334878, "vram_gb": 6.442267648, "lambda": 0.12674504642685255, "S_mean": 0.3197835286458333, "G_mean": 0.41418295974532765, "G_std": 0.20088934370627007, "skipped": 0} +{"event": "eval", "step": 400, "val_loss": 6.391335022449494, "val_ppl": 596.6525928280578, "lambda": 0.12675445899367332, "S_mean": 0.33465576171875, "G_mean": 0.4107739217579365, "G_std": 0.20222124426315227, "tokens_seen": 3284992} +{"event": "step", "step": 420, "lr": 0.0005955463702155432, "train_loss": 6.3861095905303955, "grad_norm": 0.6592063903808594, "tokens_seen": 3448832, "tokens_per_sec": 14859.94086382336, "vram_gb": 6.442267648, "lambda": 0.1268323635061582, "S_mean": 0.39794921875, "G_mean": 0.43083703393737477, "G_std": 0.20641051201770702, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005947025907861985, "train_loss": 6.340380907058716, "grad_norm": 0.5294669270515442, "tokens_seen": 3612672, "tokens_per_sec": 14884.47551995655, "vram_gb": 6.442267648, "lambda": 0.1269418727606535, "S_mean": 0.3649190266927083, "G_mean": 0.4135419428348541, "G_std": 0.20853857696056366, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005937864443115718, "train_loss": 6.273029327392578, "grad_norm": 0.49917635321617126, "tokens_seen": 3776512, "tokens_per_sec": 14906.933440559942, "vram_gb": 6.442267648, "lambda": 0.12701714721818766, "S_mean": 0.3820597330729167, "G_mean": 0.42971306045850116, "G_std": 0.21567310951650143, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005927981812569615, "train_loss": 6.366248607635498, "grad_norm": 0.48466044664382935, "tokens_seen": 3940352, "tokens_per_sec": 14927.006870494086, "vram_gb": 6.442267648, "lambda": 0.12716670148074627, "S_mean": 0.39788818359375, "G_mean": 0.43850669637322426, "G_std": 0.22153685335069895, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005917380718036191, "train_loss": 6.292417526245117, "grad_norm": 0.49576812982559204, "tokens_seen": 4104192, "tokens_per_sec": 14946.077305678778, "vram_gb": 6.442267648, "lambda": 0.12731456570327282, "S_mean": 0.3658345540364583, "G_mean": 0.4456465107699235, "G_std": 0.22521749138832092, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005906064057748841, "train_loss": 6.151647329330444, "grad_norm": 0.5173494815826416, "tokens_seen": 4268032, "tokens_per_sec": 14962.815762565238, "vram_gb": 6.442267648, "lambda": 0.12751984472076097, "S_mean": 0.3544108072916667, "G_mean": 0.4579197404285272, "G_std": 0.22025402386983237, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.000589403492556949, "train_loss": 6.171376705169678, "grad_norm": 0.48038309812545776, "tokens_seen": 4431872, "tokens_per_sec": 14981.230956384907, "vram_gb": 6.442267648, "lambda": 0.12762735784053802, "S_mean": 0.3943379720052083, "G_mean": 0.46552255749702454, "G_std": 0.22254571070273718, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.0005881296610142765, "train_loss": 6.021725654602051, "grad_norm": 0.5069782733917236, "tokens_seen": 4595712, "tokens_per_sec": 14997.719030276869, "vram_gb": 6.442267648, "lambda": 0.127838846296072, "S_mean": 0.373809814453125, "G_mean": 0.4710191413760185, "G_std": 0.2286540207763513, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005867852593996914, "train_loss": 6.194086074829102, "grad_norm": 0.5277597904205322, "tokens_seen": 4759552, "tokens_per_sec": 15011.616873623914, "vram_gb": 6.442267648, "lambda": 0.1280172864596049, "S_mean": 0.35546875, "G_mean": 0.4879833509524663, "G_std": 0.2279368738333384, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005853706552591712, "train_loss": 5.949712753295898, "grad_norm": 0.4799976348876953, "tokens_seen": 4923392, "tokens_per_sec": 15022.70418584511, "vram_gb": 6.442267648, "lambda": 0.1282313127691547, "S_mean": 0.391937255859375, "G_mean": 0.5293142658968767, "G_std": 0.22468060938020548, "skipped": 0} +{"event": "eval", "step": 600, "val_loss": 6.15944002866745, "val_ppl": 473.16304289697194, "lambda": 0.12824411317706108, "S_mean": 0.3597310384114583, "G_mean": 0.49930787707368535, "G_std": 0.2299194366981586, "tokens_seen": 4923392} +{"event": "step", "step": 620, "lr": 0.0005838862353313635, "train_loss": 6.015196800231934, "grad_norm": 0.4714614450931549, "tokens_seen": 5087232, "tokens_per_sec": 14856.760768982333, "vram_gb": 6.442267648, "lambda": 0.1284641269594431, "S_mean": 0.3969217936197917, "G_mean": 0.5108194835484028, "G_std": 0.22358550814290842, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005823324054418544, "train_loss": 6.097868204116821, "grad_norm": 0.5972551107406616, "tokens_seen": 5251072, "tokens_per_sec": 14871.573545955283, "vram_gb": 6.442267648, "lambda": 0.12870528859396776, "S_mean": 0.3539835611979167, "G_mean": 0.5306381694972515, "G_std": 0.22862366338570914, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005807095903922209, "train_loss": 5.932755470275879, "grad_norm": 0.5194063782691956, "tokens_seen": 5414912, "tokens_per_sec": 14885.286359280983, "vram_gb": 6.442267648, "lambda": 0.12897684735556444, "S_mean": 0.3624471028645833, "G_mean": 0.569448359310627, "G_std": 0.21819603629410267, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005790182338438938, "train_loss": 5.943108797073364, "grad_norm": 0.4997767508029938, "tokens_seen": 5578752, "tokens_per_sec": 14900.006594245366, "vram_gb": 6.442267648, "lambda": 0.12917988343785206, "S_mean": 0.3720499674479167, "G_mean": 0.5352130594352881, "G_std": 0.22445140344401202, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005772587981968655, "train_loss": 6.019903659820557, "grad_norm": 0.4530823528766632, "tokens_seen": 5742592, "tokens_per_sec": 14916.574125597637, "vram_gb": 6.442267648, "lambda": 0.12941528173784414, "S_mean": 0.3690999348958333, "G_mean": 0.5478734436134497, "G_std": 0.2242983958373467, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005754317644632739, "train_loss": 6.00521183013916, "grad_norm": 0.4973089396953583, "tokens_seen": 5906432, "tokens_per_sec": 14927.903805709315, "vram_gb": 6.442267648, "lambda": 0.12969180072347322, "S_mean": 0.3727620442708333, "G_mean": 0.5475347439448038, "G_std": 0.22381079259018102, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005735376321358989, "train_loss": 6.026705980300903, "grad_norm": 0.4943159818649292, "tokens_seen": 6070272, "tokens_per_sec": 14940.06111850611, "vram_gb": 6.442267648, "lambda": 0.13002433255314827, "S_mean": 0.3659261067708333, "G_mean": 0.5399082625905672, "G_std": 0.21828990429639816, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.000571576919051606, "train_loss": 5.89639949798584, "grad_norm": 0.5357763767242432, "tokens_seen": 6234112, "tokens_per_sec": 14953.698371970344, "vram_gb": 6.442267648, "lambda": 0.13041084135572115, "S_mean": 0.3721923828125, "G_mean": 0.5374414511024952, "G_std": 0.22152527794241905, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.0005695501612497752, "train_loss": 5.909939765930176, "grad_norm": 0.4712146520614624, "tokens_seen": 6397952, "tokens_per_sec": 14965.83880914147, "vram_gb": 6.442267648, "lambda": 0.13078376495589814, "S_mean": 0.330078125, "G_mean": 0.5692521668970585, "G_std": 0.21250928627947965, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000567457912825752, "train_loss": 5.864057540893555, "grad_norm": 0.4830041229724884, "tokens_seen": 6561792, "tokens_per_sec": 14976.858250900246, "vram_gb": 6.442267648, "lambda": 0.13108982487271229, "S_mean": 0.365966796875, "G_mean": 0.5646844009558359, "G_std": 0.21123078341285387, "skipped": 0} +{"event": "eval", "step": 800, "val_loss": 5.969862556457519, "val_ppl": 391.4518645213431, "lambda": 0.1311080645148953, "S_mean": 0.33685302734375, "G_mean": 0.5744916958113512, "G_std": 0.21350263400624195, "tokens_seen": 6561792} +{"event": "step", "step": 820, "lr": 0.0005653007457793648, "train_loss": 5.946488857269287, "grad_norm": 0.49574118852615356, "tokens_seen": 6725632, "tokens_per_sec": 14852.602168949461, "vram_gb": 6.442267648, "lambda": 0.13149288886537155, "S_mean": 0.3365885416666667, "G_mean": 0.5422952758769194, "G_std": 0.21813346600780883, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.000563079249858545, "train_loss": 6.123433828353882, "grad_norm": 0.4949234426021576, "tokens_seen": 6889472, "tokens_per_sec": 14863.75297774907, "vram_gb": 6.442267648, "lambda": 0.13181905820965767, "S_mean": 0.3338216145833333, "G_mean": 0.5669704626003901, "G_std": 0.21351838173965612, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005607940323980955, "train_loss": 5.937444686889648, "grad_norm": 0.535386323928833, "tokens_seen": 7053312, "tokens_per_sec": 14873.222608559063, "vram_gb": 6.442267648, "lambda": 0.1321754902601242, "S_mean": 0.3346150716145833, "G_mean": 0.5561091440419356, "G_std": 0.211072678056856, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005584457181536525, "train_loss": 5.927929162979126, "grad_norm": 0.5878754258155823, "tokens_seen": 7217152, "tokens_per_sec": 14883.407063793053, "vram_gb": 6.442267648, "lambda": 0.13254042839010557, "S_mean": 0.3527018229166667, "G_mean": 0.563983965665102, "G_std": 0.21260740204403797, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005560349491308827, "train_loss": 5.827876806259155, "grad_norm": 0.48097407817840576, "tokens_seen": 7380992, "tokens_per_sec": 14894.847847954034, "vram_gb": 6.442267648, "lambda": 0.13288607758780321, "S_mean": 0.35198974609375, "G_mean": 0.565286461263895, "G_std": 0.21128999690214792, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005535623844099654, "train_loss": 5.734800577163696, "grad_norm": 0.49217450618743896, "tokens_seen": 7544832, "tokens_per_sec": 14905.996848278974, "vram_gb": 6.442267648, "lambda": 0.13325560869028172, "S_mean": 0.36102294921875, "G_mean": 0.5610882913072904, "G_std": 0.20931372698396444, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.000551028699965407, "train_loss": 5.779804706573486, "grad_norm": 0.5187776684761047, "tokens_seen": 7708672, "tokens_per_sec": 14916.212402950363, "vram_gb": 6.442267648, "lambda": 0.13368192439277968, "S_mean": 0.30975341796875, "G_mean": 0.5797739451130232, "G_std": 0.20640257404496273, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005484345884812357, "train_loss": 5.778264045715332, "grad_norm": 0.5763224363327026, "tokens_seen": 7872512, "tokens_per_sec": 14925.816130046318, "vram_gb": 6.442267648, "lambda": 0.1340254961202542, "S_mean": 0.35247802734375, "G_mean": 0.5871407128870487, "G_std": 0.19892091366151968, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005457807591616296, "train_loss": 5.676121473312378, "grad_norm": 0.49401530623435974, "tokens_seen": 8036352, "tokens_per_sec": 14935.295057482268, "vram_gb": 6.442267648, "lambda": 0.13437930028885603, "S_mean": 0.3411051432291667, "G_mean": 0.563348725438118, "G_std": 0.20916269719600677, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.0005430679375370263, "train_loss": 5.792867660522461, "grad_norm": 0.5185261368751526, "tokens_seen": 8200192, "tokens_per_sec": 14945.683827758941, "vram_gb": 6.442267648, "lambda": 0.13478029984980822, "S_mean": 0.3387247721354167, "G_mean": 0.5567546946307024, "G_std": 0.20476983208209276, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.816040241718293, "val_ppl": 335.64036407441796, "lambda": 0.13480224770804247, "S_mean": 0.35235595703125, "G_mean": 0.5547167975455523, "G_std": 0.2071179955576857, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005402968652657703, "train_loss": 5.533315896987915, "grad_norm": 0.49728652834892273, "tokens_seen": 8364032, "tokens_per_sec": 14848.09072489041, "vram_gb": 6.442267648, "lambda": 0.13519225362688303, "S_mean": 0.34820556640625, "G_mean": 0.5625596257547537, "G_std": 0.2006139342362682, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005374682999313519, "train_loss": 5.776583671569824, "grad_norm": 0.5156252980232239, "tokens_seen": 8527872, "tokens_per_sec": 14856.462904448901, "vram_gb": 6.442267648, "lambda": 0.13549139443784952, "S_mean": 0.3170166015625, "G_mean": 0.57581627368927, "G_std": 0.2027895621334513, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005345830148352904, "train_loss": 5.49694299697876, "grad_norm": 0.5135908722877502, "tokens_seen": 8691712, "tokens_per_sec": 14865.717018935713, "vram_gb": 6.442267648, "lambda": 0.13587642088532448, "S_mean": 0.357666015625, "G_mean": 0.532783313964804, "G_std": 0.2080947089319428, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005316417987857215, "train_loss": 5.696583032608032, "grad_norm": 0.5302608609199524, "tokens_seen": 8855552, "tokens_per_sec": 14873.979516833033, "vram_gb": 6.442267648, "lambda": 0.13626651465892792, "S_mean": 0.3226420084635417, "G_mean": 0.5369457434862852, "G_std": 0.20461232277254263, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005286454558817454, "train_loss": 5.6651763916015625, "grad_norm": 0.6029629707336426, "tokens_seen": 9019392, "tokens_per_sec": 14882.652621099154, "vram_gb": 6.442267648, "lambda": 0.13660530528674522, "S_mean": 0.3177388509114583, "G_mean": 0.5775514940420786, "G_std": 0.20568437408655882, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.000525594805293594, "train_loss": 5.598934173583984, "grad_norm": 0.5290517210960388, "tokens_seen": 9183232, "tokens_per_sec": 14891.92475146329, "vram_gb": 6.442267648, "lambda": 0.13700281300892433, "S_mean": 0.3252766927083333, "G_mean": 0.5345892341186603, "G_std": 0.2053002572307984, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005224906810386772, "train_loss": 5.85844087600708, "grad_norm": 0.5295674204826355, "tokens_seen": 9347072, "tokens_per_sec": 14899.308500193198, "vram_gb": 6.442267648, "lambda": 0.1373259505877892, "S_mean": 0.3228759765625, "G_mean": 0.5394855334113041, "G_std": 0.20267565455287695, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005193339317535718, "train_loss": 5.591250419616699, "grad_norm": 0.48536035418510437, "tokens_seen": 9510912, "tokens_per_sec": 14907.499109006729, "vram_gb": 6.442267648, "lambda": 0.1377501521880428, "S_mean": 0.3446044921875, "G_mean": 0.54696986079216, "G_std": 0.20268137815097967, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005161254204620127, "train_loss": 5.659907817840576, "grad_norm": 0.5145034790039062, "tokens_seen": 9674752, "tokens_per_sec": 14915.896931669935, "vram_gb": 6.442267648, "lambda": 0.13811772099385658, "S_mean": 0.3374430338541667, "G_mean": 0.5348725412040949, "G_std": 0.20591481464604536, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005128660243389501, "train_loss": 5.553635358810425, "grad_norm": 0.5007375478744507, "tokens_seen": 9838592, "tokens_per_sec": 14924.137527873081, "vram_gb": 6.442267648, "lambda": 0.13851364267369112, "S_mean": 0.3275553385416667, "G_mean": 0.5457661300897598, "G_std": 0.2040491821244359, "skipped": 0} +{"event": "eval", "step": 1200, "val_loss": 5.666674482822418, "val_ppl": 289.07162154453937, "lambda": 0.13853390545894703, "S_mean": 0.3564046223958333, "G_mean": 0.545887304469943, "G_std": 0.2014425089582801, "tokens_seen": 9838592} +{"event": "step", "step": 1220, "lr": 0.000509556634470739, "train_loss": 5.561213254928589, "grad_norm": 0.5437570214271545, "tokens_seen": 10002432, "tokens_per_sec": 14842.011255868023, "vram_gb": 6.442267648, "lambda": 0.13890980711827675, "S_mean": 0.3364461263020833, "G_mean": 0.5422924067825079, "G_std": 0.20183626314004263, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005061981556115254, "train_loss": 5.4509100914001465, "grad_norm": 0.5164843201637268, "tokens_seen": 10166272, "tokens_per_sec": 14850.574856588542, "vram_gb": 6.442267648, "lambda": 0.13927926247318587, "S_mean": 0.3325602213541667, "G_mean": 0.5455869063735008, "G_std": 0.20312013694395623, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005027915059358954, "train_loss": 5.4327006340026855, "grad_norm": 0.49477270245552063, "tokens_seen": 10330112, "tokens_per_sec": 14858.49457891609, "vram_gb": 6.442267648, "lambda": 0.13965522777289152, "S_mean": 0.3310139973958333, "G_mean": 0.5358441267162561, "G_std": 0.19914772237340608, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0004993376167878549, "train_loss": 5.577130079269409, "grad_norm": 0.5671594142913818, "tokens_seen": 10493952, "tokens_per_sec": 14866.314173918638, "vram_gb": 6.442267648, "lambda": 0.140042661378781, "S_mean": 0.3285725911458333, "G_mean": 0.5416806563735008, "G_std": 0.20075249268362919, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0004958374324262102, "train_loss": 5.671081304550171, "grad_norm": 0.5690954923629761, "tokens_seen": 10657792, "tokens_per_sec": 14873.982745906756, "vram_gb": 6.442267648, "lambda": 0.14043778398384651, "S_mean": 0.3547566731770833, "G_mean": 0.539463714386026, "G_std": 0.2019032733514905, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0004922919097664167, "train_loss": 5.622482538223267, "grad_norm": 0.5627391934394836, "tokens_seen": 10821632, "tokens_per_sec": 14881.456220017983, "vram_gb": 6.442267648, "lambda": 0.14081103230516115, "S_mean": 0.33203125, "G_mean": 0.534868306790789, "G_std": 0.20513329189270735, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0004887020181189677, "train_loss": 5.738431453704834, "grad_norm": 0.5538545250892639, "tokens_seen": 10985472, "tokens_per_sec": 14888.922897512686, "vram_gb": 6.442267648, "lambda": 0.1412075301632285, "S_mean": 0.3426717122395833, "G_mean": 0.537356911227107, "G_std": 0.20178338636954626, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.00048506873892439467, "train_loss": 5.5309765338897705, "grad_norm": 0.5704749822616577, "tokens_seen": 11149312, "tokens_per_sec": 14896.95825174596, "vram_gb": 6.442267648, "lambda": 0.14158294132600227, "S_mean": 0.3231608072916667, "G_mean": 0.5233028481403986, "G_std": 0.201949089144667, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0004813930654849516, "train_loss": 5.409682273864746, "grad_norm": 0.5288364291191101, "tokens_seen": 11313152, "tokens_per_sec": 14903.962147973076, "vram_gb": 6.442267648, "lambda": 0.14195760743071636, "S_mean": 0.3400065104166667, "G_mean": 0.5073543048153321, "G_std": 0.20137027961512408, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0004776760026930552, "train_loss": 5.423781871795654, "grad_norm": 0.5672229528427124, "tokens_seen": 11476992, "tokens_per_sec": 14909.723403850088, "vram_gb": 6.442267648, "lambda": 0.14229492781062922, "S_mean": 0.3434041341145833, "G_mean": 0.524269043157498, "G_std": 0.20113504584878683, "skipped": 0} +{"event": "eval", "step": 1400, "val_loss": 5.559950363636017, "val_ppl": 259.8099399821424, "lambda": 0.14231197504947582, "S_mean": 0.3185323079427083, "G_mean": 0.5178466147432724, "G_std": 0.20058413812269768, "tokens_seen": 11476992} +{"event": "step", "step": 1420, "lr": 0.00047391856675655823, "train_loss": 5.399591445922852, "grad_norm": 0.5608125925064087, "tokens_seen": 11640832, "tokens_per_sec": 14840.068812259364, "vram_gb": 6.442267648, "lambda": 0.14264872639129558, "S_mean": 0.3261922200520833, "G_mean": 0.5051169519623121, "G_std": 0.20308939026047787, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.00047012178492092805, "train_loss": 5.394753932952881, "grad_norm": 0.567459762096405, "tokens_seen": 11804672, "tokens_per_sec": 14846.73099571069, "vram_gb": 6.442267648, "lambda": 0.14301192946732044, "S_mean": 0.3330891927083333, "G_mean": 0.5213417652994394, "G_std": 0.20236268763740858, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.00046628669518840776, "train_loss": 5.514447212219238, "grad_norm": 0.5923293232917786, "tokens_seen": 11968512, "tokens_per_sec": 14853.14919103156, "vram_gb": 6.442267648, "lambda": 0.1434003453080853, "S_mean": 0.3236083984375, "G_mean": 0.5063190547128519, "G_std": 0.20700944680720568, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.00046241434603423715, "train_loss": 5.5659544467926025, "grad_norm": 0.5601904392242432, "tokens_seen": 12132352, "tokens_per_sec": 14859.761485347504, "vram_gb": 6.442267648, "lambda": 0.1437656677638491, "S_mean": 0.3729654947916667, "G_mean": 0.5025668038676182, "G_std": 0.2033011494204402, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0004585057961200098, "train_loss": 5.460009336471558, "grad_norm": 0.5720506310462952, "tokens_seen": 12296192, "tokens_per_sec": 14865.64467289376, "vram_gb": 6.442267648, "lambda": 0.14413854635010162, "S_mean": 0.32379150390625, "G_mean": 0.4969680073360602, "G_std": 0.20426641839245954, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.00045456211400424565, "train_loss": 5.445913791656494, "grad_norm": 0.5427185297012329, "tokens_seen": 12460032, "tokens_per_sec": 14872.806812916358, "vram_gb": 6.442267648, "lambda": 0.14453936771800122, "S_mean": 0.3244425455729167, "G_mean": 0.5143021817008654, "G_std": 0.20330276805907488, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0004505843778502577, "train_loss": 5.34141993522644, "grad_norm": 0.5131933093070984, "tokens_seen": 12623872, "tokens_per_sec": 14879.67497369527, "vram_gb": 6.442267648, "lambda": 0.1449366994202137, "S_mean": 0.3218180338541667, "G_mean": 0.5127804862956206, "G_std": 0.20361579147477946, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.000446573675131393, "train_loss": 5.364423513412476, "grad_norm": 0.5593886971473694, "tokens_seen": 12787712, "tokens_per_sec": 14885.636960903079, "vram_gb": 6.442267648, "lambda": 0.14528835751116276, "S_mean": 0.3405965169270833, "G_mean": 0.4950882631043593, "G_std": 0.19999298950036368, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0004425311023337283, "train_loss": 5.432316303253174, "grad_norm": 0.6242905855178833, "tokens_seen": 12951552, "tokens_per_sec": 14891.946504067873, "vram_gb": 6.442267648, "lambda": 0.14564090967178345, "S_mean": 0.3337198893229167, "G_mean": 0.49802837893366814, "G_std": 0.2031185363108913, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0004384577646563017, "train_loss": 5.41972279548645, "grad_norm": 0.5526007413864136, "tokens_seen": 13115392, "tokens_per_sec": 14897.963956965426, "vram_gb": 6.442267648, "lambda": 0.14599672239273787, "S_mean": 0.3507080078125, "G_mean": 0.5047087588657936, "G_std": 0.20512275708218417, "skipped": 0} +{"event": "eval", "step": 1600, "val_loss": 5.478936398029328, "val_ppl": 239.5917415587405, "lambda": 0.14601524670918783, "S_mean": 0.3335367838541667, "G_mean": 0.505358261987567, "G_std": 0.20555959486713013, "tokens_seen": 13115392} +{"event": "step", "step": 1620, "lr": 0.00043435477570896204, "train_loss": 5.390178680419922, "grad_norm": 0.5447746515274048, "tokens_seen": 13279232, "tokens_per_sec": 14836.223705040373, "vram_gb": 6.442267648, "lambda": 0.14638349320739508, "S_mean": 0.3260091145833333, "G_mean": 0.5047999446590742, "G_std": 0.20510524604469538, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0004302232572079195, "train_loss": 5.262455463409424, "grad_norm": 0.5343047380447388, "tokens_seen": 13443072, "tokens_per_sec": 14842.941369651639, "vram_gb": 6.442267648, "lambda": 0.14677999913692474, "S_mean": 0.3223470052083333, "G_mean": 0.5010045052816471, "G_std": 0.20386168671151003, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0004260643386690794, "train_loss": 5.442059755325317, "grad_norm": 0.5404598116874695, "tokens_seen": 13606912, "tokens_per_sec": 14849.672515162843, "vram_gb": 6.442267648, "lambda": 0.14713385639091334, "S_mean": 0.345947265625, "G_mean": 0.501681626153489, "G_std": 0.20323590468615294, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.00042187915709924356, "train_loss": 5.5906171798706055, "grad_norm": 0.6456348896026611, "tokens_seen": 13770752, "tokens_per_sec": 14856.439164878951, "vram_gb": 6.442267648, "lambda": 0.1474885701512297, "S_mean": 0.3182779947916667, "G_mean": 0.5060627665370703, "G_std": 0.2021297076717019, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0004176688566852645, "train_loss": 5.446439743041992, "grad_norm": 0.5363678932189941, "tokens_seen": 13934592, "tokens_per_sec": 14862.271200033887, "vram_gb": 6.442267648, "lambda": 0.14782526809722185, "S_mean": 0.302978515625, "G_mean": 0.4984721277529995, "G_std": 0.20467048852394024, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.00041343458848123576, "train_loss": 5.211294412612915, "grad_norm": 0.559410810470581, "tokens_seen": 14098432, "tokens_per_sec": 14867.410895679812, "vram_gb": 6.442267648, "lambda": 0.14819134771823883, "S_mean": 0.3193461100260417, "G_mean": 0.4783928568164508, "G_std": 0.2001457642763853, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.00040917751009380545, "train_loss": 5.266463041305542, "grad_norm": 0.5917953848838806, "tokens_seen": 14262272, "tokens_per_sec": 14872.677997206318, "vram_gb": 6.442267648, "lambda": 0.1485797017812729, "S_mean": 0.3406778971354167, "G_mean": 0.4868131143351396, "G_std": 0.2014396789794167, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.000404898785365698, "train_loss": 5.33448600769043, "grad_norm": 0.5338597893714905, "tokens_seen": 14426112, "tokens_per_sec": 14878.272065015504, "vram_gb": 6.442267648, "lambda": 0.14889246287445226, "S_mean": 0.3252360026041667, "G_mean": 0.49005475516120595, "G_std": 0.20076519095649323, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.00040059958405753125, "train_loss": 5.219848871231079, "grad_norm": 0.6391025185585022, "tokens_seen": 14589952, "tokens_per_sec": 14883.365285356733, "vram_gb": 6.442267648, "lambda": 0.14913748173664013, "S_mean": 0.31890869140625, "G_mean": 0.484699172899127, "G_std": 0.20278549985960126, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.00039628108152801574, "train_loss": 5.291542291641235, "grad_norm": 0.6191651225090027, "tokens_seen": 14753792, "tokens_per_sec": 14888.175945623916, "vram_gb": 6.442267648, "lambda": 0.14946761603156725, "S_mean": 0.3202921549479167, "G_mean": 0.48724480501065653, "G_std": 0.19867978834857544, "skipped": 0} +{"event": "eval", "step": 1800, "val_loss": 5.4096620082855225, "val_ppl": 223.55601482903282, "lambda": 0.14947859073678652, "S_mean": 0.3542887369791667, "G_mean": 0.4867114822069804, "G_std": 0.20257033438732228, "tokens_seen": 14753792} +{"event": "step", "step": 1820, "lr": 0.0003919444584126229, "train_loss": 5.366289138793945, "grad_norm": 0.577948808670044, "tokens_seen": 14917632, "tokens_per_sec": 14833.668350202048, "vram_gb": 6.442267648, "lambda": 0.14975205467393002, "S_mean": 0.3281656901041667, "G_mean": 0.48330537664393586, "G_std": 0.20303481662025055, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0003875909003008116, "train_loss": 5.204590082168579, "grad_norm": 0.5737607479095459, "tokens_seen": 15081472, "tokens_per_sec": 14839.658817318224, "vram_gb": 6.442267648, "lambda": 0.15008515336861214, "S_mean": 0.3204752604166667, "G_mean": 0.47825769148766994, "G_std": 0.1977179548703134, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.00038322159741189957, "train_loss": 5.264942407608032, "grad_norm": 0.6027557849884033, "tokens_seen": 15245312, "tokens_per_sec": 14845.748815413177, "vram_gb": 6.442267648, "lambda": 0.15041279854873815, "S_mean": 0.3360188802083333, "G_mean": 0.4925144451359908, "G_std": 0.20301646335671344, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.00037883774426966964, "train_loss": 5.358943223953247, "grad_norm": 0.7410222887992859, "tokens_seen": 15409152, "tokens_per_sec": 14851.296213186595, "vram_gb": 6.442267648, "lambda": 0.15075742484380802, "S_mean": 0.3293863932291667, "G_mean": 0.5020403222491344, "G_std": 0.19930236305420598, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0003744405393757981, "train_loss": 5.31959342956543, "grad_norm": 0.6160460114479065, "tokens_seen": 15572992, "tokens_per_sec": 14857.535250813467, "vram_gb": 6.442267648, "lambda": 0.15108068535725275, "S_mean": 0.3235270182291667, "G_mean": 0.474533099681139, "G_std": 0.19594584063937268, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0003700311848821973, "train_loss": 5.249807834625244, "grad_norm": 0.6127752661705017, "tokens_seen": 15736832, "tokens_per_sec": 14862.22189627994, "vram_gb": 6.442267648, "lambda": 0.15135481270651022, "S_mean": 0.3219401041666667, "G_mean": 0.4734061478326718, "G_std": 0.20167956982428828, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0003656108862623586, "train_loss": 5.313121557235718, "grad_norm": 0.5792352557182312, "tokens_seen": 15900672, "tokens_per_sec": 14866.039121533458, "vram_gb": 6.442267648, "lambda": 0.1516401826714476, "S_mean": 0.321044921875, "G_mean": 0.4780598379050692, "G_std": 0.19987581127012768, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.00036118085198178824, "train_loss": 5.517450332641602, "grad_norm": 0.6575813293457031, "tokens_seen": 16064512, "tokens_per_sec": 14870.697975828889, "vram_gb": 6.442267648, "lambda": 0.151948948080341, "S_mean": 0.31951904296875, "G_mean": 0.478161768677334, "G_std": 0.19909347066034874, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.00035674229316762495, "train_loss": 5.4044153690338135, "grad_norm": 0.5802215337753296, "tokens_seen": 16228352, "tokens_per_sec": 14875.297008608006, "vram_gb": 6.442267648, "lambda": 0.1522312369197607, "S_mean": 0.3236490885416667, "G_mean": 0.4881879823903243, "G_std": 0.19993014369780818, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0003522964232775297, "train_loss": 5.254288196563721, "grad_norm": 0.5830281972885132, "tokens_seen": 16392192, "tokens_per_sec": 14879.968296915282, "vram_gb": 6.442267648, "lambda": 0.15254930996646485, "S_mean": 0.3146464029947917, "G_mean": 0.4732829745238026, "G_std": 0.1965254091968139, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.356456935405731, "val_ppl": 211.9725818487084, "lambda": 0.15256781596690416, "S_mean": 0.3260091145833333, "G_mean": 0.47392165039976436, "G_std": 0.196431546472013, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0003478444577679388, "train_loss": 5.21638298034668, "grad_norm": 0.60104900598526, "tokens_seen": 16556032, "tokens_per_sec": 14830.99093820091, "vram_gb": 6.442267648, "lambda": 0.1528791462381681, "S_mean": 0.3126424153645833, "G_mean": 0.4744356938948234, "G_std": 0.20032907339433828, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.00034338761376176993, "train_loss": 5.348584175109863, "grad_norm": 0.6428558230400085, "tokens_seen": 16719872, "tokens_per_sec": 14836.29085240971, "vram_gb": 6.442267648, "lambda": 0.15319231462975344, "S_mean": 0.33624267578125, "G_mean": 0.4752217202136914, "G_std": 0.19880206836387515, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0003389271097156731, "train_loss": 5.092561483383179, "grad_norm": 0.6614310145378113, "tokens_seen": 16883712, "tokens_per_sec": 14841.306570016433, "vram_gb": 6.442267648, "lambda": 0.15346764431645474, "S_mean": 0.3352864583333333, "G_mean": 0.4769606015955408, "G_std": 0.20134046068415046, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.00033446416508691677, "train_loss": 5.250831842422485, "grad_norm": 0.61954265832901, "tokens_seen": 17047552, "tokens_per_sec": 14846.899621456281, "vram_gb": 6.442267648, "lambda": 0.15376037017752728, "S_mean": 0.3120524088541667, "G_mean": 0.47535418905317783, "G_std": 0.20009471662342548, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.00032999999999999994, "train_loss": 5.25242805480957, "grad_norm": 0.586717963218689, "tokens_seen": 17211392, "tokens_per_sec": 14851.374086483876, "vram_gb": 6.442267648, "lambda": 0.1540603842586279, "S_mean": 0.3334859212239583, "G_mean": 0.4680200172588229, "G_std": 0.19987603199357787, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0003255358349130832, "train_loss": 5.14538311958313, "grad_norm": 0.5982330441474915, "tokens_seen": 17375232, "tokens_per_sec": 14855.979555088083, "vram_gb": 6.442267648, "lambda": 0.15434814927478632, "S_mean": 0.324462890625, "G_mean": 0.46248059688756865, "G_std": 0.19809290999546647, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0003210728902843267, "train_loss": 5.0801708698272705, "grad_norm": 0.5672221779823303, "tokens_seen": 17539072, "tokens_per_sec": 14860.283376888914, "vram_gb": 6.442267648, "lambda": 0.15463932634641728, "S_mean": 0.3314208984375, "G_mean": 0.47031607913474244, "G_std": 0.20000904969250163, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0003166123862382299, "train_loss": 5.132593393325806, "grad_norm": 0.6371153593063354, "tokens_seen": 17702912, "tokens_per_sec": 14864.875725205977, "vram_gb": 6.442267648, "lambda": 0.15491167145470777, "S_mean": 0.32611083984375, "G_mean": 0.46407177827010554, "G_std": 0.19988673956443867, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0003121555422320611, "train_loss": 5.167336463928223, "grad_norm": 0.6543046236038208, "tokens_seen": 17866752, "tokens_per_sec": 14869.789790366587, "vram_gb": 6.442267648, "lambda": 0.15516483535369238, "S_mean": 0.3072102864583333, "G_mean": 0.4632302640626828, "G_std": 0.20182768379648527, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.00030770357672247023, "train_loss": 5.316154479980469, "grad_norm": 0.6200842261314392, "tokens_seen": 18030592, "tokens_per_sec": 14874.563955861298, "vram_gb": 6.442267648, "lambda": 0.15542515770842633, "S_mean": 0.32611083984375, "G_mean": 0.47018811851739883, "G_std": 0.20271936043476066, "skipped": 0} +{"event": "eval", "step": 2200, "val_loss": 5.271997618675232, "val_ppl": 194.80471956770342, "lambda": 0.15543611192454895, "S_mean": 0.3468831380208333, "G_mean": 0.4647287307307124, "G_std": 0.2019317795832952, "tokens_seen": 18030592} +{"event": "step", "step": 2220, "lr": 0.00030325770683237505, "train_loss": 5.19379997253418, "grad_norm": 0.5998491048812866, "tokens_seen": 18194432, "tokens_per_sec": 14829.86648676394, "vram_gb": 6.442267648, "lambda": 0.15566586796194315, "S_mean": 0.3159993489583333, "G_mean": 0.4673734738801916, "G_std": 0.20080053775260845, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0002988191480182117, "train_loss": 5.144690036773682, "grad_norm": 0.583525538444519, "tokens_seen": 18358272, "tokens_per_sec": 14834.71797967139, "vram_gb": 6.442267648, "lambda": 0.1559303660566608, "S_mean": 0.3219197591145833, "G_mean": 0.46014175936579704, "G_std": 0.20052340642238656, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0002943891137376413, "train_loss": 5.143761873245239, "grad_norm": 0.5817037224769592, "tokens_seen": 18522112, "tokens_per_sec": 14839.50321799692, "vram_gb": 6.442267648, "lambda": 0.15621465692917505, "S_mean": 0.3221435546875, "G_mean": 0.4625430420661966, "G_std": 0.20192541150997081, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.0002899688151178026, "train_loss": 5.350404262542725, "grad_norm": 0.6307884454727173, "tokens_seen": 18685952, "tokens_per_sec": 14843.953786823167, "vram_gb": 6.442267648, "lambda": 0.15644852196176848, "S_mean": 0.3274739583333333, "G_mean": 0.45796986234684783, "G_std": 0.19927139300853014, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.00028555946062420177, "train_loss": 5.1144609451293945, "grad_norm": 0.6040140390396118, "tokens_seen": 18849792, "tokens_per_sec": 14848.617393925404, "vram_gb": 6.442267648, "lambda": 0.15674412685136, "S_mean": 0.3152872721354167, "G_mean": 0.4564528313154976, "G_std": 0.1993087576702237, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.00028116225573033036, "train_loss": 5.221200704574585, "grad_norm": 0.6162301898002625, "tokens_seen": 19013632, "tokens_per_sec": 14853.134120161183, "vram_gb": 6.442267648, "lambda": 0.1569712401057283, "S_mean": 0.32562255859375, "G_mean": 0.45699896446118754, "G_std": 0.19626160897314548, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0002767784025881003, "train_loss": 5.159387111663818, "grad_norm": 0.6569058299064636, "tokens_seen": 19177472, "tokens_per_sec": 14856.934248819405, "vram_gb": 6.442267648, "lambda": 0.15721193235367537, "S_mean": 0.32904052734375, "G_mean": 0.4585708274195592, "G_std": 0.1979352959121267, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.00027240909969918834, "train_loss": 5.232915878295898, "grad_norm": 0.5944669842720032, "tokens_seen": 19341312, "tokens_per_sec": 14861.559967949515, "vram_gb": 6.442267648, "lambda": 0.1574559115494291, "S_mean": 0.3220621744791667, "G_mean": 0.4535028813406825, "G_std": 0.19798388006165624, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.00026805554158737706, "train_loss": 5.123185157775879, "grad_norm": 0.661548912525177, "tokens_seen": 19505152, "tokens_per_sec": 14864.927540881788, "vram_gb": 6.442267648, "lambda": 0.1576677868142724, "S_mean": 0.3252766927083333, "G_mean": 0.4569100613395373, "G_std": 0.1980899527358512, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.0002637189184719842, "train_loss": 5.012186288833618, "grad_norm": 0.6939494013786316, "tokens_seen": 19668992, "tokens_per_sec": 14868.167637687855, "vram_gb": 6.442267648, "lambda": 0.15789554237077633, "S_mean": 0.3125406901041667, "G_mean": 0.46349408601721126, "G_std": 0.19798631950592002, "skipped": 0} +{"event": "eval", "step": 2400, "val_loss": 5.196418046951294, "val_ppl": 180.624094725049, "lambda": 0.15790792119999728, "S_mean": 0.3318277994791667, "G_mean": 0.45949235279113054, "G_std": 0.19870127147684494, "tokens_seen": 19668992} +{"event": "step", "step": 2420, "lr": 0.00025940041594246864, "train_loss": 5.001947641372681, "grad_norm": 0.6516705751419067, "tokens_seen": 19832832, "tokens_per_sec": 14826.314332061238, "vram_gb": 6.442267648, "lambda": 0.15812122200926146, "S_mean": 0.3201497395833333, "G_mean": 0.4450850347056985, "G_std": 0.20050533503914872, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.000255101214634302, "train_loss": 5.12986946105957, "grad_norm": 0.6365577578544617, "tokens_seen": 19996672, "tokens_per_sec": 14830.635990493001, "vram_gb": 6.442267648, "lambda": 0.15837650404622158, "S_mean": 0.3282877604166667, "G_mean": 0.44899142626672983, "G_std": 0.1984728348130981, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.00025082248990619454, "train_loss": 5.221479177474976, "grad_norm": 0.6396617889404297, "tokens_seen": 20160512, "tokens_per_sec": 14835.036523507586, "vram_gb": 6.442267648, "lambda": 0.15859906499584517, "S_mean": 0.3075968424479167, "G_mean": 0.4574682153761387, "G_std": 0.19969665445387363, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.0002465654115187642, "train_loss": 5.110007047653198, "grad_norm": 0.628183901309967, "tokens_seen": 20324352, "tokens_per_sec": 14839.340909724648, "vram_gb": 6.442267648, "lambda": 0.15882964866856733, "S_mean": 0.3284708658854167, "G_mean": 0.447203541174531, "G_std": 0.19675511587411165, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.00024233114331473544, "train_loss": 4.94791054725647, "grad_norm": 0.6572820544242859, "tokens_seen": 20488192, "tokens_per_sec": 14843.083825764785, "vram_gb": 6.442267648, "lambda": 0.15902110282331705, "S_mean": 0.3190714518229167, "G_mean": 0.4507615032295386, "G_std": 0.1965322570564846, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.00023812084290075633, "train_loss": 5.024427890777588, "grad_norm": 0.6008152961730957, "tokens_seen": 20652032, "tokens_per_sec": 14847.52264078149, "vram_gb": 6.442267648, "lambda": 0.15924381371587515, "S_mean": 0.3222859700520833, "G_mean": 0.4480586371695002, "G_std": 0.1962517489058276, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.00023393566133092048, "train_loss": 5.084470510482788, "grad_norm": 0.6365312933921814, "tokens_seen": 20815872, "tokens_per_sec": 14851.616136745215, "vram_gb": 6.442267648, "lambda": 0.15946097454677025, "S_mean": 0.316162109375, "G_mean": 0.4542156746610999, "G_std": 0.1973722642287612, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.00022977674279208032, "train_loss": 4.879136085510254, "grad_norm": 0.6334409713745117, "tokens_seen": 20979712, "tokens_per_sec": 14855.873543189527, "vram_gb": 6.442267648, "lambda": 0.15967680917431912, "S_mean": 0.356201171875, "G_mean": 0.4438647028679649, "G_std": 0.19627348224942884, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.00022564522429103793, "train_loss": 5.163697957992554, "grad_norm": 0.6904861927032471, "tokens_seen": 21143552, "tokens_per_sec": 14859.96930992384, "vram_gb": 6.442267648, "lambda": 0.15990279925366244, "S_mean": 0.3212483723958333, "G_mean": 0.4438582655663292, "G_std": 0.19620043179020286, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00022154223534369825, "train_loss": 4.939334869384766, "grad_norm": 0.657217264175415, "tokens_seen": 21307392, "tokens_per_sec": 14863.449118028862, "vram_gb": 6.442267648, "lambda": 0.1600928260013461, "S_mean": 0.3165486653645833, "G_mean": 0.4401321733991305, "G_std": 0.19646124417583147, "skipped": 0} +{"event": "eval", "step": 2600, "val_loss": 5.159240305423737, "val_ppl": 174.03219405836063, "lambda": 0.1601025036846598, "S_mean": 0.32952880859375, "G_mean": 0.4523120218267043, "G_std": 0.196817667533954, "tokens_seen": 21307392} +{"event": "step", "step": 2620, "lr": 0.00021746889766627162, "train_loss": 5.134489059448242, "grad_norm": 0.702821671962738, "tokens_seen": 21471232, "tokens_per_sec": 14825.420140658707, "vram_gb": 6.442267648, "lambda": 0.16027720520893732, "S_mean": 0.3202616373697917, "G_mean": 0.44931645939747494, "G_std": 0.19501375717421374, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0002134263248686069, "train_loss": 5.113766193389893, "grad_norm": 0.692633867263794, "tokens_seen": 21635072, "tokens_per_sec": 14828.79216983475, "vram_gb": 6.442267648, "lambda": 0.16046080024292073, "S_mean": 0.3257242838541667, "G_mean": 0.4500795149554809, "G_std": 0.19690127220625678, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.00020941562214974222, "train_loss": 4.98924446105957, "grad_norm": 0.6617489457130432, "tokens_seen": 21798912, "tokens_per_sec": 14832.7467463107, "vram_gb": 6.442267648, "lambda": 0.1606252252434691, "S_mean": 0.327239990234375, "G_mean": 0.44443822465837, "G_std": 0.19447811444600424, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0002054378859957543, "train_loss": 4.832056045532227, "grad_norm": 0.805416464805603, "tokens_seen": 21962752, "tokens_per_sec": 14836.580247563152, "vram_gb": 6.442267648, "lambda": 0.1608069073408842, "S_mean": 0.3241373697916667, "G_mean": 0.4425751141582926, "G_std": 0.19733314231658974, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0002014942038799901, "train_loss": 5.0881993770599365, "grad_norm": 0.6750401854515076, "tokens_seen": 22126592, "tokens_per_sec": 14840.19804536029, "vram_gb": 6.442267648, "lambda": 0.16100376565009356, "S_mean": 0.32037353515625, "G_mean": 0.4432884758959214, "G_std": 0.19582880226274332, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.00019758565396576285, "train_loss": 5.0654518604278564, "grad_norm": 0.6890210509300232, "tokens_seen": 22290432, "tokens_per_sec": 14844.35750642067, "vram_gb": 6.442267648, "lambda": 0.16119438720246157, "S_mean": 0.3271484375, "G_mean": 0.4416095061848561, "G_std": 0.19554826462020478, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.00019371330481159223, "train_loss": 5.050111770629883, "grad_norm": 0.6524158120155334, "tokens_seen": 22454272, "tokens_per_sec": 14847.813979073499, "vram_gb": 6.442267648, "lambda": 0.16136763679484525, "S_mean": 0.3208414713541667, "G_mean": 0.44492132123559713, "G_std": 0.19661091609547535, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.00018987821507907194, "train_loss": 4.895708322525024, "grad_norm": 0.6551705002784729, "tokens_seen": 22618112, "tokens_per_sec": 14851.102609411717, "vram_gb": 6.442267648, "lambda": 0.1615272182971239, "S_mean": 0.3185017903645833, "G_mean": 0.44701481858889264, "G_std": 0.193485289812088, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.00018608143324344171, "train_loss": 5.0218400955200195, "grad_norm": 0.6632121205329895, "tokens_seen": 22781952, "tokens_per_sec": 14854.752768620448, "vram_gb": 6.442267648, "lambda": 0.16169719553242126, "S_mean": 0.3076578776041667, "G_mean": 0.4440632878492276, "G_std": 0.19407524013270935, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.00018232399730694476, "train_loss": 4.97717022895813, "grad_norm": 0.7520787119865417, "tokens_seen": 22945792, "tokens_per_sec": 14858.314809126889, "vram_gb": 6.442267648, "lambda": 0.1618487909436226, "S_mean": 0.3079020182291667, "G_mean": 0.4365478853384654, "G_std": 0.19688900001347065, "skipped": 0} +{"event": "eval", "step": 2800, "val_loss": 5.090602135658264, "val_ppl": 162.48767222451275, "lambda": 0.16185519471764565, "S_mean": 0.3184611002604167, "G_mean": 0.42752663729091483, "G_std": 0.19605638071273765, "tokens_seen": 22945792} +{"event": "step", "step": 2820, "lr": 0.00017860693451504837, "train_loss": 5.018094539642334, "grad_norm": 0.6664199829101562, "tokens_seen": 23109632, "tokens_per_sec": 14823.598929146838, "vram_gb": 6.442267648, "lambda": 0.16200900978098312, "S_mean": 0.3169148763020833, "G_mean": 0.4376520048826933, "G_std": 0.1958003753485779, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.00017493126107560522, "train_loss": 4.8274946212768555, "grad_norm": 0.6696504354476929, "tokens_seen": 23273472, "tokens_per_sec": 14827.035252364658, "vram_gb": 6.442267648, "lambda": 0.16216511682917675, "S_mean": 0.3174235026041667, "G_mean": 0.43590452677259844, "G_std": 0.19407433240363994, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.00017129798188103226, "train_loss": 4.993183612823486, "grad_norm": 0.6263960003852844, "tokens_seen": 23437312, "tokens_per_sec": 14831.007117637153, "vram_gb": 6.442267648, "lambda": 0.16232836867372194, "S_mean": 0.33258056640625, "G_mean": 0.4350059659530719, "G_std": 0.1927152325709661, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.00016770809023358322, "train_loss": 4.992556810379028, "grad_norm": 0.6723331809043884, "tokens_seen": 23601152, "tokens_per_sec": 14834.444217527582, "vram_gb": 6.442267648, "lambda": 0.16248036300142607, "S_mean": 0.3330485026041667, "G_mean": 0.4299806446457903, "G_std": 0.1975610110287865, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.00016416256757378968, "train_loss": 4.835402965545654, "grad_norm": 0.7249350547790527, "tokens_seen": 23764992, "tokens_per_sec": 14838.121158330629, "vram_gb": 6.442267648, "lambda": 0.16262648720294237, "S_mean": 0.3256022135416667, "G_mean": 0.4372340639432271, "G_std": 0.19571868206063905, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.000160662383212145, "train_loss": 5.04963755607605, "grad_norm": 0.67976313829422, "tokens_seen": 23928832, "tokens_per_sec": 14842.10213125718, "vram_gb": 6.442267648, "lambda": 0.16277169901877642, "S_mean": 0.3380330403645833, "G_mean": 0.44400854377696913, "G_std": 0.19550793897360563, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.00015720849406410448, "train_loss": 4.949907302856445, "grad_norm": 0.6398596167564392, "tokens_seen": 24092672, "tokens_per_sec": 14845.470771021028, "vram_gb": 6.442267648, "lambda": 0.16289791526893774, "S_mean": 0.3282877604166667, "G_mean": 0.43647429657479125, "G_std": 0.19457052368670702, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.00015380184438847444, "train_loss": 4.956882953643799, "grad_norm": 0.6980310678482056, "tokens_seen": 24256512, "tokens_per_sec": 14848.99918931142, "vram_gb": 6.442267648, "lambda": 0.16302600068350634, "S_mean": 0.3196818033854167, "G_mean": 0.43787237629294395, "G_std": 0.19515169722338518, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.00015044336552926094, "train_loss": 4.9083251953125, "grad_norm": 0.6356950402259827, "tokens_seen": 24420352, "tokens_per_sec": 14852.40327483416, "vram_gb": 6.442267648, "lambda": 0.16315330937504768, "S_mean": 0.3218180338541667, "G_mean": 0.43096340727061033, "G_std": 0.19273544987663627, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.00014713397566104995, "train_loss": 4.844050407409668, "grad_norm": 0.658645510673523, "tokens_seen": 24584192, "tokens_per_sec": 14855.106374898998, "vram_gb": 6.442267648, "lambda": 0.16329338184247413, "S_mean": 0.31640625, "G_mean": 0.4300241045032938, "G_std": 0.19375793123617768, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.106582450866699, "val_ppl": 165.10513468003953, "lambda": 0.1632997632647554, "S_mean": 0.3410441080729167, "G_mean": 0.44313390584041673, "G_std": 0.19483599501351515, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.00014387457953798726, "train_loss": 4.9191813468933105, "grad_norm": 0.6443852782249451, "tokens_seen": 24748032, "tokens_per_sec": 14822.015935543812, "vram_gb": 6.442267648, "lambda": 0.16343002735326687, "S_mean": 0.32415771484375, "G_mean": 0.4366761473938823, "G_std": 0.19382250681519508, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.00014066606824642808, "train_loss": 4.983794450759888, "grad_norm": 0.6792364120483398, "tokens_seen": 24911872, "tokens_per_sec": 14825.565791948442, "vram_gb": 6.442267648, "lambda": 0.1635501366108656, "S_mean": 0.3211466471354167, "G_mean": 0.4348375468204419, "G_std": 0.19372082889700928, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.00013750931896132278, "train_loss": 4.957403659820557, "grad_norm": 0.6925153136253357, "tokens_seen": 25075712, "tokens_per_sec": 14828.970843403562, "vram_gb": 6.442267648, "lambda": 0.163682596758008, "S_mean": 0.303466796875, "G_mean": 0.43888544384390116, "G_std": 0.1950989724136889, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.00013440519470640596, "train_loss": 4.806356906890869, "grad_norm": 0.7236127257347107, "tokens_seen": 25239552, "tokens_per_sec": 14832.704511820057, "vram_gb": 6.442267648, "lambda": 0.16378284866611162, "S_mean": 0.315185546875, "G_mean": 0.42993523366749287, "G_std": 0.19265236457188925, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.00013135454411825447, "train_loss": 5.010875463485718, "grad_norm": 0.7469953298568726, "tokens_seen": 25403392, "tokens_per_sec": 14836.05862631957, "vram_gb": 6.442267648, "lambda": 0.1638804872830709, "S_mean": 0.3240152994791667, "G_mean": 0.4370205185065667, "G_std": 0.1943389419466257, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.00012835820121427852, "train_loss": 5.036855936050415, "grad_norm": 0.791403591632843, "tokens_seen": 25567232, "tokens_per_sec": 14839.016160511575, "vram_gb": 6.442267648, "lambda": 0.1640077323342363, "S_mean": 0.3604532877604167, "G_mean": 0.43461609942217666, "G_std": 0.19326361827552319, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0001254169851647097, "train_loss": 4.879674673080444, "grad_norm": 0.6710543632507324, "tokens_seen": 25731072, "tokens_per_sec": 14842.588115939096, "vram_gb": 6.442267648, "lambda": 0.16411774698644876, "S_mean": 0.3303629557291667, "G_mean": 0.43366697058081627, "G_std": 0.194715012020121, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.00012253170006864812, "train_loss": 5.027895450592041, "grad_norm": 0.7714133858680725, "tokens_seen": 25894912, "tokens_per_sec": 14846.118727927907, "vram_gb": 6.442267648, "lambda": 0.1642256739238898, "S_mean": 0.3208516438802083, "G_mean": 0.4341176695500811, "G_std": 0.1948940735310316, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.00011970313473422966, "train_loss": 4.881396055221558, "grad_norm": 0.6811913847923279, "tokens_seen": 26058752, "tokens_per_sec": 14849.535582086875, "vram_gb": 6.442267648, "lambda": 0.16433581709861755, "S_mean": 0.3131815592447917, "G_mean": 0.4314587600529194, "G_std": 0.1939950610200564, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.00011693206246297376, "train_loss": 4.890825271606445, "grad_norm": 0.7375786900520325, "tokens_seen": 26222592, "tokens_per_sec": 14852.283495797978, "vram_gb": 6.442267648, "lambda": 0.1644469192251563, "S_mean": 0.3254496256510417, "G_mean": 0.43080346534649533, "G_std": 0.19339506685112914, "skipped": 0} +{"event": "eval", "step": 3200, "val_loss": 5.0128332614898685, "val_ppl": 150.33005770420044, "lambda": 0.16445368466277918, "S_mean": 0.3292439778645833, "G_mean": 0.4349699147666494, "G_std": 0.19432132598012686, "tokens_seen": 26222592} +{"event": "step", "step": 3220, "lr": 0.00011421924083837032, "train_loss": 5.009815454483032, "grad_norm": 0.7520045042037964, "tokens_seen": 26386432, "tokens_per_sec": 14821.848082974131, "vram_gb": 6.442267648, "lambda": 0.16455033235251904, "S_mean": 0.320068359375, "G_mean": 0.423968060563008, "G_std": 0.1943183233961463, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.00011156541151876421, "train_loss": 5.012120723724365, "grad_norm": 0.6796030402183533, "tokens_seen": 26550272, "tokens_per_sec": 14824.569870751102, "vram_gb": 6.442267648, "lambda": 0.164661834637324, "S_mean": 0.31884765625, "G_mean": 0.4364341401184599, "G_std": 0.19307467015460134, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.00010897130003459306, "train_loss": 4.767894506454468, "grad_norm": 0.6808634996414185, "tokens_seen": 26714112, "tokens_per_sec": 14827.300199297584, "vram_gb": 6.442267648, "lambda": 0.16475263237953186, "S_mean": 0.3316446940104167, "G_mean": 0.42899767123162746, "G_std": 0.19583002182965478, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.00010643761559003458, "train_loss": 5.035125970840454, "grad_norm": 0.7540623545646667, "tokens_seen": 26877952, "tokens_per_sec": 14830.44734218077, "vram_gb": 6.442267648, "lambda": 0.16484248265624046, "S_mean": 0.3135172526041667, "G_mean": 0.42665910472472507, "G_std": 0.194008719176054, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0001039650508691173, "train_loss": 4.842902898788452, "grad_norm": 0.6890145540237427, "tokens_seen": 27041792, "tokens_per_sec": 14833.55631904649, "vram_gb": 6.442267648, "lambda": 0.1649355192979177, "S_mean": 0.3212076822916667, "G_mean": 0.4216356985270977, "G_std": 0.19253811861077944, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0001015542818463474, "train_loss": 5.014467716217041, "grad_norm": 0.7137332558631897, "tokens_seen": 27205632, "tokens_per_sec": 14836.742300684618, "vram_gb": 6.442267648, "lambda": 0.16502051676313081, "S_mean": 0.325927734375, "G_mean": 0.4199893881256382, "G_std": 0.19515333402281007, "skipped": 0} +{"event": "step", "step": 3340, "lr": 9.920596760190444e-05, "train_loss": 4.842265605926514, "grad_norm": 0.674251139163971, "tokens_seen": 27369472, "tokens_per_sec": 14839.93508378092, "vram_gb": 6.442267648, "lambda": 0.16509454355885586, "S_mean": 0.3245442708333333, "G_mean": 0.4251132579520345, "G_std": 0.19251733692362905, "skipped": 0} +{"event": "step", "step": 3360, "lr": 9.692075014145504e-05, "train_loss": 5.000671148300171, "grad_norm": 0.705901563167572, "tokens_seen": 27533312, "tokens_per_sec": 14843.057714339135, "vram_gb": 6.442267648, "lambda": 0.16518759479125342, "S_mean": 0.31585693359375, "G_mean": 0.42819516733288765, "G_std": 0.19391279543439546, "skipped": 0} +{"event": "step", "step": 3380, "lr": 9.46992542206351e-05, "train_loss": 4.961569547653198, "grad_norm": 0.7074645757675171, "tokens_seen": 27697152, "tokens_per_sec": 14846.565989083927, "vram_gb": 6.442267648, "lambda": 0.16527666803449392, "S_mean": 0.31585693359375, "G_mean": 0.42280324424306553, "G_std": 0.19400116925438246, "skipped": 0} +{"event": "step", "step": 3400, "lr": 9.254208717424797e-05, "train_loss": 4.8588173389434814, "grad_norm": 0.6858283281326294, "tokens_seen": 27860992, "tokens_per_sec": 14849.678836042753, "vram_gb": 6.442267648, "lambda": 0.16536484193056822, "S_mean": 0.3207804361979167, "G_mean": 0.4210895566890637, "G_std": 0.19573435885831714, "skipped": 0} +{"event": "eval", "step": 3400, "val_loss": 5.010326755046845, "val_ppl": 149.95372628157997, "lambda": 0.16536854455868402, "S_mean": 0.3127237955729167, "G_mean": 0.42942119607081014, "G_std": 0.19529663352295756, "tokens_seen": 27860992} +{"event": "step", "step": 3420, "lr": 9.04498387502248e-05, "train_loss": 4.899542570114136, "grad_norm": 0.704679548740387, "tokens_seen": 28024832, "tokens_per_sec": 14820.404327675906, "vram_gb": 6.442267648, "lambda": 0.16544855292886496, "S_mean": 0.3251749674479167, "G_mean": 0.43159068406869966, "G_std": 0.19391799485310912, "skipped": 0} +{"event": "step", "step": 3440, "lr": 8.842308094839387e-05, "train_loss": 4.929529666900635, "grad_norm": 0.7188193202018738, "tokens_seen": 28188672, "tokens_per_sec": 14823.495383943367, "vram_gb": 6.442267648, "lambda": 0.1655213125050068, "S_mean": 0.3303019205729167, "G_mean": 0.432841124633948, "G_std": 0.19290496595203876, "skipped": 0} +{"event": "step", "step": 3460, "lr": 8.646236786410109e-05, "train_loss": 4.806748867034912, "grad_norm": 0.7289462685585022, "tokens_seen": 28352512, "tokens_per_sec": 14826.372987487917, "vram_gb": 6.442267648, "lambda": 0.16559547061721483, "S_mean": 0.321533203125, "G_mean": 0.42248809306571883, "G_std": 0.19287102048595747, "skipped": 0} +{"event": "step", "step": 3480, "lr": 8.456823553672612e-05, "train_loss": 4.91653847694397, "grad_norm": 0.7104772925376892, "tokens_seen": 28516352, "tokens_per_sec": 14829.508975374345, "vram_gb": 6.442267648, "lambda": 0.16566639052083096, "S_mean": 0.3375244140625, "G_mean": 0.4233105207482974, "G_std": 0.19374952434251705, "skipped": 0} +{"event": "step", "step": 3500, "lr": 8.274120180313448e-05, "train_loss": 4.829211950302124, "grad_norm": 0.6808714866638184, "tokens_seen": 28680192, "tokens_per_sec": 14832.394407898584, "vram_gb": 6.442267648, "lambda": 0.16573535868277153, "S_mean": 0.31878662109375, "G_mean": 0.424697866042455, "G_std": 0.19352459472914538, "skipped": 0} +{"event": "step", "step": 3520, "lr": 8.098176615610612e-05, "train_loss": 4.718241214752197, "grad_norm": 0.7686192989349365, "tokens_seen": 28844032, "tokens_per_sec": 14835.050954564216, "vram_gb": 6.442267648, "lambda": 0.16580549317101637, "S_mean": 0.32220458984375, "G_mean": 0.42924326844513416, "G_std": 0.19292464635024467, "skipped": 0} +{"event": "step", "step": 3540, "lr": 7.929040960777911e-05, "train_loss": 4.7144505977630615, "grad_norm": 0.7325238585472107, "tokens_seen": 29007872, "tokens_per_sec": 14838.277168830555, "vram_gb": 6.442267648, "lambda": 0.1658791173249483, "S_mean": 0.3243611653645833, "G_mean": 0.42203957090775174, "G_std": 0.1909648417495191, "skipped": 0} +{"event": "step", "step": 3560, "lr": 7.76675945581456e-05, "train_loss": 4.9728453159332275, "grad_norm": 0.6722750067710876, "tokens_seen": 29171712, "tokens_per_sec": 14841.293545367624, "vram_gb": 6.442267648, "lambda": 0.165962186952432, "S_mean": 0.312164306640625, "G_mean": 0.4248941869785388, "G_std": 0.19154442489768067, "skipped": 0} +{"event": "step", "step": 3580, "lr": 7.611376466863648e-05, "train_loss": 4.79899787902832, "grad_norm": 0.6564934849739075, "tokens_seen": 29335552, "tokens_per_sec": 14844.222236938751, "vram_gb": 6.442267648, "lambda": 0.16603116939465204, "S_mean": 0.3174235026041667, "G_mean": 0.4190280182907979, "G_std": 0.1906403286072115, "skipped": 0} +{"event": "step", "step": 3600, "lr": 7.462934474082865e-05, "train_loss": 4.823073148727417, "grad_norm": 0.7472759485244751, "tokens_seen": 29499392, "tokens_per_sec": 14846.937247175887, "vram_gb": 6.442267648, "lambda": 0.16610508256902298, "S_mean": 0.3111979166666667, "G_mean": 0.4256579965973894, "G_std": 0.19441016546140114, "skipped": 0} +{"event": "eval", "step": 3600, "val_loss": 4.978898429870606, "val_ppl": 145.3142196512229, "lambda": 0.16610852628946304, "S_mean": 0.3097635904947917, "G_mean": 0.418718286169072, "G_std": 0.19276648045827946, "tokens_seen": 29499392} +{"event": "step", "step": 3620, "lr": 7.321474060030854e-05, "train_loss": 4.878501892089844, "grad_norm": 0.7533921599388123, "tokens_seen": 29663232, "tokens_per_sec": 14819.476484937464, "vram_gb": 6.442267648, "lambda": 0.16617669692883888, "S_mean": 0.315093994140625, "G_mean": 0.4205183299879233, "G_std": 0.18999496800825, "skipped": 0} +{"event": "step", "step": 3640, "lr": 7.187033898572343e-05, "train_loss": 4.956980466842651, "grad_norm": 0.7057225704193115, "tokens_seen": 29827072, "tokens_per_sec": 14822.335705700527, "vram_gb": 6.442267648, "lambda": 0.1662404810388883, "S_mean": 0.31060791015625, "G_mean": 0.4220012916872899, "G_std": 0.1921722791157663, "skipped": 0} +{"event": "step", "step": 3660, "lr": 7.0596507443051e-05, "train_loss": 4.991813898086548, "grad_norm": 0.7961166501045227, "tokens_seen": 29990912, "tokens_per_sec": 14825.049062928436, "vram_gb": 6.442267648, "lambda": 0.16630153047541776, "S_mean": 0.320037841796875, "G_mean": 0.42589141490558785, "G_std": 0.1928348538155357, "skipped": 0} +{"event": "step", "step": 3680, "lr": 6.93935942251159e-05, "train_loss": 4.920683860778809, "grad_norm": 0.67623370885849, "tokens_seen": 30154752, "tokens_per_sec": 14827.94529562485, "vram_gb": 6.442267648, "lambda": 0.1663592647140225, "S_mean": 0.3309529622395833, "G_mean": 0.416047852486372, "G_std": 0.19305691790456572, "skipped": 0} +{"event": "step", "step": 3700, "lr": 6.82619281963808e-05, "train_loss": 4.9807140827178955, "grad_norm": 0.6983736753463745, "tokens_seen": 30318592, "tokens_per_sec": 14830.748420117654, "vram_gb": 6.442267648, "lambda": 0.1664255866780877, "S_mean": 0.3262736002604167, "G_mean": 0.4216521034638087, "G_std": 0.19320985317851105, "skipped": 0} +{"event": "step", "step": 3720, "lr": 6.720181874303845e-05, "train_loss": 4.730205535888672, "grad_norm": 0.6882070899009705, "tokens_seen": 30482432, "tokens_per_sec": 14833.324921648273, "vram_gb": 6.442267648, "lambda": 0.16649126230428615, "S_mean": 0.3319905598958333, "G_mean": 0.4164570265760024, "G_std": 0.19269141228869557, "skipped": 0} +{"event": "step", "step": 3740, "lr": 6.62135556884282e-05, "train_loss": 4.921692132949829, "grad_norm": 0.6764518022537231, "tokens_seen": 30646272, "tokens_per_sec": 14835.923329408217, "vram_gb": 6.442267648, "lambda": 0.16654821671545506, "S_mean": 0.3189290364583333, "G_mean": 0.4178890536228816, "G_std": 0.19101457266757885, "skipped": 0} +{"event": "step", "step": 3760, "lr": 6.529740921380147e-05, "train_loss": 4.870234251022339, "grad_norm": 0.7265709042549133, "tokens_seen": 30810112, "tokens_per_sec": 14838.999838075888, "vram_gb": 6.442267648, "lambda": 0.16661168665935597, "S_mean": 0.3160400390625, "G_mean": 0.41870569561918575, "G_std": 0.1897947221683959, "skipped": 0} +{"event": "step", "step": 3780, "lr": 6.445362978445676e-05, "train_loss": 4.758757591247559, "grad_norm": 0.71559739112854, "tokens_seen": 30973952, "tokens_per_sec": 14841.84712923492, "vram_gb": 6.442267648, "lambda": 0.16666329310586056, "S_mean": 0.33148193359375, "G_mean": 0.4184085186570883, "G_std": 0.19139019244660935, "skipped": 0} +{"event": "step", "step": 3800, "lr": 6.368244808126497e-05, "train_loss": 4.877610445022583, "grad_norm": 0.7747244834899902, "tokens_seen": 31137792, "tokens_per_sec": 14843.326194478572, "vram_gb": 6.442267648, "lambda": 0.16672033909708261, "S_mean": 0.3239949544270833, "G_mean": 0.42720757083346445, "G_std": 0.19262509793043137, "skipped": 0} +{"event": "eval", "step": 3800, "val_loss": 4.958336865901947, "val_ppl": 142.35684039187063, "lambda": 0.1667225214963158, "S_mean": 0.3408203125, "G_mean": 0.4152589862545331, "G_std": 0.19162867420042554, "tokens_seen": 31137792} +{"event": "step", "step": 3820, "lr": 6.29840749376035e-05, "train_loss": 4.937638759613037, "grad_norm": 0.7389752864837646, "tokens_seen": 31301632, "tokens_per_sec": 14816.333767902217, "vram_gb": 6.442267648, "lambda": 0.16677486648162207, "S_mean": 0.32373046875, "G_mean": 0.42343480822940666, "G_std": 0.19235115606958667, "skipped": 0} +{"event": "step", "step": 3840, "lr": 6.23587012817167e-05, "train_loss": 4.864835739135742, "grad_norm": 0.7662633657455444, "tokens_seen": 31465472, "tokens_per_sec": 14818.163964692953, "vram_gb": 6.442267648, "lambda": 0.16683314150820175, "S_mean": 0.3357340494791667, "G_mean": 0.41695012773076695, "G_std": 0.1911865066116055, "skipped": 0} +{"event": "step", "step": 3860, "lr": 6.18064980845177e-05, "train_loss": 4.873746156692505, "grad_norm": 0.7031118869781494, "tokens_seen": 31629312, "tokens_per_sec": 14820.35896956511, "vram_gb": 6.442267648, "lambda": 0.1668974378456672, "S_mean": 0.31585693359375, "G_mean": 0.42252734551827115, "G_std": 0.1898824886108438, "skipped": 0} +{"event": "step", "step": 3880, "lr": 6.132761631284691e-05, "train_loss": 4.9272050857543945, "grad_norm": 0.7942363619804382, "tokens_seen": 31793152, "tokens_per_sec": 14822.367263386152, "vram_gb": 6.442267648, "lambda": 0.16694826819002628, "S_mean": 0.3172607421875, "G_mean": 0.428834938754638, "G_std": 0.19003127235919237, "skipped": 0} +{"event": "step", "step": 3900, "lr": 6.092218688819914e-05, "train_loss": 4.835458993911743, "grad_norm": 0.7877920866012573, "tokens_seen": 31956992, "tokens_per_sec": 14824.80083665599, "vram_gb": 6.442267648, "lambda": 0.1669996650889516, "S_mean": 0.3224894205729167, "G_mean": 0.41820415544013184, "G_std": 0.19039108495538434, "skipped": 0} +{"event": "step", "step": 3920, "lr": 6.059032065093088e-05, "train_loss": 4.765156030654907, "grad_norm": 0.7823497653007507, "tokens_seen": 32120832, "tokens_per_sec": 14826.679049839722, "vram_gb": 6.442267648, "lambda": 0.16705392487347126, "S_mean": 0.3134765625, "G_mean": 0.4173752376809716, "G_std": 0.19071281623716155, "skipped": 0} +{"event": "step", "step": 3940, "lr": 6.033210832995772e-05, "train_loss": 4.938108921051025, "grad_norm": 0.7673657536506653, "tokens_seen": 32284672, "tokens_per_sec": 14828.977484019651, "vram_gb": 6.442267648, "lambda": 0.16710669671495756, "S_mean": 0.3165690104166667, "G_mean": 0.4203516334916155, "G_std": 0.19083221132556596, "skipped": 0} +{"event": "step", "step": 3960, "lr": 6.014762051794997e-05, "train_loss": 4.888641595840454, "grad_norm": 0.7292443513870239, "tokens_seen": 32448512, "tokens_per_sec": 14831.541222216549, "vram_gb": 6.442267648, "lambda": 0.16715015222628912, "S_mean": 0.314971923828125, "G_mean": 0.41584287708004314, "G_std": 0.1907447202441593, "skipped": 0} +{"event": "step", "step": 3980, "lr": 6.003690765203337e-05, "train_loss": 4.831624984741211, "grad_norm": 0.7757254242897034, "tokens_seen": 32612352, "tokens_per_sec": 14833.327662878333, "vram_gb": 6.442267648, "lambda": 0.16720068734139204, "S_mean": 0.3243611653645833, "G_mean": 0.4214151619623105, "G_std": 0.1925815287977457, "skipped": 0} +{"event": "eval", "step": 3999, "val_loss": 4.9527598261833194, "val_ppl": 141.5651204177534, "lambda": 0.16726008399079242, "S_mean": 0.3152872721354167, "G_mean": 0.42015269367645186, "G_std": 0.19156288945426544, "tokens_seen": 32768000} +{"event": "done", "step": 3999, "elapsed_s": 2212.9264504909515, "nan_failures": 0} diff --git a/runs_4ksteps/g_only_s0/config.json b/runs_4ksteps/g_only_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..5851c24d2ed339adfda167bcc9cd430543b29cd9 --- /dev/null +++ b/runs_4ksteps/g_only_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "g_only", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s0", + "max_steps": 4000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 200, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/runs_4ksteps/g_only_s0/final.pt b/runs_4ksteps/g_only_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..6caf03458a5341d0930ecffc598fe098c8e43551 --- /dev/null +++ b/runs_4ksteps/g_only_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f37bb38b4b32a85b35fb0fb147603e2add9f9279b950ee2549704d6819d1feac +size 253629735 diff --git a/runs_4ksteps/g_only_s0/metrics.jsonl b/runs_4ksteps/g_only_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..b9fba191fbd259be5e0aa464cfb75c3e920dee08 --- /dev/null +++ b/runs_4ksteps/g_only_s0/metrics.jsonl @@ -0,0 +1,222 @@ +{"event": "start", "params": 63385776, "variant": "g_only", "seed": 0, "config": {"variant": "g_only", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\g_only_s0", "max_steps": 4000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 200, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.911605834960938, "grad_norm": 10.423650741577148, "tokens_seen": 8192, "tokens_per_sec": 7313.95483151487, "vram_gb": 5.768167424, "lambda": null, "S_mean": null, "G_mean": 0.4969075520833333, "G_std": 0.0908203125, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.839956283569336, "grad_norm": 2.094467878341675, "tokens_seen": 172032, "tokens_per_sec": 16616.96016307019, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4551595052083333, "G_std": 0.0709686279296875, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.091047286987305, "grad_norm": 1.7767575979232788, "tokens_seen": 335872, "tokens_per_sec": 17204.817428159913, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4195963541666667, "G_std": 0.07474772135416667, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.093460083007812, "grad_norm": 1.5460402965545654, "tokens_seen": 499712, "tokens_per_sec": 17417.849692658572, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4182942708333333, "G_std": 0.08763631184895833, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.423764228820801, "grad_norm": 0.7839320302009583, "tokens_seen": 663552, "tokens_per_sec": 17526.78042746082, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4287923177083333, "G_std": 0.095458984375, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.205677032470703, "grad_norm": 0.7242382764816284, "tokens_seen": 827392, "tokens_per_sec": 17595.093977584067, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4186197916666667, "G_std": 0.10943603515625, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.0528974533081055, "grad_norm": 0.5746057033538818, "tokens_seen": 991232, "tokens_per_sec": 17642.79850096263, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4054361979166667, "G_std": 0.12180582682291667, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.845210075378418, "grad_norm": 0.8412448763847351, "tokens_seen": 1155072, "tokens_per_sec": 17671.29427979756, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4053548177083333, "G_std": 0.13011678059895834, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.911183834075928, "grad_norm": 0.5963978171348572, "tokens_seen": 1318912, "tokens_per_sec": 17691.341243113402, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4039713541666667, "G_std": 0.14375813802083334, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.794652223587036, "grad_norm": 0.492755651473999, "tokens_seen": 1482752, "tokens_per_sec": 17707.277963470202, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3907063802083333, "G_std": 0.14605712890625, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.6789391040802, "grad_norm": 0.5356420278549194, "tokens_seen": 1646592, "tokens_per_sec": 17715.525553776184, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4029947916666667, "G_std": 0.14860026041666666, "skipped": 0} +{"event": "eval", "step": 200, "val_loss": 6.7469225287437435, "val_ppl": 851.4344614101415, "lambda": null, "S_mean": null, "G_mean": 0.398681640625, "G_std": 0.1478271484375, "tokens_seen": 1646592} +{"event": "step", "step": 220, "lr": 0.0005999630923479665, "train_loss": 6.57999849319458, "grad_norm": 0.5298930406570435, "tokens_seen": 1810432, "tokens_per_sec": 17148.922711416282, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.39599609375, "G_std": 0.15995279947916666, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.00059985237948205, "train_loss": 6.627178907394409, "grad_norm": 0.5012205839157104, "tokens_seen": 1974272, "tokens_per_sec": 17204.011802197034, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3841145833333333, "G_std": 0.15840657552083334, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005996678916700422, "train_loss": 6.596156120300293, "grad_norm": 0.5116039514541626, "tokens_seen": 2138112, "tokens_per_sec": 17251.65430905436, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3935546875, "G_std": 0.16878255208333334, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.000599409679349069, "train_loss": 6.516116142272949, "grad_norm": 0.46825578808784485, "tokens_seen": 2301952, "tokens_per_sec": 17292.493563794233, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3889973958333333, "G_std": 0.16566975911458334, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005990778131118008, "train_loss": 6.343671083450317, "grad_norm": 0.48715052008628845, "tokens_seen": 2465792, "tokens_per_sec": 17324.66879456348, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4014485677083333, "G_std": 0.17360432942708334, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.0005986723836871531, "train_loss": 6.412149906158447, "grad_norm": 0.47703269124031067, "tokens_seen": 2629632, "tokens_per_sec": 17353.914484685996, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4002278645833333, "G_std": 0.17889404296875, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005981935019154823, "train_loss": 6.394064664840698, "grad_norm": 0.46153727173805237, "tokens_seen": 2793472, "tokens_per_sec": 17380.044406093777, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3993326822916667, "G_std": 0.18277994791666666, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005976412987182833, "train_loss": 6.262930393218994, "grad_norm": 0.469079852104187, "tokens_seen": 2957312, "tokens_per_sec": 17402.91094065792, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.3941243489583333, "G_std": 0.18601481119791666, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005970159250623965, "train_loss": 6.217804908752441, "grad_norm": 0.49502474069595337, "tokens_seen": 3121152, "tokens_per_sec": 17424.83290450619, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4176432291666667, "G_std": 0.19710286458333334, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.000596317551918735, "train_loss": 6.270204067230225, "grad_norm": 0.49555933475494385, "tokens_seen": 3284992, "tokens_per_sec": 17443.621901943407, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.41162109375, "G_std": 0.19584147135416666, "skipped": 0} +{"event": "eval", "step": 400, "val_loss": 6.403646516799927, "val_ppl": 604.043682174031, "lambda": null, "S_mean": null, "G_mean": 0.4152018229166667, "G_std": 0.19767252604166666, "tokens_seen": 3284992} +{"event": "step", "step": 420, "lr": 0.0005955463702155432, "train_loss": 6.402326822280884, "grad_norm": 0.7042798399925232, "tokens_seen": 3448832, "tokens_per_sec": 17163.31341518854, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4402669270833333, "G_std": 0.20574951171875, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005947025907861985, "train_loss": 6.354001045227051, "grad_norm": 0.5640040040016174, "tokens_seen": 3612672, "tokens_per_sec": 17191.999699152806, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4210611979166667, "G_std": 0.20633951822916666, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005937864443115718, "train_loss": 6.302140712738037, "grad_norm": 0.4668049216270447, "tokens_seen": 3776512, "tokens_per_sec": 17216.124662005408, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.416015625, "G_std": 0.20821126302083334, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005927981812569615, "train_loss": 6.369995832443237, "grad_norm": 0.4919949173927307, "tokens_seen": 3940352, "tokens_per_sec": 17240.076998808618, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4447428385416667, "G_std": 0.21439615885416666, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005917380718036191, "train_loss": 6.302594184875488, "grad_norm": 0.48996657133102417, "tokens_seen": 4104192, "tokens_per_sec": 17262.364698344405, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.44580078125, "G_std": 0.21687825520833334, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005906064057748841, "train_loss": 6.15567684173584, "grad_norm": 0.4727458655834198, "tokens_seen": 4268032, "tokens_per_sec": 17282.588013655484, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4806315104166667, "G_std": 0.22151692708333334, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.000589403492556949, "train_loss": 6.173834323883057, "grad_norm": 0.45824119448661804, "tokens_seen": 4431872, "tokens_per_sec": 17302.251688117867, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4768880208333333, "G_std": 0.22265625, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.0005881296610142765, "train_loss": 6.041187763214111, "grad_norm": 0.5552144646644592, "tokens_seen": 4595712, "tokens_per_sec": 17320.509420354327, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4951985677083333, "G_std": 0.22347005208333334, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005867852593996914, "train_loss": 6.206289291381836, "grad_norm": 0.5377562046051025, "tokens_seen": 4759552, "tokens_per_sec": 17337.56616527177, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5145670572916666, "G_std": 0.22403971354166666, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005853706552591712, "train_loss": 5.9606099128723145, "grad_norm": 0.47999459505081177, "tokens_seen": 4923392, "tokens_per_sec": 17353.23242248598, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.526123046875, "G_std": 0.2200927734375, "skipped": 0} +{"event": "eval", "step": 600, "val_loss": 6.168170857429504, "val_ppl": 477.31223498705754, "lambda": null, "S_mean": null, "G_mean": 0.5179036458333334, "G_std": 0.22342936197916666, "tokens_seen": 4923392} +{"event": "step", "step": 620, "lr": 0.0005838862353313635, "train_loss": 6.018044471740723, "grad_norm": 0.47550901770591736, "tokens_seen": 5087232, "tokens_per_sec": 17168.319162725467, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5377604166666666, "G_std": 0.22542317708333334, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005823324054418544, "train_loss": 6.104013442993164, "grad_norm": 0.6082955598831177, "tokens_seen": 5251072, "tokens_per_sec": 17189.271272916743, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5563151041666666, "G_std": 0.22648111979166666, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005807095903922209, "train_loss": 5.948821783065796, "grad_norm": 0.48303985595703125, "tokens_seen": 5414912, "tokens_per_sec": 17207.62865550412, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5437825520833334, "G_std": 0.22090657552083334, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005790182338438938, "train_loss": 5.954113483428955, "grad_norm": 0.48425769805908203, "tokens_seen": 5578752, "tokens_per_sec": 17225.329712647046, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5411783854166666, "G_std": 0.21687825520833334, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005772587981968655, "train_loss": 6.035567998886108, "grad_norm": 0.48839735984802246, "tokens_seen": 5742592, "tokens_per_sec": 17241.83575138512, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.581298828125, "G_std": 0.22233072916666666, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005754317644632739, "train_loss": 6.019088268280029, "grad_norm": 0.5084227323532104, "tokens_seen": 5906432, "tokens_per_sec": 17258.02157218826, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5633138020833334, "G_std": 0.22347005208333334, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005735376321358989, "train_loss": 6.033242464065552, "grad_norm": 0.5038526058197021, "tokens_seen": 6070272, "tokens_per_sec": 17273.247614661643, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5607096354166666, "G_std": 0.2193603515625, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.000571576919051606, "train_loss": 5.903595447540283, "grad_norm": 0.5103038549423218, "tokens_seen": 6234112, "tokens_per_sec": 17287.391808815068, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5413411458333334, "G_std": 0.21872965494791666, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.0005695501612497752, "train_loss": 5.923217296600342, "grad_norm": 0.509450376033783, "tokens_seen": 6397952, "tokens_per_sec": 17301.458238881598, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5846354166666666, "G_std": 0.21346028645833334, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000567457912825752, "train_loss": 5.887496471405029, "grad_norm": 0.5595383644104004, "tokens_seen": 6561792, "tokens_per_sec": 17313.549139515933, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.596923828125, "G_std": 0.2113037109375, "skipped": 0} +{"event": "eval", "step": 800, "val_loss": 5.985792970657348, "val_ppl": 397.7377906304358, "lambda": null, "S_mean": null, "G_mean": 0.5601399739583334, "G_std": 0.21278889973958334, "tokens_seen": 6561792} +{"event": "step", "step": 820, "lr": 0.0005653007457793648, "train_loss": 5.956730127334595, "grad_norm": 0.4546455442905426, "tokens_seen": 6725632, "tokens_per_sec": 17172.31845700869, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5618489583333334, "G_std": 0.21323649088541666, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.000563079249858545, "train_loss": 6.133971691131592, "grad_norm": 0.4985527992248535, "tokens_seen": 6889472, "tokens_per_sec": 17186.899494434518, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5763346354166666, "G_std": 0.2125244140625, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005607940323980955, "train_loss": 5.947144985198975, "grad_norm": 0.5299014449119568, "tokens_seen": 7053312, "tokens_per_sec": 17201.5495495537, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5804036458333334, "G_std": 0.20684814453125, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005584457181536525, "train_loss": 5.925986051559448, "grad_norm": 0.5267810225486755, "tokens_seen": 7217152, "tokens_per_sec": 17213.73393275397, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5691731770833334, "G_std": 0.20918782552083334, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005560349491308827, "train_loss": 5.842682123184204, "grad_norm": 0.4826870858669281, "tokens_seen": 7380992, "tokens_per_sec": 17225.389656367854, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5870768229166666, "G_std": 0.20393880208333334, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005535623844099654, "train_loss": 5.748337745666504, "grad_norm": 0.5268693566322327, "tokens_seen": 7544832, "tokens_per_sec": 17235.057803997268, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.591552734375, "G_std": 0.20526123046875, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.000551028699965407, "train_loss": 5.788103103637695, "grad_norm": 0.48989418148994446, "tokens_seen": 7708672, "tokens_per_sec": 17242.819575716683, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5843912760416666, "G_std": 0.201904296875, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005484345884812357, "train_loss": 5.785397291183472, "grad_norm": 0.5717576742172241, "tokens_seen": 7872512, "tokens_per_sec": 17250.346290513655, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.602783203125, "G_std": 0.19989013671875, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005457807591616296, "train_loss": 5.700261116027832, "grad_norm": 0.5040525794029236, "tokens_seen": 8036352, "tokens_per_sec": 17255.276916425184, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5874837239583334, "G_std": 0.20503743489583334, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.0005430679375370263, "train_loss": 5.797425031661987, "grad_norm": 0.5206072330474854, "tokens_seen": 8200192, "tokens_per_sec": 17263.024084531153, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5813802083333334, "G_std": 0.20003255208333334, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.825378942489624, "val_ppl": 338.78949048933714, "lambda": null, "S_mean": null, "G_mean": 0.5883382161458334, "G_std": 0.19940185546875, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005402968652657703, "train_loss": 5.541991233825684, "grad_norm": 0.4957219660282135, "tokens_seen": 8364032, "tokens_per_sec": 17146.104314640685, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.577880859375, "G_std": 0.20070393880208334, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005374682999313519, "train_loss": 5.781191349029541, "grad_norm": 0.47976335883140564, "tokens_seen": 8527872, "tokens_per_sec": 17156.29503377338, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5787353515625, "G_std": 0.20058186848958334, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005345830148352904, "train_loss": 5.496805906295776, "grad_norm": 0.4959251582622528, "tokens_seen": 8691712, "tokens_per_sec": 17165.328411652667, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5699869791666666, "G_std": 0.20174153645833334, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005316417987857215, "train_loss": 5.706320285797119, "grad_norm": 0.5307176113128662, "tokens_seen": 8855552, "tokens_per_sec": 17174.60902704551, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5565592447916666, "G_std": 0.20027669270833334, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005286454558817454, "train_loss": 5.67214298248291, "grad_norm": 0.4922501742839813, "tokens_seen": 9019392, "tokens_per_sec": 17182.584641020694, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5721435546875, "G_std": 0.20194498697916666, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.000525594805293594, "train_loss": 5.612164735794067, "grad_norm": 0.5335658192634583, "tokens_seen": 9183232, "tokens_per_sec": 17191.51357430629, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5660807291666666, "G_std": 0.197265625, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005224906810386772, "train_loss": 5.869666814804077, "grad_norm": 0.5352692604064941, "tokens_seen": 9347072, "tokens_per_sec": 17200.21151689108, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5578206380208334, "G_std": 0.20001220703125, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005193339317535718, "train_loss": 5.6151628494262695, "grad_norm": 0.5167802572250366, "tokens_seen": 9510912, "tokens_per_sec": 17207.819729979314, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5635579427083334, "G_std": 0.20048014322916666, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005161254204620127, "train_loss": 5.663480043411255, "grad_norm": 0.5051583051681519, "tokens_seen": 9674752, "tokens_per_sec": 17215.70251718877, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5602213541666666, "G_std": 0.19828287760416666, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005128660243389501, "train_loss": 5.546140193939209, "grad_norm": 0.5109161734580994, "tokens_seen": 9838592, "tokens_per_sec": 17222.983855206785, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5563557942708334, "G_std": 0.19781494140625, "skipped": 0} +{"event": "eval", "step": 1200, "val_loss": 5.672565662860871, "val_ppl": 290.7796206363278, "lambda": null, "S_mean": null, "G_mean": 0.5663248697916666, "G_std": 0.1934814453125, "tokens_seen": 9838592} +{"event": "step", "step": 1220, "lr": 0.000509556634470739, "train_loss": 5.576826095581055, "grad_norm": 0.5302457213401794, "tokens_seen": 10002432, "tokens_per_sec": 17128.550244830363, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5413818359375, "G_std": 0.19510904947916666, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005061981556115254, "train_loss": 5.464673042297363, "grad_norm": 0.5050945281982422, "tokens_seen": 10166272, "tokens_per_sec": 17137.484897208265, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5687255859375, "G_std": 0.19500732421875, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005027915059358954, "train_loss": 5.45524001121521, "grad_norm": 0.5176246762275696, "tokens_seen": 10330112, "tokens_per_sec": 17145.722172987713, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5423177083333334, "G_std": 0.18986002604166666, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0004993376167878549, "train_loss": 5.571907043457031, "grad_norm": 0.5805303454399109, "tokens_seen": 10493952, "tokens_per_sec": 17154.32982177874, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5581461588541666, "G_std": 0.1973876953125, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0004958374324262102, "train_loss": 5.672812223434448, "grad_norm": 0.5629904270172119, "tokens_seen": 10657792, "tokens_per_sec": 17161.552603503882, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5564371744791666, "G_std": 0.1959228515625, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0004922919097664167, "train_loss": 5.631197929382324, "grad_norm": 0.5654869079589844, "tokens_seen": 10821632, "tokens_per_sec": 17169.11810579161, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5655517578125, "G_std": 0.19683837890625, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0004887020181189677, "train_loss": 5.7252843379974365, "grad_norm": 0.5423831939697266, "tokens_seen": 10985472, "tokens_per_sec": 17176.264595655004, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5574544270833334, "G_std": 0.19683837890625, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.00048506873892439467, "train_loss": 5.540042161941528, "grad_norm": 0.5597848296165466, "tokens_seen": 11149312, "tokens_per_sec": 17184.124420170436, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5557861328125, "G_std": 0.19217936197916666, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0004813930654849516, "train_loss": 5.4130659103393555, "grad_norm": 0.5364397168159485, "tokens_seen": 11313152, "tokens_per_sec": 17191.927048901063, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5404052734375, "G_std": 0.19620768229166666, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0004776760026930552, "train_loss": 5.420212507247925, "grad_norm": 0.6062468886375427, "tokens_seen": 11476992, "tokens_per_sec": 17198.437690237915, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5691731770833334, "G_std": 0.19441731770833334, "skipped": 0} +{"event": "eval", "step": 1400, "val_loss": 5.558569550514221, "val_ppl": 259.45143857652585, "lambda": null, "S_mean": null, "G_mean": 0.5335286458333334, "G_std": 0.1917724609375, "tokens_seen": 11476992} +{"event": "step", "step": 1420, "lr": 0.00047391856675655823, "train_loss": 5.415208339691162, "grad_norm": 0.6038483381271362, "tokens_seen": 11640832, "tokens_per_sec": 17118.20380020093, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5450846354166666, "G_std": 0.1993408203125, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.00047012178492092805, "train_loss": 5.399306535720825, "grad_norm": 0.5587677955627441, "tokens_seen": 11804672, "tokens_per_sec": 17126.650666440964, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5409342447916666, "G_std": 0.19730631510416666, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.00046628669518840776, "train_loss": 5.515282154083252, "grad_norm": 0.5624483227729797, "tokens_seen": 11968512, "tokens_per_sec": 17134.052311787782, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5529378255208334, "G_std": 0.19649251302083334, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.00046241434603423715, "train_loss": 5.572759628295898, "grad_norm": 0.5659080147743225, "tokens_seen": 12132352, "tokens_per_sec": 17141.024399365127, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5250244140625, "G_std": 0.19673665364583334, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0004585057961200098, "train_loss": 5.483408689498901, "grad_norm": 0.5952575206756592, "tokens_seen": 12296192, "tokens_per_sec": 17149.30544276937, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5370279947916666, "G_std": 0.19325764973958334, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.00045456211400424565, "train_loss": 5.446941614151001, "grad_norm": 0.5268484354019165, "tokens_seen": 12460032, "tokens_per_sec": 17157.152611973772, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5386555989583334, "G_std": 0.19246419270833334, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0004505843778502577, "train_loss": 5.362824440002441, "grad_norm": 0.5200818181037903, "tokens_seen": 12623872, "tokens_per_sec": 17163.567932209688, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5360514322916666, "G_std": 0.18631998697916666, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.000446573675131393, "train_loss": 5.356315612792969, "grad_norm": 0.5222775340080261, "tokens_seen": 12787712, "tokens_per_sec": 17170.980792442606, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5361328125, "G_std": 0.19075520833333334, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0004425311023337283, "train_loss": 5.4303297996521, "grad_norm": 0.5808046460151672, "tokens_seen": 12951552, "tokens_per_sec": 17177.88219956918, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5266520182291666, "G_std": 0.19529215494791666, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0004384577646563017, "train_loss": 5.4288225173950195, "grad_norm": 0.550300121307373, "tokens_seen": 13115392, "tokens_per_sec": 17184.739726727064, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5325520833333334, "G_std": 0.1953125, "skipped": 0} +{"event": "eval", "step": 1600, "val_loss": 5.48638836145401, "val_ppl": 241.3838394846024, "lambda": null, "S_mean": null, "G_mean": 0.5387369791666666, "G_std": 0.1949462890625, "tokens_seen": 13115392} +{"event": "step", "step": 1620, "lr": 0.00043435477570896204, "train_loss": 5.389299392700195, "grad_norm": 0.5224184989929199, "tokens_seen": 13279232, "tokens_per_sec": 17116.042174540427, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.54119873046875, "G_std": 0.19681803385416666, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0004302232572079195, "train_loss": 5.271656513214111, "grad_norm": 0.5326958894729614, "tokens_seen": 13443072, "tokens_per_sec": 17123.71024925764, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.52777099609375, "G_std": 0.19576009114583334, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0004260643386690794, "train_loss": 5.440608024597168, "grad_norm": 0.5765966176986694, "tokens_seen": 13606912, "tokens_per_sec": 17131.80406147259, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5328776041666666, "G_std": 0.19769287109375, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.00042187915709924356, "train_loss": 5.596676588058472, "grad_norm": 0.54762864112854, "tokens_seen": 13770752, "tokens_per_sec": 17139.80161963782, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5220540364583334, "G_std": 0.19694010416666666, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0004176688566852645, "train_loss": 5.453932046890259, "grad_norm": 0.5771484375, "tokens_seen": 13934592, "tokens_per_sec": 17147.370410972104, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5311482747395834, "G_std": 0.19309488932291666, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.00041343458848123576, "train_loss": 5.21959114074707, "grad_norm": 0.5627127289772034, "tokens_seen": 14098432, "tokens_per_sec": 17154.24918326592, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5096638997395834, "G_std": 0.18747965494791666, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.00040917751009380545, "train_loss": 5.274595022201538, "grad_norm": 0.5635439157485962, "tokens_seen": 14262272, "tokens_per_sec": 17161.87537661059, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5119222005208334, "G_std": 0.18784586588541666, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.000404898785365698, "train_loss": 5.356980085372925, "grad_norm": 0.540192186832428, "tokens_seen": 14426112, "tokens_per_sec": 17168.525643703273, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.50897216796875, "G_std": 0.18973795572916666, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.00040059958405753125, "train_loss": 5.209353685379028, "grad_norm": 0.5359346270561218, "tokens_seen": 14589952, "tokens_per_sec": 17175.35406211877, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5138956705729166, "G_std": 0.19189453125, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.00039628108152801574, "train_loss": 5.277047157287598, "grad_norm": 0.5840287804603577, "tokens_seen": 14753792, "tokens_per_sec": 17181.455129615984, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5162760416666666, "G_std": 0.19260660807291666, "skipped": 0} +{"event": "eval", "step": 1800, "val_loss": 5.4124178171157835, "val_ppl": 224.17294214519245, "lambda": null, "S_mean": null, "G_mean": 0.5143025716145834, "G_std": 0.19264729817708334, "tokens_seen": 14753792} +{"event": "step", "step": 1820, "lr": 0.0003919444584126229, "train_loss": 5.373379468917847, "grad_norm": 0.5935393571853638, "tokens_seen": 14917632, "tokens_per_sec": 17119.42764393717, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5155843098958334, "G_std": 0.19283040364583334, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0003875909003008116, "train_loss": 5.216418027877808, "grad_norm": 0.5897635817527771, "tokens_seen": 15081472, "tokens_per_sec": 17125.898514860048, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5003458658854166, "G_std": 0.19049072265625, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.00038322159741189957, "train_loss": 5.2742228507995605, "grad_norm": 0.6448111534118652, "tokens_seen": 15245312, "tokens_per_sec": 17132.587045880424, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5200398763020834, "G_std": 0.191650390625, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.00037883774426966964, "train_loss": 5.351633548736572, "grad_norm": 0.5910462141036987, "tokens_seen": 15409152, "tokens_per_sec": 17138.695627176687, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5229085286458334, "G_std": 0.18988037109375, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0003744405393757981, "train_loss": 5.308071613311768, "grad_norm": 0.6174255609512329, "tokens_seen": 15572992, "tokens_per_sec": 17145.00409215397, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5034586588541666, "G_std": 0.19012451171875, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0003700311848821973, "train_loss": 5.261961221694946, "grad_norm": 0.5746513605117798, "tokens_seen": 15736832, "tokens_per_sec": 17151.471262677464, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5080362955729166, "G_std": 0.19482421875, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0003656108862623586, "train_loss": 5.311746597290039, "grad_norm": 0.5747127532958984, "tokens_seen": 15900672, "tokens_per_sec": 17157.806273836915, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5031331380208334, "G_std": 0.19019571940104166, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.00036118085198178824, "train_loss": 5.5160863399505615, "grad_norm": 0.6227937936782837, "tokens_seen": 16064512, "tokens_per_sec": 17163.421265593537, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.50909423828125, "G_std": 0.19117228190104166, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.00035674229316762495, "train_loss": 5.420614004135132, "grad_norm": 0.5718360543251038, "tokens_seen": 16228352, "tokens_per_sec": 17170.099691966534, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5130411783854166, "G_std": 0.19211832682291666, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0003522964232775297, "train_loss": 5.253361701965332, "grad_norm": 0.565953254699707, "tokens_seen": 16392192, "tokens_per_sec": 17176.339819565772, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5052286783854166, "G_std": 0.1876220703125, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.372373044490814, "val_ppl": 215.37335231060365, "lambda": null, "S_mean": null, "G_mean": 0.5068359375, "G_std": 0.18633015950520834, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0003478444577679388, "train_loss": 5.226739168167114, "grad_norm": 0.6098723411560059, "tokens_seen": 16556032, "tokens_per_sec": 17122.12788308906, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5080159505208334, "G_std": 0.19346110026041666, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.00034338761376176993, "train_loss": 5.369190692901611, "grad_norm": 0.6564699411392212, "tokens_seen": 16719872, "tokens_per_sec": 17128.763888823927, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5000203450520834, "G_std": 0.19403076171875, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0003389271097156731, "train_loss": 5.0993993282318115, "grad_norm": 0.6156439781188965, "tokens_seen": 16883712, "tokens_per_sec": 17135.686823419248, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5040283203125, "G_std": 0.19146728515625, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.00033446416508691677, "train_loss": 5.244616746902466, "grad_norm": 0.5869975686073303, "tokens_seen": 17047552, "tokens_per_sec": 17142.286878250652, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.5069580078125, "G_std": 0.191070556640625, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.00032999999999999994, "train_loss": 5.243185520172119, "grad_norm": 0.5543504953384399, "tokens_seen": 17211392, "tokens_per_sec": 17148.764659163568, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.49273681640625, "G_std": 0.191741943359375, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0003255358349130832, "train_loss": 5.152468919754028, "grad_norm": 0.5921120643615723, "tokens_seen": 17375232, "tokens_per_sec": 17155.013425781584, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4863688151041667, "G_std": 0.18560791015625, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0003210728902843267, "train_loss": 5.086758136749268, "grad_norm": 0.59932541847229, "tokens_seen": 17539072, "tokens_per_sec": 17161.412853973747, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4986979166666667, "G_std": 0.18938191731770834, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0003166123862382299, "train_loss": 5.127586603164673, "grad_norm": 0.615513265132904, "tokens_seen": 17702912, "tokens_per_sec": 17166.543057181218, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4900105794270833, "G_std": 0.19010416666666666, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0003121555422320611, "train_loss": 5.173962354660034, "grad_norm": 0.6525688767433167, "tokens_seen": 17866752, "tokens_per_sec": 17172.72617240733, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4993896484375, "G_std": 0.19225056966145834, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.00030770357672247023, "train_loss": 5.31455135345459, "grad_norm": 0.6006180644035339, "tokens_seen": 18030592, "tokens_per_sec": 17178.591853061247, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4974365234375, "G_std": 0.19343058268229166, "skipped": 0} +{"event": "eval", "step": 2200, "val_loss": 5.278232133388519, "val_ppl": 196.0230262874387, "lambda": null, "S_mean": null, "G_mean": 0.4930013020833333, "G_std": 0.1898193359375, "tokens_seen": 18030592} +{"event": "step", "step": 2220, "lr": 0.00030325770683237505, "train_loss": 5.20708966255188, "grad_norm": 0.5952531099319458, "tokens_seen": 18194432, "tokens_per_sec": 17128.803581632852, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4936726888020833, "G_std": 0.19148763020833334, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0002988191480182117, "train_loss": 5.148871898651123, "grad_norm": 0.5748616456985474, "tokens_seen": 18358272, "tokens_per_sec": 17134.35098543873, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4821573893229167, "G_std": 0.1893310546875, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0002943891137376413, "train_loss": 5.157868385314941, "grad_norm": 0.5733730792999268, "tokens_seen": 18522112, "tokens_per_sec": 17140.430342535987, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4826456705729167, "G_std": 0.189117431640625, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.0002899688151178026, "train_loss": 5.33675742149353, "grad_norm": 0.6393439173698425, "tokens_seen": 18685952, "tokens_per_sec": 17146.370835718983, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.483154296875, "G_std": 0.187225341796875, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.00028555946062420177, "train_loss": 5.1231160163879395, "grad_norm": 0.6320329904556274, "tokens_seen": 18849792, "tokens_per_sec": 17151.76395399015, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4803059895833333, "G_std": 0.189300537109375, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.00028116225573033036, "train_loss": 5.2337095737457275, "grad_norm": 0.6345316171646118, "tokens_seen": 19013632, "tokens_per_sec": 17157.018312368647, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.478759765625, "G_std": 0.18959554036458334, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0002767784025881003, "train_loss": 5.167039632797241, "grad_norm": 0.633669376373291, "tokens_seen": 19177472, "tokens_per_sec": 17156.210077751795, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.483642578125, "G_std": 0.18755086263020834, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.00027240909969918834, "train_loss": 5.251038074493408, "grad_norm": 0.6096031069755554, "tokens_seen": 19341312, "tokens_per_sec": 17150.494607168825, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4729207356770833, "G_std": 0.18616739908854166, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.00026805554158737706, "train_loss": 5.135629892349243, "grad_norm": 0.6145675778388977, "tokens_seen": 19505152, "tokens_per_sec": 17152.62764875554, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.48333740234375, "G_std": 0.18767293294270834, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.0002637189184719842, "train_loss": 5.030846834182739, "grad_norm": 0.6235900521278381, "tokens_seen": 19668992, "tokens_per_sec": 17156.0016309164, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4855143229166667, "G_std": 0.190399169921875, "skipped": 0} +{"event": "eval", "step": 2400, "val_loss": 5.21182256937027, "val_ppl": 183.42806409806855, "lambda": null, "S_mean": null, "G_mean": 0.485107421875, "G_std": 0.1895751953125, "tokens_seen": 19668992} +{"event": "step", "step": 2420, "lr": 0.00025940041594246864, "train_loss": 5.034740924835205, "grad_norm": 0.6429900527000427, "tokens_seen": 19832832, "tokens_per_sec": 17107.933341682587, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.47509765625, "G_std": 0.19132486979166666, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.000255101214634302, "train_loss": 5.139315128326416, "grad_norm": 0.6185042262077332, "tokens_seen": 19996672, "tokens_per_sec": 17111.232466030735, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.46600341796875, "G_std": 0.18586222330729166, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.00025082248990619454, "train_loss": 5.227548599243164, "grad_norm": 0.6202218532562256, "tokens_seen": 20160512, "tokens_per_sec": 17115.199330307438, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.47979736328125, "G_std": 0.18782552083333334, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.0002465654115187642, "train_loss": 5.115784406661987, "grad_norm": 0.6058593392372131, "tokens_seen": 20324352, "tokens_per_sec": 17119.273662182266, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4676513671875, "G_std": 0.18546549479166666, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.00024233114331473544, "train_loss": 4.961671352386475, "grad_norm": 0.6129927039146423, "tokens_seen": 20488192, "tokens_per_sec": 17123.466157060706, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4730021158854167, "G_std": 0.18901570638020834, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.00023812084290075633, "train_loss": 5.037380218505859, "grad_norm": 0.5843706130981445, "tokens_seen": 20652032, "tokens_per_sec": 17127.581703769083, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.46563720703125, "G_std": 0.18805948893229166, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.00023393566133092048, "train_loss": 5.092918157577515, "grad_norm": 0.6105762720108032, "tokens_seen": 20815872, "tokens_per_sec": 17132.05639241563, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4740397135416667, "G_std": 0.18754069010416666, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.00022977674279208032, "train_loss": 4.898678541183472, "grad_norm": 0.6551973223686218, "tokens_seen": 20979712, "tokens_per_sec": 17136.455095809855, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4639485677083333, "G_std": 0.18603515625, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.00022564522429103793, "train_loss": 5.188225030899048, "grad_norm": 0.6525774598121643, "tokens_seen": 21143552, "tokens_per_sec": 17141.012739572423, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4627888997395833, "G_std": 0.1871337890625, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00022154223534369825, "train_loss": 4.963178634643555, "grad_norm": 0.6131529808044434, "tokens_seen": 21307392, "tokens_per_sec": 17145.828146187967, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4612630208333333, "G_std": 0.1849365234375, "skipped": 0} +{"event": "eval", "step": 2600, "val_loss": 5.173846662044525, "val_ppl": 176.59282557200152, "lambda": null, "S_mean": null, "G_mean": 0.4725748697916667, "G_std": 0.18658447265625, "tokens_seen": 21307392} +{"event": "step", "step": 2620, "lr": 0.00021746889766627162, "train_loss": 5.140842437744141, "grad_norm": 0.6730852127075195, "tokens_seen": 21471232, "tokens_per_sec": 17103.439677396083, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4739176432291667, "G_std": 0.18548583984375, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0002134263248686069, "train_loss": 5.147828817367554, "grad_norm": 0.6925684213638306, "tokens_seen": 21635072, "tokens_per_sec": 17108.244830355863, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4818522135416667, "G_std": 0.18784586588541666, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.00020941562214974222, "train_loss": 5.008978366851807, "grad_norm": 0.6329017877578735, "tokens_seen": 21798912, "tokens_per_sec": 17112.931111494414, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4649454752604167, "G_std": 0.18291219075520834, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0002054378859957543, "train_loss": 4.847362041473389, "grad_norm": 0.7835169434547424, "tokens_seen": 21962752, "tokens_per_sec": 17117.58337399862, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4655558268229167, "G_std": 0.18603515625, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0002014942038799901, "train_loss": 5.090112924575806, "grad_norm": 0.6949713826179504, "tokens_seen": 22126592, "tokens_per_sec": 17122.21842416686, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4690958658854167, "G_std": 0.187835693359375, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.00019758565396576285, "train_loss": 5.088443040847778, "grad_norm": 0.6923709511756897, "tokens_seen": 22290432, "tokens_per_sec": 17126.86832754837, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4544677734375, "G_std": 0.18590291341145834, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.00019371330481159223, "train_loss": 5.068541526794434, "grad_norm": 0.6351603269577026, "tokens_seen": 22454272, "tokens_per_sec": 17130.941377957657, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4671630859375, "G_std": 0.18669637044270834, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.00018987821507907194, "train_loss": 4.9181718826293945, "grad_norm": 0.6568493843078613, "tokens_seen": 22618112, "tokens_per_sec": 17135.52537988495, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4716796875, "G_std": 0.18596394856770834, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.00018608143324344171, "train_loss": 5.049103498458862, "grad_norm": 0.6434805989265442, "tokens_seen": 22781952, "tokens_per_sec": 17139.81605323699, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4642130533854167, "G_std": 0.18343098958333334, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.00018232399730694476, "train_loss": 4.996208190917969, "grad_norm": 0.6680638194084167, "tokens_seen": 22945792, "tokens_per_sec": 17143.58306140298, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4649658203125, "G_std": 0.18834431966145834, "skipped": 0} +{"event": "eval", "step": 2800, "val_loss": 5.106460976600647, "val_ppl": 165.08507987307894, "lambda": null, "S_mean": null, "G_mean": 0.45648193359375, "G_std": 0.18641153971354166, "tokens_seen": 22945792} +{"event": "step", "step": 2820, "lr": 0.00017860693451504837, "train_loss": 5.031168699264526, "grad_norm": 0.6529491543769836, "tokens_seen": 23109632, "tokens_per_sec": 17103.570284283673, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.46160888671875, "G_std": 0.18650309244791666, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.00017493126107560522, "train_loss": 4.850235462188721, "grad_norm": 0.6840153932571411, "tokens_seen": 23273472, "tokens_per_sec": 17107.96017598299, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.449462890625, "G_std": 0.18410237630208334, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.00017129798188103226, "train_loss": 5.016289234161377, "grad_norm": 0.6253331303596497, "tokens_seen": 23437312, "tokens_per_sec": 17112.294941340602, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4520263671875, "G_std": 0.18190511067708334, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.00016770809023358322, "train_loss": 5.006634712219238, "grad_norm": 0.755618691444397, "tokens_seen": 23601152, "tokens_per_sec": 17116.74410517038, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4512125651041667, "G_std": 0.18951416015625, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.00016416256757378968, "train_loss": 4.851034641265869, "grad_norm": 0.6539465188980103, "tokens_seen": 23764992, "tokens_per_sec": 17121.219169986245, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4589640299479167, "G_std": 0.18593343098958334, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.000160662383212145, "train_loss": 5.060125827789307, "grad_norm": 0.6579357981681824, "tokens_seen": 23928832, "tokens_per_sec": 17125.220953560707, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4770711263020833, "G_std": 0.18938191731770834, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.00015720849406410448, "train_loss": 4.967514276504517, "grad_norm": 0.6187177896499634, "tokens_seen": 24092672, "tokens_per_sec": 17129.31564034902, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4576619466145833, "G_std": 0.1845703125, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.00015380184438847444, "train_loss": 4.9507668018341064, "grad_norm": 0.6601535677909851, "tokens_seen": 24256512, "tokens_per_sec": 17133.331370104814, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4636637369791667, "G_std": 0.18593343098958334, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.00015044336552926094, "train_loss": 4.911197185516357, "grad_norm": 0.6382002234458923, "tokens_seen": 24420352, "tokens_per_sec": 17137.269436095226, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4538981119791667, "G_std": 0.18437703450520834, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.00014713397566104995, "train_loss": 4.863725185394287, "grad_norm": 0.6832441687583923, "tokens_seen": 24584192, "tokens_per_sec": 17141.468632922235, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4453328450520833, "G_std": 0.18247477213541666, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.121176671981812, "val_ppl": 167.53238435022772, "lambda": null, "S_mean": null, "G_mean": 0.4664713541666667, "G_std": 0.18342081705729166, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.00014387457953798726, "train_loss": 4.932088375091553, "grad_norm": 0.6196199059486389, "tokens_seen": 24748032, "tokens_per_sec": 17104.492321935104, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4552815755208333, "G_std": 0.18620808919270834, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.00014066606824642808, "train_loss": 5.0088207721710205, "grad_norm": 0.7093318104743958, "tokens_seen": 24911872, "tokens_per_sec": 17108.884764303293, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4568888346354167, "G_std": 0.18316650390625, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.00013750931896132278, "train_loss": 4.971526622772217, "grad_norm": 0.6942002773284912, "tokens_seen": 25075712, "tokens_per_sec": 17113.31870225437, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4580485026041667, "G_std": 0.185791015625, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.00013440519470640596, "train_loss": 4.816200256347656, "grad_norm": 0.6351425051689148, "tokens_seen": 25239552, "tokens_per_sec": 17117.66471825739, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4505411783854167, "G_std": 0.18452962239583334, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.00013135454411825447, "train_loss": 5.0213823318481445, "grad_norm": 0.6937621235847473, "tokens_seen": 25403392, "tokens_per_sec": 17121.063680245683, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4586181640625, "G_std": 0.187042236328125, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.00012835820121427852, "train_loss": 5.055838584899902, "grad_norm": 0.7068161368370056, "tokens_seen": 25567232, "tokens_per_sec": 17125.35675909521, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4617106119791667, "G_std": 0.187530517578125, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0001254169851647097, "train_loss": 4.900604724884033, "grad_norm": 0.6451532244682312, "tokens_seen": 25731072, "tokens_per_sec": 17129.562763952148, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.456298828125, "G_std": 0.18507893880208334, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.00012253170006864812, "train_loss": 5.057351589202881, "grad_norm": 0.8039398193359375, "tokens_seen": 25894912, "tokens_per_sec": 17133.909080043697, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.462158203125, "G_std": 0.1851806640625, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.00011970313473422966, "train_loss": 4.887045860290527, "grad_norm": 0.6508256793022156, "tokens_seen": 26058752, "tokens_per_sec": 17138.081544159617, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4506429036458333, "G_std": 0.18393961588541666, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.00011693206246297376, "train_loss": 4.907451391220093, "grad_norm": 0.6901627779006958, "tokens_seen": 26222592, "tokens_per_sec": 17142.075005360253, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.45135498046875, "G_std": 0.18355305989583334, "skipped": 0} +{"event": "eval", "step": 3200, "val_loss": 5.028978729248047, "val_ppl": 152.7769064097008, "lambda": null, "S_mean": null, "G_mean": 0.458984375, "G_std": 0.18415323893229166, "tokens_seen": 26222592} +{"event": "step", "step": 3220, "lr": 0.00011421924083837032, "train_loss": 5.021888732910156, "grad_norm": 0.7422127723693848, "tokens_seen": 26386432, "tokens_per_sec": 17107.76817369697, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4473470052083333, "G_std": 0.18335978190104166, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.00011156541151876421, "train_loss": 5.0297253131866455, "grad_norm": 0.6749002933502197, "tokens_seen": 26550272, "tokens_per_sec": 17111.9178400943, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4608561197916667, "G_std": 0.18642171223958334, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.00010897130003459306, "train_loss": 4.792108058929443, "grad_norm": 0.6720587611198425, "tokens_seen": 26714112, "tokens_per_sec": 17115.840415346465, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.45697021484375, "G_std": 0.18448893229166666, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.00010643761559003458, "train_loss": 5.03830361366272, "grad_norm": 0.7002348899841309, "tokens_seen": 26877952, "tokens_per_sec": 17120.236374272234, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4521077473958333, "G_std": 0.1851806640625, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0001039650508691173, "train_loss": 4.860140323638916, "grad_norm": 0.6837826371192932, "tokens_seen": 27041792, "tokens_per_sec": 17124.072437852363, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4398193359375, "G_std": 0.180999755859375, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0001015542818463474, "train_loss": 5.028934955596924, "grad_norm": 0.7117943167686462, "tokens_seen": 27205632, "tokens_per_sec": 17127.760533231994, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4434407552083333, "G_std": 0.18402099609375, "skipped": 0} +{"event": "step", "step": 3340, "lr": 9.920596760190444e-05, "train_loss": 4.8557868003845215, "grad_norm": 0.6710282564163208, "tokens_seen": 27369472, "tokens_per_sec": 17131.46948306932, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4430745442708333, "G_std": 0.183319091796875, "skipped": 0} +{"event": "step", "step": 3360, "lr": 9.692075014145504e-05, "train_loss": 5.009390115737915, "grad_norm": 0.7153744697570801, "tokens_seen": 27533312, "tokens_per_sec": 17135.09662441313, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.445068359375, "G_std": 0.18384806315104166, "skipped": 0} +{"event": "step", "step": 3380, "lr": 9.46992542206351e-05, "train_loss": 4.999907732009888, "grad_norm": 0.6932403445243835, "tokens_seen": 27697152, "tokens_per_sec": 17138.786926251265, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4417724609375, "G_std": 0.1822509765625, "skipped": 0} +{"event": "step", "step": 3400, "lr": 9.254208717424797e-05, "train_loss": 4.889073371887207, "grad_norm": 0.6791947484016418, "tokens_seen": 27860992, "tokens_per_sec": 17142.449193253553, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4418131510416667, "G_std": 0.18456013997395834, "skipped": 0} +{"event": "eval", "step": 3400, "val_loss": 5.0327965259552006, "val_ppl": 153.36129240356925, "lambda": null, "S_mean": null, "G_mean": 0.44757080078125, "G_std": 0.18470255533854166, "tokens_seen": 27860992} +{"event": "step", "step": 3420, "lr": 9.04498387502248e-05, "train_loss": 4.919636964797974, "grad_norm": 0.6894566416740417, "tokens_seen": 28024832, "tokens_per_sec": 17110.49778672493, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4532674153645833, "G_std": 0.18464152018229166, "skipped": 0} +{"event": "step", "step": 3440, "lr": 8.842308094839387e-05, "train_loss": 4.969231128692627, "grad_norm": 0.7258201241493225, "tokens_seen": 28188672, "tokens_per_sec": 17114.243181144313, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4556070963541667, "G_std": 0.18288167317708334, "skipped": 0} +{"event": "step", "step": 3460, "lr": 8.646236786410109e-05, "train_loss": 4.833479881286621, "grad_norm": 0.6987709999084473, "tokens_seen": 28352512, "tokens_per_sec": 17118.034853177902, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4396565755208333, "G_std": 0.18316650390625, "skipped": 0} +{"event": "step", "step": 3480, "lr": 8.456823553672612e-05, "train_loss": 4.9484498500823975, "grad_norm": 0.6853471398353577, "tokens_seen": 28516352, "tokens_per_sec": 17121.857718241794, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4453125, "G_std": 0.18583170572916666, "skipped": 0} +{"event": "step", "step": 3500, "lr": 8.274120180313448e-05, "train_loss": 4.852669954299927, "grad_norm": 0.678068220615387, "tokens_seen": 28680192, "tokens_per_sec": 17125.706105611018, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4424031575520833, "G_std": 0.184356689453125, "skipped": 0} +{"event": "step", "step": 3520, "lr": 8.098176615610612e-05, "train_loss": 4.741950988769531, "grad_norm": 0.7352700233459473, "tokens_seen": 28844032, "tokens_per_sec": 17129.556929024282, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4512532552083333, "G_std": 0.18738810221354166, "skipped": 0} +{"event": "step", "step": 3540, "lr": 7.929040960777911e-05, "train_loss": 4.738071441650391, "grad_norm": 0.704932451248169, "tokens_seen": 29007872, "tokens_per_sec": 17133.355677261607, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4396565755208333, "G_std": 0.18056233723958334, "skipped": 0} +{"event": "step", "step": 3560, "lr": 7.76675945581456e-05, "train_loss": 4.989506959915161, "grad_norm": 0.6758953928947449, "tokens_seen": 29171712, "tokens_per_sec": 17137.274543465843, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4422607421875, "G_std": 0.18071492513020834, "skipped": 0} +{"event": "step", "step": 3580, "lr": 7.611376466863648e-05, "train_loss": 4.819319725036621, "grad_norm": 0.6647200584411621, "tokens_seen": 29335552, "tokens_per_sec": 17141.087194202562, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.43359375, "G_std": 0.18017578125, "skipped": 0} +{"event": "step", "step": 3600, "lr": 7.462934474082865e-05, "train_loss": 4.850670337677002, "grad_norm": 0.756624162197113, "tokens_seen": 29499392, "tokens_per_sec": 17145.13230559456, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4462483723958333, "G_std": 0.1826171875, "skipped": 0} +{"event": "eval", "step": 3600, "val_loss": 4.997407293319702, "val_ppl": 148.0288657088242, "lambda": null, "S_mean": null, "G_mean": 0.4366048177083333, "G_std": 0.17948404947916666, "tokens_seen": 29499392} +{"event": "step", "step": 3620, "lr": 7.321474060030854e-05, "train_loss": 4.905128002166748, "grad_norm": 0.7215244174003601, "tokens_seen": 29663232, "tokens_per_sec": 17115.282477123037, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.435791015625, "G_std": 0.18195597330729166, "skipped": 0} +{"event": "step", "step": 3640, "lr": 7.187033898572343e-05, "train_loss": 4.977303504943848, "grad_norm": 0.7003995776176453, "tokens_seen": 29827072, "tokens_per_sec": 17119.012755716325, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4398600260416667, "G_std": 0.18022664388020834, "skipped": 0} +{"event": "step", "step": 3660, "lr": 7.0596507443051e-05, "train_loss": 5.018312215805054, "grad_norm": 0.79936283826828, "tokens_seen": 29990912, "tokens_per_sec": 17122.87325891007, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4494222005208333, "G_std": 0.184112548828125, "skipped": 0} +{"event": "step", "step": 3680, "lr": 6.93935942251159e-05, "train_loss": 4.93916654586792, "grad_norm": 0.6864333748817444, "tokens_seen": 30154752, "tokens_per_sec": 17126.877786628036, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.435791015625, "G_std": 0.18211873372395834, "skipped": 0} +{"event": "step", "step": 3700, "lr": 6.82619281963808e-05, "train_loss": 4.98549222946167, "grad_norm": 0.675516664981842, "tokens_seen": 30318592, "tokens_per_sec": 17130.419571096132, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4390665690104167, "G_std": 0.1829833984375, "skipped": 0} +{"event": "step", "step": 3720, "lr": 6.720181874303845e-05, "train_loss": 4.760695695877075, "grad_norm": 0.6770106554031372, "tokens_seen": 30482432, "tokens_per_sec": 17134.247711484102, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.43572998046875, "G_std": 0.18349202473958334, "skipped": 0} +{"event": "step", "step": 3740, "lr": 6.62135556884282e-05, "train_loss": 4.94133448600769, "grad_norm": 0.7522312998771667, "tokens_seen": 30646272, "tokens_per_sec": 17137.524178122694, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.43365478515625, "G_std": 0.18117268880208334, "skipped": 0} +{"event": "step", "step": 3760, "lr": 6.529740921380147e-05, "train_loss": 4.873022079467773, "grad_norm": 0.6978594064712524, "tokens_seen": 30810112, "tokens_per_sec": 17140.530770345154, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4378662109375, "G_std": 0.17961629231770834, "skipped": 0} +{"event": "step", "step": 3780, "lr": 6.445362978445676e-05, "train_loss": 4.783512353897095, "grad_norm": 0.7067244648933411, "tokens_seen": 30973952, "tokens_per_sec": 17144.443060403046, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4366455078125, "G_std": 0.17896525065104166, "skipped": 0} +{"event": "step", "step": 3800, "lr": 6.368244808126497e-05, "train_loss": 4.900389194488525, "grad_norm": 0.7266732454299927, "tokens_seen": 31137792, "tokens_per_sec": 17148.130282167374, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4463297526041667, "G_std": 0.181915283203125, "skipped": 0} +{"event": "eval", "step": 3800, "val_loss": 4.975602662563324, "val_ppl": 144.8360861380076, "lambda": null, "S_mean": null, "G_mean": 0.43267822265625, "G_std": 0.17845662434895834, "tokens_seen": 31137792} +{"event": "step", "step": 3820, "lr": 6.29840749376035e-05, "train_loss": 4.945023536682129, "grad_norm": 0.6832239031791687, "tokens_seen": 31301632, "tokens_per_sec": 17119.01428487174, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4392903645833333, "G_std": 0.18108113606770834, "skipped": 0} +{"event": "step", "step": 3840, "lr": 6.23587012817167e-05, "train_loss": 4.877219200134277, "grad_norm": 0.6931920647621155, "tokens_seen": 31465472, "tokens_per_sec": 17122.671830444855, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4384765625, "G_std": 0.18076578776041666, "skipped": 0} +{"event": "step", "step": 3860, "lr": 6.18064980845177e-05, "train_loss": 4.890400648117065, "grad_norm": 0.7316928505897522, "tokens_seen": 31629312, "tokens_per_sec": 17126.14310025294, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4364217122395833, "G_std": 0.18004353841145834, "skipped": 0} +{"event": "step", "step": 3880, "lr": 6.132761631284691e-05, "train_loss": 4.957690954208374, "grad_norm": 0.7352874875068665, "tokens_seen": 31793152, "tokens_per_sec": 17129.494232423207, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.445556640625, "G_std": 0.179779052734375, "skipped": 0} +{"event": "step", "step": 3900, "lr": 6.092218688819914e-05, "train_loss": 4.848414421081543, "grad_norm": 0.7141496539115906, "tokens_seen": 31956992, "tokens_per_sec": 17133.015936843683, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4344685872395833, "G_std": 0.180419921875, "skipped": 0} +{"event": "step", "step": 3920, "lr": 6.059032065093088e-05, "train_loss": 4.785126686096191, "grad_norm": 0.7542188167572021, "tokens_seen": 32120832, "tokens_per_sec": 17136.42935171679, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.433837890625, "G_std": 0.17963663736979166, "skipped": 0} +{"event": "step", "step": 3940, "lr": 6.033210832995772e-05, "train_loss": 4.961305379867554, "grad_norm": 0.7421402931213379, "tokens_seen": 32284672, "tokens_per_sec": 17139.869729535705, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4343668619791667, "G_std": 0.17948404947916666, "skipped": 0} +{"event": "step", "step": 3960, "lr": 6.014762051794997e-05, "train_loss": 4.907829284667969, "grad_norm": 0.7166607975959778, "tokens_seen": 32448512, "tokens_per_sec": 17143.277066845654, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4307047526041667, "G_std": 0.18060302734375, "skipped": 0} +{"event": "step", "step": 3980, "lr": 6.003690765203337e-05, "train_loss": 4.85357141494751, "grad_norm": 0.7629654407501221, "tokens_seen": 32612352, "tokens_per_sec": 17146.8889112771, "vram_gb": 6.28956416, "lambda": null, "S_mean": null, "G_mean": 0.4358113606770833, "G_std": 0.18165079752604166, "skipped": 0} +{"event": "eval", "step": 3999, "val_loss": 4.976549911499023, "val_ppl": 144.973346966276, "lambda": null, "S_mean": null, "G_mean": 0.4402058919270833, "G_std": 0.18248494466145834, "tokens_seen": 32768000} +{"event": "done", "step": 3999, "elapsed_s": 1914.1351833343506, "nan_failures": 0} diff --git a/runs_4ksteps/s_only_s0/config.json b/runs_4ksteps/s_only_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..81ccda67b077f8d1828da56bcde7b68f1eb66b72 --- /dev/null +++ b/runs_4ksteps/s_only_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s0", + "max_steps": 4000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 200, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/runs_4ksteps/s_only_s0/final.pt b/runs_4ksteps/s_only_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..40caa2a2e8953447c9e000bcbe81dec22144583d --- /dev/null +++ b/runs_4ksteps/s_only_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c9c337328b4f9303910f8f4745ebcd61baeab4952ccf0c2fa115e8c824e09af4 +size 255945415 diff --git a/runs_4ksteps/s_only_s0/metrics.jsonl b/runs_4ksteps/s_only_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..75294c7fc1581b91d6b6cd8aee210fa7cc042ec7 --- /dev/null +++ b/runs_4ksteps/s_only_s0/metrics.jsonl @@ -0,0 +1,222 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 0, "config": {"variant": "s_only", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s0", "max_steps": 4000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 200, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.919593811035156, "grad_norm": 16.242082595825195, "tokens_seen": 8192, "tokens_per_sec": 7865.5417897860725, "vram_gb": 5.63817984, "lambda": 0.12692801654338837, "S_mean": 0.3408559163411458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.820077896118164, "grad_norm": 1.886579155921936, "tokens_seen": 172032, "tokens_per_sec": 15859.249816516956, "vram_gb": 6.154519552, "lambda": 0.1269285697489977, "S_mean": 0.4690144856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.075931549072266, "grad_norm": 1.8170305490493774, "tokens_seen": 335872, "tokens_per_sec": 16243.599218249792, "vram_gb": 6.154519552, "lambda": 0.126861909404397, "S_mean": 0.5243937174479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.09058952331543, "grad_norm": 1.5147713422775269, "tokens_seen": 499712, "tokens_per_sec": 16370.040208719523, "vram_gb": 6.154519552, "lambda": 0.1267800386995077, "S_mean": 0.5086568196614584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.4280595779418945, "grad_norm": 0.735983669757843, "tokens_seen": 663552, "tokens_per_sec": 16457.922014806176, "vram_gb": 6.154519552, "lambda": 0.12668094101051489, "S_mean": 0.3618672688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.217446804046631, "grad_norm": 0.6781662106513977, "tokens_seen": 827392, "tokens_per_sec": 16511.308248659676, "vram_gb": 6.154519552, "lambda": 0.1265610239158074, "S_mean": 0.268402099609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.076810121536255, "grad_norm": 0.7957208752632141, "tokens_seen": 991232, "tokens_per_sec": 16546.323120754958, "vram_gb": 6.154519552, "lambda": 0.12644990781943002, "S_mean": 0.26434326171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.8256025314331055, "grad_norm": 0.5386276245117188, "tokens_seen": 1155072, "tokens_per_sec": 16557.811975261877, "vram_gb": 6.154519552, "lambda": 0.12640769282976785, "S_mean": 0.255157470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.909908294677734, "grad_norm": 0.7910153269767761, "tokens_seen": 1318912, "tokens_per_sec": 16571.74948575738, "vram_gb": 6.154519552, "lambda": 0.12637964511911073, "S_mean": 0.23397318522135416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.812398195266724, "grad_norm": 0.7713411450386047, "tokens_seen": 1482752, "tokens_per_sec": 16584.163953232546, "vram_gb": 6.154519552, "lambda": 0.12634567295511565, "S_mean": 0.2824910481770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.698545455932617, "grad_norm": 0.7676857113838196, "tokens_seen": 1646592, "tokens_per_sec": 16593.469988871384, "vram_gb": 6.154519552, "lambda": 0.12624020067354044, "S_mean": 0.3038736979166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 200, "val_loss": 6.750462174415588, "val_ppl": 854.4535778653467, "lambda": 0.1262423557539781, "S_mean": 0.2534993489583333, "G_mean": null, "G_std": null, "tokens_seen": 1646592} +{"event": "step", "step": 220, "lr": 0.0005999630923479665, "train_loss": 6.587653636932373, "grad_norm": 0.587562620639801, "tokens_seen": 1810432, "tokens_per_sec": 16050.705841033758, "vram_gb": 6.154519552, "lambda": 0.12625185896952948, "S_mean": 0.259429931640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.00059985237948205, "train_loss": 6.628092288970947, "grad_norm": 0.5433539748191833, "tokens_seen": 1974272, "tokens_per_sec": 16097.028614764478, "vram_gb": 6.154519552, "lambda": 0.1262657322610418, "S_mean": 0.3388468424479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005996678916700422, "train_loss": 6.6135475635528564, "grad_norm": 0.5810588002204895, "tokens_seen": 2138112, "tokens_per_sec": 16137.45820031691, "vram_gb": 6.154519552, "lambda": 0.12628101371228695, "S_mean": 0.2993570963541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.000599409679349069, "train_loss": 6.512814044952393, "grad_norm": 0.5428544282913208, "tokens_seen": 2301952, "tokens_per_sec": 16173.300787027143, "vram_gb": 6.154519552, "lambda": 0.12627421288440624, "S_mean": 0.25018310546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005990778131118008, "train_loss": 6.341583728790283, "grad_norm": 0.5384073257446289, "tokens_seen": 2465792, "tokens_per_sec": 16202.876633893942, "vram_gb": 6.154519552, "lambda": 0.12628612201660872, "S_mean": 0.2724507649739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.0005986723836871531, "train_loss": 6.406653165817261, "grad_norm": 0.4830692708492279, "tokens_seen": 2629632, "tokens_per_sec": 16230.851427445747, "vram_gb": 6.154519552, "lambda": 0.1263035830731193, "S_mean": 0.309417724609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005981935019154823, "train_loss": 6.398892879486084, "grad_norm": 0.5299336910247803, "tokens_seen": 2793472, "tokens_per_sec": 16252.082634517838, "vram_gb": 6.154519552, "lambda": 0.126261239250501, "S_mean": 0.2884623209635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005976412987182833, "train_loss": 6.249614238739014, "grad_norm": 0.5303296446800232, "tokens_seen": 2957312, "tokens_per_sec": 16270.18275919345, "vram_gb": 6.154519552, "lambda": 0.12632379370431104, "S_mean": 0.3086344401041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005970159250623965, "train_loss": 6.225820302963257, "grad_norm": 0.5481662750244141, "tokens_seen": 3121152, "tokens_per_sec": 16290.40251731415, "vram_gb": 6.154519552, "lambda": 0.12632035743445158, "S_mean": 0.3420308430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.000596317551918735, "train_loss": 6.267168045043945, "grad_norm": 0.5582903623580933, "tokens_seen": 3284992, "tokens_per_sec": 16305.99531467296, "vram_gb": 6.154519552, "lambda": 0.12638370475421348, "S_mean": 0.313507080078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 400, "val_loss": 6.398098909854889, "val_ppl": 600.701963090071, "lambda": 0.12639418554802737, "S_mean": 0.3267415364583333, "G_mean": null, "G_std": null, "tokens_seen": 3284992} +{"event": "step", "step": 420, "lr": 0.0005955463702155432, "train_loss": 6.389300584793091, "grad_norm": 0.7712118029594421, "tokens_seen": 3448832, "tokens_per_sec": 16042.322730649656, "vram_gb": 6.154519552, "lambda": 0.1264245609442393, "S_mean": 0.3840128580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005947025907861985, "train_loss": 6.3582603931427, "grad_norm": 0.660672128200531, "tokens_seen": 3612672, "tokens_per_sec": 16068.041899233778, "vram_gb": 6.154519552, "lambda": 0.12652567743013302, "S_mean": 0.3241373697916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005937864443115718, "train_loss": 6.295156002044678, "grad_norm": 0.5301492214202881, "tokens_seen": 3776512, "tokens_per_sec": 16090.627031669745, "vram_gb": 6.154519552, "lambda": 0.1265407477815946, "S_mean": 0.3528188069661458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005927981812569615, "train_loss": 6.386673212051392, "grad_norm": 0.5241534113883972, "tokens_seen": 3940352, "tokens_per_sec": 16112.140825064731, "vram_gb": 6.154519552, "lambda": 0.12667018050948778, "S_mean": 0.385223388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005917380718036191, "train_loss": 6.316304922103882, "grad_norm": 0.5405471324920654, "tokens_seen": 4104192, "tokens_per_sec": 16131.784580003117, "vram_gb": 6.154519552, "lambda": 0.1267355838790536, "S_mean": 0.3172607421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005906064057748841, "train_loss": 6.170685052871704, "grad_norm": 0.5597842931747437, "tokens_seen": 4268032, "tokens_per_sec": 16152.074008749561, "vram_gb": 6.154519552, "lambda": 0.12688766761372486, "S_mean": 0.3484700520833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.000589403492556949, "train_loss": 6.183544635772705, "grad_norm": 0.5253093838691711, "tokens_seen": 4431872, "tokens_per_sec": 16169.708293143098, "vram_gb": 6.154519552, "lambda": 0.12706252994636694, "S_mean": 0.363861083984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.0005881296610142765, "train_loss": 6.040666103363037, "grad_norm": 0.5582141280174255, "tokens_seen": 4595712, "tokens_per_sec": 16189.039251281292, "vram_gb": 6.154519552, "lambda": 0.12717174366116524, "S_mean": 0.354888916015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005867852593996914, "train_loss": 6.209366083145142, "grad_norm": 0.5902380347251892, "tokens_seen": 4759552, "tokens_per_sec": 16204.804666737704, "vram_gb": 6.154519552, "lambda": 0.12731845180193582, "S_mean": 0.3054148356119792, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005853706552591712, "train_loss": 5.960181474685669, "grad_norm": 0.5583432912826538, "tokens_seen": 4923392, "tokens_per_sec": 16219.654211841376, "vram_gb": 6.154519552, "lambda": 0.12746825131277242, "S_mean": 0.3393351236979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 600, "val_loss": 6.17175452709198, "val_ppl": 479.0258330138068, "lambda": 0.12747944239526987, "S_mean": 0.3070068359375, "G_mean": null, "G_std": null, "tokens_seen": 4923392} +{"event": "step", "step": 620, "lr": 0.0005838862353313635, "train_loss": 6.025485515594482, "grad_norm": 0.4982314705848694, "tokens_seen": 5087232, "tokens_per_sec": 16045.715471374515, "vram_gb": 6.154519552, "lambda": 0.1276232535019517, "S_mean": 0.3570963541666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005823324054418544, "train_loss": 6.123433828353882, "grad_norm": 0.6382611393928528, "tokens_seen": 5251072, "tokens_per_sec": 16063.763623844789, "vram_gb": 6.154519552, "lambda": 0.12782887276262045, "S_mean": 0.3178202311197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005807095903922209, "train_loss": 5.941244840621948, "grad_norm": 0.5853946208953857, "tokens_seen": 5414912, "tokens_per_sec": 16079.095133873507, "vram_gb": 6.154519552, "lambda": 0.12805543839931488, "S_mean": 0.3174336751302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005790182338438938, "train_loss": 5.950453519821167, "grad_norm": 0.5208508372306824, "tokens_seen": 5578752, "tokens_per_sec": 16096.192714073588, "vram_gb": 6.154519552, "lambda": 0.1282186039413015, "S_mean": 0.3467763264973958, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005772587981968655, "train_loss": 6.049641370773315, "grad_norm": 0.5777992010116577, "tokens_seen": 5742592, "tokens_per_sec": 16110.355441343252, "vram_gb": 6.154519552, "lambda": 0.12838227239747843, "S_mean": 0.3238627115885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005754317644632739, "train_loss": 6.018137216567993, "grad_norm": 0.5333870053291321, "tokens_seen": 5906432, "tokens_per_sec": 16124.776808460974, "vram_gb": 6.154519552, "lambda": 0.12860822770744562, "S_mean": 0.32501220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005735376321358989, "train_loss": 6.050051927566528, "grad_norm": 0.581946611404419, "tokens_seen": 6070272, "tokens_per_sec": 16138.193454242135, "vram_gb": 6.154519552, "lambda": 0.1289047651613752, "S_mean": 0.3107401529947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.000571576919051606, "train_loss": 5.914849281311035, "grad_norm": 0.5249657034873962, "tokens_seen": 6234112, "tokens_per_sec": 16150.697611573429, "vram_gb": 6.154519552, "lambda": 0.12920729536563158, "S_mean": 0.3397928873697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.0005695501612497752, "train_loss": 5.928347110748291, "grad_norm": 0.5219926834106445, "tokens_seen": 6397952, "tokens_per_sec": 16162.988864120756, "vram_gb": 6.154519552, "lambda": 0.12947439402341843, "S_mean": 0.326995849609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000567457912825752, "train_loss": 5.888119697570801, "grad_norm": 0.5409494638442993, "tokens_seen": 6561792, "tokens_per_sec": 16174.79101379646, "vram_gb": 6.154519552, "lambda": 0.12977179295072952, "S_mean": 0.34320068359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 800, "val_loss": 5.981451761722565, "val_ppl": 396.01487026455237, "lambda": 0.1297842881952723, "S_mean": 0.322174072265625, "G_mean": null, "G_std": null, "tokens_seen": 6561792} +{"event": "step", "step": 820, "lr": 0.0005653007457793648, "train_loss": 5.966612339019775, "grad_norm": 0.5255451798439026, "tokens_seen": 6725632, "tokens_per_sec": 16042.347995770688, "vram_gb": 6.154519552, "lambda": 0.13008342559138933, "S_mean": 0.3064982096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.000563079249858545, "train_loss": 6.137436866760254, "grad_norm": 0.5351489782333374, "tokens_seen": 6889472, "tokens_per_sec": 16055.734975299967, "vram_gb": 6.154519552, "lambda": 0.1303543041770657, "S_mean": 0.319122314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005607940323980955, "train_loss": 5.95181941986084, "grad_norm": 0.5563442707061768, "tokens_seen": 7053312, "tokens_per_sec": 16070.379828966734, "vram_gb": 6.154519552, "lambda": 0.13066580053418875, "S_mean": 0.3179117838541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005584457181536525, "train_loss": 5.932007074356079, "grad_norm": 0.6170052289962769, "tokens_seen": 7217152, "tokens_per_sec": 16083.01834969238, "vram_gb": 6.154519552, "lambda": 0.13096548213313022, "S_mean": 0.350921630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005560349491308827, "train_loss": 5.85567831993103, "grad_norm": 0.5164368152618408, "tokens_seen": 7380992, "tokens_per_sec": 16094.362881449653, "vram_gb": 6.154519552, "lambda": 0.1312484402830402, "S_mean": 0.3311258951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005535623844099654, "train_loss": 5.737001895904541, "grad_norm": 0.537044882774353, "tokens_seen": 7544832, "tokens_per_sec": 16104.099184438523, "vram_gb": 6.154519552, "lambda": 0.1315185483545065, "S_mean": 0.3565266927083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.000551028699965407, "train_loss": 5.798852920532227, "grad_norm": 0.5535750389099121, "tokens_seen": 7708672, "tokens_per_sec": 16114.942196454953, "vram_gb": 6.154519552, "lambda": 0.13187734379122654, "S_mean": 0.3029276529947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005484345884812357, "train_loss": 5.784021854400635, "grad_norm": 0.5765553712844849, "tokens_seen": 7872512, "tokens_per_sec": 16123.969805120412, "vram_gb": 6.154519552, "lambda": 0.13223367060224214, "S_mean": 0.3155008951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005457807591616296, "train_loss": 5.710877180099487, "grad_norm": 0.5739303231239319, "tokens_seen": 8036352, "tokens_per_sec": 16133.55427120676, "vram_gb": 6.154519552, "lambda": 0.13256636603424946, "S_mean": 0.3251851399739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.0005430679375370263, "train_loss": 5.804469108581543, "grad_norm": 0.5793725252151489, "tokens_seen": 8200192, "tokens_per_sec": 16142.566161764442, "vram_gb": 6.154519552, "lambda": 0.13293919060379267, "S_mean": 0.3327738444010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.830336010456085, "val_ppl": 340.47306236265075, "lambda": 0.1329573361823956, "S_mean": 0.328765869140625, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005402968652657703, "train_loss": 5.551600217819214, "grad_norm": 0.5231625437736511, "tokens_seen": 8364032, "tokens_per_sec": 16037.391004962372, "vram_gb": 6.154519552, "lambda": 0.13332742483665547, "S_mean": 0.3073832194010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005374682999313519, "train_loss": 5.803956031799316, "grad_norm": 0.4986833333969116, "tokens_seen": 8527872, "tokens_per_sec": 16049.289330836993, "vram_gb": 6.154519552, "lambda": 0.1336649345854918, "S_mean": 0.28564453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005345830148352904, "train_loss": 5.513277530670166, "grad_norm": 0.5439568161964417, "tokens_seen": 8691712, "tokens_per_sec": 16060.246194938089, "vram_gb": 6.154519552, "lambda": 0.13397249206900597, "S_mean": 0.3126424153645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005316417987857215, "train_loss": 5.726963520050049, "grad_norm": 0.6540611386299133, "tokens_seen": 8855552, "tokens_per_sec": 16071.179543576896, "vram_gb": 6.154519552, "lambda": 0.13436876454701027, "S_mean": 0.304962158203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005286454558817454, "train_loss": 5.677356958389282, "grad_norm": 0.5802360773086548, "tokens_seen": 9019392, "tokens_per_sec": 16082.257368098346, "vram_gb": 6.154519552, "lambda": 0.1346825072541833, "S_mean": 0.290679931640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.000525594805293594, "train_loss": 5.618464469909668, "grad_norm": 0.5714879035949707, "tokens_seen": 9183232, "tokens_per_sec": 16091.882986404307, "vram_gb": 6.154519552, "lambda": 0.13510261848568916, "S_mean": 0.3033345540364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005224906810386772, "train_loss": 5.867502212524414, "grad_norm": 0.5576197504997253, "tokens_seen": 9347072, "tokens_per_sec": 16100.165858210952, "vram_gb": 6.154519552, "lambda": 0.1354632107540965, "S_mean": 0.3012797037760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005193339317535718, "train_loss": 5.61969256401062, "grad_norm": 0.5443897247314453, "tokens_seen": 9510912, "tokens_per_sec": 16109.041413267476, "vram_gb": 6.154519552, "lambda": 0.1358909821137786, "S_mean": 0.3170267740885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005161254204620127, "train_loss": 5.676989555358887, "grad_norm": 0.5600603818893433, "tokens_seen": 9674752, "tokens_per_sec": 16117.013404196989, "vram_gb": 6.154519552, "lambda": 0.1362527391562859, "S_mean": 0.3006388346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005128660243389501, "train_loss": 5.560107231140137, "grad_norm": 0.515430212020874, "tokens_seen": 9838592, "tokens_per_sec": 16124.47349031634, "vram_gb": 6.154519552, "lambda": 0.13655376713722944, "S_mean": 0.29461669921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1200, "val_loss": 5.680997133255005, "val_ppl": 293.2416852256867, "lambda": 0.1365662201618155, "S_mean": 0.3016866048177083, "G_mean": null, "G_std": null, "tokens_seen": 9838592} +{"event": "step", "step": 1220, "lr": 0.000509556634470739, "train_loss": 5.5829901695251465, "grad_norm": 0.6020843386650085, "tokens_seen": 10002432, "tokens_per_sec": 16035.94167305411, "vram_gb": 6.154519552, "lambda": 0.13686958079536757, "S_mean": 0.3241170247395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005061981556115254, "train_loss": 5.4571216106414795, "grad_norm": 0.5335522294044495, "tokens_seen": 10166272, "tokens_per_sec": 16045.258981970808, "vram_gb": 6.154519552, "lambda": 0.13723452016711235, "S_mean": 0.2968037923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005027915059358954, "train_loss": 5.460075378417969, "grad_norm": 0.5649682283401489, "tokens_seen": 10330112, "tokens_per_sec": 16053.102112237919, "vram_gb": 6.154519552, "lambda": 0.1376482384900252, "S_mean": 0.2987467447916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0004993376167878549, "train_loss": 5.586447238922119, "grad_norm": 0.5518506169319153, "tokens_seen": 10493952, "tokens_per_sec": 16061.56659818779, "vram_gb": 6.154519552, "lambda": 0.13803729942689338, "S_mean": 0.2910563151041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0004958374324262102, "train_loss": 5.688790321350098, "grad_norm": 0.6089926958084106, "tokens_seen": 10657792, "tokens_per_sec": 16070.156039988378, "vram_gb": 6.154519552, "lambda": 0.1383867928137382, "S_mean": 0.303070068359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0004922919097664167, "train_loss": 5.639514446258545, "grad_norm": 0.5769636631011963, "tokens_seen": 10821632, "tokens_per_sec": 16078.540681419116, "vram_gb": 6.154519552, "lambda": 0.1387265526379148, "S_mean": 0.2890421549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0004887020181189677, "train_loss": 5.757676124572754, "grad_norm": 0.5670664310455322, "tokens_seen": 10985472, "tokens_per_sec": 16087.103797430686, "vram_gb": 6.154519552, "lambda": 0.13911849415550628, "S_mean": 0.2984415690104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.00048506873892439467, "train_loss": 5.5488996505737305, "grad_norm": 0.5843585729598999, "tokens_seen": 11149312, "tokens_per_sec": 16094.669119301996, "vram_gb": 6.154519552, "lambda": 0.13947931056221327, "S_mean": 0.2875569661458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0004813930654849516, "train_loss": 5.42434287071228, "grad_norm": 0.5659754276275635, "tokens_seen": 11313152, "tokens_per_sec": 16101.412435629876, "vram_gb": 6.154519552, "lambda": 0.13979739509522915, "S_mean": 0.3043314615885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0004776760026930552, "train_loss": 5.440374135971069, "grad_norm": 0.5832152962684631, "tokens_seen": 11476992, "tokens_per_sec": 16105.620599837912, "vram_gb": 6.154519552, "lambda": 0.14012548172225556, "S_mean": 0.3166300455729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1400, "val_loss": 5.570456850528717, "val_ppl": 262.5540197880673, "lambda": 0.14014238491654396, "S_mean": 0.3018290201822917, "G_mean": null, "G_std": null, "tokens_seen": 11476992} +{"event": "step", "step": 1420, "lr": 0.00047391856675655823, "train_loss": 5.424355983734131, "grad_norm": 0.6256735920906067, "tokens_seen": 11640832, "tokens_per_sec": 16030.028131048824, "vram_gb": 6.154519552, "lambda": 0.1404859513665239, "S_mean": 0.2938741048177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.00047012178492092805, "train_loss": 5.408387184143066, "grad_norm": 0.6174618005752563, "tokens_seen": 11804672, "tokens_per_sec": 16038.021959917616, "vram_gb": 6.154519552, "lambda": 0.140826303511858, "S_mean": 0.3070882161458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.00046628669518840776, "train_loss": 5.529391765594482, "grad_norm": 0.6300767064094543, "tokens_seen": 11968512, "tokens_per_sec": 16045.795600986015, "vram_gb": 6.154519552, "lambda": 0.14116969301054874, "S_mean": 0.2715810139973958, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.00046241434603423715, "train_loss": 5.581990718841553, "grad_norm": 0.5661997199058533, "tokens_seen": 12132352, "tokens_per_sec": 16052.94809271283, "vram_gb": 6.154519552, "lambda": 0.14155861859520277, "S_mean": 0.3088785807291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0004585057961200098, "train_loss": 5.477790355682373, "grad_norm": 0.6343227028846741, "tokens_seen": 12296192, "tokens_per_sec": 16060.667233792457, "vram_gb": 6.154519552, "lambda": 0.1419260473921895, "S_mean": 0.2834879557291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.00045456211400424565, "train_loss": 5.4671783447265625, "grad_norm": 0.5828719735145569, "tokens_seen": 12460032, "tokens_per_sec": 16067.286576348897, "vram_gb": 6.154519552, "lambda": 0.14222985816498598, "S_mean": 0.3004353841145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0004505843778502577, "train_loss": 5.371294975280762, "grad_norm": 0.6054646968841553, "tokens_seen": 12623872, "tokens_per_sec": 16074.259735870157, "vram_gb": 6.154519552, "lambda": 0.14253480080515146, "S_mean": 0.2915242513020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.000446573675131393, "train_loss": 5.37671160697937, "grad_norm": 0.5748475193977356, "tokens_seen": 12787712, "tokens_per_sec": 16081.425421964303, "vram_gb": 6.154519552, "lambda": 0.14288722444325686, "S_mean": 0.300445556640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0004425311023337283, "train_loss": 5.4438793659210205, "grad_norm": 0.6472764015197754, "tokens_seen": 12951552, "tokens_per_sec": 16087.680793437283, "vram_gb": 6.154519552, "lambda": 0.14320783875882626, "S_mean": 0.2858174641927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0004384577646563017, "train_loss": 5.429971218109131, "grad_norm": 0.5652442574501038, "tokens_seen": 13115392, "tokens_per_sec": 16092.657575992002, "vram_gb": 6.154519552, "lambda": 0.14355705057581267, "S_mean": 0.3065897623697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1600, "val_loss": 5.494125485420227, "val_ppl": 243.25869983470895, "lambda": 0.14357468920449415, "S_mean": 0.3021647135416667, "G_mean": null, "G_std": null, "tokens_seen": 13115392} +{"event": "step", "step": 1620, "lr": 0.00043435477570896204, "train_loss": 5.411472797393799, "grad_norm": 0.6373422145843506, "tokens_seen": 13279232, "tokens_per_sec": 16027.33249226929, "vram_gb": 6.154519552, "lambda": 0.14388109495242438, "S_mean": 0.2859293619791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0004302232572079195, "train_loss": 5.288194894790649, "grad_norm": 0.5879696607589722, "tokens_seen": 13443072, "tokens_per_sec": 16033.845499326488, "vram_gb": 6.154519552, "lambda": 0.1442484213039279, "S_mean": 0.290924072265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0004260643386690794, "train_loss": 5.457664966583252, "grad_norm": 0.5912243723869324, "tokens_seen": 13606912, "tokens_per_sec": 16040.722328717804, "vram_gb": 6.154519552, "lambda": 0.14454306599994501, "S_mean": 0.2941487630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.00042187915709924356, "train_loss": 5.604820489883423, "grad_norm": 0.5878295302391052, "tokens_seen": 13770752, "tokens_per_sec": 16046.751170840358, "vram_gb": 6.154519552, "lambda": 0.14485854376107454, "S_mean": 0.2823893229166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0004176688566852645, "train_loss": 5.470637321472168, "grad_norm": 0.5773929357528687, "tokens_seen": 13934592, "tokens_per_sec": 16052.964804939964, "vram_gb": 6.154519552, "lambda": 0.14520258642733097, "S_mean": 0.2703653971354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.00041343458848123576, "train_loss": 5.2282328605651855, "grad_norm": 0.5962106585502625, "tokens_seen": 14098432, "tokens_per_sec": 16059.086335589383, "vram_gb": 6.154519552, "lambda": 0.14549836268027624, "S_mean": 0.28631591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.00040917751009380545, "train_loss": 5.2963995933532715, "grad_norm": 0.5963074564933777, "tokens_seen": 14262272, "tokens_per_sec": 16065.179726484801, "vram_gb": 6.154519552, "lambda": 0.1458452930673957, "S_mean": 0.3032328287760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.000404898785365698, "train_loss": 5.362212896347046, "grad_norm": 0.6068029999732971, "tokens_seen": 14426112, "tokens_per_sec": 16071.551334698335, "vram_gb": 6.154519552, "lambda": 0.14619173482060432, "S_mean": 0.2948811848958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.00040059958405753125, "train_loss": 5.238745927810669, "grad_norm": 0.6292857527732849, "tokens_seen": 14589952, "tokens_per_sec": 16077.291599817156, "vram_gb": 6.154519552, "lambda": 0.14650953530023494, "S_mean": 0.2731221516927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.00039628108152801574, "train_loss": 5.297290802001953, "grad_norm": 0.6111298203468323, "tokens_seen": 14753792, "tokens_per_sec": 16082.783366820913, "vram_gb": 6.154519552, "lambda": 0.14679855667054653, "S_mean": 0.2750345865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1800, "val_loss": 5.423682105541229, "val_ppl": 226.71236637516768, "lambda": 0.14680707703034082, "S_mean": 0.3173014322916667, "G_mean": null, "G_std": null, "tokens_seen": 14753792} +{"event": "step", "step": 1820, "lr": 0.0003919444584126229, "train_loss": 5.383754253387451, "grad_norm": 0.556212306022644, "tokens_seen": 14917632, "tokens_per_sec": 16024.61814475664, "vram_gb": 6.154519552, "lambda": 0.14706969012816748, "S_mean": 0.2752176920572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0003875909003008116, "train_loss": 5.233007431030273, "grad_norm": 0.6342540383338928, "tokens_seen": 15081472, "tokens_per_sec": 16030.024998626814, "vram_gb": 6.154519552, "lambda": 0.1473601134493947, "S_mean": 0.28533935546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.00038322159741189957, "train_loss": 5.274621486663818, "grad_norm": 0.5902026891708374, "tokens_seen": 15245312, "tokens_per_sec": 16036.045768507674, "vram_gb": 6.154519552, "lambda": 0.14768042291204134, "S_mean": 0.3062642415364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.00037883774426966964, "train_loss": 5.368350267410278, "grad_norm": 0.6515896320343018, "tokens_seen": 15409152, "tokens_per_sec": 16041.744692361652, "vram_gb": 6.154519552, "lambda": 0.14795536641031504, "S_mean": 0.309814453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0003744405393757981, "train_loss": 5.338687181472778, "grad_norm": 0.6551464200019836, "tokens_seen": 15572992, "tokens_per_sec": 16046.988526521322, "vram_gb": 6.154519552, "lambda": 0.14826269975552955, "S_mean": 0.2742513020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0003700311848821973, "train_loss": 5.272947311401367, "grad_norm": 0.5953201055526733, "tokens_seen": 15736832, "tokens_per_sec": 16052.592016546463, "vram_gb": 6.154519552, "lambda": 0.1485254509995381, "S_mean": 0.2857767740885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0003656108862623586, "train_loss": 5.337001323699951, "grad_norm": 0.6209555864334106, "tokens_seen": 15900672, "tokens_per_sec": 16057.688211931858, "vram_gb": 6.154519552, "lambda": 0.1487580065925916, "S_mean": 0.27789306640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.00036118085198178824, "train_loss": 5.522840738296509, "grad_norm": 0.6615057587623596, "tokens_seen": 16064512, "tokens_per_sec": 16063.203277008859, "vram_gb": 6.154519552, "lambda": 0.1491000996902585, "S_mean": 0.2825724283854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.00035674229316762495, "train_loss": 5.432572841644287, "grad_norm": 0.6224022507667542, "tokens_seen": 16228352, "tokens_per_sec": 16068.339435098931, "vram_gb": 6.154519552, "lambda": 0.1493905217697223, "S_mean": 0.295318603515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0003522964232775297, "train_loss": 5.266852140426636, "grad_norm": 0.5988137722015381, "tokens_seen": 16392192, "tokens_per_sec": 16073.772916303724, "vram_gb": 6.154519552, "lambda": 0.14968158894528946, "S_mean": 0.2735799153645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.378534960746765, "val_ppl": 216.7045620607252, "lambda": 0.14969413044552007, "S_mean": 0.2893168131510417, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0003478444577679388, "train_loss": 5.226250648498535, "grad_norm": 0.5943301320075989, "tokens_seen": 16556032, "tokens_per_sec": 16021.675715514584, "vram_gb": 6.154519552, "lambda": 0.14995385675380626, "S_mean": 0.2728373209635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.00034338761376176993, "train_loss": 5.3614091873168945, "grad_norm": 0.6712439656257629, "tokens_seen": 16719872, "tokens_per_sec": 16027.422222450126, "vram_gb": 6.154519552, "lambda": 0.15023998698840538, "S_mean": 0.3235066731770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0003389271097156731, "train_loss": 5.118818998336792, "grad_norm": 0.6669472455978394, "tokens_seen": 16883712, "tokens_per_sec": 16033.242223855521, "vram_gb": 6.154519552, "lambda": 0.15052441755930582, "S_mean": 0.30572509765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.00033446416508691677, "train_loss": 5.263031005859375, "grad_norm": 0.6320888996124268, "tokens_seen": 17047552, "tokens_per_sec": 16037.840535238445, "vram_gb": 6.154519552, "lambda": 0.15079285173366466, "S_mean": 0.2885640462239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.00032999999999999994, "train_loss": 5.261996030807495, "grad_norm": 0.5956965684890747, "tokens_seen": 17211392, "tokens_per_sec": 16043.025151541327, "vram_gb": 6.154519552, "lambda": 0.15108702021340528, "S_mean": 0.2974650065104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0003255358349130832, "train_loss": 5.177993059158325, "grad_norm": 0.6050276756286621, "tokens_seen": 17375232, "tokens_per_sec": 16047.185202557057, "vram_gb": 6.154519552, "lambda": 0.15135678462684155, "S_mean": 0.2878316243489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0003210728902843267, "train_loss": 5.099453449249268, "grad_norm": 0.62972491979599, "tokens_seen": 17539072, "tokens_per_sec": 16052.356947784478, "vram_gb": 6.154519552, "lambda": 0.15164329887678227, "S_mean": 0.2798055013020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0003166123862382299, "train_loss": 5.150311470031738, "grad_norm": 0.6198303699493408, "tokens_seen": 17702912, "tokens_per_sec": 16057.06927842326, "vram_gb": 6.154519552, "lambda": 0.15192982678612074, "S_mean": 0.2804667154947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0003121555422320611, "train_loss": 5.189667701721191, "grad_norm": 0.684391975402832, "tokens_seen": 17866752, "tokens_per_sec": 16061.393469165443, "vram_gb": 6.154519552, "lambda": 0.152193579201897, "S_mean": 0.2868855794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.00030770357672247023, "train_loss": 5.33546781539917, "grad_norm": 0.6262090802192688, "tokens_seen": 18030592, "tokens_per_sec": 16065.920836287709, "vram_gb": 6.154519552, "lambda": 0.15246522768090168, "S_mean": 0.2790425618489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2200, "val_loss": 5.292816865444183, "val_ppl": 198.90291979073925, "lambda": 0.15247871074825525, "S_mean": 0.29547119140625, "G_mean": null, "G_std": null, "tokens_seen": 18030592} +{"event": "step", "step": 2220, "lr": 0.00030325770683237505, "train_loss": 5.225836992263794, "grad_norm": 0.652269184589386, "tokens_seen": 18194432, "tokens_per_sec": 16017.91458337693, "vram_gb": 6.154519552, "lambda": 0.1527368761599064, "S_mean": 0.2896728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0002988191480182117, "train_loss": 5.154954671859741, "grad_norm": 0.6020411252975464, "tokens_seen": 18358272, "tokens_per_sec": 16022.672442103745, "vram_gb": 6.154519552, "lambda": 0.15297844167798758, "S_mean": 0.286712646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0002943891137376413, "train_loss": 5.166842699050903, "grad_norm": 0.5930964946746826, "tokens_seen": 18522112, "tokens_per_sec": 16027.798287833817, "vram_gb": 6.154519552, "lambda": 0.15325090506424507, "S_mean": 0.2750956217447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.0002899688151178026, "train_loss": 5.370299577713013, "grad_norm": 0.7324725985527039, "tokens_seen": 18685952, "tokens_per_sec": 16032.312472267946, "vram_gb": 6.154519552, "lambda": 0.15346690950294337, "S_mean": 0.3030192057291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.00028555946062420177, "train_loss": 5.134362459182739, "grad_norm": 0.6828411817550659, "tokens_seen": 18849792, "tokens_per_sec": 16037.014693888394, "vram_gb": 6.154519552, "lambda": 0.15374540848036608, "S_mean": 0.264190673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.00028116225573033036, "train_loss": 5.255957126617432, "grad_norm": 0.6783632636070251, "tokens_seen": 19013632, "tokens_per_sec": 16041.336352365219, "vram_gb": 6.154519552, "lambda": 0.15396491096665463, "S_mean": 0.3024088541666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0002767784025881003, "train_loss": 5.1864259243011475, "grad_norm": 0.66922926902771, "tokens_seen": 19177472, "tokens_per_sec": 16045.916413453759, "vram_gb": 6.154519552, "lambda": 0.1541658298422893, "S_mean": 0.2977091471354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.00027240909969918834, "train_loss": 5.270910978317261, "grad_norm": 0.6359841823577881, "tokens_seen": 19341312, "tokens_per_sec": 16049.953853485622, "vram_gb": 6.154519552, "lambda": 0.15444632650663456, "S_mean": 0.29638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.00026805554158737706, "train_loss": 5.143517017364502, "grad_norm": 0.7062222957611084, "tokens_seen": 19505152, "tokens_per_sec": 16054.499069814101, "vram_gb": 6.154519552, "lambda": 0.15467509596298137, "S_mean": 0.2908426920572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.0002637189184719842, "train_loss": 5.0452258586883545, "grad_norm": 0.6624132394790649, "tokens_seen": 19668992, "tokens_per_sec": 16058.529807215598, "vram_gb": 6.154519552, "lambda": 0.1549065876752138, "S_mean": 0.2614542643229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2400, "val_loss": 5.22437846660614, "val_ppl": 185.74568749384835, "lambda": 0.15491734258830547, "S_mean": 0.2842203776041667, "G_mean": null, "G_std": null, "tokens_seen": 19668992} +{"event": "step", "step": 2420, "lr": 0.00025940041594246864, "train_loss": 5.037837982177734, "grad_norm": 0.6501566767692566, "tokens_seen": 19832832, "tokens_per_sec": 16014.739362077356, "vram_gb": 6.154519552, "lambda": 0.1551545119533936, "S_mean": 0.273223876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.000255101214634302, "train_loss": 5.160506963729858, "grad_norm": 0.6669596433639526, "tokens_seen": 19996672, "tokens_per_sec": 16019.593290402692, "vram_gb": 6.154519552, "lambda": 0.15540442988276482, "S_mean": 0.2832132975260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.00025082248990619454, "train_loss": 5.233983516693115, "grad_norm": 0.6696785688400269, "tokens_seen": 20160512, "tokens_per_sec": 16024.23740761764, "vram_gb": 6.154519552, "lambda": 0.15560725517570972, "S_mean": 0.283477783203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.0002465654115187642, "train_loss": 5.135564804077148, "grad_norm": 0.6338163018226624, "tokens_seen": 20324352, "tokens_per_sec": 16028.682630317711, "vram_gb": 6.154519552, "lambda": 0.15582442035277685, "S_mean": 0.2818806966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.00024233114331473544, "train_loss": 4.974042177200317, "grad_norm": 0.6510593891143799, "tokens_seen": 20488192, "tokens_per_sec": 16033.468430890067, "vram_gb": 6.154519552, "lambda": 0.15603429606805244, "S_mean": 0.2716776529947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.00023812084290075633, "train_loss": 5.031066179275513, "grad_norm": 0.6261394619941711, "tokens_seen": 20652032, "tokens_per_sec": 16038.223712260786, "vram_gb": 6.154519552, "lambda": 0.15623840938011804, "S_mean": 0.2757059733072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.00023393566133092048, "train_loss": 5.113039016723633, "grad_norm": 0.6411056518554688, "tokens_seen": 20815872, "tokens_per_sec": 16042.149222202035, "vram_gb": 6.154519552, "lambda": 0.15647442545741796, "S_mean": 0.273101806640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.00022977674279208032, "train_loss": 4.919046401977539, "grad_norm": 0.708196759223938, "tokens_seen": 20979712, "tokens_per_sec": 16046.576755726002, "vram_gb": 6.154519552, "lambda": 0.15665741016467413, "S_mean": 0.3010457356770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.00022564522429103793, "train_loss": 5.194380760192871, "grad_norm": 0.7063701748847961, "tokens_seen": 21143552, "tokens_per_sec": 16050.570618124302, "vram_gb": 6.154519552, "lambda": 0.15685578901320696, "S_mean": 0.2570343017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00022154223534369825, "train_loss": 4.977371454238892, "grad_norm": 0.6532168984413147, "tokens_seen": 21307392, "tokens_per_sec": 16054.76981851435, "vram_gb": 6.154519552, "lambda": 0.1570213104908665, "S_mean": 0.2830912272135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2600, "val_loss": 5.187380135059357, "val_ppl": 178.99898493300782, "lambda": 0.15702975541353226, "S_mean": 0.2924601236979167, "G_mean": null, "G_std": null, "tokens_seen": 21307392} +{"event": "step", "step": 2620, "lr": 0.00021746889766627162, "train_loss": 5.154656648635864, "grad_norm": 0.7170519232749939, "tokens_seen": 21471232, "tokens_per_sec": 16014.201442639707, "vram_gb": 6.154519552, "lambda": 0.15720977572103342, "S_mean": 0.2804972330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0002134263248686069, "train_loss": 5.146814584732056, "grad_norm": 0.7693564295768738, "tokens_seen": 21635072, "tokens_per_sec": 16018.19308741698, "vram_gb": 6.154519552, "lambda": 0.1574209677055478, "S_mean": 0.2940673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.00020941562214974222, "train_loss": 5.01505708694458, "grad_norm": 0.6471207141876221, "tokens_seen": 21798912, "tokens_per_sec": 16022.701698157383, "vram_gb": 6.154519552, "lambda": 0.15761489141732454, "S_mean": 0.2889302571614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0002054378859957543, "train_loss": 4.851718902587891, "grad_norm": 0.8571721911430359, "tokens_seen": 21962752, "tokens_per_sec": 16026.676324508528, "vram_gb": 6.154519552, "lambda": 0.15779203393807015, "S_mean": 0.2804361979166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0002014942038799901, "train_loss": 5.102889537811279, "grad_norm": 0.6810380220413208, "tokens_seen": 22126592, "tokens_per_sec": 16030.624167156144, "vram_gb": 6.154519552, "lambda": 0.1579585187137127, "S_mean": 0.2747802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.00019758565396576285, "train_loss": 5.090231418609619, "grad_norm": 0.6802136301994324, "tokens_seen": 22290432, "tokens_per_sec": 16034.629854828228, "vram_gb": 6.154519552, "lambda": 0.15815662002811828, "S_mean": 0.2774658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.00019371330481159223, "train_loss": 5.085364818572998, "grad_norm": 0.6522641777992249, "tokens_seen": 22454272, "tokens_per_sec": 16038.810675748848, "vram_gb": 6.154519552, "lambda": 0.15834523302813372, "S_mean": 0.2763773600260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.00018987821507907194, "train_loss": 4.931299686431885, "grad_norm": 0.6634652018547058, "tokens_seen": 22618112, "tokens_per_sec": 16042.715203478387, "vram_gb": 6.154519552, "lambda": 0.15851814641306797, "S_mean": 0.268798828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.00018608143324344171, "train_loss": 5.064481258392334, "grad_norm": 0.643841564655304, "tokens_seen": 22781952, "tokens_per_sec": 16046.255407811295, "vram_gb": 6.154519552, "lambda": 0.15868712433924279, "S_mean": 0.265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.00018232399730694476, "train_loss": 5.011410713195801, "grad_norm": 0.7190777063369751, "tokens_seen": 22945792, "tokens_per_sec": 16049.984086089122, "vram_gb": 6.154519552, "lambda": 0.15886648620168367, "S_mean": 0.270965576171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2800, "val_loss": 5.1254523396492, "val_ppl": 168.25023069183226, "lambda": 0.15887460112571716, "S_mean": 0.2751668294270833, "G_mean": null, "G_std": null, "tokens_seen": 22945792} +{"event": "step", "step": 2820, "lr": 0.00017860693451504837, "train_loss": 5.053393363952637, "grad_norm": 0.6543769836425781, "tokens_seen": 23109632, "tokens_per_sec": 16012.57496128552, "vram_gb": 6.154519552, "lambda": 0.1590363153566917, "S_mean": 0.2711588541666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.00017493126107560522, "train_loss": 4.850782871246338, "grad_norm": 0.6891620755195618, "tokens_seen": 23273472, "tokens_per_sec": 16016.615606800779, "vram_gb": 6.154519552, "lambda": 0.1591800656169653, "S_mean": 0.2590535481770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.00017129798188103226, "train_loss": 5.036581039428711, "grad_norm": 0.6424465775489807, "tokens_seen": 23437312, "tokens_per_sec": 16020.469850008896, "vram_gb": 6.154519552, "lambda": 0.15933304311086735, "S_mean": 0.2737019856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.00016770809023358322, "train_loss": 5.025433301925659, "grad_norm": 0.6619417071342468, "tokens_seen": 23601152, "tokens_per_sec": 16024.181835125537, "vram_gb": 6.154519552, "lambda": 0.15949062475313744, "S_mean": 0.272857666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.00016416256757378968, "train_loss": 4.873651027679443, "grad_norm": 0.7159934639930725, "tokens_seen": 23764992, "tokens_per_sec": 16028.243341999245, "vram_gb": 6.154519552, "lambda": 0.15964605690290531, "S_mean": 0.279083251953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.000160662383212145, "train_loss": 5.076171398162842, "grad_norm": 0.7399501800537109, "tokens_seen": 23928832, "tokens_per_sec": 16032.20812630705, "vram_gb": 6.154519552, "lambda": 0.1597970174625516, "S_mean": 0.2778422037760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.00015720849406410448, "train_loss": 4.990732192993164, "grad_norm": 0.6782010197639465, "tokens_seen": 24092672, "tokens_per_sec": 16036.200265880581, "vram_gb": 6.154519552, "lambda": 0.15993541137625775, "S_mean": 0.2803446451822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.00015380184438847444, "train_loss": 4.978413820266724, "grad_norm": 0.6770049929618835, "tokens_seen": 24256512, "tokens_per_sec": 16039.990162297287, "vram_gb": 6.154519552, "lambda": 0.16006658226251602, "S_mean": 0.2781473795572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.00015044336552926094, "train_loss": 4.939317226409912, "grad_norm": 0.6427668333053589, "tokens_seen": 24420352, "tokens_per_sec": 16043.926477239324, "vram_gb": 6.154519552, "lambda": 0.16020564020921788, "S_mean": 0.25946044921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.00014713397566104995, "train_loss": 4.894949913024902, "grad_norm": 0.6939587593078613, "tokens_seen": 24584192, "tokens_per_sec": 16047.724178484363, "vram_gb": 6.154519552, "lambda": 0.1603458309546113, "S_mean": 0.2691853841145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.143673479557037, "val_ppl": 171.34404247592894, "lambda": 0.1603525262326002, "S_mean": 0.3034566243489583, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.00014387457953798726, "train_loss": 4.944443225860596, "grad_norm": 0.6768608689308167, "tokens_seen": 24748032, "tokens_per_sec": 16012.633259652072, "vram_gb": 6.154519552, "lambda": 0.1604727047185103, "S_mean": 0.280181884765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.00014066606824642808, "train_loss": 5.031642198562622, "grad_norm": 0.6888409852981567, "tokens_seen": 24911872, "tokens_per_sec": 16016.268916219255, "vram_gb": 6.154519552, "lambda": 0.16060379706323147, "S_mean": 0.272430419921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.00013750931896132278, "train_loss": 4.987033367156982, "grad_norm": 0.7447900176048279, "tokens_seen": 25075712, "tokens_per_sec": 16019.581435551898, "vram_gb": 6.154519552, "lambda": 0.1607404612004757, "S_mean": 0.24842325846354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.00013440519470640596, "train_loss": 4.82954740524292, "grad_norm": 0.661358118057251, "tokens_seen": 25239552, "tokens_per_sec": 16023.278565222354, "vram_gb": 6.154519552, "lambda": 0.16087450087070465, "S_mean": 0.2677205403645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.00013135454411825447, "train_loss": 5.0510969161987305, "grad_norm": 0.6898619532585144, "tokens_seen": 25403392, "tokens_per_sec": 16026.84784266673, "vram_gb": 6.154519552, "lambda": 0.16098162159323692, "S_mean": 0.276123046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.00012835820121427852, "train_loss": 5.093709945678711, "grad_norm": 1.6149394512176514, "tokens_seen": 25567232, "tokens_per_sec": 16030.119631754613, "vram_gb": 6.154519552, "lambda": 0.1611141012981534, "S_mean": 0.2979736328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0001254169851647097, "train_loss": 4.912811756134033, "grad_norm": 0.6933885216712952, "tokens_seen": 25731072, "tokens_per_sec": 16033.766082109056, "vram_gb": 6.154519552, "lambda": 0.16125171103825173, "S_mean": 0.269195556640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.00012253170006864812, "train_loss": 5.075261116027832, "grad_norm": 0.6975211501121521, "tokens_seen": 25894912, "tokens_per_sec": 16037.246908888268, "vram_gb": 6.154519552, "lambda": 0.16136677749454975, "S_mean": 0.2743631998697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.00011970313473422966, "train_loss": 4.898503303527832, "grad_norm": 0.650119960308075, "tokens_seen": 26058752, "tokens_per_sec": 16040.834144645005, "vram_gb": 6.154519552, "lambda": 0.16147833845267692, "S_mean": 0.259521484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.00011693206246297376, "train_loss": 4.937174081802368, "grad_norm": 0.7332515716552734, "tokens_seen": 26222592, "tokens_per_sec": 16044.419303486704, "vram_gb": 6.154519552, "lambda": 0.16159397643059492, "S_mean": 0.2662353515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3200, "val_loss": 5.050759446620941, "val_ppl": 156.14100022492596, "lambda": 0.16159946161011854, "S_mean": 0.2737019856770833, "G_mean": null, "G_std": null, "tokens_seen": 26222592} +{"event": "step", "step": 3220, "lr": 0.00011421924083837032, "train_loss": 5.042072534561157, "grad_norm": 0.7670690417289734, "tokens_seen": 26386432, "tokens_per_sec": 16011.799330744865, "vram_gb": 6.154519552, "lambda": 0.16170350648462772, "S_mean": 0.2578837076822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.00011156541151876421, "train_loss": 5.055638074874878, "grad_norm": 0.6817185282707214, "tokens_seen": 26550272, "tokens_per_sec": 16014.95292293126, "vram_gb": 6.154519552, "lambda": 0.16180988680571318, "S_mean": 0.2802937825520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.00010897130003459306, "train_loss": 4.804340839385986, "grad_norm": 0.6882858276367188, "tokens_seen": 26714112, "tokens_per_sec": 16018.410275472143, "vram_gb": 6.154519552, "lambda": 0.16191981422404447, "S_mean": 0.2765401204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.00010643761559003458, "train_loss": 5.073801040649414, "grad_norm": 0.7749873995780945, "tokens_seen": 26877952, "tokens_per_sec": 16021.67475805285, "vram_gb": 6.154519552, "lambda": 0.1620280851299564, "S_mean": 0.2607421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0001039650508691173, "train_loss": 4.886696100234985, "grad_norm": 0.7296509742736816, "tokens_seen": 27041792, "tokens_per_sec": 16024.933047321769, "vram_gb": 6.154519552, "lambda": 0.16213232837617397, "S_mean": 0.2599589029947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0001015542818463474, "train_loss": 5.038439989089966, "grad_norm": 0.7277916073799133, "tokens_seen": 27205632, "tokens_per_sec": 16028.162158205443, "vram_gb": 6.154519552, "lambda": 0.16222489128510156, "S_mean": 0.2616678873697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 9.920596760190444e-05, "train_loss": 4.88080620765686, "grad_norm": 0.6752232313156128, "tokens_seen": 27369472, "tokens_per_sec": 16031.44502435361, "vram_gb": 6.154519552, "lambda": 0.16232431990404925, "S_mean": 0.2654520670572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 9.692075014145504e-05, "train_loss": 5.032386779785156, "grad_norm": 0.688566267490387, "tokens_seen": 27533312, "tokens_per_sec": 16034.341740093232, "vram_gb": 6.154519552, "lambda": 0.162418513558805, "S_mean": 0.2650858561197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 9.46992542206351e-05, "train_loss": 5.0187578201293945, "grad_norm": 0.7294942140579224, "tokens_seen": 27697152, "tokens_per_sec": 16037.641330700344, "vram_gb": 6.154519552, "lambda": 0.16249898045013347, "S_mean": 0.2537129720052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 9.254208717424797e-05, "train_loss": 4.890284538269043, "grad_norm": 0.6810927987098694, "tokens_seen": 27860992, "tokens_per_sec": 16040.96042544972, "vram_gb": 6.154519552, "lambda": 0.1626018676906824, "S_mean": 0.26788330078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3400, "val_loss": 5.054001724720001, "val_ppl": 156.64807436379562, "lambda": 0.1626071206604441, "S_mean": 0.2689005533854167, "G_mean": null, "G_std": null, "tokens_seen": 27860992} +{"event": "step", "step": 3420, "lr": 9.04498387502248e-05, "train_loss": 4.9337780475616455, "grad_norm": 0.7144514322280884, "tokens_seen": 28024832, "tokens_per_sec": 16010.170509375082, "vram_gb": 6.154519552, "lambda": 0.1626981602360805, "S_mean": 0.28369140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 8.842308094839387e-05, "train_loss": 4.973560333251953, "grad_norm": 0.7910426259040833, "tokens_seen": 28188672, "tokens_per_sec": 16013.556230442315, "vram_gb": 6.154519552, "lambda": 0.16279214837898812, "S_mean": 0.2785237630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 8.646236786410109e-05, "train_loss": 4.86721658706665, "grad_norm": 0.7446473836898804, "tokens_seen": 28352512, "tokens_per_sec": 16016.984128507394, "vram_gb": 6.154519552, "lambda": 0.1628744974732399, "S_mean": 0.2619730631510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 8.456823553672612e-05, "train_loss": 4.957941770553589, "grad_norm": 0.7413833141326904, "tokens_seen": 28516352, "tokens_per_sec": 16020.270392383683, "vram_gb": 6.154519552, "lambda": 0.16295635203520456, "S_mean": 0.2657063802083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 8.274120180313448e-05, "train_loss": 4.861666679382324, "grad_norm": 0.7100507020950317, "tokens_seen": 28680192, "tokens_per_sec": 16023.48195887356, "vram_gb": 6.154519552, "lambda": 0.16303630731999874, "S_mean": 0.263092041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 8.098176615610612e-05, "train_loss": 4.777354955673218, "grad_norm": 0.7366724610328674, "tokens_seen": 28844032, "tokens_per_sec": 16026.554050503117, "vram_gb": 6.154519552, "lambda": 0.16311296044538418, "S_mean": 0.28143310546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 7.929040960777911e-05, "train_loss": 4.758236885070801, "grad_norm": 0.6964825391769409, "tokens_seen": 29007872, "tokens_per_sec": 16029.579041215102, "vram_gb": 6.154519552, "lambda": 0.16320041629175344, "S_mean": 0.272369384765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 7.76675945581456e-05, "train_loss": 5.001392602920532, "grad_norm": 0.6975379586219788, "tokens_seen": 29171712, "tokens_per_sec": 16032.625342967183, "vram_gb": 6.154519552, "lambda": 0.16328275483101606, "S_mean": 0.254241943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 7.611376466863648e-05, "train_loss": 4.839605331420898, "grad_norm": 0.6866869330406189, "tokens_seen": 29335552, "tokens_per_sec": 16035.670873192075, "vram_gb": 6.154519552, "lambda": 0.163363772444427, "S_mean": 0.2598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 7.462934474082865e-05, "train_loss": 4.86929726600647, "grad_norm": 0.8042006492614746, "tokens_seen": 29499392, "tokens_per_sec": 16038.564711987692, "vram_gb": 6.154519552, "lambda": 0.16343969789644083, "S_mean": 0.2730000813802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3600, "val_loss": 5.021624600887298, "val_ppl": 151.65748665284252, "lambda": 0.16344331422199807, "S_mean": 0.26153564453125, "G_mean": null, "G_std": null, "tokens_seen": 29499392} +{"event": "step", "step": 3620, "lr": 7.321474060030854e-05, "train_loss": 4.918807506561279, "grad_norm": 0.7458661794662476, "tokens_seen": 29663232, "tokens_per_sec": 16009.183614741318, "vram_gb": 6.154519552, "lambda": 0.16351322084665298, "S_mean": 0.261199951171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 7.187033898572343e-05, "train_loss": 4.990954399108887, "grad_norm": 0.7242737412452698, "tokens_seen": 29827072, "tokens_per_sec": 16012.16167095624, "vram_gb": 6.154519552, "lambda": 0.16358474300553402, "S_mean": 0.2503255208333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 7.0596507443051e-05, "train_loss": 5.03786039352417, "grad_norm": 0.8026186227798462, "tokens_seen": 29990912, "tokens_per_sec": 16015.537641967268, "vram_gb": 6.154519552, "lambda": 0.16365407779812813, "S_mean": 0.2654215494791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 6.93935942251159e-05, "train_loss": 4.952725887298584, "grad_norm": 0.7051458954811096, "tokens_seen": 30154752, "tokens_per_sec": 16018.500986554633, "vram_gb": 6.154519552, "lambda": 0.16372491512447596, "S_mean": 0.2631123860677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 6.82619281963808e-05, "train_loss": 5.013772964477539, "grad_norm": 0.6938650012016296, "tokens_seen": 30318592, "tokens_per_sec": 16021.403997124551, "vram_gb": 6.154519552, "lambda": 0.16379747508714596, "S_mean": 0.265899658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 6.720181874303845e-05, "train_loss": 4.77271032333374, "grad_norm": 0.7032396793365479, "tokens_seen": 30482432, "tokens_per_sec": 16024.326038023943, "vram_gb": 6.154519552, "lambda": 0.16386881253371635, "S_mean": 0.271087646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 6.62135556884282e-05, "train_loss": 4.955537557601929, "grad_norm": 0.7061375975608826, "tokens_seen": 30646272, "tokens_per_sec": 16027.112990085789, "vram_gb": 6.154519552, "lambda": 0.1639383789151907, "S_mean": 0.265869140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 6.529740921380147e-05, "train_loss": 4.901541233062744, "grad_norm": 0.7111294865608215, "tokens_seen": 30810112, "tokens_per_sec": 16029.91394913022, "vram_gb": 6.154519552, "lambda": 0.16399934142827988, "S_mean": 0.2575276692708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 6.445362978445676e-05, "train_loss": 4.811633586883545, "grad_norm": 0.7257888317108154, "tokens_seen": 30973952, "tokens_per_sec": 16032.788640593855, "vram_gb": 6.154519552, "lambda": 0.16407388728111982, "S_mean": 0.2745259602864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 6.368244808126497e-05, "train_loss": 4.923191785812378, "grad_norm": 0.7689898014068604, "tokens_seen": 31137792, "tokens_per_sec": 16035.603807529593, "vram_gb": 6.154519552, "lambda": 0.16412982872376838, "S_mean": 0.270416259765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3800, "val_loss": 5.001268577575684, "val_ppl": 148.60155217850138, "lambda": 0.16413324978202581, "S_mean": 0.2752278645833333, "G_mean": null, "G_std": null, "tokens_seen": 31137792} +{"event": "step", "step": 3820, "lr": 6.29840749376035e-05, "train_loss": 4.979934453964233, "grad_norm": 0.7554792761802673, "tokens_seen": 31301632, "tokens_per_sec": 16006.34600193865, "vram_gb": 6.154519552, "lambda": 0.1641976566364368, "S_mean": 0.2696634928385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 6.23587012817167e-05, "train_loss": 4.897418022155762, "grad_norm": 0.749420166015625, "tokens_seen": 31465472, "tokens_per_sec": 16007.926642712793, "vram_gb": 6.154519552, "lambda": 0.16426111416270336, "S_mean": 0.271881103515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 6.18064980845177e-05, "train_loss": 4.908336877822876, "grad_norm": 0.7046741247177124, "tokens_seen": 31629312, "tokens_per_sec": 16009.092991450234, "vram_gb": 6.154519552, "lambda": 0.1643272421012322, "S_mean": 0.264190673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 6.132761631284691e-05, "train_loss": 4.9749298095703125, "grad_norm": 0.7789618968963623, "tokens_seen": 31793152, "tokens_per_sec": 16010.955576289045, "vram_gb": 6.154519552, "lambda": 0.1643884942556421, "S_mean": 0.2693379720052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 6.092218688819914e-05, "train_loss": 4.877171516418457, "grad_norm": 0.7684378027915955, "tokens_seen": 31956992, "tokens_per_sec": 16012.215822588769, "vram_gb": 6.154519552, "lambda": 0.164447876624763, "S_mean": 0.26171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 6.059032065093088e-05, "train_loss": 4.805212736129761, "grad_norm": 0.7828719019889832, "tokens_seen": 32120832, "tokens_per_sec": 16013.29050166757, "vram_gb": 6.154519552, "lambda": 0.16450831635544697, "S_mean": 0.2579345703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 6.033210832995772e-05, "train_loss": 4.98388671875, "grad_norm": 0.7720547318458557, "tokens_seen": 32284672, "tokens_per_sec": 16013.533166512078, "vram_gb": 6.154519552, "lambda": 0.1645667472233375, "S_mean": 0.263031005859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 6.014762051794997e-05, "train_loss": 4.924776077270508, "grad_norm": 0.7707194089889526, "tokens_seen": 32448512, "tokens_per_sec": 16013.703864242174, "vram_gb": 6.154519552, "lambda": 0.16462658531963825, "S_mean": 0.2539265950520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 6.003690765203337e-05, "train_loss": 4.893288850784302, "grad_norm": 0.8201964497566223, "tokens_seen": 32612352, "tokens_per_sec": 16014.325679282236, "vram_gb": 6.154519552, "lambda": 0.16468850150704384, "S_mean": 0.276458740234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3999, "val_loss": 4.996470749378204, "val_ppl": 147.89029507035235, "lambda": 0.16474913991987705, "S_mean": 0.2511393229166667, "G_mean": null, "G_std": null, "tokens_seen": 32768000} +{"event": "done", "step": 3999, "elapsed_s": 2049.8153450489044, "nan_failures": 0} diff --git a/runs_full.log b/runs_full.log new file mode 100644 index 0000000000000000000000000000000000000000..b6be235098995d5431f9be62f1509c6628628970 --- /dev/null +++ b/runs_full.log @@ -0,0 +1,5178 @@ +[init] variant=baseline seed=0 params=63.38M +[init] use_S=False use_G=False +[baseline s0] step 0/12000 loss=10.8906 gn=14.00 lr=3.00e-06 tps=6667 mem=5.5G lam=None Gm=None +[baseline s0] step 20/12000 loss=9.7979 gn=2.47 lr=6.30e-05 tps=17646 mem=6.0G lam=None Gm=None +[baseline s0] step 40/12000 loss=9.0640 gn=1.79 lr=1.23e-04 tps=18332 mem=6.0G lam=None Gm=None +[baseline s0] step 60/12000 loss=8.1033 gn=1.50 lr=1.83e-04 tps=18576 mem=6.0G lam=None Gm=None +[baseline s0] step 80/12000 loss=7.5715 gn=0.82 lr=2.43e-04 tps=18702 mem=6.0G lam=None Gm=None +[baseline s0] step 100/12000 loss=7.3209 gn=0.74 lr=3.03e-04 tps=18757 mem=6.0G lam=None Gm=None +[baseline s0] step 120/12000 loss=7.0971 gn=0.63 lr=3.63e-04 tps=18797 mem=6.0G lam=None Gm=None +[baseline s0] step 140/12000 loss=6.9852 gn=0.56 lr=4.23e-04 tps=18828 mem=6.0G lam=None Gm=None +[baseline s0] step 160/12000 loss=6.8020 gn=0.78 lr=4.83e-04 tps=18851 mem=6.0G lam=None Gm=None +[baseline s0] step 180/12000 loss=6.8511 gn=0.55 lr=5.43e-04 tps=18868 mem=6.0G lam=None Gm=None +[baseline s0] step 200/12000 loss=6.6884 gn=0.64 lr=6.00e-04 tps=18882 mem=6.0G lam=None Gm=None +[baseline s0] step 220/12000 loss=6.8355 gn=0.58 lr=6.00e-04 tps=18895 mem=6.0G lam=None Gm=None +[baseline s0] step 240/12000 loss=6.5594 gn=0.60 lr=6.00e-04 tps=18900 mem=6.0G lam=None Gm=None +[baseline s0] step 260/12000 loss=6.6564 gn=0.58 lr=6.00e-04 tps=18906 mem=6.0G lam=None Gm=None +[baseline s0] step 280/12000 loss=6.4235 gn=0.66 lr=6.00e-04 tps=18907 mem=6.0G lam=None Gm=None +[baseline s0] step 300/12000 loss=6.5648 gn=0.55 lr=6.00e-04 tps=18912 mem=6.0G lam=None Gm=None +[baseline s0] step 320/12000 loss=6.3605 gn=0.59 lr=6.00e-04 tps=18915 mem=6.0G lam=None Gm=None +[baseline s0] step 340/12000 loss=6.4030 gn=0.55 lr=6.00e-04 tps=18917 mem=6.0G lam=None Gm=None +[baseline s0] step 360/12000 loss=6.5522 gn=0.52 lr=6.00e-04 tps=18921 mem=6.0G lam=None Gm=None +[baseline s0] step 380/12000 loss=6.4029 gn=0.55 lr=6.00e-04 tps=18924 mem=6.0G lam=None Gm=None +[baseline s0] step 400/12000 loss=6.3057 gn=0.54 lr=6.00e-04 tps=18926 mem=6.0G lam=None Gm=None +[baseline s0] step 420/12000 loss=6.4277 gn=0.56 lr=6.00e-04 tps=18928 mem=6.0G lam=None Gm=None +[baseline s0] step 440/12000 loss=6.3644 gn=0.54 lr=5.99e-04 tps=18929 mem=6.0G lam=None Gm=None +[baseline s0] step 460/12000 loss=6.4158 gn=0.52 lr=5.99e-04 tps=18930 mem=6.0G lam=None Gm=None +[baseline s0] step 480/12000 loss=6.2019 gn=0.49 lr=5.99e-04 tps=18931 mem=6.0G lam=None Gm=None +[baseline s0] step 500/12000 loss=6.0828 gn=0.53 lr=5.99e-04 tps=18932 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 500: val_loss=6.3037 val_ppl=546.61 +[baseline s0] step 520/12000 loss=6.2691 gn=0.54 lr=5.99e-04 tps=18675 mem=6.0G lam=None Gm=None +[baseline s0] step 540/12000 loss=6.1950 gn=0.55 lr=5.99e-04 tps=18682 mem=6.0G lam=None Gm=None +[baseline s0] step 560/12000 loss=6.1852 gn=0.56 lr=5.99e-04 tps=18692 mem=6.0G lam=None Gm=None +[baseline s0] step 580/12000 loss=6.2196 gn=0.61 lr=5.99e-04 tps=18702 mem=6.0G lam=None Gm=None +[baseline s0] step 600/12000 loss=6.1668 gn=0.50 lr=5.98e-04 tps=18710 mem=6.0G lam=None Gm=None +[baseline s0] step 620/12000 loss=6.0696 gn=0.49 lr=5.98e-04 tps=18716 mem=6.0G lam=None Gm=None +[baseline s0] step 640/12000 loss=6.1462 gn=0.54 lr=5.98e-04 tps=18724 mem=6.0G lam=None Gm=None +[baseline s0] step 660/12000 loss=6.0421 gn=0.56 lr=5.98e-04 tps=18729 mem=6.0G lam=None Gm=None +[baseline s0] step 680/12000 loss=6.0313 gn=0.51 lr=5.98e-04 tps=18735 mem=6.0G lam=None Gm=None +[baseline s0] step 700/12000 loss=5.9985 gn=0.54 lr=5.98e-04 tps=18740 mem=6.0G lam=None Gm=None +[baseline s0] step 720/12000 loss=6.0923 gn=0.53 lr=5.97e-04 tps=18744 mem=6.0G lam=None Gm=None +[baseline s0] step 740/12000 loss=5.9365 gn=0.56 lr=5.97e-04 tps=18749 mem=6.0G lam=None Gm=None +[baseline s0] step 760/12000 loss=5.9347 gn=0.61 lr=5.97e-04 tps=18753 mem=6.0G lam=None Gm=None +[baseline s0] step 780/12000 loss=5.9418 gn=0.51 lr=5.97e-04 tps=18758 mem=6.0G lam=None Gm=None +[baseline s0] step 800/12000 loss=5.8355 gn=0.55 lr=5.97e-04 tps=18762 mem=6.0G lam=None Gm=None +[baseline s0] step 820/12000 loss=5.8155 gn=0.53 lr=5.96e-04 tps=18766 mem=6.0G lam=None Gm=None +[baseline s0] step 840/12000 loss=5.7998 gn=0.56 lr=5.96e-04 tps=18770 mem=6.0G lam=None Gm=None +[baseline s0] step 860/12000 loss=5.7586 gn=0.55 lr=5.96e-04 tps=18773 mem=6.0G lam=None Gm=None +[baseline s0] step 880/12000 loss=5.9184 gn=0.54 lr=5.96e-04 tps=18776 mem=6.0G lam=None Gm=None +[baseline s0] step 900/12000 loss=5.7441 gn=0.52 lr=5.95e-04 tps=18779 mem=6.0G lam=None Gm=None +[baseline s0] step 920/12000 loss=5.7875 gn=0.55 lr=5.95e-04 tps=18783 mem=6.0G lam=None Gm=None +[baseline s0] step 940/12000 loss=5.7619 gn=0.53 lr=5.95e-04 tps=18785 mem=6.0G lam=None Gm=None +[baseline s0] step 960/12000 loss=5.7905 gn=0.55 lr=5.94e-04 tps=18788 mem=6.0G lam=None Gm=None +[baseline s0] step 980/12000 loss=5.8956 gn=0.59 lr=5.94e-04 tps=18790 mem=6.0G lam=None Gm=None +[baseline s0] step 1000/12000 loss=5.8392 gn=0.59 lr=5.94e-04 tps=18793 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1000: val_loss=5.8236 val_ppl=338.20 +[baseline s0] step 1020/12000 loss=5.6326 gn=0.51 lr=5.94e-04 tps=18663 mem=6.0G lam=None Gm=None +[baseline s0] step 1040/12000 loss=5.7049 gn=0.52 lr=5.93e-04 tps=18667 mem=6.0G lam=None Gm=None +[baseline s0] step 1060/12000 loss=5.7521 gn=0.54 lr=5.93e-04 tps=18672 mem=6.0G lam=None Gm=None +[baseline s0] step 1080/12000 loss=5.7357 gn=0.57 lr=5.93e-04 tps=18676 mem=6.0G lam=None Gm=None +[baseline s0] step 1100/12000 loss=5.6329 gn=0.54 lr=5.92e-04 tps=18680 mem=6.0G lam=None Gm=None +[baseline s0] step 1120/12000 loss=5.7277 gn=0.50 lr=5.92e-04 tps=18683 mem=6.0G lam=None Gm=None +[baseline s0] step 1140/12000 loss=5.5540 gn=0.57 lr=5.92e-04 tps=18687 mem=6.0G lam=None Gm=None +[baseline s0] step 1160/12000 loss=5.5822 gn=0.53 lr=5.91e-04 tps=18690 mem=6.0G lam=None Gm=None +[baseline s0] step 1180/12000 loss=5.6485 gn=0.51 lr=5.91e-04 tps=18694 mem=6.0G lam=None Gm=None +[baseline s0] step 1200/12000 loss=5.5272 gn=0.56 lr=5.90e-04 tps=18696 mem=6.0G lam=None Gm=None +[baseline s0] step 1220/12000 loss=5.7117 gn=0.53 lr=5.90e-04 tps=18698 mem=6.0G lam=None Gm=None +[baseline s0] step 1240/12000 loss=5.6763 gn=0.53 lr=5.90e-04 tps=18699 mem=6.0G lam=None Gm=None +[baseline s0] step 1260/12000 loss=5.6450 gn=0.63 lr=5.89e-04 tps=18700 mem=6.0G lam=None Gm=None +[baseline s0] step 1280/12000 loss=5.4911 gn=0.58 lr=5.89e-04 tps=18699 mem=6.0G lam=None Gm=None +[baseline s0] step 1300/12000 loss=5.5996 gn=0.58 lr=5.89e-04 tps=18701 mem=6.0G lam=None Gm=None +[baseline s0] step 1320/12000 loss=5.5407 gn=0.56 lr=5.88e-04 tps=18703 mem=6.0G lam=None Gm=None +[baseline s0] step 1340/12000 loss=5.6305 gn=0.56 lr=5.88e-04 tps=18706 mem=6.0G lam=None Gm=None +[baseline s0] step 1360/12000 loss=5.4731 gn=0.54 lr=5.87e-04 tps=18708 mem=6.0G lam=None Gm=None +[baseline s0] step 1380/12000 loss=5.3011 gn=0.58 lr=5.87e-04 tps=18708 mem=6.0G lam=None Gm=None +[baseline s0] step 1400/12000 loss=5.5879 gn=0.56 lr=5.86e-04 tps=18710 mem=6.0G lam=None Gm=None +[baseline s0] step 1420/12000 loss=5.5230 gn=0.55 lr=5.86e-04 tps=18712 mem=6.0G lam=None Gm=None +[baseline s0] step 1440/12000 loss=5.4377 gn=0.58 lr=5.85e-04 tps=18714 mem=6.0G lam=None Gm=None +[baseline s0] step 1460/12000 loss=5.5309 gn=0.55 lr=5.85e-04 tps=18716 mem=6.0G lam=None Gm=None +[baseline s0] step 1480/12000 loss=5.4392 gn=0.56 lr=5.84e-04 tps=18718 mem=6.0G lam=None Gm=None +[baseline s0] step 1500/12000 loss=5.6700 gn=0.62 lr=5.84e-04 tps=18720 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 1500: val_loss=5.5417 val_ppl=255.11 +[baseline s0] step 1520/12000 loss=5.4467 gn=0.59 lr=5.83e-04 tps=18634 mem=6.0G lam=None Gm=None +[baseline s0] step 1540/12000 loss=5.3014 gn=0.56 lr=5.83e-04 tps=18637 mem=6.0G lam=None Gm=None +[baseline s0] step 1560/12000 loss=5.5465 gn=0.60 lr=5.82e-04 tps=18640 mem=6.0G lam=None Gm=None +[baseline s0] step 1580/12000 loss=5.4340 gn=0.58 lr=5.82e-04 tps=18642 mem=6.0G lam=None Gm=None +[baseline s0] step 1600/12000 loss=5.5623 gn=0.59 lr=5.81e-04 tps=18645 mem=6.0G lam=None Gm=None +[baseline s0] step 1620/12000 loss=5.3427 gn=0.55 lr=5.81e-04 tps=18645 mem=6.0G lam=None Gm=None +[baseline s0] step 1640/12000 loss=5.4088 gn=0.57 lr=5.80e-04 tps=18647 mem=6.0G lam=None Gm=None +[baseline s0] step 1660/12000 loss=5.3999 gn=0.61 lr=5.80e-04 tps=18649 mem=6.0G lam=None Gm=None +[baseline s0] step 1680/12000 loss=5.5024 gn=0.58 lr=5.79e-04 tps=18651 mem=6.0G lam=None Gm=None +[baseline s0] step 1700/12000 loss=5.4286 gn=0.64 lr=5.79e-04 tps=18653 mem=6.0G lam=None Gm=None +[baseline s0] step 1720/12000 loss=5.3568 gn=0.64 lr=5.78e-04 tps=18654 mem=6.0G lam=None Gm=None +[baseline s0] step 1740/12000 loss=5.4529 gn=0.70 lr=5.78e-04 tps=18656 mem=6.0G lam=None Gm=None +[baseline s0] step 1760/12000 loss=5.4133 gn=0.60 lr=5.77e-04 tps=18658 mem=6.0G lam=None Gm=None +[baseline s0] step 1780/12000 loss=5.3399 gn=0.56 lr=5.76e-04 tps=18660 mem=6.0G lam=None Gm=None +[baseline s0] step 1800/12000 loss=5.3877 gn=0.69 lr=5.76e-04 tps=18661 mem=6.0G lam=None Gm=None +[baseline s0] step 1820/12000 loss=5.3093 gn=0.56 lr=5.75e-04 tps=18663 mem=6.0G lam=None Gm=None +[baseline s0] step 1840/12000 loss=5.4398 gn=0.62 lr=5.75e-04 tps=18665 mem=6.0G lam=None Gm=None +[baseline s0] step 1860/12000 loss=5.3951 gn=0.61 lr=5.74e-04 tps=18667 mem=6.0G lam=None Gm=None +[baseline s0] step 1880/12000 loss=5.3402 gn=0.62 lr=5.73e-04 tps=18669 mem=6.0G lam=None Gm=None +[baseline s0] step 1900/12000 loss=5.2814 gn=0.62 lr=5.73e-04 tps=18670 mem=6.0G lam=None Gm=None +[baseline s0] step 1920/12000 loss=5.2185 gn=0.63 lr=5.72e-04 tps=18672 mem=6.0G lam=None Gm=None +[baseline s0] step 1940/12000 loss=5.2060 gn=0.60 lr=5.72e-04 tps=18674 mem=6.0G lam=None Gm=None +[baseline s0] step 1960/12000 loss=5.1960 gn=0.61 lr=5.71e-04 tps=18674 mem=6.0G lam=None Gm=None +[baseline s0] step 1980/12000 loss=5.0817 gn=0.61 lr=5.70e-04 tps=18676 mem=6.0G lam=None Gm=None +[baseline s0] step 2000/12000 loss=5.2380 gn=0.59 lr=5.70e-04 tps=18677 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2000: val_loss=5.3548 val_ppl=211.63 +[baseline s0] step 2020/12000 loss=5.1532 gn=0.66 lr=5.69e-04 tps=18612 mem=6.0G lam=None Gm=None +[baseline s0] step 2040/12000 loss=5.3276 gn=0.59 lr=5.68e-04 tps=18614 mem=6.0G lam=None Gm=None +[baseline s0] step 2060/12000 loss=5.3216 gn=0.68 lr=5.68e-04 tps=18616 mem=6.0G lam=None Gm=None +[baseline s0] step 2080/12000 loss=5.2661 gn=0.65 lr=5.67e-04 tps=18618 mem=6.0G lam=None Gm=None +[baseline s0] step 2100/12000 loss=5.2616 gn=0.63 lr=5.66e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 2120/12000 loss=5.2263 gn=0.67 lr=5.65e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 2140/12000 loss=5.2854 gn=0.62 lr=5.65e-04 tps=18623 mem=6.0G lam=None Gm=None +[baseline s0] step 2160/12000 loss=5.3385 gn=0.62 lr=5.64e-04 tps=18625 mem=6.0G lam=None Gm=None +[baseline s0] step 2180/12000 loss=5.3075 gn=0.62 lr=5.63e-04 tps=18627 mem=6.0G lam=None Gm=None +[baseline s0] step 2200/12000 loss=5.2478 gn=0.61 lr=5.63e-04 tps=18629 mem=6.0G lam=None Gm=None +[baseline s0] step 2220/12000 loss=5.2238 gn=0.61 lr=5.62e-04 tps=18631 mem=6.0G lam=None Gm=None +[baseline s0] step 2240/12000 loss=5.1347 gn=0.59 lr=5.61e-04 tps=18633 mem=6.0G lam=None Gm=None +[baseline s0] step 2260/12000 loss=5.2502 gn=0.62 lr=5.60e-04 tps=18635 mem=6.0G lam=None Gm=None +[baseline s0] step 2280/12000 loss=5.1784 gn=0.66 lr=5.60e-04 tps=18637 mem=6.0G lam=None Gm=None +[baseline s0] step 2300/12000 loss=5.1868 gn=0.65 lr=5.59e-04 tps=18638 mem=6.0G lam=None Gm=None +[baseline s0] step 2320/12000 loss=5.1629 gn=0.63 lr=5.58e-04 tps=18640 mem=6.0G lam=None Gm=None +[baseline s0] step 2340/12000 loss=5.1081 gn=0.63 lr=5.57e-04 tps=18641 mem=6.0G lam=None Gm=None +[baseline s0] step 2360/12000 loss=5.2008 gn=0.68 lr=5.57e-04 tps=18643 mem=6.0G lam=None Gm=None +[baseline s0] step 2380/12000 loss=5.2855 gn=0.63 lr=5.56e-04 tps=18645 mem=6.0G lam=None Gm=None +[baseline s0] step 2400/12000 loss=5.1805 gn=0.60 lr=5.55e-04 tps=18646 mem=6.0G lam=None Gm=None +[baseline s0] step 2420/12000 loss=5.2646 gn=0.65 lr=5.54e-04 tps=18648 mem=6.0G lam=None Gm=None +[baseline s0] step 2440/12000 loss=5.0850 gn=0.68 lr=5.53e-04 tps=18650 mem=6.0G lam=None Gm=None +[baseline s0] step 2460/12000 loss=5.1138 gn=0.69 lr=5.53e-04 tps=18651 mem=6.0G lam=None Gm=None +[baseline s0] step 2480/12000 loss=5.0729 gn=0.65 lr=5.52e-04 tps=18653 mem=6.0G lam=None Gm=None +[baseline s0] step 2500/12000 loss=5.2348 gn=0.68 lr=5.51e-04 tps=18654 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 2500: val_loss=5.2090 val_ppl=182.91 +[baseline s0] step 2520/12000 loss=5.1440 gn=0.63 lr=5.50e-04 tps=18603 mem=6.0G lam=None Gm=None +[baseline s0] step 2540/12000 loss=5.0019 gn=0.66 lr=5.49e-04 tps=18605 mem=6.0G lam=None Gm=None +[baseline s0] step 2560/12000 loss=5.2428 gn=0.70 lr=5.48e-04 tps=18606 mem=6.0G lam=None Gm=None +[baseline s0] step 2580/12000 loss=5.1422 gn=0.67 lr=5.48e-04 tps=18651 mem=6.0G lam=None Gm=None +[baseline s0] step 2600/12000 loss=4.9855 gn=0.64 lr=5.47e-04 tps=18652 mem=6.0G lam=None Gm=None +[baseline s0] step 2620/12000 loss=5.0011 gn=0.67 lr=5.46e-04 tps=18654 mem=6.0G lam=None Gm=None +[baseline s0] step 2640/12000 loss=5.1417 gn=0.67 lr=5.45e-04 tps=18655 mem=6.0G lam=None Gm=None +[baseline s0] step 2660/12000 loss=5.1118 gn=0.68 lr=5.44e-04 tps=18656 mem=6.0G lam=None Gm=None +[baseline s0] step 2680/12000 loss=5.0961 gn=0.68 lr=5.43e-04 tps=18658 mem=6.0G lam=None Gm=None +[baseline s0] step 2700/12000 loss=5.0977 gn=0.66 lr=5.42e-04 tps=18659 mem=6.0G lam=None Gm=None +[baseline s0] step 2720/12000 loss=5.0585 gn=0.67 lr=5.41e-04 tps=18660 mem=6.0G lam=None Gm=None +[baseline s0] step 2740/12000 loss=5.1334 gn=0.72 lr=5.41e-04 tps=18662 mem=6.0G lam=None Gm=None +[baseline s0] step 2760/12000 loss=5.2603 gn=0.63 lr=5.40e-04 tps=18663 mem=6.0G lam=None Gm=None +[baseline s0] step 2780/12000 loss=5.2544 gn=0.73 lr=5.39e-04 tps=18664 mem=6.0G lam=None Gm=None +[baseline s0] step 2800/12000 loss=5.0197 gn=0.66 lr=5.38e-04 tps=18665 mem=6.0G lam=None Gm=None +[baseline s0] step 2820/12000 loss=5.2950 gn=0.77 lr=5.37e-04 tps=18666 mem=6.0G lam=None Gm=None +[baseline s0] step 2840/12000 loss=5.0151 gn=0.71 lr=5.36e-04 tps=18667 mem=6.0G lam=None Gm=None +[baseline s0] step 2860/12000 loss=5.0484 gn=0.69 lr=5.35e-04 tps=18669 mem=6.0G lam=None Gm=None +[baseline s0] step 2880/12000 loss=5.0958 gn=0.70 lr=5.34e-04 tps=18670 mem=6.0G lam=None Gm=None +[baseline s0] step 2900/12000 loss=4.9132 gn=0.67 lr=5.33e-04 tps=18671 mem=6.0G lam=None Gm=None +[baseline s0] step 2920/12000 loss=5.0519 gn=0.66 lr=5.32e-04 tps=18673 mem=6.0G lam=None Gm=None +[baseline s0] step 2940/12000 loss=5.1136 gn=0.75 lr=5.31e-04 tps=18674 mem=6.0G lam=None Gm=None +[baseline s0] step 2960/12000 loss=4.9660 gn=0.77 lr=5.30e-04 tps=18675 mem=6.0G lam=None Gm=None +[baseline s0] step 2980/12000 loss=5.1204 gn=0.73 lr=5.29e-04 tps=18676 mem=6.0G lam=None Gm=None +[baseline s0] step 3000/12000 loss=5.0981 gn=0.69 lr=5.28e-04 tps=18678 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3000: val_loss=5.0483 val_ppl=155.76 +[baseline s0] step 3020/12000 loss=5.0546 gn=0.88 lr=5.27e-04 tps=18634 mem=6.0G lam=None Gm=None +[baseline s0] step 3040/12000 loss=5.1706 gn=0.65 lr=5.26e-04 tps=18636 mem=6.0G lam=None Gm=None +[baseline s0] step 3060/12000 loss=4.9128 gn=0.68 lr=5.25e-04 tps=18637 mem=6.0G lam=None Gm=None +[baseline s0] step 3080/12000 loss=4.9875 gn=0.81 lr=5.24e-04 tps=18639 mem=6.0G lam=None Gm=None +[baseline s0] step 3100/12000 loss=4.9276 gn=0.77 lr=5.23e-04 tps=18640 mem=6.0G lam=None Gm=None +[baseline s0] step 3120/12000 loss=4.8869 gn=0.70 lr=5.22e-04 tps=18641 mem=6.0G lam=None Gm=None +[baseline s0] step 3140/12000 loss=5.0678 gn=0.71 lr=5.21e-04 tps=18643 mem=6.0G lam=None Gm=None +[baseline s0] step 3160/12000 loss=4.8544 gn=0.70 lr=5.20e-04 tps=18644 mem=6.0G lam=None Gm=None +[baseline s0] step 3180/12000 loss=4.8843 gn=0.77 lr=5.19e-04 tps=18645 mem=6.0G lam=None Gm=None +[baseline s0] step 3200/12000 loss=4.8414 gn=0.67 lr=5.18e-04 tps=18647 mem=6.0G lam=None Gm=None +[baseline s0] step 3220/12000 loss=4.8266 gn=0.81 lr=5.17e-04 tps=18648 mem=6.0G lam=None Gm=None +[baseline s0] step 3240/12000 loss=4.8790 gn=0.68 lr=5.16e-04 tps=18649 mem=6.0G lam=None Gm=None +[baseline s0] step 3260/12000 loss=5.0030 gn=0.72 lr=5.15e-04 tps=18650 mem=6.0G lam=None Gm=None +[baseline s0] step 3280/12000 loss=4.8685 gn=0.71 lr=5.14e-04 tps=18652 mem=6.0G lam=None Gm=None +[baseline s0] step 3300/12000 loss=5.0462 gn=0.74 lr=5.13e-04 tps=18653 mem=6.0G lam=None Gm=None +[baseline s0] step 3320/12000 loss=4.9234 gn=0.78 lr=5.12e-04 tps=18654 mem=6.0G lam=None Gm=None +[baseline s0] step 3340/12000 loss=4.9812 gn=0.72 lr=5.11e-04 tps=18655 mem=6.0G lam=None Gm=None +[baseline s0] step 3360/12000 loss=4.7854 gn=0.74 lr=5.10e-04 tps=18656 mem=6.0G lam=None Gm=None +[baseline s0] step 3380/12000 loss=4.9662 gn=0.98 lr=5.09e-04 tps=18657 mem=6.0G lam=None Gm=None +[baseline s0] step 3400/12000 loss=4.9917 gn=0.77 lr=5.08e-04 tps=18658 mem=6.0G lam=None Gm=None +[baseline s0] step 3420/12000 loss=4.8157 gn=0.71 lr=5.07e-04 tps=18659 mem=6.0G lam=None Gm=None +[baseline s0] step 3440/12000 loss=4.9150 gn=0.72 lr=5.06e-04 tps=18660 mem=6.0G lam=None Gm=None +[baseline s0] step 3460/12000 loss=4.8272 gn=0.70 lr=5.05e-04 tps=18661 mem=6.0G lam=None Gm=None +[baseline s0] step 3480/12000 loss=4.7964 gn=0.69 lr=5.03e-04 tps=18662 mem=6.0G lam=None Gm=None +[baseline s0] step 3500/12000 loss=4.9704 gn=0.83 lr=5.02e-04 tps=18663 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 3500: val_loss=4.9356 val_ppl=139.16 +[baseline s0] step 3520/12000 loss=4.9426 gn=0.67 lr=5.01e-04 tps=18627 mem=6.0G lam=None Gm=None +[baseline s0] step 3540/12000 loss=4.8093 gn=0.71 lr=5.00e-04 tps=18627 mem=6.0G lam=None Gm=None +[baseline s0] step 3560/12000 loss=4.8575 gn=0.76 lr=4.99e-04 tps=18629 mem=6.0G lam=None Gm=None +[baseline s0] step 3580/12000 loss=4.9189 gn=0.75 lr=4.98e-04 tps=18630 mem=6.0G lam=None Gm=None +[baseline s0] step 3600/12000 loss=4.7883 gn=0.71 lr=4.97e-04 tps=18631 mem=6.0G lam=None Gm=None +[baseline s0] step 3620/12000 loss=4.8739 gn=0.72 lr=4.96e-04 tps=18632 mem=6.0G lam=None Gm=None +[baseline s0] step 3640/12000 loss=4.9285 gn=0.71 lr=4.94e-04 tps=18633 mem=6.0G lam=None Gm=None +[baseline s0] step 3660/12000 loss=4.8100 gn=0.69 lr=4.93e-04 tps=18634 mem=6.0G lam=None Gm=None +[baseline s0] step 3680/12000 loss=4.9491 gn=0.74 lr=4.92e-04 tps=18636 mem=6.0G lam=None Gm=None +[baseline s0] step 3700/12000 loss=4.7929 gn=0.76 lr=4.91e-04 tps=18637 mem=6.0G lam=None Gm=None +[baseline s0] step 3720/12000 loss=4.9241 gn=0.84 lr=4.90e-04 tps=18638 mem=6.0G lam=None Gm=None +[baseline s0] step 3740/12000 loss=4.8684 gn=0.72 lr=4.89e-04 tps=18639 mem=6.0G lam=None Gm=None +[baseline s0] step 3760/12000 loss=4.9506 gn=0.75 lr=4.88e-04 tps=18640 mem=6.0G lam=None Gm=None +[baseline s0] step 3780/12000 loss=4.9263 gn=0.79 lr=4.86e-04 tps=18641 mem=6.0G lam=None Gm=None +[baseline s0] step 3800/12000 loss=4.9013 gn=0.79 lr=4.85e-04 tps=18642 mem=6.0G lam=None Gm=None +[baseline s0] step 3820/12000 loss=4.6945 gn=0.89 lr=4.84e-04 tps=18643 mem=6.0G lam=None Gm=None +[baseline s0] step 3840/12000 loss=4.9473 gn=0.75 lr=4.83e-04 tps=18644 mem=6.0G lam=None Gm=None +[baseline s0] step 3860/12000 loss=4.8646 gn=0.83 lr=4.82e-04 tps=18645 mem=6.0G lam=None Gm=None +[baseline s0] step 3880/12000 loss=4.7655 gn=0.76 lr=4.80e-04 tps=18646 mem=6.0G lam=None Gm=None +[baseline s0] step 3900/12000 loss=4.7127 gn=0.78 lr=4.79e-04 tps=18647 mem=6.0G lam=None Gm=None +[baseline s0] step 3920/12000 loss=4.8885 gn=1.37 lr=4.78e-04 tps=18648 mem=6.0G lam=None Gm=None +[baseline s0] step 3940/12000 loss=4.6799 gn=0.73 lr=4.77e-04 tps=18649 mem=6.0G lam=None Gm=None +[baseline s0] step 3960/12000 loss=4.7992 gn=0.77 lr=4.76e-04 tps=18650 mem=6.0G lam=None Gm=None +[baseline s0] step 3980/12000 loss=4.6827 gn=0.70 lr=4.74e-04 tps=18651 mem=6.0G lam=None Gm=None +[baseline s0] step 4000/12000 loss=4.7646 gn=0.72 lr=4.73e-04 tps=18652 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 4000: val_loss=4.8916 val_ppl=133.16 +[baseline s0] step 4020/12000 loss=4.9418 gn=0.75 lr=4.72e-04 tps=18619 mem=6.0G lam=None Gm=None +[baseline s0] step 4040/12000 loss=4.8966 gn=0.73 lr=4.71e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4060/12000 loss=4.7766 gn=0.78 lr=4.70e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 4080/12000 loss=4.8194 gn=0.77 lr=4.68e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 4100/12000 loss=4.7156 gn=0.82 lr=4.67e-04 tps=18624 mem=6.0G lam=None Gm=None +[baseline s0] step 4120/12000 loss=4.6408 gn=0.92 lr=4.66e-04 tps=18625 mem=6.0G lam=None Gm=None +[baseline s0] step 4140/12000 loss=4.7545 gn=0.77 lr=4.65e-04 tps=18626 mem=6.0G lam=None Gm=None +[baseline s0] step 4160/12000 loss=4.7485 gn=0.82 lr=4.63e-04 tps=18627 mem=6.0G lam=None Gm=None +[baseline s0] step 4180/12000 loss=4.9272 gn=0.82 lr=4.62e-04 tps=18628 mem=6.0G lam=None Gm=None +[baseline s0] step 4200/12000 loss=4.7507 gn=0.84 lr=4.61e-04 tps=18629 mem=6.0G lam=None Gm=None +[baseline s0] step 4220/12000 loss=4.8140 gn=0.72 lr=4.60e-04 tps=18630 mem=6.0G lam=None Gm=None +[baseline s0] step 4240/12000 loss=4.8231 gn=0.74 lr=4.58e-04 tps=18630 mem=6.0G lam=None Gm=None +[baseline s0] step 4260/12000 loss=4.8609 gn=0.76 lr=4.57e-04 tps=18631 mem=6.0G lam=None Gm=None +[baseline s0] step 4280/12000 loss=4.8800 gn=0.83 lr=4.56e-04 tps=18632 mem=6.0G lam=None Gm=None +[baseline s0] step 4300/12000 loss=4.8544 gn=0.78 lr=4.54e-04 tps=18633 mem=6.0G lam=None Gm=None +[baseline s0] step 4320/12000 loss=4.6919 gn=0.84 lr=4.53e-04 tps=18634 mem=6.0G lam=None Gm=None +[baseline s0] step 4340/12000 loss=4.7236 gn=0.83 lr=4.52e-04 tps=18635 mem=6.0G lam=None Gm=None +[baseline s0] step 4360/12000 loss=4.8616 gn=0.78 lr=4.51e-04 tps=18636 mem=6.0G lam=None Gm=None +[baseline s0] step 4380/12000 loss=4.8548 gn=0.77 lr=4.49e-04 tps=18636 mem=6.0G lam=None Gm=None +[baseline s0] step 4400/12000 loss=4.5996 gn=0.79 lr=4.48e-04 tps=18637 mem=6.0G lam=None Gm=None +[baseline s0] step 4420/12000 loss=4.8777 gn=0.76 lr=4.47e-04 tps=18638 mem=6.0G lam=None Gm=None +[baseline s0] step 4440/12000 loss=4.7108 gn=0.84 lr=4.45e-04 tps=18639 mem=6.0G lam=None Gm=None +[baseline s0] step 4460/12000 loss=4.7344 gn=0.79 lr=4.44e-04 tps=18639 mem=6.0G lam=None Gm=None +[baseline s0] step 4480/12000 loss=4.7013 gn=0.72 lr=4.43e-04 tps=18640 mem=6.0G lam=None Gm=None +[baseline s0] step 4500/12000 loss=4.7293 gn=0.92 lr=4.42e-04 tps=18641 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 4500: val_loss=4.8242 val_ppl=124.49 +[baseline s0] step 4520/12000 loss=4.7026 gn=0.77 lr=4.40e-04 tps=18612 mem=6.0G lam=None Gm=None +[baseline s0] step 4540/12000 loss=4.7621 gn=0.76 lr=4.39e-04 tps=18613 mem=6.0G lam=None Gm=None +[baseline s0] step 4560/12000 loss=4.7833 gn=0.81 lr=4.38e-04 tps=18614 mem=6.0G lam=None Gm=None +[baseline s0] step 4580/12000 loss=4.7398 gn=0.81 lr=4.36e-04 tps=18615 mem=6.0G lam=None Gm=None +[baseline s0] step 4600/12000 loss=4.7381 gn=0.81 lr=4.35e-04 tps=18616 mem=6.0G lam=None Gm=None +[baseline s0] step 4620/12000 loss=4.7503 gn=0.81 lr=4.34e-04 tps=18616 mem=6.0G lam=None Gm=None +[baseline s0] step 4640/12000 loss=4.8133 gn=0.88 lr=4.32e-04 tps=18617 mem=6.0G lam=None Gm=None +[baseline s0] step 4660/12000 loss=4.6980 gn=0.80 lr=4.31e-04 tps=18618 mem=6.0G lam=None Gm=None +[baseline s0] step 4680/12000 loss=4.7577 gn=0.90 lr=4.30e-04 tps=18619 mem=6.0G lam=None Gm=None +[baseline s0] step 4700/12000 loss=4.7020 gn=0.91 lr=4.28e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 4720/12000 loss=4.7367 gn=0.79 lr=4.27e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 4740/12000 loss=4.6210 gn=0.85 lr=4.26e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 4760/12000 loss=4.7306 gn=0.94 lr=4.24e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 4780/12000 loss=4.7554 gn=0.86 lr=4.23e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4800/12000 loss=4.8066 gn=0.78 lr=4.22e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4820/12000 loss=4.7258 gn=0.77 lr=4.20e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4840/12000 loss=4.4762 gn=0.84 lr=4.19e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4860/12000 loss=4.7409 gn=0.81 lr=4.18e-04 tps=18620 mem=6.0G lam=None Gm=None +[baseline s0] step 4880/12000 loss=4.7273 gn=0.79 lr=4.16e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4900/12000 loss=4.5456 gn=0.74 lr=4.15e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4920/12000 loss=4.7799 gn=0.85 lr=4.13e-04 tps=18621 mem=6.0G lam=None Gm=None +[baseline s0] step 4940/12000 loss=4.6447 gn=0.77 lr=4.12e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 4960/12000 loss=4.7186 gn=0.84 lr=4.11e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 4980/12000 loss=4.5803 gn=0.81 lr=4.09e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] step 5000/12000 loss=4.7852 gn=0.96 lr=4.08e-04 tps=18622 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 5000: val_loss=4.7797 val_ppl=119.07 +[baseline s0] step 5020/12000 loss=4.6721 gn=0.90 lr=4.07e-04 tps=18595 mem=6.0G lam=None Gm=None +[baseline s0] step 5040/12000 loss=4.6498 gn=0.82 lr=4.05e-04 tps=18595 mem=6.0G lam=None Gm=None +[baseline s0] step 5060/12000 loss=4.5732 gn=1.01 lr=4.04e-04 tps=18596 mem=6.0G lam=None Gm=None +[baseline s0] step 5080/12000 loss=4.6904 gn=0.85 lr=4.02e-04 tps=18596 mem=6.0G lam=None Gm=None +[baseline s0] step 5100/12000 loss=4.6648 gn=0.75 lr=4.01e-04 tps=18596 mem=6.0G lam=None Gm=None +[baseline s0] step 5120/12000 loss=4.6045 gn=0.81 lr=4.00e-04 tps=18597 mem=6.0G lam=None Gm=None +[baseline s0] step 5140/12000 loss=4.7116 gn=0.78 lr=3.98e-04 tps=18597 mem=6.0G lam=None Gm=None +[baseline s0] step 5160/12000 loss=4.5561 gn=0.82 lr=3.97e-04 tps=18597 mem=6.0G lam=None Gm=None +[baseline s0] step 5180/12000 loss=4.5872 gn=0.87 lr=3.95e-04 tps=18598 mem=6.0G lam=None Gm=None +[baseline s0] step 5200/12000 loss=4.5324 gn=0.89 lr=3.94e-04 tps=18598 mem=6.0G lam=None Gm=None +[baseline s0] step 5220/12000 loss=4.6178 gn=0.80 lr=3.93e-04 tps=18599 mem=6.0G lam=None Gm=None +[baseline s0] step 5240/12000 loss=4.5994 gn=0.84 lr=3.91e-04 tps=18599 mem=6.0G lam=None Gm=None +[baseline s0] step 5260/12000 loss=4.5629 gn=0.90 lr=3.90e-04 tps=18599 mem=6.0G lam=None Gm=None +[baseline s0] step 5280/12000 loss=4.6182 gn=0.87 lr=3.88e-04 tps=18600 mem=6.0G lam=None Gm=None +[baseline s0] step 5300/12000 loss=4.7206 gn=0.84 lr=3.87e-04 tps=18600 mem=6.0G lam=None Gm=None +[baseline s0] step 5320/12000 loss=4.6038 gn=0.77 lr=3.86e-04 tps=18600 mem=6.0G lam=None Gm=None +[baseline s0] step 5340/12000 loss=4.6830 gn=0.86 lr=3.84e-04 tps=18600 mem=6.0G lam=None Gm=None +[baseline s0] step 5360/12000 loss=4.5190 gn=0.83 lr=3.83e-04 tps=18601 mem=6.0G lam=None Gm=None +[baseline s0] step 5380/12000 loss=4.7830 gn=0.78 lr=3.81e-04 tps=18601 mem=6.0G lam=None Gm=None +[baseline s0] step 5400/12000 loss=4.4823 gn=0.83 lr=3.80e-04 tps=18601 mem=6.0G lam=None Gm=None +[baseline s0] step 5420/12000 loss=4.6542 gn=0.82 lr=3.79e-04 tps=18602 mem=6.0G lam=None Gm=None +[baseline s0] step 5440/12000 loss=4.5200 gn=0.90 lr=3.77e-04 tps=18602 mem=6.0G lam=None Gm=None +[baseline s0] step 5460/12000 loss=4.6652 gn=0.92 lr=3.76e-04 tps=18603 mem=6.0G lam=None Gm=None +[baseline s0] step 5480/12000 loss=4.5554 gn=0.83 lr=3.74e-04 tps=18603 mem=6.0G lam=None Gm=None +[baseline s0] step 5500/12000 loss=4.6322 gn=0.85 lr=3.73e-04 tps=18603 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 5500: val_loss=4.7067 val_ppl=110.69 +[baseline s0] step 5520/12000 loss=4.4449 gn=0.82 lr=3.72e-04 tps=18580 mem=6.0G lam=None Gm=None +[baseline s0] step 5540/12000 loss=4.6775 gn=0.83 lr=3.70e-04 tps=18580 mem=6.0G lam=None Gm=None +[baseline s0] step 5560/12000 loss=4.6909 gn=0.81 lr=3.69e-04 tps=18580 mem=6.0G lam=None Gm=None +[baseline s0] step 5580/12000 loss=4.7376 gn=0.79 lr=3.67e-04 tps=18580 mem=6.0G lam=None Gm=None +[baseline s0] step 5600/12000 loss=4.5590 gn=0.83 lr=3.66e-04 tps=18581 mem=6.0G lam=None Gm=None +[baseline s0] step 5620/12000 loss=4.6387 gn=0.93 lr=3.64e-04 tps=18581 mem=6.0G lam=None Gm=None +[baseline s0] step 5640/12000 loss=4.7402 gn=0.92 lr=3.63e-04 tps=18582 mem=6.0G lam=None Gm=None +[baseline s0] step 5660/12000 loss=4.5091 gn=0.78 lr=3.62e-04 tps=18582 mem=6.0G lam=None Gm=None +[baseline s0] step 5680/12000 loss=4.5878 gn=0.87 lr=3.60e-04 tps=18583 mem=6.0G lam=None Gm=None +[baseline s0] step 5700/12000 loss=4.7258 gn=0.96 lr=3.59e-04 tps=18583 mem=6.0G lam=None Gm=None +[baseline s0] step 5720/12000 loss=4.5681 gn=0.84 lr=3.57e-04 tps=18584 mem=6.0G lam=None Gm=None +[baseline s0] step 5740/12000 loss=4.7217 gn=0.83 lr=3.56e-04 tps=18584 mem=6.0G lam=None Gm=None +[baseline s0] step 5760/12000 loss=4.6030 gn=0.89 lr=3.54e-04 tps=18585 mem=6.0G lam=None Gm=None +[baseline s0] step 5780/12000 loss=4.6341 gn=0.85 lr=3.53e-04 tps=18585 mem=6.0G lam=None Gm=None +[baseline s0] step 5800/12000 loss=4.4998 gn=0.87 lr=3.52e-04 tps=18585 mem=6.0G lam=None Gm=None +[baseline s0] step 5820/12000 loss=4.7303 gn=0.82 lr=3.50e-04 tps=18586 mem=6.0G lam=None Gm=None +[baseline s0] step 5840/12000 loss=4.5410 gn=0.83 lr=3.49e-04 tps=18586 mem=6.0G lam=None Gm=None +[baseline s0] step 5860/12000 loss=4.6327 gn=0.83 lr=3.47e-04 tps=18586 mem=6.0G lam=None Gm=None +[baseline s0] step 5880/12000 loss=4.6711 gn=0.88 lr=3.46e-04 tps=18587 mem=6.0G lam=None Gm=None +[baseline s0] step 5900/12000 loss=4.5782 gn=0.87 lr=3.44e-04 tps=18587 mem=6.0G lam=None Gm=None +[baseline s0] step 5920/12000 loss=4.5809 gn=0.84 lr=3.43e-04 tps=18587 mem=6.0G lam=None Gm=None +[baseline s0] step 5940/12000 loss=4.4978 gn=0.82 lr=3.41e-04 tps=18588 mem=6.0G lam=None Gm=None +[baseline s0] step 5960/12000 loss=4.5709 gn=0.94 lr=3.40e-04 tps=18588 mem=6.0G lam=None Gm=None +[baseline s0] step 5980/12000 loss=4.8044 gn=0.86 lr=3.39e-04 tps=18589 mem=6.0G lam=None Gm=None +[baseline s0] step 6000/12000 loss=4.5734 gn=0.88 lr=3.37e-04 tps=18589 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 6000: val_loss=4.6557 val_ppl=105.18 +[baseline s0] step 6020/12000 loss=4.4932 gn=0.81 lr=3.36e-04 tps=18567 mem=6.0G lam=None Gm=None +[baseline s0] step 6040/12000 loss=4.5192 gn=0.91 lr=3.34e-04 tps=18568 mem=6.0G lam=None Gm=None +[baseline s0] step 6060/12000 loss=4.6716 gn=0.82 lr=3.33e-04 tps=18568 mem=6.0G lam=None Gm=None +[baseline s0] step 6080/12000 loss=4.6065 gn=0.88 lr=3.31e-04 tps=18568 mem=6.0G lam=None Gm=None +[baseline s0] step 6100/12000 loss=4.5258 gn=0.84 lr=3.30e-04 tps=18569 mem=6.0G lam=None Gm=None +[baseline s0] step 6120/12000 loss=4.6838 gn=0.95 lr=3.29e-04 tps=18569 mem=6.0G lam=None Gm=None +[baseline s0] step 6140/12000 loss=4.5907 gn=0.86 lr=3.27e-04 tps=18570 mem=6.0G lam=None Gm=None +[baseline s0] step 6160/12000 loss=4.4610 gn=0.83 lr=3.26e-04 tps=18570 mem=6.0G lam=None Gm=None +[baseline s0] step 6180/12000 loss=4.6383 gn=0.95 lr=3.24e-04 tps=18571 mem=6.0G lam=None Gm=None +[baseline s0] step 6200/12000 loss=4.5698 gn=0.90 lr=3.23e-04 tps=18571 mem=6.0G lam=None Gm=None +[baseline s0] step 6220/12000 loss=4.5467 gn=0.88 lr=3.21e-04 tps=18571 mem=6.0G lam=None Gm=None +[baseline s0] step 6240/12000 loss=4.4848 gn=0.81 lr=3.20e-04 tps=18572 mem=6.0G lam=None Gm=None +[baseline s0] step 6260/12000 loss=4.6177 gn=0.90 lr=3.19e-04 tps=18572 mem=6.0G lam=None Gm=None +[baseline s0] step 6280/12000 loss=4.5630 gn=0.87 lr=3.17e-04 tps=18572 mem=6.0G lam=None Gm=None +[baseline s0] step 6300/12000 loss=4.5074 gn=0.92 lr=3.16e-04 tps=18573 mem=6.0G lam=None Gm=None +[baseline s0] step 6320/12000 loss=4.4739 gn=0.89 lr=3.14e-04 tps=18573 mem=6.0G lam=None Gm=None +[baseline s0] step 6340/12000 loss=4.5595 gn=0.90 lr=3.13e-04 tps=18574 mem=6.0G lam=None Gm=None +[baseline s0] step 6360/12000 loss=4.5909 gn=0.91 lr=3.11e-04 tps=18574 mem=6.0G lam=None Gm=None +[baseline s0] step 6380/12000 loss=4.5909 gn=0.90 lr=3.10e-04 tps=18574 mem=6.0G lam=None Gm=None +[baseline s0] step 6400/12000 loss=4.4411 gn=0.89 lr=3.08e-04 tps=18575 mem=6.0G lam=None Gm=None +[baseline s0] step 6420/12000 loss=4.6060 gn=0.89 lr=3.07e-04 tps=18575 mem=6.0G lam=None Gm=None +[baseline s0] step 6440/12000 loss=4.6567 gn=0.91 lr=3.06e-04 tps=18575 mem=6.0G lam=None Gm=None +[baseline s0] step 6460/12000 loss=4.4973 gn=0.93 lr=3.04e-04 tps=18576 mem=6.0G lam=None Gm=None +[baseline s0] step 6480/12000 loss=4.5014 gn=0.89 lr=3.03e-04 tps=18576 mem=6.0G lam=None Gm=None +[baseline s0] step 6500/12000 loss=4.5293 gn=0.89 lr=3.01e-04 tps=18576 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 6500: val_loss=4.6319 val_ppl=102.71 +[baseline s0] step 6520/12000 loss=4.6949 gn=0.91 lr=3.00e-04 tps=18556 mem=6.0G lam=None Gm=None +[baseline s0] step 6540/12000 loss=4.4790 gn=0.89 lr=2.98e-04 tps=18557 mem=6.0G lam=None Gm=None +[baseline s0] step 6560/12000 loss=4.6873 gn=0.92 lr=2.97e-04 tps=18557 mem=6.0G lam=None Gm=None +[baseline s0] step 6580/12000 loss=4.6303 gn=0.96 lr=2.96e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s0] step 6600/12000 loss=4.3768 gn=0.96 lr=2.94e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s0] step 6620/12000 loss=4.3712 gn=0.88 lr=2.93e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s0] step 6640/12000 loss=4.6871 gn=0.90 lr=2.91e-04 tps=18559 mem=6.0G lam=None Gm=None +[baseline s0] step 6660/12000 loss=4.6658 gn=0.88 lr=2.90e-04 tps=18559 mem=6.0G lam=None Gm=None +[baseline s0] step 6680/12000 loss=4.5191 gn=0.85 lr=2.88e-04 tps=18560 mem=6.0G lam=None Gm=None +[baseline s0] step 6700/12000 loss=4.4853 gn=0.94 lr=2.87e-04 tps=18560 mem=6.0G lam=None Gm=None +[baseline s0] step 6720/12000 loss=4.5775 gn=0.91 lr=2.86e-04 tps=18561 mem=6.0G lam=None Gm=None +[baseline s0] step 6740/12000 loss=4.4772 gn=0.95 lr=2.84e-04 tps=18561 mem=6.0G lam=None Gm=None +[baseline s0] step 6760/12000 loss=4.5020 gn=0.94 lr=2.83e-04 tps=18561 mem=6.0G lam=None Gm=None +[baseline s0] step 6780/12000 loss=4.4824 gn=1.02 lr=2.81e-04 tps=18562 mem=6.0G lam=None Gm=None +[baseline s0] step 6800/12000 loss=4.4255 gn=0.84 lr=2.80e-04 tps=18562 mem=6.0G lam=None Gm=None +[baseline s0] step 6820/12000 loss=4.4363 gn=0.88 lr=2.79e-04 tps=18563 mem=6.0G lam=None Gm=None +[baseline s0] step 6840/12000 loss=4.3635 gn=0.90 lr=2.77e-04 tps=18563 mem=6.0G lam=None Gm=None +[baseline s0] step 6860/12000 loss=4.5673 gn=0.91 lr=2.76e-04 tps=18563 mem=6.0G lam=None Gm=None +[baseline s0] step 6880/12000 loss=4.5163 gn=0.86 lr=2.74e-04 tps=18564 mem=6.0G lam=None Gm=None +[baseline s0] step 6900/12000 loss=4.6515 gn=1.03 lr=2.73e-04 tps=18564 mem=6.0G lam=None Gm=None +[baseline s0] step 6920/12000 loss=4.5797 gn=0.92 lr=2.72e-04 tps=18564 mem=6.0G lam=None Gm=None +[baseline s0] step 6940/12000 loss=4.4206 gn=0.84 lr=2.70e-04 tps=18565 mem=6.0G lam=None Gm=None +[baseline s0] step 6960/12000 loss=4.5281 gn=0.94 lr=2.69e-04 tps=18565 mem=6.0G lam=None Gm=None +[baseline s0] step 6980/12000 loss=4.3630 gn=0.95 lr=2.67e-04 tps=18566 mem=6.0G lam=None Gm=None +[baseline s0] step 7000/12000 loss=4.6522 gn=0.93 lr=2.66e-04 tps=18566 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 7000: val_loss=4.5767 val_ppl=97.19 +[baseline s0] step 7020/12000 loss=4.4590 gn=0.91 lr=2.65e-04 tps=18547 mem=6.0G lam=None Gm=None +[baseline s0] step 7040/12000 loss=4.4620 gn=0.95 lr=2.63e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s0] step 7060/12000 loss=4.5508 gn=0.88 lr=2.62e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s0] step 7080/12000 loss=4.5586 gn=0.93 lr=2.60e-04 tps=18545 mem=6.0G lam=None Gm=None +[baseline s0] step 7100/12000 loss=4.5408 gn=0.92 lr=2.59e-04 tps=18545 mem=6.0G lam=None Gm=None +[baseline s0] step 7120/12000 loss=4.7478 gn=0.99 lr=2.58e-04 tps=18546 mem=6.0G lam=None Gm=None +[baseline s0] step 7140/12000 loss=4.5156 gn=0.95 lr=2.56e-04 tps=18546 mem=6.0G lam=None Gm=None +[baseline s0] step 7160/12000 loss=4.4579 gn=0.90 lr=2.55e-04 tps=18547 mem=6.0G lam=None Gm=None +[baseline s0] step 7180/12000 loss=4.5782 gn=0.91 lr=2.53e-04 tps=18547 mem=6.0G lam=None Gm=None +[baseline s0] step 7200/12000 loss=4.6369 gn=0.98 lr=2.52e-04 tps=18547 mem=6.0G lam=None Gm=None +[baseline s0] step 7220/12000 loss=4.5872 gn=0.94 lr=2.51e-04 tps=18548 mem=6.0G lam=None Gm=None +[baseline s0] step 7240/12000 loss=4.3991 gn=0.96 lr=2.49e-04 tps=18548 mem=6.0G lam=None Gm=None +[baseline s0] step 7260/12000 loss=4.4472 gn=0.93 lr=2.48e-04 tps=18548 mem=6.0G lam=None Gm=None +[baseline s0] step 7280/12000 loss=4.5047 gn=1.05 lr=2.47e-04 tps=18549 mem=6.0G lam=None Gm=None +[baseline s0] step 7300/12000 loss=4.3615 gn=0.92 lr=2.45e-04 tps=18549 mem=6.0G lam=None Gm=None +[baseline s0] step 7320/12000 loss=4.6048 gn=0.92 lr=2.44e-04 tps=18550 mem=6.0G lam=None Gm=None +[baseline s0] step 7340/12000 loss=4.4221 gn=0.95 lr=2.42e-04 tps=18550 mem=6.0G lam=None Gm=None +[baseline s0] step 7360/12000 loss=4.3882 gn=0.89 lr=2.41e-04 tps=18551 mem=6.0G lam=None Gm=None +[baseline s0] step 7380/12000 loss=4.3508 gn=0.95 lr=2.40e-04 tps=18551 mem=6.0G lam=None Gm=None +[baseline s0] step 7400/12000 loss=4.4751 gn=0.91 lr=2.38e-04 tps=18551 mem=6.0G lam=None Gm=None +[baseline s0] step 7420/12000 loss=4.5885 gn=0.96 lr=2.37e-04 tps=18552 mem=6.0G lam=None Gm=None +[baseline s0] step 7440/12000 loss=4.6247 gn=0.98 lr=2.36e-04 tps=18552 mem=6.0G lam=None Gm=None +[baseline s0] step 7460/12000 loss=4.4207 gn=0.93 lr=2.34e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s0] step 7480/12000 loss=4.4802 gn=0.98 lr=2.33e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s0] step 7500/12000 loss=4.5344 gn=0.94 lr=2.32e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 7500: val_loss=4.6005 val_ppl=99.53 +[baseline s0] step 7520/12000 loss=4.5113 gn=0.88 lr=2.30e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 7540/12000 loss=4.5841 gn=0.95 lr=2.29e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s0] step 7560/12000 loss=4.2763 gn=0.88 lr=2.28e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s0] step 7580/12000 loss=4.5162 gn=0.96 lr=2.26e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 7600/12000 loss=4.3627 gn=0.98 lr=2.25e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 7620/12000 loss=4.5433 gn=1.01 lr=2.24e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 7640/12000 loss=4.3629 gn=0.99 lr=2.22e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 7660/12000 loss=4.5124 gn=0.91 lr=2.21e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 7680/12000 loss=4.4767 gn=1.03 lr=2.20e-04 tps=18540 mem=6.0G lam=None Gm=None +[baseline s0] step 7700/12000 loss=4.5321 gn=0.98 lr=2.18e-04 tps=18540 mem=6.0G lam=None Gm=None +[baseline s0] step 7720/12000 loss=4.4970 gn=0.99 lr=2.17e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s0] step 7740/12000 loss=4.3513 gn=0.95 lr=2.16e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s0] step 7760/12000 loss=4.4816 gn=1.04 lr=2.15e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s0] step 7780/12000 loss=4.4231 gn=0.98 lr=2.13e-04 tps=18542 mem=6.0G lam=None Gm=None +[baseline s0] step 7800/12000 loss=4.4907 gn=0.96 lr=2.12e-04 tps=18542 mem=6.0G lam=None Gm=None +[baseline s0] step 7820/12000 loss=4.5537 gn=0.98 lr=2.11e-04 tps=18542 mem=6.0G lam=None Gm=None +[baseline s0] step 7840/12000 loss=4.3124 gn=0.95 lr=2.09e-04 tps=18543 mem=6.0G lam=None Gm=None +[baseline s0] step 7860/12000 loss=4.4694 gn=1.01 lr=2.08e-04 tps=18543 mem=6.0G lam=None Gm=None +[baseline s0] step 7880/12000 loss=4.5248 gn=1.01 lr=2.07e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s0] step 7900/12000 loss=4.3011 gn=0.99 lr=2.06e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s0] step 7920/12000 loss=4.3623 gn=0.99 lr=2.04e-04 tps=18545 mem=6.0G lam=None Gm=None +[baseline s0] step 7940/12000 loss=4.4721 gn=1.01 lr=2.03e-04 tps=18545 mem=6.0G lam=None Gm=None +[baseline s0] step 7960/12000 loss=4.5157 gn=0.97 lr=2.02e-04 tps=18546 mem=6.0G lam=None Gm=None +[baseline s0] step 7980/12000 loss=4.4175 gn=0.91 lr=2.00e-04 tps=18546 mem=6.0G lam=None Gm=None +[baseline s0] step 8000/12000 loss=4.3924 gn=1.02 lr=1.99e-04 tps=18547 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 8000: val_loss=4.5244 val_ppl=92.24 +[baseline s0] step 8020/12000 loss=4.4241 gn=0.92 lr=1.98e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 8040/12000 loss=4.5963 gn=1.06 lr=1.97e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 8060/12000 loss=4.5171 gn=1.03 lr=1.95e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 8080/12000 loss=4.4052 gn=1.03 lr=1.94e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 8100/12000 loss=4.5251 gn=0.97 lr=1.93e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 8120/12000 loss=4.3498 gn=0.89 lr=1.92e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 8140/12000 loss=4.5102 gn=0.96 lr=1.90e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 8160/12000 loss=4.4934 gn=1.04 lr=1.89e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 8180/12000 loss=4.5656 gn=0.99 lr=1.88e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 8200/12000 loss=4.2223 gn=0.94 lr=1.87e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 8220/12000 loss=4.3963 gn=0.97 lr=1.86e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 8240/12000 loss=4.2669 gn=0.94 lr=1.84e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 8260/12000 loss=4.4105 gn=1.01 lr=1.83e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 8280/12000 loss=4.3288 gn=0.99 lr=1.82e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s0] step 8300/12000 loss=4.3627 gn=0.94 lr=1.81e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s0] step 8320/12000 loss=4.4254 gn=1.06 lr=1.80e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 8340/12000 loss=4.4471 gn=1.05 lr=1.78e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 8360/12000 loss=4.4944 gn=1.05 lr=1.77e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 8380/12000 loss=4.4316 gn=0.93 lr=1.76e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 8400/12000 loss=4.3864 gn=0.87 lr=1.75e-04 tps=18540 mem=6.0G lam=None Gm=None +[baseline s0] step 8420/12000 loss=4.5247 gn=1.02 lr=1.74e-04 tps=18540 mem=6.0G lam=None Gm=None +[baseline s0] step 8440/12000 loss=4.1947 gn=0.93 lr=1.72e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s0] step 8460/12000 loss=4.3625 gn=1.07 lr=1.71e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s0] step 8480/12000 loss=4.3072 gn=0.92 lr=1.70e-04 tps=18542 mem=6.0G lam=None Gm=None +[baseline s0] step 8500/12000 loss=4.6167 gn=0.98 lr=1.69e-04 tps=18542 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 8500: val_loss=4.5128 val_ppl=91.17 +[baseline s0] step 8520/12000 loss=4.4268 gn=0.95 lr=1.68e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 8540/12000 loss=4.4600 gn=1.07 lr=1.67e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 8560/12000 loss=4.5243 gn=1.05 lr=1.66e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 8580/12000 loss=4.3689 gn=0.94 lr=1.64e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 8600/12000 loss=4.4969 gn=1.04 lr=1.63e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 8620/12000 loss=4.4529 gn=1.00 lr=1.62e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 8640/12000 loss=4.4830 gn=1.14 lr=1.61e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 8660/12000 loss=4.3365 gn=1.05 lr=1.60e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 8680/12000 loss=4.3955 gn=0.94 lr=1.59e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 8700/12000 loss=4.3754 gn=1.05 lr=1.58e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 8720/12000 loss=4.3422 gn=1.01 lr=1.57e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 8740/12000 loss=4.4313 gn=0.96 lr=1.55e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 8760/12000 loss=4.3260 gn=1.15 lr=1.54e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 8780/12000 loss=4.4357 gn=0.94 lr=1.53e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 8800/12000 loss=4.4379 gn=1.09 lr=1.52e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 8820/12000 loss=4.4215 gn=1.00 lr=1.51e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 8840/12000 loss=4.4264 gn=1.02 lr=1.50e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 8860/12000 loss=4.4592 gn=0.97 lr=1.49e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 8880/12000 loss=4.2800 gn=1.02 lr=1.48e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 8900/12000 loss=4.3819 gn=1.06 lr=1.47e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s0] step 8920/12000 loss=4.4698 gn=1.14 lr=1.46e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 8940/12000 loss=4.2678 gn=1.09 lr=1.45e-04 tps=18538 mem=6.0G lam=None Gm=None +[baseline s0] step 8960/12000 loss=4.5048 gn=1.06 lr=1.44e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 8980/12000 loss=4.3727 gn=1.06 lr=1.43e-04 tps=18539 mem=6.0G lam=None Gm=None +[baseline s0] step 9000/12000 loss=4.3275 gn=1.01 lr=1.42e-04 tps=18540 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 9000: val_loss=4.4812 val_ppl=88.34 +[baseline s0] step 9020/12000 loss=4.3610 gn=1.04 lr=1.41e-04 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 9040/12000 loss=4.3208 gn=1.00 lr=1.40e-04 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 9060/12000 loss=4.3409 gn=1.08 lr=1.39e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 9080/12000 loss=4.3098 gn=0.98 lr=1.38e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 9100/12000 loss=4.2663 gn=0.96 lr=1.37e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 9120/12000 loss=4.3542 gn=1.08 lr=1.36e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 9140/12000 loss=4.4994 gn=1.05 lr=1.35e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 9160/12000 loss=4.3219 gn=1.05 lr=1.34e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9180/12000 loss=4.4657 gn=1.07 lr=1.33e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9200/12000 loss=4.4225 gn=1.28 lr=1.32e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 9220/12000 loss=4.4676 gn=1.13 lr=1.31e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9240/12000 loss=4.4681 gn=1.08 lr=1.30e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9260/12000 loss=4.3932 gn=1.00 lr=1.29e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9280/12000 loss=4.3387 gn=0.95 lr=1.28e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 9300/12000 loss=4.4581 gn=1.03 lr=1.27e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 9320/12000 loss=4.3252 gn=0.99 lr=1.26e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 9340/12000 loss=4.4799 gn=1.07 lr=1.25e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 9360/12000 loss=4.5417 gn=1.12 lr=1.24e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 9380/12000 loss=4.4493 gn=1.06 lr=1.23e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 9400/12000 loss=4.3057 gn=1.09 lr=1.22e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 9420/12000 loss=4.5391 gn=1.19 lr=1.21e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 9440/12000 loss=4.4165 gn=1.06 lr=1.20e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 9460/12000 loss=4.3522 gn=1.04 lr=1.19e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s0] step 9480/12000 loss=4.4137 gn=1.04 lr=1.19e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] step 9500/12000 loss=4.3500 gn=1.01 lr=1.18e-04 tps=18536 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 9500: val_loss=4.4473 val_ppl=85.40 +[baseline s0] step 9520/12000 loss=4.2063 gn=0.99 lr=1.17e-04 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 9540/12000 loss=4.5277 gn=1.08 lr=1.16e-04 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 9560/12000 loss=4.4985 gn=1.00 lr=1.15e-04 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 9580/12000 loss=4.2679 gn=1.13 lr=1.14e-04 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 9600/12000 loss=4.3517 gn=1.13 lr=1.13e-04 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 9620/12000 loss=4.4529 gn=1.06 lr=1.12e-04 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 9640/12000 loss=4.3182 gn=1.19 lr=1.12e-04 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 9660/12000 loss=4.4327 gn=1.03 lr=1.11e-04 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 9680/12000 loss=4.6218 gn=1.14 lr=1.10e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 9700/12000 loss=4.2662 gn=0.95 lr=1.09e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 9720/12000 loss=4.2763 gn=0.97 lr=1.08e-04 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 9740/12000 loss=4.4463 gn=1.12 lr=1.07e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 9760/12000 loss=4.2887 gn=1.01 lr=1.07e-04 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 9780/12000 loss=4.3682 gn=1.08 lr=1.06e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 9800/12000 loss=4.3298 gn=1.12 lr=1.05e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 9820/12000 loss=4.4885 gn=1.11 lr=1.04e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9840/12000 loss=4.1213 gn=1.02 lr=1.03e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 9860/12000 loss=4.4605 gn=1.11 lr=1.03e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 9880/12000 loss=4.5103 gn=1.13 lr=1.02e-04 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 9900/12000 loss=4.3027 gn=1.03 lr=1.01e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 9920/12000 loss=4.4369 gn=1.02 lr=1.00e-04 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 9940/12000 loss=4.5785 gn=1.15 lr=9.96e-05 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 9960/12000 loss=4.4837 gn=1.03 lr=9.89e-05 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] step 9980/12000 loss=4.3510 gn=1.02 lr=9.81e-05 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] step 10000/12000 loss=4.4632 gn=1.05 lr=9.74e-05 tps=18534 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 10000: val_loss=4.4380 val_ppl=84.61 +[baseline s0] step 10020/12000 loss=4.2623 gn=1.07 lr=9.67e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 10040/12000 loss=4.2463 gn=1.02 lr=9.59e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 10060/12000 loss=4.2705 gn=0.98 lr=9.52e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 10080/12000 loss=4.6041 gn=1.10 lr=9.45e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 10100/12000 loss=4.5744 gn=1.08 lr=9.38e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 10120/12000 loss=4.3201 gn=1.31 lr=9.31e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 10140/12000 loss=4.3347 gn=1.18 lr=9.24e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 10160/12000 loss=4.3109 gn=1.18 lr=9.18e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 10180/12000 loss=4.3007 gn=1.11 lr=9.11e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 10200/12000 loss=4.3223 gn=1.04 lr=9.04e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 10220/12000 loss=4.3634 gn=1.28 lr=8.98e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 10240/12000 loss=4.3424 gn=1.11 lr=8.91e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 10260/12000 loss=4.4135 gn=1.16 lr=8.85e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 10280/12000 loss=4.3889 gn=1.16 lr=8.78e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 10300/12000 loss=4.2880 gn=1.06 lr=8.72e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 10320/12000 loss=4.2487 gn=1.05 lr=8.66e-05 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 10340/12000 loss=4.3159 gn=1.17 lr=8.59e-05 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 10360/12000 loss=4.4130 gn=1.20 lr=8.53e-05 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 10380/12000 loss=4.2615 gn=1.12 lr=8.47e-05 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 10400/12000 loss=4.3641 gn=1.13 lr=8.41e-05 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 10420/12000 loss=4.4171 gn=1.24 lr=8.35e-05 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 10440/12000 loss=4.4333 gn=1.21 lr=8.30e-05 tps=18531 mem=6.0G lam=None Gm=None +[baseline s0] step 10460/12000 loss=4.4505 gn=1.24 lr=8.24e-05 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 10480/12000 loss=4.3401 gn=1.10 lr=8.18e-05 tps=18532 mem=6.0G lam=None Gm=None +[baseline s0] step 10500/12000 loss=4.4372 gn=1.10 lr=8.12e-05 tps=18533 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 10500: val_loss=4.4456 val_ppl=85.25 +[baseline s0] step 10520/12000 loss=4.2424 gn=1.22 lr=8.07e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 10540/12000 loss=4.4211 gn=1.37 lr=8.01e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 10560/12000 loss=4.3578 gn=1.17 lr=7.96e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 10580/12000 loss=4.3625 gn=1.11 lr=7.91e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 10600/12000 loss=4.3857 gn=1.05 lr=7.85e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 10620/12000 loss=4.2077 gn=1.08 lr=7.80e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 10640/12000 loss=4.2168 gn=1.10 lr=7.75e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 10660/12000 loss=4.3594 gn=1.17 lr=7.70e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 10680/12000 loss=4.4426 gn=1.04 lr=7.65e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 10700/12000 loss=4.2647 gn=1.20 lr=7.60e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 10720/12000 loss=4.4696 gn=1.07 lr=7.55e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 10740/12000 loss=4.4054 gn=1.05 lr=7.50e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 10760/12000 loss=4.2985 gn=1.12 lr=7.46e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 10780/12000 loss=4.3981 gn=1.16 lr=7.41e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 10800/12000 loss=4.4760 gn=1.21 lr=7.37e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 10820/12000 loss=4.2626 gn=1.28 lr=7.32e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 10840/12000 loss=4.3661 gn=1.06 lr=7.28e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 10860/12000 loss=4.4598 gn=1.17 lr=7.23e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 10880/12000 loss=4.2943 gn=1.10 lr=7.19e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 10900/12000 loss=4.3923 gn=1.18 lr=7.15e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 10920/12000 loss=4.3150 gn=1.12 lr=7.11e-05 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 10940/12000 loss=4.3970 gn=1.17 lr=7.07e-05 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 10960/12000 loss=4.4461 gn=1.18 lr=7.03e-05 tps=18529 mem=6.0G lam=None Gm=None +[baseline s0] step 10980/12000 loss=4.3537 gn=1.28 lr=6.99e-05 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] step 11000/12000 loss=4.3740 gn=1.15 lr=6.95e-05 tps=18530 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 11000: val_loss=4.4020 val_ppl=81.61 +[baseline s0] step 11020/12000 loss=4.2243 gn=1.15 lr=6.91e-05 tps=18518 mem=6.0G lam=None Gm=None +[baseline s0] step 11040/12000 loss=4.2250 gn=1.04 lr=6.88e-05 tps=18519 mem=6.0G lam=None Gm=None +[baseline s0] step 11060/12000 loss=4.4457 gn=1.23 lr=6.84e-05 tps=18519 mem=6.0G lam=None Gm=None +[baseline s0] step 11080/12000 loss=4.3700 gn=1.19 lr=6.81e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 11100/12000 loss=4.4416 gn=1.16 lr=6.77e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 11120/12000 loss=4.3198 gn=1.04 lr=6.74e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 11140/12000 loss=4.1966 gn=1.09 lr=6.70e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 11160/12000 loss=4.2716 gn=1.45 lr=6.67e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 11180/12000 loss=4.3577 gn=1.06 lr=6.64e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11200/12000 loss=4.3613 gn=1.09 lr=6.61e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11220/12000 loss=4.3546 gn=1.05 lr=6.58e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 11240/12000 loss=4.1941 gn=1.04 lr=6.55e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 11260/12000 loss=4.3848 gn=1.26 lr=6.52e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11280/12000 loss=4.3834 gn=1.18 lr=6.49e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11300/12000 loss=4.3904 gn=1.12 lr=6.47e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11320/12000 loss=4.2389 gn=1.16 lr=6.44e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 11340/12000 loss=4.3053 gn=1.20 lr=6.42e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 11360/12000 loss=4.2086 gn=1.15 lr=6.39e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 11380/12000 loss=4.3957 gn=1.11 lr=6.37e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 11400/12000 loss=4.2900 gn=1.08 lr=6.34e-05 tps=18526 mem=6.0G lam=None Gm=None +[baseline s0] step 11420/12000 loss=4.3235 gn=1.12 lr=6.32e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 11440/12000 loss=4.1684 gn=1.23 lr=6.30e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 11460/12000 loss=4.4011 gn=1.12 lr=6.28e-05 tps=18527 mem=6.0G lam=None Gm=None +[baseline s0] step 11480/12000 loss=4.2809 gn=1.44 lr=6.26e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] step 11500/12000 loss=4.2935 gn=1.16 lr=6.24e-05 tps=18528 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 11500: val_loss=4.4083 val_ppl=82.13 +[baseline s0] step 11520/12000 loss=4.3158 gn=1.12 lr=6.22e-05 tps=18517 mem=6.0G lam=None Gm=None +[baseline s0] step 11540/12000 loss=4.4412 gn=1.24 lr=6.20e-05 tps=18517 mem=6.0G lam=None Gm=None +[baseline s0] step 11560/12000 loss=4.3818 gn=1.11 lr=6.19e-05 tps=18518 mem=6.0G lam=None Gm=None +[baseline s0] step 11580/12000 loss=4.1854 gn=1.13 lr=6.17e-05 tps=18518 mem=6.0G lam=None Gm=None +[baseline s0] step 11600/12000 loss=4.3095 gn=1.19 lr=6.15e-05 tps=18518 mem=6.0G lam=None Gm=None +[baseline s0] step 11620/12000 loss=4.3766 gn=1.15 lr=6.14e-05 tps=18519 mem=6.0G lam=None Gm=None +[baseline s0] step 11640/12000 loss=4.1438 gn=1.03 lr=6.12e-05 tps=18519 mem=6.0G lam=None Gm=None +[baseline s0] step 11660/12000 loss=4.2908 gn=1.18 lr=6.11e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 11680/12000 loss=4.2674 gn=1.11 lr=6.10e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 11700/12000 loss=4.2794 gn=1.13 lr=6.09e-05 tps=18520 mem=6.0G lam=None Gm=None +[baseline s0] step 11720/12000 loss=4.2484 gn=1.09 lr=6.07e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 11740/12000 loss=4.4237 gn=1.12 lr=6.06e-05 tps=18521 mem=6.0G lam=None Gm=None +[baseline s0] step 11760/12000 loss=4.3582 gn=1.16 lr=6.06e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11780/12000 loss=4.3567 gn=1.25 lr=6.05e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11800/12000 loss=4.4003 gn=1.12 lr=6.04e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11820/12000 loss=4.3092 gn=1.25 lr=6.03e-05 tps=18522 mem=6.0G lam=None Gm=None +[baseline s0] step 11840/12000 loss=4.3483 gn=1.18 lr=6.02e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 11860/12000 loss=4.1406 gn=1.19 lr=6.02e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 11880/12000 loss=4.4873 gn=1.41 lr=6.01e-05 tps=18523 mem=6.0G lam=None Gm=None +[baseline s0] step 11900/12000 loss=4.2680 gn=1.34 lr=6.01e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11920/12000 loss=4.4411 gn=1.26 lr=6.01e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11940/12000 loss=4.1499 gn=1.25 lr=6.00e-05 tps=18524 mem=6.0G lam=None Gm=None +[baseline s0] step 11960/12000 loss=4.3426 gn=1.17 lr=6.00e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] step 11980/12000 loss=4.2262 gn=1.14 lr=6.00e-05 tps=18525 mem=6.0G lam=None Gm=None +[baseline s0] EVAL step 11999: val_loss=4.3609 val_ppl=78.32 +[init] variant=s_only seed=0 params=63.97M +[init] use_S=True use_G=False +[s_only s0] step 0/12000 loss=10.9256 gn=14.69 lr=3.00e-06 tps=7703 mem=5.6G lam=0.12692801654338837 Gm=None +[s_only s0] step 20/12000 loss=9.7610 gn=2.20 lr=6.30e-05 tps=15735 mem=6.2G lam=0.1269291490316391 Gm=None +[s_only s0] step 40/12000 loss=9.0366 gn=1.75 lr=1.23e-04 tps=16149 mem=6.2G lam=0.1268723625689745 Gm=None +[s_only s0] step 60/12000 loss=8.0866 gn=1.55 lr=1.83e-04 tps=16270 mem=6.2G lam=0.12678918428719044 Gm=None +[s_only s0] step 80/12000 loss=7.5602 gn=0.72 lr=2.43e-04 tps=16351 mem=6.2G lam=0.12669831266005835 Gm=None +[s_only s0] step 100/12000 loss=7.3254 gn=1.05 lr=3.03e-04 tps=16413 mem=6.2G lam=0.12659604599078497 Gm=None +[s_only s0] step 120/12000 loss=7.0915 gn=0.61 lr=3.63e-04 tps=16448 mem=6.2G lam=0.12653159288068613 Gm=None +[s_only s0] step 140/12000 loss=6.9781 gn=0.65 lr=4.23e-04 tps=16472 mem=6.2G lam=0.1264833795527617 Gm=None +[s_only s0] step 160/12000 loss=6.7904 gn=0.73 lr=4.83e-04 tps=16494 mem=6.2G lam=0.12647119164466858 Gm=None +[s_only s0] step 180/12000 loss=6.8486 gn=0.55 lr=5.43e-04 tps=16504 mem=6.2G lam=0.12654991447925568 Gm=None +[s_only s0] step 200/12000 loss=6.6927 gn=0.62 lr=6.00e-04 tps=16518 mem=6.2G lam=0.12651852642496428 Gm=None +[s_only s0] step 220/12000 loss=6.8431 gn=0.57 lr=6.00e-04 tps=16520 mem=6.2G lam=0.12652307624618211 Gm=None +[s_only s0] step 240/12000 loss=6.5568 gn=0.51 lr=6.00e-04 tps=16522 mem=6.2G lam=0.12653816243012747 Gm=None +[s_only s0] step 260/12000 loss=6.6635 gn=0.61 lr=6.00e-04 tps=16527 mem=6.2G lam=0.126507175154984 Gm=None +[s_only s0] step 280/12000 loss=6.4218 gn=0.62 lr=6.00e-04 tps=16523 mem=6.2G lam=0.1264537163078785 Gm=None +[s_only s0] step 300/12000 loss=6.5542 gn=0.56 lr=6.00e-04 tps=16530 mem=6.2G lam=0.1264992943033576 Gm=None +[s_only s0] step 320/12000 loss=6.3604 gn=0.61 lr=6.00e-04 tps=16533 mem=6.2G lam=0.12649505771696568 Gm=None +[s_only s0] step 340/12000 loss=6.4236 gn=0.58 lr=6.00e-04 tps=16532 mem=6.2G lam=0.1265031366298596 Gm=None +[s_only s0] step 360/12000 loss=6.5499 gn=0.49 lr=6.00e-04 tps=16536 mem=6.2G lam=0.12650990641365448 Gm=None +[s_only s0] step 380/12000 loss=6.4058 gn=0.63 lr=6.00e-04 tps=16539 mem=6.2G lam=0.12655804896106324 Gm=None +[s_only s0] step 400/12000 loss=6.3090 gn=0.51 lr=6.00e-04 tps=16541 mem=6.2G lam=0.1266178029278914 Gm=None +[s_only s0] step 420/12000 loss=6.4235 gn=0.54 lr=6.00e-04 tps=16542 mem=6.2G lam=0.12665767719348273 Gm=None +[s_only s0] step 440/12000 loss=6.3801 gn=0.56 lr=5.99e-04 tps=16543 mem=6.2G lam=0.1267284316321214 Gm=None +[s_only s0] step 460/12000 loss=6.4337 gn=0.57 lr=5.99e-04 tps=16545 mem=6.2G lam=0.12682047113776207 Gm=None +[s_only s0] step 480/12000 loss=6.1905 gn=0.48 lr=5.99e-04 tps=16547 mem=6.2G lam=0.12688480038195848 Gm=None +[s_only s0] step 500/12000 loss=6.0882 gn=0.55 lr=5.99e-04 tps=16547 mem=6.2G lam=0.12700997106730938 Gm=None +[s_only s0] EVAL step 500: val_loss=6.3018 val_ppl=545.54 +[s_only s0] step 520/12000 loss=6.2738 gn=0.55 lr=5.99e-04 tps=16311 mem=6.2G lam=0.12714012960592905 Gm=None +[s_only s0] step 540/12000 loss=6.1985 gn=0.55 lr=5.99e-04 tps=16317 mem=6.2G lam=0.12725414087375006 Gm=None +[s_only s0] step 560/12000 loss=6.1768 gn=0.52 lr=5.99e-04 tps=16322 mem=6.2G lam=0.12743254657834768 Gm=None +[s_only s0] step 580/12000 loss=6.2261 gn=0.54 lr=5.99e-04 tps=16331 mem=6.2G lam=0.12754579167813063 Gm=None +[s_only s0] step 600/12000 loss=6.1704 gn=0.52 lr=5.98e-04 tps=16341 mem=6.2G lam=0.1276875389739871 Gm=None +[s_only s0] step 620/12000 loss=6.0588 gn=0.49 lr=5.98e-04 tps=16349 mem=6.2G lam=0.12788229497770467 Gm=None +[s_only s0] step 640/12000 loss=6.1576 gn=0.53 lr=5.98e-04 tps=16355 mem=6.2G lam=0.12811957268665233 Gm=None +[s_only s0] step 660/12000 loss=6.0542 gn=0.59 lr=5.98e-04 tps=16366 mem=6.2G lam=0.12820511466513076 Gm=None +[s_only s0] step 680/12000 loss=6.0388 gn=0.53 lr=5.98e-04 tps=16370 mem=6.2G lam=0.12838880127916732 Gm=None +[s_only s0] step 700/12000 loss=6.0082 gn=0.58 lr=5.98e-04 tps=16368 mem=6.2G lam=0.12858952147265276 Gm=None +[s_only s0] step 720/12000 loss=6.1128 gn=0.62 lr=5.97e-04 tps=16366 mem=6.2G lam=0.12882865810145935 Gm=None +[s_only s0] step 740/12000 loss=5.9496 gn=0.52 lr=5.97e-04 tps=16364 mem=6.2G lam=0.12914348921428123 Gm=None +[s_only s0] step 760/12000 loss=5.9350 gn=0.54 lr=5.97e-04 tps=16363 mem=6.2G lam=0.12943589904656014 Gm=None +[s_only s0] step 780/12000 loss=5.9469 gn=0.52 lr=5.97e-04 tps=16361 mem=6.2G lam=0.12967557397981486 Gm=None +[s_only s0] step 800/12000 loss=5.8616 gn=0.61 lr=5.97e-04 tps=16359 mem=6.2G lam=0.12991350578765073 Gm=None +[s_only s0] step 820/12000 loss=5.8306 gn=0.57 lr=5.96e-04 tps=16358 mem=6.2G lam=0.13019642885774374 Gm=None +[s_only s0] step 840/12000 loss=5.8162 gn=0.55 lr=5.96e-04 tps=16357 mem=6.2G lam=0.13047212238113085 Gm=None +[s_only s0] step 860/12000 loss=5.7629 gn=0.56 lr=5.96e-04 tps=16357 mem=6.2G lam=0.1307267645994822 Gm=None +[s_only s0] step 880/12000 loss=5.9425 gn=0.56 lr=5.96e-04 tps=16357 mem=6.2G lam=0.13104286634673676 Gm=None +[s_only s0] step 900/12000 loss=5.7673 gn=0.53 lr=5.95e-04 tps=16358 mem=6.2G lam=0.13141013123095036 Gm=None +[s_only s0] step 920/12000 loss=5.7994 gn=0.54 lr=5.95e-04 tps=16359 mem=6.2G lam=0.1317999962096413 Gm=None +[s_only s0] step 940/12000 loss=5.7589 gn=0.52 lr=5.95e-04 tps=16358 mem=6.2G lam=0.13219300843775272 Gm=None +[s_only s0] step 960/12000 loss=5.7951 gn=0.56 lr=5.94e-04 tps=16357 mem=6.2G lam=0.13255574026455483 Gm=None +[s_only s0] step 980/12000 loss=5.9091 gn=0.57 lr=5.94e-04 tps=16357 mem=6.2G lam=0.13292946728567281 Gm=None +[s_only s0] step 1000/12000 loss=5.8388 gn=0.59 lr=5.94e-04 tps=16353 mem=6.2G lam=0.13331958558410406 Gm=None +[s_only s0] EVAL step 1000: val_loss=5.8344 val_ppl=341.86 +[s_only s0] step 1020/12000 loss=5.6500 gn=0.55 lr=5.94e-04 tps=16233 mem=6.2G lam=0.13371922448277473 Gm=None +[s_only s0] step 1040/12000 loss=5.7214 gn=0.56 lr=5.93e-04 tps=16233 mem=6.2G lam=0.13419483602046967 Gm=None +[s_only s0] step 1060/12000 loss=5.7629 gn=0.54 lr=5.93e-04 tps=16235 mem=6.2G lam=0.13460853043943644 Gm=None +[s_only s0] step 1080/12000 loss=5.7501 gn=0.64 lr=5.93e-04 tps=16237 mem=6.2G lam=0.13512087613344193 Gm=None +[s_only s0] step 1100/12000 loss=5.6367 gn=0.53 lr=5.92e-04 tps=16238 mem=6.2G lam=0.135609638877213 Gm=None +[s_only s0] step 1120/12000 loss=5.7414 gn=0.56 lr=5.92e-04 tps=16238 mem=6.2G lam=0.13602933008223772 Gm=None +[s_only s0] step 1140/12000 loss=5.5556 gn=0.55 lr=5.92e-04 tps=16241 mem=6.2G lam=0.13641619806488356 Gm=None +[s_only s0] step 1160/12000 loss=5.6024 gn=0.59 lr=5.91e-04 tps=16243 mem=6.2G lam=0.13685524898270765 Gm=None +[s_only s0] step 1180/12000 loss=5.6684 gn=0.51 lr=5.91e-04 tps=16244 mem=6.2G lam=0.137272611570855 Gm=None +[s_only s0] step 1200/12000 loss=5.5357 gn=0.65 lr=5.90e-04 tps=16245 mem=6.2G lam=0.13768705042699972 Gm=None +[s_only s0] step 1220/12000 loss=5.7341 gn=0.57 lr=5.90e-04 tps=16245 mem=6.2G lam=0.13817448013772568 Gm=None +[s_only s0] step 1240/12000 loss=5.6890 gn=0.54 lr=5.90e-04 tps=16247 mem=6.2G lam=0.13863906636834145 Gm=None +[s_only s0] step 1260/12000 loss=5.6541 gn=0.79 lr=5.89e-04 tps=16248 mem=6.2G lam=0.13909072522073984 Gm=None +[s_only s0] step 1280/12000 loss=5.4901 gn=0.54 lr=5.89e-04 tps=16248 mem=6.2G lam=0.13960564974695444 Gm=None +[s_only s0] step 1300/12000 loss=5.5880 gn=0.56 lr=5.89e-04 tps=16250 mem=6.2G lam=0.14002052570382753 Gm=None +[s_only s0] step 1320/12000 loss=5.5451 gn=0.56 lr=5.88e-04 tps=16251 mem=6.2G lam=0.14044411045809588 Gm=None +[s_only s0] step 1340/12000 loss=5.6546 gn=0.60 lr=5.88e-04 tps=16253 mem=6.2G lam=0.14091022374729315 Gm=None +[s_only s0] step 1360/12000 loss=5.4739 gn=0.56 lr=5.87e-04 tps=16255 mem=6.2G lam=0.1413696048160394 Gm=None +[s_only s0] step 1380/12000 loss=5.3003 gn=0.58 lr=5.87e-04 tps=16256 mem=6.2G lam=0.14189015049487352 Gm=None +[s_only s0] step 1400/12000 loss=5.5889 gn=0.55 lr=5.86e-04 tps=16258 mem=6.2G lam=0.1423905324190855 Gm=None +[s_only s0] step 1420/12000 loss=5.5367 gn=0.55 lr=5.86e-04 tps=16259 mem=6.2G lam=0.14287546804795662 Gm=None +[s_only s0] step 1440/12000 loss=5.4111 gn=0.53 lr=5.85e-04 tps=16260 mem=6.2G lam=0.143426896383365 Gm=None +[s_only s0] step 1460/12000 loss=5.5308 gn=0.57 lr=5.85e-04 tps=16261 mem=6.2G lam=0.14388229014972845 Gm=None +[s_only s0] step 1480/12000 loss=5.4336 gn=0.56 lr=5.84e-04 tps=16262 mem=6.2G lam=0.14435840335985026 Gm=None +[s_only s0] step 1500/12000 loss=5.6808 gn=0.60 lr=5.84e-04 tps=16261 mem=6.2G lam=0.1448592577750484 Gm=None +[s_only s0] EVAL step 1500: val_loss=5.5503 val_ppl=257.31 +[s_only s0] step 1520/12000 loss=5.4572 gn=0.64 lr=5.83e-04 tps=16183 mem=6.2G lam=0.14535083683828512 Gm=None +[s_only s0] step 1540/12000 loss=5.3083 gn=0.54 lr=5.83e-04 tps=16185 mem=6.2G lam=0.14577612529198328 Gm=None +[s_only s0] step 1560/12000 loss=5.5564 gn=0.69 lr=5.82e-04 tps=16188 mem=6.2G lam=0.14625812321901321 Gm=None +[s_only s0] step 1580/12000 loss=5.4485 gn=0.61 lr=5.82e-04 tps=16190 mem=6.2G lam=0.14671009592711926 Gm=None +[s_only s0] step 1600/12000 loss=5.5708 gn=0.60 lr=5.81e-04 tps=16192 mem=6.2G lam=0.14719284946719804 Gm=None +[s_only s0] step 1620/12000 loss=5.3609 gn=0.57 lr=5.81e-04 tps=16195 mem=6.2G lam=0.1477067389835914 Gm=None +[s_only s0] step 1640/12000 loss=5.4107 gn=0.57 lr=5.80e-04 tps=16197 mem=6.2G lam=0.1481638333449761 Gm=None +[s_only s0] step 1660/12000 loss=5.4023 gn=0.62 lr=5.80e-04 tps=16199 mem=6.2G lam=0.14868756011128426 Gm=None +[s_only s0] step 1680/12000 loss=5.4933 gn=0.59 lr=5.79e-04 tps=16199 mem=6.2G lam=0.14921100592861572 Gm=None +[s_only s0] step 1700/12000 loss=5.4323 gn=0.66 lr=5.79e-04 tps=16201 mem=6.2G lam=0.14970508000502983 Gm=None +[s_only s0] step 1720/12000 loss=5.3890 gn=0.81 lr=5.78e-04 tps=16202 mem=6.2G lam=0.1502062166109681 Gm=None +[s_only s0] step 1740/12000 loss=5.4424 gn=0.76 lr=5.78e-04 tps=16203 mem=6.2G lam=0.15072913623104492 Gm=None +[s_only s0] step 1760/12000 loss=5.4009 gn=0.57 lr=5.77e-04 tps=16204 mem=6.2G lam=0.1511542359367013 Gm=None +[s_only s0] step 1780/12000 loss=5.3510 gn=0.60 lr=5.76e-04 tps=16205 mem=6.2G lam=0.1516593744357427 Gm=None +[s_only s0] step 1800/12000 loss=5.3817 gn=0.62 lr=5.76e-04 tps=16206 mem=6.2G lam=0.15221729936699072 Gm=None +[s_only s0] step 1820/12000 loss=5.3037 gn=0.57 lr=5.75e-04 tps=16207 mem=6.2G lam=0.152794297474126 Gm=None +[s_only s0] step 1840/12000 loss=5.4450 gn=0.64 lr=5.75e-04 tps=16209 mem=6.2G lam=0.15329882968217134 Gm=None +[s_only s0] step 1860/12000 loss=5.4057 gn=0.64 lr=5.74e-04 tps=16211 mem=6.2G lam=0.15386295039206743 Gm=None +[s_only s0] step 1880/12000 loss=5.3374 gn=0.60 lr=5.73e-04 tps=16212 mem=6.2G lam=0.15437764674425125 Gm=None +[s_only s0] step 1900/12000 loss=5.2762 gn=0.64 lr=5.73e-04 tps=16214 mem=6.2G lam=0.15489212019989887 Gm=None +[s_only s0] step 1920/12000 loss=5.2363 gn=0.59 lr=5.72e-04 tps=16215 mem=6.2G lam=0.155486432214578 Gm=None +[s_only s0] step 1940/12000 loss=5.2090 gn=0.59 lr=5.72e-04 tps=16217 mem=6.2G lam=0.1559885237365961 Gm=None +[s_only s0] step 1960/12000 loss=5.1999 gn=0.65 lr=5.71e-04 tps=16218 mem=6.2G lam=0.15653540287166834 Gm=None +[s_only s0] step 1980/12000 loss=5.0823 gn=0.64 lr=5.70e-04 tps=16219 mem=6.2G lam=0.15712275231877962 Gm=None +[s_only s0] step 2000/12000 loss=5.2553 gn=0.63 lr=5.70e-04 tps=16220 mem=6.2G lam=0.1575803036491076 Gm=None +[s_only s0] EVAL step 2000: val_loss=5.3601 val_ppl=212.75 +[s_only s0] step 2020/12000 loss=5.1400 gn=0.68 lr=5.69e-04 tps=16162 mem=6.2G lam=0.1581548529987534 Gm=None +[s_only s0] step 2040/12000 loss=5.3224 gn=0.60 lr=5.68e-04 tps=16164 mem=6.2G lam=0.15864485315978527 Gm=None +[s_only s0] step 2060/12000 loss=5.3243 gn=0.61 lr=5.68e-04 tps=16165 mem=6.2G lam=0.15910970916350684 Gm=None +[s_only s0] step 2080/12000 loss=5.2623 gn=0.71 lr=5.67e-04 tps=16168 mem=6.2G lam=0.15963038491706052 Gm=None +[s_only s0] step 2100/12000 loss=5.2664 gn=0.63 lr=5.66e-04 tps=16169 mem=6.2G lam=0.16016417586555085 Gm=None +[s_only s0] step 2120/12000 loss=5.2259 gn=0.70 lr=5.65e-04 tps=16171 mem=6.2G lam=0.16073347721248865 Gm=None +[s_only s0] step 2140/12000 loss=5.3009 gn=0.63 lr=5.65e-04 tps=16173 mem=6.2G lam=0.1612242196376125 Gm=None +[s_only s0] step 2160/12000 loss=5.3402 gn=0.71 lr=5.64e-04 tps=16175 mem=6.2G lam=0.16176896231869856 Gm=None +[s_only s0] step 2180/12000 loss=5.2976 gn=0.64 lr=5.63e-04 tps=16176 mem=6.2G lam=0.1622620935862263 Gm=None +[s_only s0] step 2200/12000 loss=5.2439 gn=0.61 lr=5.63e-04 tps=16178 mem=6.2G lam=0.16275164764374495 Gm=None +[s_only s0] step 2220/12000 loss=5.2439 gn=0.66 lr=5.62e-04 tps=16179 mem=6.2G lam=0.16329598737259707 Gm=None +[s_only s0] step 2240/12000 loss=5.1437 gn=0.66 lr=5.61e-04 tps=16180 mem=6.2G lam=0.16387491455922523 Gm=None +[s_only s0] step 2260/12000 loss=5.2504 gn=0.65 lr=5.60e-04 tps=16181 mem=6.2G lam=0.16447194665670395 Gm=None +[s_only s0] step 2280/12000 loss=5.1769 gn=0.70 lr=5.60e-04 tps=16182 mem=6.2G lam=0.16499119562407336 Gm=None +[s_only s0] step 2300/12000 loss=5.2037 gn=0.65 lr=5.59e-04 tps=16183 mem=6.2G lam=0.1655031287421783 Gm=None +[s_only s0] step 2320/12000 loss=5.1659 gn=0.65 lr=5.58e-04 tps=16184 mem=6.2G lam=0.16607369109988213 Gm=None +[s_only s0] step 2340/12000 loss=5.1266 gn=0.65 lr=5.57e-04 tps=16185 mem=6.2G lam=0.16658391834547123 Gm=None +[s_only s0] step 2360/12000 loss=5.2048 gn=0.66 lr=5.57e-04 tps=16187 mem=6.2G lam=0.16710987128317356 Gm=None +[s_only s0] step 2380/12000 loss=5.2976 gn=0.79 lr=5.56e-04 tps=16188 mem=6.2G lam=0.1676124402632316 Gm=None +[s_only s0] step 2400/12000 loss=5.1768 gn=0.65 lr=5.55e-04 tps=16188 mem=6.2G lam=0.16816102030376592 Gm=None +[s_only s0] step 2420/12000 loss=5.2781 gn=0.65 lr=5.54e-04 tps=16190 mem=6.2G lam=0.1687153841679295 Gm=None +[s_only s0] step 2440/12000 loss=5.0696 gn=0.69 lr=5.53e-04 tps=16191 mem=6.2G lam=0.16925037807474533 Gm=None +[s_only s0] step 2460/12000 loss=5.1115 gn=0.72 lr=5.53e-04 tps=16192 mem=6.2G lam=0.16977348178625107 Gm=None +[s_only s0] step 2480/12000 loss=5.0896 gn=0.64 lr=5.52e-04 tps=16193 mem=6.2G lam=0.17023571642736593 Gm=None +[s_only s0] step 2500/12000 loss=5.2228 gn=0.65 lr=5.51e-04 tps=16194 mem=6.2G lam=0.17075345230599245 Gm=None +[s_only s0] EVAL step 2500: val_loss=5.2138 val_ppl=183.79 +[s_only s0] step 2520/12000 loss=5.1292 gn=0.64 lr=5.50e-04 tps=16149 mem=6.2G lam=0.1712930944437782 Gm=None +[s_only s0] step 2540/12000 loss=5.0139 gn=0.63 lr=5.49e-04 tps=16150 mem=6.2G lam=0.17185424516598383 Gm=None +[s_only s0] step 2560/12000 loss=5.2535 gn=0.76 lr=5.48e-04 tps=16151 mem=6.2G lam=0.17237655725330114 Gm=None +[s_only s0] step 2580/12000 loss=5.1443 gn=0.67 lr=5.48e-04 tps=16153 mem=6.2G lam=0.17285159106055895 Gm=None +[s_only s0] step 2600/12000 loss=4.9839 gn=0.61 lr=5.47e-04 tps=16153 mem=6.2G lam=0.1733904123927156 Gm=None +[s_only s0] step 2620/12000 loss=5.0097 gn=0.69 lr=5.46e-04 tps=16154 mem=6.2G lam=0.1739516311014692 Gm=None +[s_only s0] step 2640/12000 loss=5.1497 gn=0.66 lr=5.45e-04 tps=16156 mem=6.2G lam=0.17445672551790872 Gm=None +[s_only s0] step 2660/12000 loss=5.0909 gn=0.70 lr=5.44e-04 tps=16157 mem=6.2G lam=0.17500938288867474 Gm=None +[s_only s0] step 2680/12000 loss=5.0821 gn=0.65 lr=5.43e-04 tps=16158 mem=6.2G lam=0.17556125267098346 Gm=None +[s_only s0] step 2700/12000 loss=5.0918 gn=0.70 lr=5.42e-04 tps=16159 mem=6.2G lam=0.1761414504920443 Gm=None +[s_only s0] step 2720/12000 loss=5.0552 gn=0.72 lr=5.41e-04 tps=16161 mem=6.2G lam=0.1767077585682273 Gm=None +[s_only s0] step 2740/12000 loss=5.1216 gn=0.71 lr=5.41e-04 tps=16162 mem=6.2G lam=0.1772888901953896 Gm=None +[s_only s0] step 2760/12000 loss=5.2703 gn=0.66 lr=5.40e-04 tps=16163 mem=6.2G lam=0.1778625544781486 Gm=None +[s_only s0] step 2780/12000 loss=5.2451 gn=0.68 lr=5.39e-04 tps=16164 mem=6.2G lam=0.1784231293325623 Gm=None +[s_only s0] step 2800/12000 loss=5.0151 gn=0.64 lr=5.38e-04 tps=16166 mem=6.2G lam=0.17898354027420282 Gm=None +[s_only s0] step 2820/12000 loss=5.2810 gn=0.75 lr=5.37e-04 tps=16167 mem=6.2G lam=0.179576115993162 Gm=None +[s_only s0] step 2840/12000 loss=5.0248 gn=0.74 lr=5.36e-04 tps=16168 mem=6.2G lam=0.18015736465652785 Gm=None +[s_only s0] step 2860/12000 loss=5.0505 gn=0.67 lr=5.35e-04 tps=16169 mem=6.2G lam=0.18060852338870367 Gm=None +[s_only s0] step 2880/12000 loss=5.1017 gn=0.71 lr=5.34e-04 tps=16170 mem=6.2G lam=0.18111553757141033 Gm=None +[s_only s0] step 2900/12000 loss=4.9150 gn=0.67 lr=5.33e-04 tps=16171 mem=6.2G lam=0.18161660339683294 Gm=None +[s_only s0] step 2920/12000 loss=5.0558 gn=0.72 lr=5.32e-04 tps=16172 mem=6.2G lam=0.18219109748800597 Gm=None +[s_only s0] step 2940/12000 loss=5.1108 gn=0.74 lr=5.31e-04 tps=16174 mem=6.2G lam=0.1827462576329708 Gm=None +[s_only s0] step 2960/12000 loss=4.9618 gn=0.73 lr=5.30e-04 tps=16176 mem=6.2G lam=0.18327279606213173 Gm=None +[s_only s0] step 2980/12000 loss=5.1028 gn=0.66 lr=5.29e-04 tps=16177 mem=6.2G lam=0.1837539408976833 Gm=None +[s_only s0] step 3000/12000 loss=5.0964 gn=0.72 lr=5.28e-04 tps=16179 mem=6.2G lam=0.18421449698507786 Gm=None +[s_only s0] EVAL step 3000: val_loss=5.0580 val_ppl=157.27 +[s_only s0] step 3020/12000 loss=5.0648 gn=0.70 lr=5.27e-04 tps=16140 mem=6.2G lam=0.1847746204584837 Gm=None +[s_only s0] step 3040/12000 loss=5.1722 gn=0.68 lr=5.26e-04 tps=16142 mem=6.2G lam=0.18537040334194899 Gm=None +[s_only s0] step 3060/12000 loss=4.9138 gn=0.64 lr=5.25e-04 tps=16143 mem=6.2G lam=0.18592176058640084 Gm=None +[s_only s0] step 3080/12000 loss=4.9928 gn=0.74 lr=5.24e-04 tps=16144 mem=6.2G lam=0.18644982296973467 Gm=None +[s_only s0] step 3100/12000 loss=4.9182 gn=0.76 lr=5.23e-04 tps=16145 mem=6.2G lam=0.1869327879200379 Gm=None +[s_only s0] step 3120/12000 loss=4.8905 gn=0.68 lr=5.22e-04 tps=16147 mem=6.2G lam=0.1874176071335872 Gm=None +[s_only s0] step 3140/12000 loss=5.0580 gn=0.71 lr=5.21e-04 tps=16148 mem=6.2G lam=0.1879212474450469 Gm=None +[s_only s0] step 3160/12000 loss=4.8619 gn=0.74 lr=5.20e-04 tps=16149 mem=6.2G lam=0.18845899713536104 Gm=None +[s_only s0] step 3180/12000 loss=4.8853 gn=0.76 lr=5.19e-04 tps=16151 mem=6.2G lam=0.18896493346740803 Gm=None +[s_only s0] step 3200/12000 loss=4.8330 gn=0.68 lr=5.18e-04 tps=16152 mem=6.2G lam=0.18949866201728582 Gm=None +[s_only s0] step 3220/12000 loss=4.8118 gn=0.74 lr=5.17e-04 tps=16153 mem=6.2G lam=0.19001475628465414 Gm=None +[s_only s0] step 3240/12000 loss=4.8837 gn=0.74 lr=5.16e-04 tps=16154 mem=6.2G lam=0.19054187182337046 Gm=None +[s_only s0] step 3260/12000 loss=5.0045 gn=0.78 lr=5.15e-04 tps=16155 mem=6.2G lam=0.19109232847889265 Gm=None +[s_only s0] step 3280/12000 loss=4.8702 gn=0.68 lr=5.14e-04 tps=16156 mem=6.2G lam=0.1916848849505186 Gm=None +[s_only s0] step 3300/12000 loss=5.0483 gn=0.72 lr=5.13e-04 tps=16157 mem=6.2G lam=0.19218628481030464 Gm=None +[s_only s0] step 3320/12000 loss=4.9126 gn=0.69 lr=5.12e-04 tps=16159 mem=6.2G lam=0.19261476366470257 Gm=None +[s_only s0] step 3340/12000 loss=4.9914 gn=0.73 lr=5.11e-04 tps=16159 mem=6.2G lam=0.19318069579700628 Gm=None +[s_only s0] step 3360/12000 loss=4.7873 gn=0.73 lr=5.10e-04 tps=16160 mem=6.2G lam=0.1936395252123475 Gm=None +[s_only s0] step 3380/12000 loss=4.9907 gn=0.92 lr=5.09e-04 tps=16161 mem=6.2G lam=0.19413988385349512 Gm=None +[s_only s0] step 3400/12000 loss=4.9975 gn=0.75 lr=5.08e-04 tps=16162 mem=6.2G lam=0.19463006996860108 Gm=None +[s_only s0] step 3420/12000 loss=4.8166 gn=0.69 lr=5.07e-04 tps=16163 mem=6.2G lam=0.19516540008286634 Gm=None +[s_only s0] step 3440/12000 loss=4.9386 gn=0.72 lr=5.06e-04 tps=16165 mem=6.2G lam=0.19572093648215136 Gm=None +[s_only s0] step 3460/12000 loss=4.8379 gn=0.77 lr=5.05e-04 tps=16166 mem=6.2G lam=0.19626024613777796 Gm=None +[s_only s0] step 3480/12000 loss=4.8089 gn=0.71 lr=5.03e-04 tps=16167 mem=6.2G lam=0.1968085989356041 Gm=None +[s_only s0] step 3500/12000 loss=4.9759 gn=0.81 lr=5.02e-04 tps=16169 mem=6.2G lam=0.1974132585649689 Gm=None +[s_only s0] EVAL step 3500: val_loss=4.9419 val_ppl=140.03 +[s_only s0] step 3520/12000 loss=4.9489 gn=0.74 lr=5.01e-04 tps=16136 mem=6.2G lam=0.1979413339868188 Gm=None +[s_only s0] step 3540/12000 loss=4.8186 gn=0.69 lr=5.00e-04 tps=16137 mem=6.2G lam=0.1984906168654561 Gm=None +[s_only s0] step 3560/12000 loss=4.8558 gn=0.72 lr=4.99e-04 tps=16138 mem=6.2G lam=0.1990638313194116 Gm=None +[s_only s0] step 3580/12000 loss=4.9269 gn=0.72 lr=4.98e-04 tps=16140 mem=6.2G lam=0.19960614200681448 Gm=None +[s_only s0] step 3600/12000 loss=4.7922 gn=0.71 lr=4.97e-04 tps=16141 mem=6.2G lam=0.20009012271960577 Gm=None +[s_only s0] step 3620/12000 loss=4.8797 gn=0.73 lr=4.96e-04 tps=16142 mem=6.2G lam=0.2006464945152402 Gm=None +[s_only s0] step 3640/12000 loss=4.9337 gn=0.75 lr=4.94e-04 tps=16143 mem=6.2G lam=0.2011732952669263 Gm=None +[s_only s0] step 3660/12000 loss=4.8105 gn=0.69 lr=4.93e-04 tps=16143 mem=6.2G lam=0.2016546931117773 Gm=None +[s_only s0] step 3680/12000 loss=4.9483 gn=0.72 lr=4.92e-04 tps=16144 mem=6.2G lam=0.20222297900666794 Gm=None +[s_only s0] step 3700/12000 loss=4.7869 gn=0.70 lr=4.91e-04 tps=16145 mem=6.2G lam=0.20278107188642025 Gm=None +[s_only s0] step 3720/12000 loss=4.9360 gn=0.90 lr=4.90e-04 tps=16147 mem=6.2G lam=0.20334272366017103 Gm=None +[s_only s0] step 3740/12000 loss=4.8879 gn=0.77 lr=4.89e-04 tps=16148 mem=6.2G lam=0.20382423481593528 Gm=None +[s_only s0] step 3760/12000 loss=4.9460 gn=0.73 lr=4.88e-04 tps=16149 mem=6.2G lam=0.20428088183204332 Gm=None +[s_only s0] step 3780/12000 loss=4.9107 gn=0.72 lr=4.86e-04 tps=16150 mem=6.2G lam=0.20476394860694805 Gm=None +[s_only s0] step 3800/12000 loss=4.9090 gn=0.76 lr=4.85e-04 tps=16151 mem=6.2G lam=0.20527261961251497 Gm=None +[s_only s0] step 3820/12000 loss=4.6941 gn=0.91 lr=4.84e-04 tps=16152 mem=6.2G lam=0.20580366626381874 Gm=None +[s_only s0] step 3840/12000 loss=4.9655 gn=0.73 lr=4.83e-04 tps=16153 mem=6.2G lam=0.20623450446873903 Gm=None +[s_only s0] step 3860/12000 loss=4.8627 gn=0.83 lr=4.82e-04 tps=16154 mem=6.2G lam=0.20679831225425005 Gm=None +[s_only s0] step 3880/12000 loss=4.7749 gn=0.75 lr=4.80e-04 tps=16155 mem=6.2G lam=0.20726905297487974 Gm=None +[s_only s0] step 3900/12000 loss=4.7193 gn=0.74 lr=4.79e-04 tps=16156 mem=6.2G lam=0.20780920734008154 Gm=None +[s_only s0] step 3920/12000 loss=4.8890 gn=1.22 lr=4.78e-04 tps=16157 mem=6.2G lam=0.2083284022907416 Gm=None +[s_only s0] step 3940/12000 loss=4.6701 gn=0.71 lr=4.77e-04 tps=16159 mem=6.2G lam=0.20892691984772682 Gm=None +[s_only s0] step 3960/12000 loss=4.7925 gn=0.75 lr=4.76e-04 tps=16160 mem=6.2G lam=0.20948684805383286 Gm=None +[s_only s0] step 3980/12000 loss=4.6761 gn=0.68 lr=4.74e-04 tps=16161 mem=6.2G lam=0.20997001510113478 Gm=None +[s_only s0] step 4000/12000 loss=4.7757 gn=0.75 lr=4.73e-04 tps=16162 mem=6.2G lam=0.210554842526714 Gm=None +[s_only s0] EVAL step 4000: val_loss=4.8994 val_ppl=134.20 +[s_only s0] step 4020/12000 loss=4.9498 gn=0.74 lr=4.72e-04 tps=16133 mem=6.2G lam=0.21111888128022352 Gm=None +[s_only s0] step 4040/12000 loss=4.9222 gn=0.69 lr=4.71e-04 tps=16134 mem=6.2G lam=0.2115729625026385 Gm=None +[s_only s0] step 4060/12000 loss=4.7867 gn=0.76 lr=4.70e-04 tps=16135 mem=6.2G lam=0.21207218430936337 Gm=None +[s_only s0] step 4080/12000 loss=4.8019 gn=0.75 lr=4.68e-04 tps=16135 mem=6.2G lam=0.21255518527080616 Gm=None +[s_only s0] step 4100/12000 loss=4.7154 gn=0.79 lr=4.67e-04 tps=16136 mem=6.2G lam=0.2130913178746899 Gm=None +[s_only s0] step 4120/12000 loss=4.6537 gn=0.96 lr=4.66e-04 tps=16137 mem=6.2G lam=0.21349258441478014 Gm=None +[s_only s0] step 4140/12000 loss=4.7530 gn=0.74 lr=4.65e-04 tps=16138 mem=6.2G lam=0.21397934729854265 Gm=None +[s_only s0] step 4160/12000 loss=4.7568 gn=0.82 lr=4.63e-04 tps=16139 mem=6.2G lam=0.21449112767974535 Gm=None +[s_only s0] step 4180/12000 loss=4.9338 gn=0.82 lr=4.62e-04 tps=16140 mem=6.2G lam=0.2150273428608974 Gm=None +[s_only s0] step 4200/12000 loss=4.7576 gn=0.80 lr=4.61e-04 tps=16141 mem=6.2G lam=0.21556786571939787 Gm=None +[s_only s0] step 4220/12000 loss=4.8228 gn=0.76 lr=4.60e-04 tps=16142 mem=6.2G lam=0.21601732540875673 Gm=None +[s_only s0] step 4240/12000 loss=4.8232 gn=0.73 lr=4.58e-04 tps=16143 mem=6.2G lam=0.21646190155297518 Gm=None +[s_only s0] step 4260/12000 loss=4.8497 gn=0.73 lr=4.57e-04 tps=16144 mem=6.2G lam=0.21695900615304708 Gm=None +[s_only s0] step 4280/12000 loss=4.8860 gn=0.84 lr=4.56e-04 tps=16144 mem=6.2G lam=0.21744878496974707 Gm=None +[s_only s0] step 4300/12000 loss=4.8456 gn=0.79 lr=4.54e-04 tps=16145 mem=6.2G lam=0.21787717348585525 Gm=None +[s_only s0] step 4320/12000 loss=4.7048 gn=0.79 lr=4.53e-04 tps=16146 mem=6.2G lam=0.21842833422124386 Gm=None +[s_only s0] step 4340/12000 loss=4.7380 gn=0.86 lr=4.52e-04 tps=16147 mem=6.2G lam=0.218914441143473 Gm=None +[s_only s0] step 4360/12000 loss=4.8701 gn=0.75 lr=4.51e-04 tps=16148 mem=6.2G lam=0.21934434367964664 Gm=None +[s_only s0] step 4380/12000 loss=4.8615 gn=0.77 lr=4.49e-04 tps=16149 mem=6.2G lam=0.21977617622663578 Gm=None +[s_only s0] step 4400/12000 loss=4.6141 gn=0.78 lr=4.48e-04 tps=16150 mem=6.2G lam=0.22030633222311735 Gm=None +[s_only s0] step 4420/12000 loss=4.9030 gn=0.79 lr=4.47e-04 tps=16151 mem=6.2G lam=0.22080085054039955 Gm=None +[s_only s0] step 4440/12000 loss=4.7176 gn=0.86 lr=4.45e-04 tps=16152 mem=6.2G lam=0.22125315200537443 Gm=None +[s_only s0] step 4460/12000 loss=4.7352 gn=0.77 lr=4.44e-04 tps=16153 mem=6.2G lam=0.221769320157667 Gm=None +[s_only s0] step 4480/12000 loss=4.7158 gn=0.78 lr=4.43e-04 tps=16155 mem=6.2G lam=0.22233792239179215 Gm=None +[s_only s0] step 4500/12000 loss=4.7371 gn=0.84 lr=4.42e-04 tps=16155 mem=6.2G lam=0.2228744880606731 Gm=None +[s_only s0] EVAL step 4500: val_loss=4.8303 val_ppl=125.25 +[s_only s0] step 4520/12000 loss=4.7304 gn=0.77 lr=4.40e-04 tps=16130 mem=6.2G lam=0.2233520494773984 Gm=None +[s_only s0] step 4540/12000 loss=4.7723 gn=0.79 lr=4.39e-04 tps=16130 mem=6.2G lam=0.22376834383855262 Gm=None +[s_only s0] step 4560/12000 loss=4.7712 gn=0.76 lr=4.38e-04 tps=16131 mem=6.2G lam=0.2242610106865565 Gm=None +[s_only s0] step 4580/12000 loss=4.7315 gn=0.79 lr=4.36e-04 tps=16132 mem=6.2G lam=0.2247393016392986 Gm=None +[s_only s0] step 4600/12000 loss=4.7387 gn=0.79 lr=4.35e-04 tps=16133 mem=6.2G lam=0.2251900608340899 Gm=None +[s_only s0] step 4620/12000 loss=4.7496 gn=0.82 lr=4.34e-04 tps=16133 mem=6.2G lam=0.22558404753605524 Gm=None +[s_only s0] step 4640/12000 loss=4.8194 gn=0.82 lr=4.32e-04 tps=16134 mem=6.2G lam=0.22604842856526375 Gm=None +[s_only s0] step 4660/12000 loss=4.6933 gn=0.75 lr=4.31e-04 tps=16135 mem=6.2G lam=0.22652095276862383 Gm=None +[s_only s0] step 4680/12000 loss=4.7540 gn=0.81 lr=4.30e-04 tps=16136 mem=6.2G lam=0.22703739690283933 Gm=None +[s_only s0] step 4700/12000 loss=4.6995 gn=0.86 lr=4.28e-04 tps=16137 mem=6.2G lam=0.22749096093078455 Gm=None +[s_only s0] step 4720/12000 loss=4.7401 gn=0.80 lr=4.27e-04 tps=16138 mem=6.2G lam=0.22802438121289015 Gm=None +[s_only s0] step 4740/12000 loss=4.6182 gn=0.82 lr=4.26e-04 tps=16139 mem=6.2G lam=0.22852983387808004 Gm=None +[s_only s0] step 4760/12000 loss=4.7311 gn=0.87 lr=4.24e-04 tps=16139 mem=6.2G lam=0.228978686966002 Gm=None +[s_only s0] step 4780/12000 loss=4.7596 gn=0.75 lr=4.23e-04 tps=16140 mem=6.2G lam=0.2294976394623518 Gm=None +[s_only s0] step 4800/12000 loss=4.8197 gn=0.77 lr=4.22e-04 tps=16141 mem=6.2G lam=0.2300543443610271 Gm=None +[s_only s0] step 4820/12000 loss=4.7158 gn=0.85 lr=4.20e-04 tps=16142 mem=6.2G lam=0.23049756636222204 Gm=None +[s_only s0] step 4840/12000 loss=4.4817 gn=0.82 lr=4.19e-04 tps=16143 mem=6.2G lam=0.23090325699498257 Gm=None +[s_only s0] step 4860/12000 loss=4.7350 gn=0.79 lr=4.18e-04 tps=16144 mem=6.2G lam=0.23133533913642168 Gm=None +[s_only s0] step 4880/12000 loss=4.7213 gn=0.77 lr=4.16e-04 tps=16145 mem=6.2G lam=0.23179953824728727 Gm=None +[s_only s0] step 4900/12000 loss=4.5495 gn=0.74 lr=4.15e-04 tps=16146 mem=6.2G lam=0.23226681165397167 Gm=None +[s_only s0] step 4920/12000 loss=4.7877 gn=0.83 lr=4.13e-04 tps=16146 mem=6.2G lam=0.23271866360058388 Gm=None +[s_only s0] step 4940/12000 loss=4.6567 gn=0.75 lr=4.12e-04 tps=16147 mem=6.2G lam=0.23308005277067423 Gm=None +[s_only s0] step 4960/12000 loss=4.7104 gn=0.81 lr=4.11e-04 tps=16148 mem=6.2G lam=0.2335965841387709 Gm=None +[s_only s0] step 4980/12000 loss=4.5755 gn=0.77 lr=4.09e-04 tps=16149 mem=6.2G lam=0.23406649194657803 Gm=None +[s_only s0] step 5000/12000 loss=4.7850 gn=1.02 lr=4.08e-04 tps=16149 mem=6.2G lam=0.23458139908810458 Gm=None +[s_only s0] EVAL step 5000: val_loss=4.7904 val_ppl=120.35 +[s_only s0] step 5020/12000 loss=4.6639 gn=0.89 lr=4.07e-04 tps=16126 mem=6.2G lam=0.23510567812869945 Gm=None +[s_only s0] step 5040/12000 loss=4.6718 gn=0.83 lr=4.05e-04 tps=16127 mem=6.2G lam=0.23558136355131865 Gm=None +[s_only s0] step 5060/12000 loss=4.5665 gn=0.86 lr=4.04e-04 tps=16128 mem=6.2G lam=0.23603251234938702 Gm=None +[s_only s0] step 5080/12000 loss=4.6749 gn=0.80 lr=4.02e-04 tps=16129 mem=6.2G lam=0.2364618411908547 Gm=None +[s_only s0] step 5100/12000 loss=4.6960 gn=0.77 lr=4.01e-04 tps=16130 mem=6.2G lam=0.23687253861377636 Gm=None +[s_only s0] step 5120/12000 loss=4.6173 gn=0.80 lr=4.00e-04 tps=16131 mem=6.2G lam=0.2373960828408599 Gm=None +[s_only s0] step 5140/12000 loss=4.7165 gn=0.78 lr=3.98e-04 tps=16132 mem=6.2G lam=0.23792181257158518 Gm=None +[s_only s0] step 5160/12000 loss=4.5660 gn=0.80 lr=3.97e-04 tps=16132 mem=6.2G lam=0.23831176726768413 Gm=None +[s_only s0] step 5180/12000 loss=4.5946 gn=0.82 lr=3.95e-04 tps=16133 mem=6.2G lam=0.23875507029394308 Gm=None +[s_only s0] step 5200/12000 loss=4.5337 gn=0.83 lr=3.94e-04 tps=16134 mem=6.2G lam=0.23933975491672754 Gm=None +[s_only s0] step 5220/12000 loss=4.6324 gn=0.76 lr=3.93e-04 tps=16135 mem=6.2G lam=0.23975173570215702 Gm=None +[s_only s0] step 5240/12000 loss=4.6159 gn=0.89 lr=3.91e-04 tps=16136 mem=6.2G lam=0.24012123855451742 Gm=None +[s_only s0] step 5260/12000 loss=4.5719 gn=0.75 lr=3.90e-04 tps=16137 mem=6.2G lam=0.24047718166063228 Gm=None +[s_only s0] step 5280/12000 loss=4.6066 gn=0.84 lr=3.88e-04 tps=16138 mem=6.2G lam=0.24095913426329693 Gm=None +[s_only s0] step 5300/12000 loss=4.7285 gn=0.82 lr=3.87e-04 tps=16139 mem=6.2G lam=0.24136390319714943 Gm=None +[s_only s0] step 5320/12000 loss=4.6019 gn=0.80 lr=3.86e-04 tps=16140 mem=6.2G lam=0.2418892669181029 Gm=None +[s_only s0] step 5340/12000 loss=4.6715 gn=0.83 lr=3.84e-04 tps=16141 mem=6.2G lam=0.24226428413142762 Gm=None +[s_only s0] step 5360/12000 loss=4.5251 gn=0.84 lr=3.83e-04 tps=16142 mem=6.2G lam=0.24271236546337605 Gm=None +[s_only s0] step 5380/12000 loss=4.7984 gn=0.81 lr=3.81e-04 tps=16143 mem=6.2G lam=0.24320416369785866 Gm=None +[s_only s0] step 5400/12000 loss=4.4953 gn=0.80 lr=3.80e-04 tps=16145 mem=6.2G lam=0.24369302547226349 Gm=None +[s_only s0] step 5420/12000 loss=4.6752 gn=0.89 lr=3.79e-04 tps=16146 mem=6.2G lam=0.2441459676871697 Gm=None +[s_only s0] step 5440/12000 loss=4.5292 gn=0.86 lr=3.77e-04 tps=16146 mem=6.2G lam=0.24451540472606817 Gm=None +[s_only s0] step 5460/12000 loss=4.6754 gn=0.88 lr=3.76e-04 tps=16148 mem=6.2G lam=0.24500690152247748 Gm=None +[s_only s0] step 5480/12000 loss=4.5606 gn=0.79 lr=3.74e-04 tps=16148 mem=6.2G lam=0.2454573983947436 Gm=None +[s_only s0] step 5500/12000 loss=4.6216 gn=0.77 lr=3.73e-04 tps=16150 mem=6.2G lam=0.2459756676107645 Gm=None +[s_only s0] EVAL step 5500: val_loss=4.7094 val_ppl=110.98 +[s_only s0] step 5520/12000 loss=4.4479 gn=0.81 lr=3.72e-04 tps=16129 mem=6.2G lam=0.24639331921935081 Gm=None +[s_only s0] step 5540/12000 loss=4.6668 gn=0.89 lr=3.70e-04 tps=16130 mem=6.2G lam=0.24690922566999993 Gm=None +[s_only s0] step 5560/12000 loss=4.6960 gn=0.80 lr=3.69e-04 tps=16130 mem=6.2G lam=0.24734488502144814 Gm=None +[s_only s0] step 5580/12000 loss=4.7398 gn=0.83 lr=3.67e-04 tps=16131 mem=6.2G lam=0.24773935228586197 Gm=None +[s_only s0] step 5600/12000 loss=4.5539 gn=0.78 lr=3.66e-04 tps=16132 mem=6.2G lam=0.2482747072353959 Gm=None +[s_only s0] step 5620/12000 loss=4.6380 gn=0.91 lr=3.64e-04 tps=16133 mem=6.2G lam=0.24876787109921375 Gm=None +[s_only s0] step 5640/12000 loss=4.7254 gn=0.83 lr=3.63e-04 tps=16134 mem=6.2G lam=0.2493281296143929 Gm=None +[s_only s0] step 5660/12000 loss=4.5169 gn=0.80 lr=3.62e-04 tps=16135 mem=6.2G lam=0.2497758191699783 Gm=None +[s_only s0] step 5680/12000 loss=4.5861 gn=0.80 lr=3.60e-04 tps=16135 mem=6.2G lam=0.2501976250981291 Gm=None +[s_only s0] step 5700/12000 loss=4.7191 gn=0.95 lr=3.59e-04 tps=16136 mem=6.2G lam=0.25061630364507437 Gm=None +[s_only s0] step 5720/12000 loss=4.5923 gn=0.85 lr=3.57e-04 tps=16137 mem=6.2G lam=0.25099703048666316 Gm=None +[s_only s0] step 5740/12000 loss=4.7264 gn=0.87 lr=3.56e-04 tps=16138 mem=6.2G lam=0.2514810236170888 Gm=None +[s_only s0] step 5760/12000 loss=4.5914 gn=0.85 lr=3.54e-04 tps=16138 mem=6.2G lam=0.2518500291431944 Gm=None +[s_only s0] step 5780/12000 loss=4.6367 gn=0.84 lr=3.53e-04 tps=16139 mem=6.2G lam=0.25223508073637885 Gm=None +[s_only s0] step 5800/12000 loss=4.4962 gn=0.82 lr=3.52e-04 tps=16140 mem=6.2G lam=0.25256541060904664 Gm=None +[s_only s0] step 5820/12000 loss=4.7429 gn=0.85 lr=3.50e-04 tps=16141 mem=6.2G lam=0.2530466482664148 Gm=None +[s_only s0] step 5840/12000 loss=4.5461 gn=0.80 lr=3.49e-04 tps=16142 mem=6.2G lam=0.25343132484704256 Gm=None +[s_only s0] step 5860/12000 loss=4.6343 gn=0.84 lr=3.47e-04 tps=16143 mem=6.2G lam=0.2538413793469469 Gm=None +[s_only s0] step 5880/12000 loss=4.6913 gn=0.91 lr=3.46e-04 tps=16143 mem=6.2G lam=0.25414797974129516 Gm=None +[s_only s0] step 5900/12000 loss=4.5938 gn=0.86 lr=3.44e-04 tps=16144 mem=6.2G lam=0.25460825363794964 Gm=None +[s_only s0] step 5920/12000 loss=4.5824 gn=0.83 lr=3.43e-04 tps=16145 mem=6.2G lam=0.2550163685033719 Gm=None +[s_only s0] step 5940/12000 loss=4.4968 gn=0.82 lr=3.41e-04 tps=16145 mem=6.2G lam=0.2553661344572902 Gm=None +[s_only s0] step 5960/12000 loss=4.5720 gn=0.85 lr=3.40e-04 tps=16146 mem=6.2G lam=0.25568556661407155 Gm=None +[s_only s0] step 5980/12000 loss=4.8083 gn=0.83 lr=3.39e-04 tps=16147 mem=6.2G lam=0.25609944729755324 Gm=None +[s_only s0] step 6000/12000 loss=4.5787 gn=0.80 lr=3.37e-04 tps=16148 mem=6.2G lam=0.25658431835472584 Gm=None +[s_only s0] EVAL step 6000: val_loss=4.6656 val_ppl=106.23 +[s_only s0] step 6020/12000 loss=4.5143 gn=0.78 lr=3.36e-04 tps=16129 mem=6.2G lam=0.2569734662150343 Gm=None +[s_only s0] step 6040/12000 loss=4.5056 gn=0.81 lr=3.34e-04 tps=16130 mem=6.2G lam=0.25745457131415606 Gm=None +[s_only s0] step 6060/12000 loss=4.6819 gn=0.84 lr=3.33e-04 tps=16131 mem=6.2G lam=0.25780809484422207 Gm=None +[s_only s0] step 6080/12000 loss=4.6180 gn=0.83 lr=3.31e-04 tps=16131 mem=6.2G lam=0.2581539597983162 Gm=None +[s_only s0] step 6100/12000 loss=4.5484 gn=0.88 lr=3.30e-04 tps=16132 mem=6.2G lam=0.258528473166128 Gm=None +[s_only s0] step 6120/12000 loss=4.6886 gn=0.89 lr=3.29e-04 tps=16133 mem=6.2G lam=0.25887297187000513 Gm=None +[s_only s0] step 6140/12000 loss=4.5905 gn=0.90 lr=3.27e-04 tps=16134 mem=6.2G lam=0.259189138499399 Gm=None +[s_only s0] step 6160/12000 loss=4.4846 gn=0.86 lr=3.26e-04 tps=16135 mem=6.2G lam=0.25953735255946714 Gm=None +[s_only s0] step 6180/12000 loss=4.6429 gn=0.86 lr=3.24e-04 tps=16136 mem=6.2G lam=0.2600186796238025 Gm=None +[s_only s0] step 6200/12000 loss=4.5664 gn=0.92 lr=3.23e-04 tps=16136 mem=6.2G lam=0.26035670408358175 Gm=None +[s_only s0] step 6220/12000 loss=4.5456 gn=0.87 lr=3.21e-04 tps=16137 mem=6.2G lam=0.2608103146776557 Gm=None +[s_only s0] step 6240/12000 loss=4.4753 gn=0.80 lr=3.20e-04 tps=16138 mem=6.2G lam=0.26122718894233304 Gm=None +[s_only s0] step 6260/12000 loss=4.6133 gn=0.93 lr=3.19e-04 tps=16139 mem=6.2G lam=0.2615890757491191 Gm=None +[s_only s0] step 6280/12000 loss=4.5746 gn=0.85 lr=3.17e-04 tps=16139 mem=6.2G lam=0.2619721883287032 Gm=None +[s_only s0] step 6300/12000 loss=4.5197 gn=0.90 lr=3.16e-04 tps=16140 mem=6.2G lam=0.2622793447226286 Gm=None +[s_only s0] step 6320/12000 loss=4.4728 gn=0.94 lr=3.14e-04 tps=16141 mem=6.2G lam=0.26265330457439023 Gm=None +[s_only s0] step 6340/12000 loss=4.5646 gn=0.84 lr=3.13e-04 tps=16142 mem=6.2G lam=0.262972737972935 Gm=None +[s_only s0] step 6360/12000 loss=4.5673 gn=0.81 lr=3.11e-04 tps=16143 mem=6.2G lam=0.26328661230703193 Gm=None +[s_only s0] step 6380/12000 loss=4.5938 gn=0.85 lr=3.10e-04 tps=16144 mem=6.2G lam=0.2636836841702461 Gm=None +[s_only s0] step 6400/12000 loss=4.4460 gn=0.89 lr=3.08e-04 tps=16144 mem=6.2G lam=0.26405732488880557 Gm=None +[s_only s0] step 6420/12000 loss=4.6228 gn=0.90 lr=3.07e-04 tps=16145 mem=6.2G lam=0.2644826766724388 Gm=None +[s_only s0] step 6440/12000 loss=4.6558 gn=0.87 lr=3.06e-04 tps=16146 mem=6.2G lam=0.2648964145531257 Gm=None +[s_only s0] step 6460/12000 loss=4.4960 gn=0.89 lr=3.04e-04 tps=16146 mem=6.2G lam=0.26526647216329974 Gm=None +[s_only s0] step 6480/12000 loss=4.5259 gn=0.84 lr=3.03e-04 tps=16147 mem=6.2G lam=0.26567857588330906 Gm=None +[s_only s0] step 6500/12000 loss=4.5260 gn=0.87 lr=3.01e-04 tps=16148 mem=6.2G lam=0.2660436149065693 Gm=None +[s_only s0] EVAL step 6500: val_loss=4.6381 val_ppl=103.35 +[s_only s0] step 6520/12000 loss=4.6883 gn=0.86 lr=3.00e-04 tps=16130 mem=6.2G lam=0.26638095329205197 Gm=None +[s_only s0] step 6540/12000 loss=4.4947 gn=0.84 lr=2.98e-04 tps=16131 mem=6.2G lam=0.26664580839375657 Gm=None +[s_only s0] step 6560/12000 loss=4.6959 gn=0.90 lr=2.97e-04 tps=16133 mem=6.2G lam=0.2670180688922604 Gm=None +[s_only s0] step 6580/12000 loss=4.6391 gn=0.93 lr=2.96e-04 tps=16133 mem=6.2G lam=0.26743728170792264 Gm=None +[s_only s0] step 6600/12000 loss=4.3719 gn=0.96 lr=2.94e-04 tps=16134 mem=6.2G lam=0.2677856758236885 Gm=None +[s_only s0] step 6620/12000 loss=4.3928 gn=0.90 lr=2.93e-04 tps=16135 mem=6.2G lam=0.268091787584126 Gm=None +[s_only s0] step 6640/12000 loss=4.6901 gn=0.86 lr=2.91e-04 tps=16135 mem=6.2G lam=0.26836703221003216 Gm=None +[s_only s0] step 6660/12000 loss=4.6629 gn=0.83 lr=2.90e-04 tps=16136 mem=6.2G lam=0.268718325222532 Gm=None +[s_only s0] step 6680/12000 loss=4.5314 gn=0.87 lr=2.88e-04 tps=16137 mem=6.2G lam=0.2690673191100359 Gm=None +[s_only s0] step 6700/12000 loss=4.5002 gn=0.93 lr=2.87e-04 tps=16137 mem=6.2G lam=0.26939949424316484 Gm=None +[s_only s0] step 6720/12000 loss=4.5894 gn=0.89 lr=2.86e-04 tps=16137 mem=6.2G lam=0.26969714121272165 Gm=None +[s_only s0] step 6740/12000 loss=4.4840 gn=0.86 lr=2.84e-04 tps=16136 mem=6.2G lam=0.270036765995125 Gm=None +[s_only s0] step 6760/12000 loss=4.5024 gn=0.91 lr=2.83e-04 tps=16135 mem=6.2G lam=0.2703805410613616 Gm=None +[s_only s0] step 6780/12000 loss=4.5016 gn=0.97 lr=2.81e-04 tps=16134 mem=6.2G lam=0.27067977966119844 Gm=None +[s_only s0] step 6800/12000 loss=4.4354 gn=0.85 lr=2.80e-04 tps=16132 mem=6.2G lam=0.2710682429994146 Gm=None +[s_only s0] step 6820/12000 loss=4.4436 gn=0.87 lr=2.79e-04 tps=16131 mem=6.2G lam=0.2714441018179059 Gm=None +[s_only s0] step 6840/12000 loss=4.3696 gn=0.85 lr=2.77e-04 tps=16130 mem=6.2G lam=0.27181279721359414 Gm=None +[s_only s0] step 6860/12000 loss=4.5705 gn=0.90 lr=2.76e-04 tps=16128 mem=6.2G lam=0.27209067686150473 Gm=None +[s_only s0] step 6880/12000 loss=4.5014 gn=0.85 lr=2.74e-04 tps=16127 mem=6.2G lam=0.27237822860479355 Gm=None +[s_only s0] step 6900/12000 loss=4.6439 gn=0.91 lr=2.73e-04 tps=16127 mem=6.2G lam=0.2726485515013337 Gm=None +[s_only s0] step 6920/12000 loss=4.5946 gn=0.96 lr=2.72e-04 tps=16126 mem=6.2G lam=0.2729827808216214 Gm=None +[s_only s0] step 6940/12000 loss=4.4160 gn=0.82 lr=2.70e-04 tps=16126 mem=6.2G lam=0.2733424073085189 Gm=None +[s_only s0] step 6960/12000 loss=4.5359 gn=0.96 lr=2.69e-04 tps=16125 mem=6.2G lam=0.2736334915583332 Gm=None +[s_only s0] step 6980/12000 loss=4.3588 gn=0.86 lr=2.67e-04 tps=16125 mem=6.2G lam=0.27394513972103596 Gm=None +[s_only s0] step 7000/12000 loss=4.6726 gn=0.96 lr=2.66e-04 tps=16124 mem=6.2G lam=0.27427069656550884 Gm=None +[s_only s0] EVAL step 7000: val_loss=4.5847 val_ppl=97.98 +[s_only s0] step 7020/12000 loss=4.4502 gn=0.88 lr=2.65e-04 tps=16107 mem=6.2G lam=0.2746367147192359 Gm=None +[s_only s0] step 7040/12000 loss=4.4897 gn=0.88 lr=2.63e-04 tps=16108 mem=6.2G lam=0.2749103515719374 Gm=None +[s_only s0] step 7060/12000 loss=4.5623 gn=0.96 lr=2.62e-04 tps=16109 mem=6.2G lam=0.2752482444047928 Gm=None +[s_only s0] step 7080/12000 loss=4.5895 gn=0.94 lr=2.60e-04 tps=16110 mem=6.2G lam=0.27555989536146325 Gm=None +[s_only s0] step 7100/12000 loss=4.5361 gn=0.91 lr=2.59e-04 tps=16110 mem=6.2G lam=0.27586640138179064 Gm=None +[s_only s0] step 7120/12000 loss=4.7432 gn=0.92 lr=2.58e-04 tps=16111 mem=6.2G lam=0.276191510570546 Gm=None +[s_only s0] step 7140/12000 loss=4.5188 gn=0.99 lr=2.56e-04 tps=16112 mem=6.2G lam=0.276406048797071 Gm=None +[s_only s0] step 7160/12000 loss=4.4620 gn=0.89 lr=2.55e-04 tps=16113 mem=6.2G lam=0.2767351033786933 Gm=None +[s_only s0] step 7180/12000 loss=4.5916 gn=0.87 lr=2.53e-04 tps=16114 mem=6.2G lam=0.2769890542452534 Gm=None +[s_only s0] step 7200/12000 loss=4.6348 gn=0.95 lr=2.52e-04 tps=16115 mem=6.2G lam=0.27736977611978847 Gm=None +[s_only s0] step 7220/12000 loss=4.5895 gn=0.93 lr=2.51e-04 tps=16116 mem=6.2G lam=0.27765374816954136 Gm=None +[s_only s0] step 7240/12000 loss=4.3930 gn=0.91 lr=2.49e-04 tps=16117 mem=6.2G lam=0.27793166041374207 Gm=None +[s_only s0] step 7260/12000 loss=4.4409 gn=0.94 lr=2.48e-04 tps=16118 mem=6.2G lam=0.2782314857468009 Gm=None +[s_only s0] step 7280/12000 loss=4.5136 gn=0.92 lr=2.47e-04 tps=16118 mem=6.2G lam=0.2785340202972293 Gm=None +[s_only s0] step 7300/12000 loss=4.3627 gn=0.93 lr=2.45e-04 tps=16119 mem=6.2G lam=0.2788186551382144 Gm=None +[s_only s0] step 7320/12000 loss=4.6183 gn=0.87 lr=2.44e-04 tps=16120 mem=6.2G lam=0.2791850545133154 Gm=None +[s_only s0] step 7340/12000 loss=4.4328 gn=0.91 lr=2.42e-04 tps=16121 mem=6.2G lam=0.2795635983347893 Gm=None +[s_only s0] step 7360/12000 loss=4.3884 gn=0.90 lr=2.41e-04 tps=16122 mem=6.2G lam=0.27983981433014077 Gm=None +[s_only s0] step 7380/12000 loss=4.3591 gn=0.97 lr=2.40e-04 tps=16123 mem=6.2G lam=0.2800653586164117 Gm=None +[s_only s0] step 7400/12000 loss=4.4771 gn=0.88 lr=2.38e-04 tps=16123 mem=6.2G lam=0.280342352266113 Gm=None +[s_only s0] step 7420/12000 loss=4.5912 gn=1.00 lr=2.37e-04 tps=16124 mem=6.2G lam=0.28066197130829096 Gm=None +[s_only s0] step 7440/12000 loss=4.6272 gn=0.93 lr=2.36e-04 tps=16125 mem=6.2G lam=0.2809149483218789 Gm=None +[s_only s0] step 7460/12000 loss=4.4216 gn=0.91 lr=2.34e-04 tps=16126 mem=6.2G lam=0.28117808420211077 Gm=None +[s_only s0] step 7480/12000 loss=4.4769 gn=0.91 lr=2.33e-04 tps=16126 mem=6.2G lam=0.2814760835220416 Gm=None +[s_only s0] step 7500/12000 loss=4.5337 gn=0.90 lr=2.32e-04 tps=16127 mem=6.2G lam=0.28182004981984693 Gm=None +[s_only s0] EVAL step 7500: val_loss=4.5993 val_ppl=99.42 +[s_only s0] step 7520/12000 loss=4.5220 gn=0.92 lr=2.30e-04 tps=16113 mem=6.2G lam=0.28211297436306876 Gm=None +[s_only s0] step 7540/12000 loss=4.5837 gn=0.87 lr=2.29e-04 tps=16113 mem=6.2G lam=0.28235764894634485 Gm=None +[s_only s0] step 7560/12000 loss=4.2890 gn=0.86 lr=2.28e-04 tps=16114 mem=6.2G lam=0.28262589623530704 Gm=None +[s_only s0] step 7580/12000 loss=4.5014 gn=0.87 lr=2.26e-04 tps=16115 mem=6.2G lam=0.2829236580679814 Gm=None +[s_only s0] step 7600/12000 loss=4.3696 gn=0.94 lr=2.25e-04 tps=16115 mem=6.2G lam=0.28318110077331465 Gm=None +[s_only s0] step 7620/12000 loss=4.5523 gn=0.97 lr=2.24e-04 tps=16116 mem=6.2G lam=0.28345720563083887 Gm=None +[s_only s0] step 7640/12000 loss=4.3701 gn=0.93 lr=2.22e-04 tps=16117 mem=6.2G lam=0.28370180757095415 Gm=None +[s_only s0] step 7660/12000 loss=4.5281 gn=0.93 lr=2.21e-04 tps=16117 mem=6.2G lam=0.28389049352457124 Gm=None +[s_only s0] step 7680/12000 loss=4.4864 gn=0.96 lr=2.20e-04 tps=16118 mem=6.2G lam=0.28412259959926206 Gm=None +[s_only s0] step 7700/12000 loss=4.5408 gn=0.92 lr=2.18e-04 tps=16119 mem=6.2G lam=0.2843953262393673 Gm=None +[s_only s0] step 7720/12000 loss=4.5142 gn=1.04 lr=2.17e-04 tps=16120 mem=6.2G lam=0.2846067106972138 Gm=None +[s_only s0] step 7740/12000 loss=4.3542 gn=0.91 lr=2.16e-04 tps=16121 mem=6.2G lam=0.28483075990031165 Gm=None +[s_only s0] step 7760/12000 loss=4.4746 gn=1.03 lr=2.15e-04 tps=16123 mem=6.2G lam=0.2850479284922282 Gm=None +[s_only s0] step 7780/12000 loss=4.4214 gn=1.04 lr=2.13e-04 tps=16123 mem=6.2G lam=0.2853077305480838 Gm=None +[s_only s0] step 7800/12000 loss=4.5001 gn=0.90 lr=2.12e-04 tps=16124 mem=6.2G lam=0.28561234287917614 Gm=None +[s_only s0] step 7820/12000 loss=4.5611 gn=1.00 lr=2.11e-04 tps=16124 mem=6.2G lam=0.28590454751004774 Gm=None +[s_only s0] step 7840/12000 loss=4.3224 gn=0.93 lr=2.09e-04 tps=16125 mem=6.2G lam=0.28618164360523224 Gm=None +[s_only s0] step 7860/12000 loss=4.4725 gn=1.09 lr=2.08e-04 tps=16126 mem=6.2G lam=0.28646007645875216 Gm=None +[s_only s0] step 7880/12000 loss=4.5309 gn=0.98 lr=2.07e-04 tps=16126 mem=6.2G lam=0.28668402569989365 Gm=None +[s_only s0] step 7900/12000 loss=4.3059 gn=0.97 lr=2.06e-04 tps=16127 mem=6.2G lam=0.2869128854945302 Gm=None +[s_only s0] step 7920/12000 loss=4.3630 gn=0.90 lr=2.04e-04 tps=16128 mem=6.2G lam=0.2871545773620407 Gm=None +[s_only s0] step 7940/12000 loss=4.4943 gn=0.89 lr=2.03e-04 tps=16128 mem=6.2G lam=0.28738891116033 Gm=None +[s_only s0] step 7960/12000 loss=4.5192 gn=0.98 lr=2.02e-04 tps=16129 mem=6.2G lam=0.2876265387361248 Gm=None +[s_only s0] step 7980/12000 loss=4.4255 gn=0.91 lr=2.00e-04 tps=16130 mem=6.2G lam=0.28791743361701566 Gm=None +[s_only s0] step 8000/12000 loss=4.3984 gn=1.03 lr=1.99e-04 tps=16130 mem=6.2G lam=0.2882212145874898 Gm=None +[s_only s0] EVAL step 8000: val_loss=4.5275 val_ppl=92.53 +[s_only s0] step 8020/12000 loss=4.4413 gn=0.91 lr=1.98e-04 tps=16116 mem=6.2G lam=0.28852348184833926 Gm=None +[s_only s0] step 8040/12000 loss=4.5891 gn=0.96 lr=1.97e-04 tps=16117 mem=6.2G lam=0.28873354227592546 Gm=None +[s_only s0] step 8060/12000 loss=4.5123 gn=0.94 lr=1.95e-04 tps=16117 mem=6.2G lam=0.28898813140889007 Gm=None +[s_only s0] step 8080/12000 loss=4.3905 gn=0.95 lr=1.94e-04 tps=16118 mem=6.2G lam=0.2892303755506873 Gm=None +[s_only s0] step 8100/12000 loss=4.5345 gn=0.98 lr=1.93e-04 tps=16119 mem=6.2G lam=0.2894964615503947 Gm=None +[s_only s0] step 8120/12000 loss=4.3651 gn=0.86 lr=1.92e-04 tps=16119 mem=6.2G lam=0.2897577394420902 Gm=None +[s_only s0] step 8140/12000 loss=4.5280 gn=0.97 lr=1.90e-04 tps=16120 mem=6.2G lam=0.2899898402392864 Gm=None +[s_only s0] step 8160/12000 loss=4.4980 gn=1.00 lr=1.89e-04 tps=16121 mem=6.2G lam=0.29022093831251067 Gm=None +[s_only s0] step 8180/12000 loss=4.5791 gn=0.91 lr=1.88e-04 tps=16122 mem=6.2G lam=0.2904351207738121 Gm=None +[s_only s0] step 8200/12000 loss=4.2441 gn=0.92 lr=1.87e-04 tps=16122 mem=6.2G lam=0.2906233662118514 Gm=None +[s_only s0] step 8220/12000 loss=4.4063 gn=0.95 lr=1.86e-04 tps=16123 mem=6.2G lam=0.29084122522423667 Gm=None +[s_only s0] step 8240/12000 loss=4.2596 gn=0.96 lr=1.84e-04 tps=16124 mem=6.2G lam=0.29107419432451326 Gm=None +[s_only s0] step 8260/12000 loss=4.3982 gn=0.91 lr=1.83e-04 tps=16125 mem=6.2G lam=0.29126286537696916 Gm=None +[s_only s0] step 8280/12000 loss=4.3406 gn=0.92 lr=1.82e-04 tps=16125 mem=6.2G lam=0.29142970498651266 Gm=None +[s_only s0] step 8300/12000 loss=4.3641 gn=1.02 lr=1.81e-04 tps=16126 mem=6.2G lam=0.29166133391360444 Gm=None +[s_only s0] step 8320/12000 loss=4.4118 gn=0.97 lr=1.80e-04 tps=16127 mem=6.2G lam=0.2918169867868225 Gm=None +[s_only s0] step 8340/12000 loss=4.4442 gn=0.90 lr=1.78e-04 tps=16127 mem=6.2G lam=0.2920383792370558 Gm=None +[s_only s0] step 8360/12000 loss=4.4907 gn=0.94 lr=1.77e-04 tps=16128 mem=6.2G lam=0.2922313576564193 Gm=None +[s_only s0] step 8380/12000 loss=4.4370 gn=0.89 lr=1.76e-04 tps=16129 mem=6.2G lam=0.2924636627237002 Gm=None +[s_only s0] step 8400/12000 loss=4.3901 gn=0.87 lr=1.75e-04 tps=16129 mem=6.2G lam=0.2926519817362229 Gm=None +[s_only s0] step 8420/12000 loss=4.5158 gn=0.90 lr=1.74e-04 tps=16130 mem=6.2G lam=0.29286807247747976 Gm=None +[s_only s0] step 8440/12000 loss=4.1920 gn=0.98 lr=1.72e-04 tps=16131 mem=6.2G lam=0.29309562624742586 Gm=None +[s_only s0] step 8460/12000 loss=4.3577 gn=0.99 lr=1.71e-04 tps=16131 mem=6.2G lam=0.2932690183321635 Gm=None +[s_only s0] step 8480/12000 loss=4.3174 gn=0.97 lr=1.70e-04 tps=16132 mem=6.2G lam=0.29344680936386186 Gm=None +[s_only s0] step 8500/12000 loss=4.6087 gn=0.97 lr=1.69e-04 tps=16132 mem=6.2G lam=0.29361132842799026 Gm=None +[s_only s0] EVAL step 8500: val_loss=4.5196 val_ppl=91.80 +[s_only s0] step 8520/12000 loss=4.4197 gn=0.95 lr=1.68e-04 tps=16119 mem=6.2G lam=0.2938387679557006 Gm=None +[s_only s0] step 8540/12000 loss=4.4765 gn=1.02 lr=1.67e-04 tps=16119 mem=6.2G lam=0.29404696480681497 Gm=None +[s_only s0] step 8560/12000 loss=4.5315 gn=0.96 lr=1.66e-04 tps=16120 mem=6.2G lam=0.2942091707761089 Gm=None +[s_only s0] step 8580/12000 loss=4.3859 gn=0.95 lr=1.64e-04 tps=16121 mem=6.2G lam=0.294420891130964 Gm=None +[s_only s0] step 8600/12000 loss=4.4942 gn=1.09 lr=1.63e-04 tps=16121 mem=6.2G lam=0.2946446466570099 Gm=None +[s_only s0] step 8620/12000 loss=4.4495 gn=1.02 lr=1.62e-04 tps=16122 mem=6.2G lam=0.2948167234038313 Gm=None +[s_only s0] step 8640/12000 loss=4.4908 gn=0.98 lr=1.61e-04 tps=16123 mem=6.2G lam=0.2950603763262431 Gm=None +[s_only s0] step 8660/12000 loss=4.3346 gn=1.01 lr=1.60e-04 tps=16124 mem=6.2G lam=0.29521816068639356 Gm=None +[s_only s0] step 8680/12000 loss=4.3943 gn=0.92 lr=1.59e-04 tps=16124 mem=6.2G lam=0.2954234428082903 Gm=None +[s_only s0] step 8700/12000 loss=4.3808 gn=0.92 lr=1.58e-04 tps=16125 mem=6.2G lam=0.29561913789560396 Gm=None +[s_only s0] step 8720/12000 loss=4.3462 gn=1.02 lr=1.57e-04 tps=16126 mem=6.2G lam=0.2957868929952383 Gm=None +[s_only s0] step 8740/12000 loss=4.4466 gn=0.93 lr=1.55e-04 tps=16126 mem=6.2G lam=0.29597753286361694 Gm=None +[s_only s0] step 8760/12000 loss=4.3204 gn=1.13 lr=1.54e-04 tps=16127 mem=6.2G lam=0.29616829566657543 Gm=None +[s_only s0] step 8780/12000 loss=4.4361 gn=1.07 lr=1.53e-04 tps=16127 mem=6.2G lam=0.2963369864349564 Gm=None +[s_only s0] step 8800/12000 loss=4.4352 gn=0.99 lr=1.52e-04 tps=16128 mem=6.2G lam=0.29647776670753956 Gm=None +[s_only s0] step 8820/12000 loss=4.4450 gn=0.98 lr=1.51e-04 tps=16129 mem=6.2G lam=0.2966680508106947 Gm=None +[s_only s0] step 8840/12000 loss=4.4266 gn=1.01 lr=1.50e-04 tps=16129 mem=6.2G lam=0.2968801188593109 Gm=None +[s_only s0] step 8860/12000 loss=4.4712 gn=0.98 lr=1.49e-04 tps=16130 mem=6.2G lam=0.2971161361783743 Gm=None +[s_only s0] step 8880/12000 loss=4.2829 gn=0.95 lr=1.48e-04 tps=16131 mem=6.2G lam=0.29728054410467547 Gm=None +[s_only s0] step 8900/12000 loss=4.3924 gn=0.98 lr=1.47e-04 tps=16131 mem=6.2G lam=0.29742020554840565 Gm=None +[s_only s0] step 8920/12000 loss=4.4721 gn=1.13 lr=1.46e-04 tps=16132 mem=6.2G lam=0.29759298140803975 Gm=None +[s_only s0] step 8940/12000 loss=4.2929 gn=1.03 lr=1.45e-04 tps=16132 mem=6.2G lam=0.29777022016545135 Gm=None +[s_only s0] step 8960/12000 loss=4.5136 gn=0.99 lr=1.44e-04 tps=16133 mem=6.2G lam=0.29792562561730546 Gm=None +[s_only s0] step 8980/12000 loss=4.3859 gn=1.03 lr=1.43e-04 tps=16133 mem=6.2G lam=0.2980987538273136 Gm=None +[s_only s0] step 9000/12000 loss=4.3277 gn=0.95 lr=1.42e-04 tps=16134 mem=6.2G lam=0.29826742441703874 Gm=None +[s_only s0] EVAL step 9000: val_loss=4.4839 val_ppl=88.58 +[s_only s0] step 9020/12000 loss=4.3696 gn=0.95 lr=1.41e-04 tps=16121 mem=6.2G lam=0.2984508974477649 Gm=None +[s_only s0] step 9040/12000 loss=4.3183 gn=0.94 lr=1.40e-04 tps=16122 mem=6.2G lam=0.2985867454359929 Gm=None +[s_only s0] step 9060/12000 loss=4.3380 gn=1.01 lr=1.39e-04 tps=16122 mem=6.2G lam=0.29872787930071354 Gm=None +[s_only s0] step 9080/12000 loss=4.3190 gn=0.98 lr=1.38e-04 tps=16123 mem=6.2G lam=0.29888784047216177 Gm=None +[s_only s0] step 9100/12000 loss=4.2759 gn=0.91 lr=1.37e-04 tps=16124 mem=6.2G lam=0.2990593835711479 Gm=None +[s_only s0] step 9120/12000 loss=4.3609 gn=0.99 lr=1.36e-04 tps=16124 mem=6.2G lam=0.2992296653489272 Gm=None +[s_only s0] step 9140/12000 loss=4.4855 gn=0.95 lr=1.35e-04 tps=16125 mem=6.2G lam=0.2993747644747297 Gm=None +[s_only s0] step 9160/12000 loss=4.3172 gn=1.00 lr=1.34e-04 tps=16126 mem=6.2G lam=0.2994982100402315 Gm=None +[s_only s0] step 9180/12000 loss=4.4755 gn=1.01 lr=1.33e-04 tps=16126 mem=6.2G lam=0.29968910571187735 Gm=None +[s_only s0] step 9200/12000 loss=4.4156 gn=1.05 lr=1.32e-04 tps=16127 mem=6.2G lam=0.29986171517521143 Gm=None +[s_only s0] step 9220/12000 loss=4.4736 gn=1.13 lr=1.31e-04 tps=16127 mem=6.2G lam=0.30000419324884814 Gm=None +[s_only s0] step 9240/12000 loss=4.4817 gn=1.05 lr=1.30e-04 tps=16128 mem=6.2G lam=0.3001790726557374 Gm=None +[s_only s0] step 9260/12000 loss=4.4239 gn=0.95 lr=1.29e-04 tps=16129 mem=6.2G lam=0.30033059728642303 Gm=None +[s_only s0] step 9280/12000 loss=4.3528 gn=0.96 lr=1.28e-04 tps=16129 mem=6.2G lam=0.3004874673982461 Gm=None +[s_only s0] step 9300/12000 loss=4.4547 gn=0.93 lr=1.27e-04 tps=16130 mem=6.2G lam=0.30062550865113735 Gm=None +[s_only s0] step 9320/12000 loss=4.3185 gn=0.95 lr=1.26e-04 tps=16130 mem=6.2G lam=0.3007676613827546 Gm=None +[s_only s0] step 9340/12000 loss=4.4814 gn=1.01 lr=1.25e-04 tps=16131 mem=6.2G lam=0.30092645188172656 Gm=None +[s_only s0] step 9360/12000 loss=4.5415 gn=1.18 lr=1.24e-04 tps=16131 mem=6.2G lam=0.3010989126438896 Gm=None +[s_only s0] step 9380/12000 loss=4.4376 gn=1.02 lr=1.23e-04 tps=16132 mem=6.2G lam=0.3012465611100197 Gm=None +[s_only s0] step 9400/12000 loss=4.3045 gn=0.94 lr=1.22e-04 tps=16132 mem=6.2G lam=0.3014300012340148 Gm=None +[s_only s0] step 9420/12000 loss=4.5429 gn=1.16 lr=1.21e-04 tps=16133 mem=6.2G lam=0.30156367427359027 Gm=None +[s_only s0] step 9440/12000 loss=4.4201 gn=1.04 lr=1.20e-04 tps=16133 mem=6.2G lam=0.3016795888543129 Gm=None +[s_only s0] step 9460/12000 loss=4.3530 gn=0.97 lr=1.19e-04 tps=16134 mem=6.2G lam=0.3018177195141713 Gm=None +[s_only s0] step 9480/12000 loss=4.4123 gn=0.95 lr=1.19e-04 tps=16134 mem=6.2G lam=0.30197125642249983 Gm=None +[s_only s0] step 9500/12000 loss=4.3521 gn=0.99 lr=1.18e-04 tps=16135 mem=6.2G lam=0.3021107396731774 Gm=None +[s_only s0] EVAL step 9500: val_loss=4.4471 val_ppl=85.38 +[s_only s0] step 9520/12000 loss=4.2176 gn=0.95 lr=1.17e-04 tps=16123 mem=6.2G lam=0.30225958799322444 Gm=None +[s_only s0] step 9540/12000 loss=4.5227 gn=1.11 lr=1.16e-04 tps=16124 mem=6.2G lam=0.3023770597452919 Gm=None +[s_only s0] step 9560/12000 loss=4.4930 gn=1.00 lr=1.15e-04 tps=16124 mem=6.2G lam=0.3025206609939535 Gm=None +[s_only s0] step 9580/12000 loss=4.2774 gn=1.01 lr=1.14e-04 tps=16125 mem=6.2G lam=0.30265992507338524 Gm=None +[s_only s0] step 9600/12000 loss=4.3507 gn=0.99 lr=1.13e-04 tps=16125 mem=6.2G lam=0.30279487123092014 Gm=None +[s_only s0] step 9620/12000 loss=4.4666 gn=1.00 lr=1.12e-04 tps=16126 mem=6.2G lam=0.302939726660649 Gm=None +[s_only s0] step 9640/12000 loss=4.3188 gn=1.10 lr=1.12e-04 tps=16127 mem=6.2G lam=0.30306793935596943 Gm=None +[s_only s0] step 9660/12000 loss=4.4338 gn=1.00 lr=1.11e-04 tps=16127 mem=6.2G lam=0.30318890667210024 Gm=None +[s_only s0] step 9680/12000 loss=4.6153 gn=1.05 lr=1.10e-04 tps=16128 mem=6.2G lam=0.303310786994795 Gm=None +[s_only s0] step 9700/12000 loss=4.2755 gn=0.94 lr=1.09e-04 tps=16128 mem=6.2G lam=0.30346473089108866 Gm=None +[s_only s0] step 9720/12000 loss=4.2710 gn=1.01 lr=1.08e-04 tps=16129 mem=6.2G lam=0.30359996265421313 Gm=None +[s_only s0] step 9740/12000 loss=4.4418 gn=1.04 lr=1.07e-04 tps=16129 mem=6.2G lam=0.30373777480175096 Gm=None +[s_only s0] step 9760/12000 loss=4.2935 gn=1.00 lr=1.07e-04 tps=16130 mem=6.2G lam=0.3038620452086131 Gm=None +[s_only s0] step 9780/12000 loss=4.3698 gn=1.07 lr=1.06e-04 tps=16131 mem=6.2G lam=0.30398966496189433 Gm=None +[s_only s0] step 9800/12000 loss=4.3165 gn=1.00 lr=1.05e-04 tps=16131 mem=6.2G lam=0.30412731133401394 Gm=None +[s_only s0] step 9820/12000 loss=4.4819 gn=1.09 lr=1.04e-04 tps=16132 mem=6.2G lam=0.30422808043658733 Gm=None +[s_only s0] step 9840/12000 loss=4.1539 gn=1.03 lr=1.03e-04 tps=16132 mem=6.2G lam=0.30434557888656855 Gm=None +[s_only s0] step 9860/12000 loss=4.4455 gn=0.98 lr=1.03e-04 tps=16133 mem=6.2G lam=0.30449891556054354 Gm=None +[s_only s0] step 9880/12000 loss=4.5144 gn=1.06 lr=1.02e-04 tps=16133 mem=6.2G lam=0.3046369568134348 Gm=None +[s_only s0] step 9900/12000 loss=4.3192 gn=1.03 lr=1.01e-04 tps=16133 mem=6.2G lam=0.3047385923564434 Gm=None +[s_only s0] step 9920/12000 loss=4.4255 gn=0.95 lr=1.00e-04 tps=16134 mem=6.2G lam=0.30487463250756264 Gm=None +[s_only s0] step 9940/12000 loss=4.5693 gn=1.01 lr=9.96e-05 tps=16135 mem=6.2G lam=0.3050095026070873 Gm=None +[s_only s0] step 9960/12000 loss=4.4813 gn=1.00 lr=9.89e-05 tps=16135 mem=6.2G lam=0.3051137924194336 Gm=None +[s_only s0] step 9980/12000 loss=4.3533 gn=0.96 lr=9.81e-05 tps=16136 mem=6.2G lam=0.30521447646121186 Gm=None +[s_only s0] step 10000/12000 loss=4.4773 gn=1.04 lr=9.74e-05 tps=16136 mem=6.2G lam=0.30531759777416784 Gm=None +[s_only s0] EVAL step 10000: val_loss=4.4417 val_ppl=84.92 +[s_only s0] step 10020/12000 loss=4.2563 gn=1.02 lr=9.67e-05 tps=16125 mem=6.2G lam=0.3054550839588046 Gm=None +[s_only s0] step 10040/12000 loss=4.2389 gn=0.95 lr=9.59e-05 tps=16125 mem=6.2G lam=0.30555308672289055 Gm=None +[s_only s0] step 10060/12000 loss=4.2582 gn=0.95 lr=9.52e-05 tps=16126 mem=6.2G lam=0.3056671616310875 Gm=None +[s_only s0] step 10080/12000 loss=4.6050 gn=1.12 lr=9.45e-05 tps=16126 mem=6.2G lam=0.30576302856206894 Gm=None +[s_only s0] step 10100/12000 loss=4.5862 gn=1.08 lr=9.38e-05 tps=16127 mem=6.2G lam=0.30588390678167343 Gm=None +[s_only s0] step 10120/12000 loss=4.3319 gn=1.03 lr=9.31e-05 tps=16127 mem=6.2G lam=0.30598066623012227 Gm=None +[s_only s0] step 10140/12000 loss=4.3553 gn=1.13 lr=9.24e-05 tps=16128 mem=6.2G lam=0.3060676859070857 Gm=None +[s_only s0] step 10160/12000 loss=4.3133 gn=1.05 lr=9.18e-05 tps=16129 mem=6.2G lam=0.3061769533281525 Gm=None +[s_only s0] step 10180/12000 loss=4.3022 gn=1.02 lr=9.11e-05 tps=16129 mem=6.2G lam=0.30623981977502507 Gm=None +[s_only s0] step 10200/12000 loss=4.3146 gn=1.06 lr=9.04e-05 tps=16130 mem=6.2G lam=0.3063527972747882 Gm=None +[s_only s0] step 10220/12000 loss=4.3700 gn=1.12 lr=8.98e-05 tps=16130 mem=6.2G lam=0.3064659346515934 Gm=None +[s_only s0] step 10240/12000 loss=4.3411 gn=1.05 lr=8.91e-05 tps=16131 mem=6.2G lam=0.3065769240881006 Gm=None +[s_only s0] step 10260/12000 loss=4.4120 gn=1.06 lr=8.85e-05 tps=16131 mem=6.2G lam=0.3066984995578726 Gm=None +[s_only s0] step 10280/12000 loss=4.3968 gn=1.04 lr=8.78e-05 tps=16130 mem=6.2G lam=0.3068163835753997 Gm=None +[s_only s0] step 10300/12000 loss=4.2939 gn=1.01 lr=8.72e-05 tps=16130 mem=6.2G lam=0.3069254057481885 Gm=None +[s_only s0] step 10320/12000 loss=4.2411 gn=0.95 lr=8.66e-05 tps=16130 mem=6.2G lam=0.3070480947693189 Gm=None +[s_only s0] step 10340/12000 loss=4.3176 gn=1.18 lr=8.59e-05 tps=16130 mem=6.2G lam=0.3071647270893057 Gm=None +[s_only s0] step 10360/12000 loss=4.4130 gn=1.03 lr=8.53e-05 tps=16130 mem=6.2G lam=0.3072299202904105 Gm=None +[s_only s0] step 10380/12000 loss=4.2658 gn=1.12 lr=8.47e-05 tps=16130 mem=6.2G lam=0.30731492551664513 Gm=None +[s_only s0] step 10400/12000 loss=4.3634 gn=1.10 lr=8.41e-05 tps=16130 mem=6.2G lam=0.3074271818622947 Gm=None +[s_only s0] step 10420/12000 loss=4.4117 gn=1.15 lr=8.35e-05 tps=16131 mem=6.2G lam=0.3075336180627346 Gm=None +[s_only s0] step 10440/12000 loss=4.4431 gn=1.12 lr=8.30e-05 tps=16131 mem=6.2G lam=0.30764831354220706 Gm=None +[s_only s0] step 10460/12000 loss=4.4613 gn=1.03 lr=8.24e-05 tps=16132 mem=6.2G lam=0.3077319609001279 Gm=None +[s_only s0] step 10480/12000 loss=4.3438 gn=1.02 lr=8.18e-05 tps=16132 mem=6.2G lam=0.30780448733518523 Gm=None +[s_only s0] step 10500/12000 loss=4.4513 gn=1.10 lr=8.12e-05 tps=16133 mem=6.2G lam=0.30791467397163313 Gm=None +[s_only s0] EVAL step 10500: val_loss=4.4463 val_ppl=85.31 +[s_only s0] step 10520/12000 loss=4.2383 gn=1.04 lr=8.07e-05 tps=16122 mem=6.2G lam=0.3080063049371044 Gm=None +[s_only s0] step 10540/12000 loss=4.4220 gn=1.06 lr=8.01e-05 tps=16122 mem=6.2G lam=0.30813345530380803 Gm=None +[s_only s0] step 10560/12000 loss=4.3597 gn=0.99 lr=7.96e-05 tps=16123 mem=6.2G lam=0.3082221153502663 Gm=None +[s_only s0] step 10580/12000 loss=4.3688 gn=1.15 lr=7.91e-05 tps=16124 mem=6.2G lam=0.3083371141304572 Gm=None +[s_only s0] step 10600/12000 loss=4.3913 gn=1.00 lr=7.85e-05 tps=16124 mem=6.2G lam=0.3084687264636159 Gm=None +[s_only s0] step 10620/12000 loss=4.2043 gn=1.13 lr=7.80e-05 tps=16125 mem=6.2G lam=0.30855484399944544 Gm=None +[s_only s0] step 10640/12000 loss=4.2264 gn=1.03 lr=7.75e-05 tps=16125 mem=6.2G lam=0.3086452490339677 Gm=None +[s_only s0] step 10660/12000 loss=4.3622 gn=1.06 lr=7.70e-05 tps=16126 mem=6.2G lam=0.3087216891969244 Gm=None +[s_only s0] step 10680/12000 loss=4.4328 gn=1.03 lr=7.65e-05 tps=16126 mem=6.2G lam=0.30878500764568645 Gm=None +[s_only s0] step 10700/12000 loss=4.2578 gn=1.05 lr=7.60e-05 tps=16126 mem=6.2G lam=0.3088736794888973 Gm=None +[s_only s0] step 10720/12000 loss=4.4785 gn=1.13 lr=7.55e-05 tps=16127 mem=6.2G lam=0.30894628446549177 Gm=None +[s_only s0] step 10740/12000 loss=4.4056 gn=0.96 lr=7.50e-05 tps=16127 mem=6.2G lam=0.3090308668712775 Gm=None +[s_only s0] step 10760/12000 loss=4.3064 gn=1.02 lr=7.46e-05 tps=16127 mem=6.2G lam=0.3091060410564144 Gm=None +[s_only s0] step 10780/12000 loss=4.3916 gn=1.13 lr=7.41e-05 tps=16128 mem=6.2G lam=0.309191457927227 Gm=None +[s_only s0] step 10800/12000 loss=4.4801 gn=1.19 lr=7.37e-05 tps=16128 mem=6.2G lam=0.30928928777575493 Gm=None +[s_only s0] step 10820/12000 loss=4.2658 gn=1.21 lr=7.32e-05 tps=16129 mem=6.2G lam=0.3093623553092281 Gm=None +[s_only s0] step 10840/12000 loss=4.3724 gn=1.04 lr=7.28e-05 tps=16129 mem=6.2G lam=0.3094145366922021 Gm=None +[s_only s0] step 10860/12000 loss=4.4696 gn=1.15 lr=7.23e-05 tps=16129 mem=6.2G lam=0.3094859082872669 Gm=None +[s_only s0] step 10880/12000 loss=4.2939 gn=1.02 lr=7.19e-05 tps=16130 mem=6.2G lam=0.3095793429141243 Gm=None +[s_only s0] step 10900/12000 loss=4.3986 gn=1.11 lr=7.15e-05 tps=16130 mem=6.2G lam=0.3096870860705773 Gm=None +[s_only s0] step 10920/12000 loss=4.3111 gn=1.09 lr=7.11e-05 tps=16131 mem=6.2G lam=0.30977474711835384 Gm=None +[s_only s0] step 10940/12000 loss=4.4139 gn=1.13 lr=7.07e-05 tps=16131 mem=6.2G lam=0.309862667384247 Gm=None +[s_only s0] step 10960/12000 loss=4.4419 gn=1.14 lr=7.03e-05 tps=16132 mem=6.2G lam=0.3099531127760808 Gm=None +[s_only s0] step 10980/12000 loss=4.3520 gn=1.12 lr=6.99e-05 tps=16132 mem=6.2G lam=0.3100305811191599 Gm=None +[s_only s0] step 11000/12000 loss=4.3773 gn=1.09 lr=6.95e-05 tps=16133 mem=6.2G lam=0.3100839105124275 Gm=None +[s_only s0] EVAL step 11000: val_loss=4.4022 val_ppl=81.63 +[s_only s0] step 11020/12000 loss=4.2313 gn=1.06 lr=6.91e-05 tps=16123 mem=6.2G lam=0.31015896735092 Gm=None +[s_only s0] step 11040/12000 loss=4.2449 gn=1.03 lr=6.88e-05 tps=16123 mem=6.2G lam=0.3102403013035655 Gm=None +[s_only s0] step 11060/12000 loss=4.4386 gn=1.09 lr=6.84e-05 tps=16124 mem=6.2G lam=0.31032549031078815 Gm=None +[s_only s0] step 11080/12000 loss=4.3734 gn=1.00 lr=6.81e-05 tps=16124 mem=6.2G lam=0.3104364260410269 Gm=None +[s_only s0] step 11100/12000 loss=4.4417 gn=1.11 lr=6.77e-05 tps=16125 mem=6.2G lam=0.3104982062553366 Gm=None +[s_only s0] step 11120/12000 loss=4.3255 gn=1.11 lr=6.74e-05 tps=16125 mem=6.2G lam=0.3105800474683444 Gm=None +[s_only s0] step 11140/12000 loss=4.2031 gn=1.08 lr=6.70e-05 tps=16126 mem=6.2G lam=0.3106411180148522 Gm=None +[s_only s0] step 11160/12000 loss=4.2900 gn=1.26 lr=6.67e-05 tps=16127 mem=6.2G lam=0.3107173986112078 Gm=None +[s_only s0] step 11180/12000 loss=4.3696 gn=1.08 lr=6.64e-05 tps=16127 mem=6.2G lam=0.31083083990961313 Gm=None +[s_only s0] step 11200/12000 loss=4.3761 gn=1.04 lr=6.61e-05 tps=16128 mem=6.2G lam=0.310928404952089 Gm=None +[s_only s0] step 11220/12000 loss=4.3537 gn=1.05 lr=6.58e-05 tps=16128 mem=6.2G lam=0.31100600709517795 Gm=None +[s_only s0] step 11240/12000 loss=4.2108 gn=1.01 lr=6.55e-05 tps=16129 mem=6.2G lam=0.3110955087468028 Gm=None +[s_only s0] step 11260/12000 loss=4.3860 gn=1.11 lr=6.52e-05 tps=16129 mem=6.2G lam=0.3111806216960152 Gm=None +[s_only s0] step 11280/12000 loss=4.3981 gn=1.16 lr=6.49e-05 tps=16130 mem=6.2G lam=0.31125370413064957 Gm=None +[s_only s0] step 11300/12000 loss=4.3767 gn=1.06 lr=6.47e-05 tps=16130 mem=6.2G lam=0.31134983617812395 Gm=None +[s_only s0] step 11320/12000 loss=4.2401 gn=1.13 lr=6.44e-05 tps=16131 mem=6.2G lam=0.31142853666096926 Gm=None +[s_only s0] step 11340/12000 loss=4.3063 gn=1.07 lr=6.42e-05 tps=16131 mem=6.2G lam=0.31151006898532313 Gm=None +[s_only s0] step 11360/12000 loss=4.1893 gn=1.13 lr=6.39e-05 tps=16132 mem=6.2G lam=0.3116032707815369 Gm=None +[s_only s0] step 11380/12000 loss=4.3936 gn=1.06 lr=6.37e-05 tps=16132 mem=6.2G lam=0.3116645651559035 Gm=None +[s_only s0] step 11400/12000 loss=4.3090 gn=1.08 lr=6.34e-05 tps=16132 mem=6.2G lam=0.31172104304035503 Gm=None +[s_only s0] step 11420/12000 loss=4.3278 gn=1.09 lr=6.32e-05 tps=16133 mem=6.2G lam=0.3118023608500759 Gm=None +[s_only s0] step 11440/12000 loss=4.1832 gn=1.14 lr=6.30e-05 tps=16133 mem=6.2G lam=0.31188417381296557 Gm=None +[s_only s0] step 11460/12000 loss=4.4073 gn=1.15 lr=6.28e-05 tps=16134 mem=6.2G lam=0.31196018649886054 Gm=None +[s_only s0] step 11480/12000 loss=4.2714 gn=1.12 lr=6.26e-05 tps=16134 mem=6.2G lam=0.3120608953759074 Gm=None +[s_only s0] step 11500/12000 loss=4.3075 gn=1.07 lr=6.24e-05 tps=16135 mem=6.2G lam=0.3121458950142066 Gm=None +[s_only s0] EVAL step 11500: val_loss=4.4093 val_ppl=82.21 +[s_only s0] step 11520/12000 loss=4.3124 gn=1.13 lr=6.22e-05 tps=16125 mem=6.2G lam=0.3122201127310594 Gm=None +[s_only s0] step 11540/12000 loss=4.4380 gn=1.15 lr=6.20e-05 tps=16126 mem=6.2G lam=0.31230275239795446 Gm=None +[s_only s0] step 11560/12000 loss=4.3843 gn=1.12 lr=6.19e-05 tps=16126 mem=6.2G lam=0.3123857543493311 Gm=None +[s_only s0] step 11580/12000 loss=4.1990 gn=1.10 lr=6.17e-05 tps=16127 mem=6.2G lam=0.31247828745593625 Gm=None +[s_only s0] step 11600/12000 loss=4.3171 gn=1.10 lr=6.15e-05 tps=16127 mem=6.2G lam=0.31256062258034945 Gm=None +[s_only s0] step 11620/12000 loss=4.3742 gn=1.13 lr=6.14e-05 tps=16128 mem=6.2G lam=0.3126506805419922 Gm=None +[s_only s0] step 11640/12000 loss=4.1332 gn=1.08 lr=6.12e-05 tps=16128 mem=6.2G lam=0.31272664417823154 Gm=None +[s_only s0] step 11660/12000 loss=4.2999 gn=1.09 lr=6.11e-05 tps=16129 mem=6.2G lam=0.31279188642899197 Gm=None +[s_only s0] step 11680/12000 loss=4.2734 gn=1.06 lr=6.10e-05 tps=16129 mem=6.2G lam=0.3128799091403683 Gm=None +[s_only s0] step 11700/12000 loss=4.2908 gn=1.14 lr=6.09e-05 tps=16130 mem=6.2G lam=0.3129816207413872 Gm=None +[s_only s0] step 11720/12000 loss=4.2602 gn=1.07 lr=6.07e-05 tps=16130 mem=6.2G lam=0.3130528898909688 Gm=None +[s_only s0] step 11740/12000 loss=4.4307 gn=1.09 lr=6.06e-05 tps=16131 mem=6.2G lam=0.31312065074841183 Gm=None +[s_only s0] step 11760/12000 loss=4.3595 gn=1.20 lr=6.06e-05 tps=16131 mem=6.2G lam=0.31317413536210853 Gm=None +[s_only s0] step 11780/12000 loss=4.3751 gn=1.08 lr=6.05e-05 tps=16132 mem=6.2G lam=0.3132516921808322 Gm=None +[s_only s0] step 11800/12000 loss=4.4002 gn=1.07 lr=6.04e-05 tps=16132 mem=6.2G lam=0.31333007508267957 Gm=None +[s_only s0] step 11820/12000 loss=4.3176 gn=1.14 lr=6.03e-05 tps=16133 mem=6.2G lam=0.31340450793504715 Gm=None +[s_only s0] step 11840/12000 loss=4.3494 gn=1.17 lr=6.02e-05 tps=16133 mem=6.2G lam=0.3134999359026551 Gm=None +[s_only s0] step 11860/12000 loss=4.1565 gn=1.11 lr=6.02e-05 tps=16134 mem=6.2G lam=0.3135545486584306 Gm=None +[s_only s0] step 11880/12000 loss=4.4903 gn=1.24 lr=6.01e-05 tps=16134 mem=6.2G lam=0.3136437699819605 Gm=None +[s_only s0] step 11900/12000 loss=4.2865 gn=1.34 lr=6.01e-05 tps=16135 mem=6.2G lam=0.3136974945664406 Gm=None +[s_only s0] step 11920/12000 loss=4.4594 gn=1.20 lr=6.01e-05 tps=16136 mem=6.2G lam=0.3137782495468855 Gm=None +[s_only s0] step 11940/12000 loss=4.1444 gn=1.14 lr=6.00e-05 tps=16136 mem=6.2G lam=0.31386074889451265 Gm=None +[s_only s0] step 11960/12000 loss=4.3475 gn=1.10 lr=6.00e-05 tps=16137 mem=6.2G lam=0.31391672417521477 Gm=None +[s_only s0] step 11980/12000 loss=4.2326 gn=1.10 lr=6.00e-05 tps=16137 mem=6.2G lam=0.31397200624148053 Gm=None +[s_only s0] EVAL step 11999: val_loss=4.3661 val_ppl=78.74 +[init] variant=g_only seed=0 params=63.39M +[init] use_S=False use_G=True +[g_only s0] step 0/12000 loss=10.9209 gn=9.51 lr=3.00e-06 tps=8285 mem=5.8G lam=None Gm=0.4967447916666667 +[g_only s0] step 20/12000 loss=9.7765 gn=1.99 lr=6.30e-05 tps=16924 mem=6.3G lam=None Gm=0.4545084635416667 +[g_only s0] step 40/12000 loss=9.0332 gn=1.74 lr=1.23e-04 tps=17292 mem=6.3G lam=None Gm=0.4269205729166667 +[g_only s0] step 60/12000 loss=8.0810 gn=1.55 lr=1.83e-04 tps=17411 mem=6.3G lam=None Gm=0.4256184895833333 +[g_only s0] step 80/12000 loss=7.5517 gn=0.78 lr=2.43e-04 tps=17503 mem=6.3G lam=None Gm=0.419189453125 +[g_only s0] step 100/12000 loss=7.3260 gn=0.78 lr=3.03e-04 tps=17565 mem=6.3G lam=None Gm=0.4297688802083333 +[g_only s0] step 120/12000 loss=7.0941 gn=0.73 lr=3.63e-04 tps=17606 mem=6.3G lam=None Gm=0.4077962239583333 +[g_only s0] step 140/12000 loss=6.9859 gn=0.60 lr=4.23e-04 tps=17627 mem=6.3G lam=None Gm=0.4068196614583333 +[g_only s0] step 160/12000 loss=6.7662 gn=0.70 lr=4.83e-04 tps=17651 mem=6.3G lam=None Gm=0.3974609375 +[g_only s0] step 180/12000 loss=6.8373 gn=0.54 lr=5.43e-04 tps=17670 mem=6.3G lam=None Gm=0.3920084635416667 +[g_only s0] step 200/12000 loss=6.6822 gn=0.54 lr=6.00e-04 tps=17687 mem=6.3G lam=None Gm=0.3943684895833333 +[g_only s0] step 220/12000 loss=6.8444 gn=0.66 lr=6.00e-04 tps=17698 mem=6.3G lam=None Gm=0.3870442708333333 +[g_only s0] step 240/12000 loss=6.5415 gn=0.47 lr=6.00e-04 tps=17707 mem=6.3G lam=None Gm=0.388671875 +[g_only s0] step 260/12000 loss=6.6508 gn=0.51 lr=6.00e-04 tps=17716 mem=6.3G lam=None Gm=0.391845703125 +[g_only s0] step 280/12000 loss=6.4061 gn=0.49 lr=6.00e-04 tps=17723 mem=6.3G lam=None Gm=0.3846842447916667 +[g_only s0] step 300/12000 loss=6.5534 gn=0.46 lr=6.00e-04 tps=17730 mem=6.3G lam=None Gm=0.3966471354166667 +[g_only s0] step 320/12000 loss=6.3489 gn=0.44 lr=6.00e-04 tps=17736 mem=6.3G lam=None Gm=0.3968912760416667 +[g_only s0] step 340/12000 loss=6.4007 gn=0.50 lr=6.00e-04 tps=17740 mem=6.3G lam=None Gm=0.41357421875 +[g_only s0] step 360/12000 loss=6.5474 gn=0.50 lr=6.00e-04 tps=17744 mem=6.3G lam=None Gm=0.4055989583333333 +[g_only s0] step 380/12000 loss=6.4042 gn=0.51 lr=6.00e-04 tps=17749 mem=6.3G lam=None Gm=0.3932291666666667 +[g_only s0] step 400/12000 loss=6.2961 gn=0.44 lr=6.00e-04 tps=17749 mem=6.3G lam=None Gm=0.4051106770833333 +[g_only s0] step 420/12000 loss=6.4144 gn=0.44 lr=6.00e-04 tps=17748 mem=6.3G lam=None Gm=0.4239908854166667 +[g_only s0] step 440/12000 loss=6.3620 gn=0.47 lr=5.99e-04 tps=17750 mem=6.3G lam=None Gm=0.4229329427083333 +[g_only s0] step 460/12000 loss=6.4024 gn=0.47 lr=5.99e-04 tps=17750 mem=6.3G lam=None Gm=0.4178466796875 +[g_only s0] step 480/12000 loss=6.1760 gn=0.44 lr=5.99e-04 tps=17750 mem=6.3G lam=None Gm=0.4530436197916667 +[g_only s0] step 500/12000 loss=6.0632 gn=0.49 lr=5.99e-04 tps=17750 mem=6.3G lam=None Gm=0.48193359375 +[g_only s0] EVAL step 500: val_loss=6.2870 val_ppl=537.56 +[g_only s0] step 520/12000 loss=6.2521 gn=0.51 lr=5.99e-04 tps=17501 mem=6.3G lam=None Gm=0.4698079427083333 +[g_only s0] step 540/12000 loss=6.1760 gn=0.49 lr=5.99e-04 tps=17511 mem=6.3G lam=None Gm=0.4944661458333333 +[g_only s0] step 560/12000 loss=6.1480 gn=0.46 lr=5.99e-04 tps=17517 mem=6.3G lam=None Gm=0.4838053385416667 +[g_only s0] step 580/12000 loss=6.2003 gn=0.49 lr=5.99e-04 tps=17526 mem=6.3G lam=None Gm=0.49560546875 +[g_only s0] step 600/12000 loss=6.1432 gn=0.46 lr=5.98e-04 tps=17535 mem=6.3G lam=None Gm=0.5352376302083334 +[g_only s0] step 620/12000 loss=6.0342 gn=0.43 lr=5.98e-04 tps=17543 mem=6.3G lam=None Gm=0.5078125 +[g_only s0] step 640/12000 loss=6.1269 gn=0.49 lr=5.98e-04 tps=17551 mem=6.3G lam=None Gm=0.5270182291666666 +[g_only s0] step 660/12000 loss=6.0091 gn=0.52 lr=5.98e-04 tps=17559 mem=6.3G lam=None Gm=0.5542805989583334 +[g_only s0] step 680/12000 loss=6.0012 gn=0.46 lr=5.98e-04 tps=17565 mem=6.3G lam=None Gm=0.5082194010416666 +[g_only s0] step 700/12000 loss=5.9763 gn=0.51 lr=5.98e-04 tps=17571 mem=6.3G lam=None Gm=0.54541015625 +[g_only s0] step 720/12000 loss=6.0843 gn=0.52 lr=5.97e-04 tps=17578 mem=6.3G lam=None Gm=0.528076171875 +[g_only s0] step 740/12000 loss=5.9092 gn=0.47 lr=5.97e-04 tps=17584 mem=6.3G lam=None Gm=0.5430501302083334 +[g_only s0] step 760/12000 loss=5.9035 gn=0.49 lr=5.97e-04 tps=17591 mem=6.3G lam=None Gm=0.5521647135416666 +[g_only s0] step 780/12000 loss=5.9155 gn=0.49 lr=5.97e-04 tps=17597 mem=6.3G lam=None Gm=0.5515950520833334 +[g_only s0] step 800/12000 loss=5.8089 gn=0.52 lr=5.97e-04 tps=17602 mem=6.3G lam=None Gm=0.5675455729166666 +[g_only s0] step 820/12000 loss=5.7905 gn=0.49 lr=5.96e-04 tps=17607 mem=6.3G lam=None Gm=0.569091796875 +[g_only s0] step 840/12000 loss=5.7856 gn=0.51 lr=5.96e-04 tps=17611 mem=6.3G lam=None Gm=0.559326171875 +[g_only s0] step 860/12000 loss=5.7313 gn=0.52 lr=5.96e-04 tps=17615 mem=6.3G lam=None Gm=0.570068359375 +[g_only s0] step 880/12000 loss=5.8922 gn=0.47 lr=5.96e-04 tps=17620 mem=6.3G lam=None Gm=0.5540364583333334 +[g_only s0] step 900/12000 loss=5.7237 gn=0.50 lr=5.95e-04 tps=17624 mem=6.3G lam=None Gm=0.5313720703125 +[g_only s0] step 920/12000 loss=5.7638 gn=0.51 lr=5.95e-04 tps=17627 mem=6.3G lam=None Gm=0.5430501302083334 +[g_only s0] step 940/12000 loss=5.7276 gn=0.49 lr=5.95e-04 tps=17629 mem=6.3G lam=None Gm=0.5469156901041666 +[g_only s0] step 960/12000 loss=5.7675 gn=0.53 lr=5.94e-04 tps=17632 mem=6.3G lam=None Gm=0.5422770182291666 +[g_only s0] step 980/12000 loss=5.8863 gn=0.54 lr=5.94e-04 tps=17635 mem=6.3G lam=None Gm=0.5409342447916666 +[g_only s0] step 1000/12000 loss=5.8094 gn=0.56 lr=5.94e-04 tps=17637 mem=6.3G lam=None Gm=0.5491943359375 +[g_only s0] EVAL step 1000: val_loss=5.8058 val_ppl=332.23 +[g_only s0] step 1020/12000 loss=5.6155 gn=0.49 lr=5.94e-04 tps=17513 mem=6.3G lam=None Gm=0.5557454427083334 +[g_only s0] step 1040/12000 loss=5.6844 gn=0.51 lr=5.93e-04 tps=17517 mem=6.3G lam=None Gm=0.5449625651041666 +[g_only s0] step 1060/12000 loss=5.7501 gn=0.52 lr=5.93e-04 tps=17523 mem=6.3G lam=None Gm=0.5424397786458334 +[g_only s0] step 1080/12000 loss=5.7182 gn=0.53 lr=5.93e-04 tps=17526 mem=6.3G lam=None Gm=0.5396728515625 +[g_only s0] step 1100/12000 loss=5.6004 gn=0.51 lr=5.92e-04 tps=17531 mem=6.3G lam=None Gm=0.531982421875 +[g_only s0] step 1120/12000 loss=5.7146 gn=0.54 lr=5.92e-04 tps=17535 mem=6.3G lam=None Gm=0.5465087890625 +[g_only s0] step 1140/12000 loss=5.5299 gn=0.54 lr=5.92e-04 tps=17539 mem=6.3G lam=None Gm=0.5310465494791666 +[g_only s0] step 1160/12000 loss=5.5716 gn=0.50 lr=5.91e-04 tps=17543 mem=6.3G lam=None Gm=0.5498860677083334 +[g_only s0] step 1180/12000 loss=5.6297 gn=0.53 lr=5.91e-04 tps=17547 mem=6.3G lam=None Gm=0.5210774739583334 +[g_only s0] step 1200/12000 loss=5.5101 gn=0.53 lr=5.90e-04 tps=17552 mem=6.3G lam=None Gm=0.5207112630208334 +[g_only s0] step 1220/12000 loss=5.6987 gn=0.52 lr=5.90e-04 tps=17555 mem=6.3G lam=None Gm=0.5294189453125 +[g_only s0] step 1240/12000 loss=5.6557 gn=0.54 lr=5.90e-04 tps=17559 mem=6.3G lam=None Gm=0.5279134114583334 +[g_only s0] step 1260/12000 loss=5.6250 gn=0.59 lr=5.89e-04 tps=17564 mem=6.3G lam=None Gm=0.5297444661458334 +[g_only s0] step 1280/12000 loss=5.4801 gn=0.56 lr=5.89e-04 tps=17567 mem=6.3G lam=None Gm=0.5185139973958334 +[g_only s0] step 1300/12000 loss=5.5676 gn=0.56 lr=5.89e-04 tps=17570 mem=6.3G lam=None Gm=0.522705078125 +[g_only s0] step 1320/12000 loss=5.5267 gn=0.54 lr=5.88e-04 tps=17573 mem=6.3G lam=None Gm=0.5242919921875 +[g_only s0] step 1340/12000 loss=5.6132 gn=0.53 lr=5.88e-04 tps=17576 mem=6.3G lam=None Gm=0.5221761067708334 +[g_only s0] step 1360/12000 loss=5.4278 gn=0.52 lr=5.87e-04 tps=17579 mem=6.3G lam=None Gm=0.520751953125 +[g_only s0] step 1380/12000 loss=5.2792 gn=0.55 lr=5.87e-04 tps=17583 mem=6.3G lam=None Gm=0.5063069661458334 +[g_only s0] step 1400/12000 loss=5.5660 gn=0.54 lr=5.86e-04 tps=17586 mem=6.3G lam=None Gm=0.5053304036458334 +[g_only s0] step 1420/12000 loss=5.5058 gn=0.54 lr=5.86e-04 tps=17589 mem=6.3G lam=None Gm=0.5075887044270834 +[g_only s0] step 1440/12000 loss=5.4098 gn=0.55 lr=5.85e-04 tps=17591 mem=6.3G lam=None Gm=0.506591796875 +[g_only s0] step 1460/12000 loss=5.4990 gn=0.59 lr=5.85e-04 tps=17593 mem=6.3G lam=None Gm=0.5242513020833334 +[g_only s0] step 1480/12000 loss=5.4127 gn=0.55 lr=5.84e-04 tps=17595 mem=6.3G lam=None Gm=0.4999186197916667 +[g_only s0] step 1500/12000 loss=5.6498 gn=0.60 lr=5.84e-04 tps=17597 mem=6.3G lam=None Gm=0.49700927734375 +[g_only s0] EVAL step 1500: val_loss=5.5196 val_ppl=249.54 +[g_only s0] step 1520/12000 loss=5.4345 gn=0.58 lr=5.83e-04 tps=17513 mem=6.3G lam=None Gm=0.5090128580729166 +[g_only s0] step 1540/12000 loss=5.2837 gn=0.55 lr=5.83e-04 tps=17517 mem=6.3G lam=None Gm=0.4920247395833333 +[g_only s0] step 1560/12000 loss=5.5236 gn=0.65 lr=5.82e-04 tps=17520 mem=6.3G lam=None Gm=0.5075480143229166 +[g_only s0] step 1580/12000 loss=5.4258 gn=0.59 lr=5.82e-04 tps=17524 mem=6.3G lam=None Gm=0.4908040364583333 +[g_only s0] step 1600/12000 loss=5.5376 gn=0.60 lr=5.81e-04 tps=17527 mem=6.3G lam=None Gm=0.4931844075520833 +[g_only s0] step 1620/12000 loss=5.3325 gn=0.56 lr=5.81e-04 tps=17530 mem=6.3G lam=None Gm=0.4916585286458333 +[g_only s0] step 1640/12000 loss=5.3901 gn=0.52 lr=5.80e-04 tps=17533 mem=6.3G lam=None Gm=0.4882405598958333 +[g_only s0] step 1660/12000 loss=5.3795 gn=0.58 lr=5.80e-04 tps=17536 mem=6.3G lam=None Gm=0.4827473958333333 +[g_only s0] step 1680/12000 loss=5.4730 gn=0.53 lr=5.79e-04 tps=17539 mem=6.3G lam=None Gm=0.4774576822916667 +[g_only s0] step 1700/12000 loss=5.4040 gn=0.58 lr=5.79e-04 tps=17542 mem=6.3G lam=None Gm=0.484619140625 +[g_only s0] step 1720/12000 loss=5.3436 gn=0.66 lr=5.78e-04 tps=17545 mem=6.3G lam=None Gm=0.486328125 +[g_only s0] step 1740/12000 loss=5.4290 gn=0.69 lr=5.78e-04 tps=17548 mem=6.3G lam=None Gm=0.4794108072916667 +[g_only s0] step 1760/12000 loss=5.3971 gn=0.55 lr=5.77e-04 tps=17551 mem=6.3G lam=None Gm=0.4796142578125 +[g_only s0] step 1780/12000 loss=5.3164 gn=0.54 lr=5.76e-04 tps=17554 mem=6.3G lam=None Gm=0.4614054361979167 +[g_only s0] step 1800/12000 loss=5.3619 gn=0.68 lr=5.76e-04 tps=17557 mem=6.3G lam=None Gm=0.4594523111979167 +[g_only s0] step 1820/12000 loss=5.2972 gn=0.55 lr=5.75e-04 tps=17560 mem=6.3G lam=None Gm=0.4691975911458333 +[g_only s0] step 1840/12000 loss=5.4051 gn=0.67 lr=5.75e-04 tps=17563 mem=6.3G lam=None Gm=0.4773966471354167 +[g_only s0] step 1860/12000 loss=5.3593 gn=0.59 lr=5.74e-04 tps=17565 mem=6.3G lam=None Gm=0.4798787434895833 +[g_only s0] step 1880/12000 loss=5.3028 gn=0.58 lr=5.73e-04 tps=17568 mem=6.3G lam=None Gm=0.46502685546875 +[g_only s0] step 1900/12000 loss=5.2472 gn=0.57 lr=5.73e-04 tps=17570 mem=6.3G lam=None Gm=0.4611612955729167 +[g_only s0] step 1920/12000 loss=5.2073 gn=0.61 lr=5.72e-04 tps=17572 mem=6.3G lam=None Gm=0.4652506510416667 +[g_only s0] step 1940/12000 loss=5.1944 gn=0.63 lr=5.72e-04 tps=17573 mem=6.3G lam=None Gm=0.44989013671875 +[g_only s0] step 1960/12000 loss=5.1711 gn=0.62 lr=5.71e-04 tps=17573 mem=6.3G lam=None Gm=0.45770263671875 +[g_only s0] step 1980/12000 loss=5.0529 gn=0.60 lr=5.70e-04 tps=17574 mem=6.3G lam=None Gm=0.4552408854166667 +[g_only s0] step 2000/12000 loss=5.2192 gn=0.62 lr=5.70e-04 tps=17574 mem=6.3G lam=None Gm=0.45849609375 +[g_only s0] EVAL step 2000: val_loss=5.3300 val_ppl=206.44 +[g_only s0] step 2020/12000 loss=5.1103 gn=0.70 lr=5.69e-04 tps=17509 mem=6.3G lam=None Gm=0.4582926432291667 +[g_only s0] step 2040/12000 loss=5.3316 gn=1.00 lr=5.68e-04 tps=17508 mem=6.3G lam=None Gm=0.47747802734375 +[g_only s0] step 2060/12000 loss=5.3017 gn=0.57 lr=5.68e-04 tps=17506 mem=6.3G lam=None Gm=0.4581705729166667 +[g_only s0] step 2080/12000 loss=5.2231 gn=0.72 lr=5.67e-04 tps=17504 mem=6.3G lam=None Gm=0.46868896484375 +[g_only s0] step 2100/12000 loss=5.2437 gn=0.60 lr=5.66e-04 tps=17504 mem=6.3G lam=None Gm=0.4566446940104167 +[g_only s0] step 2120/12000 loss=5.2091 gn=0.69 lr=5.65e-04 tps=17503 mem=6.3G lam=None Gm=0.4367167154947917 +[g_only s0] step 2140/12000 loss=5.2685 gn=0.63 lr=5.65e-04 tps=17502 mem=6.3G lam=None Gm=0.4484456380208333 +[g_only s0] step 2160/12000 loss=5.2950 gn=0.62 lr=5.64e-04 tps=17500 mem=6.3G lam=None Gm=0.4414265950520833 +[g_only s0] step 2180/12000 loss=5.2707 gn=0.63 lr=5.63e-04 tps=17502 mem=6.3G lam=None Gm=0.441680908203125 +[g_only s0] step 2200/12000 loss=5.2073 gn=0.59 lr=5.63e-04 tps=17504 mem=6.3G lam=None Gm=0.4357808430989583 +[g_only s0] step 2220/12000 loss=5.2005 gn=0.60 lr=5.62e-04 tps=17507 mem=6.3G lam=None Gm=0.4326883951822917 +[g_only s0] step 2240/12000 loss=5.1011 gn=0.63 lr=5.61e-04 tps=17509 mem=6.3G lam=None Gm=0.449371337890625 +[g_only s0] step 2260/12000 loss=5.2164 gn=0.62 lr=5.60e-04 tps=17511 mem=6.3G lam=None Gm=0.443023681640625 +[g_only s0] step 2280/12000 loss=5.1463 gn=0.73 lr=5.60e-04 tps=17514 mem=6.3G lam=None Gm=0.4376627604166667 +[g_only s0] step 2300/12000 loss=5.1475 gn=0.68 lr=5.59e-04 tps=17516 mem=6.3G lam=None Gm=0.4483235677083333 +[g_only s0] step 2320/12000 loss=5.1225 gn=0.71 lr=5.58e-04 tps=17518 mem=6.3G lam=None Gm=0.4344380696614583 +[g_only s0] step 2340/12000 loss=5.0559 gn=0.67 lr=5.57e-04 tps=17520 mem=6.3G lam=None Gm=0.434173583984375 +[g_only s0] step 2360/12000 loss=5.1724 gn=0.73 lr=5.57e-04 tps=17522 mem=6.3G lam=None Gm=0.4397684733072917 +[g_only s0] step 2380/12000 loss=5.2440 gn=0.68 lr=5.56e-04 tps=17524 mem=6.3G lam=None Gm=0.4410909016927083 +[g_only s0] step 2400/12000 loss=5.1251 gn=0.63 lr=5.55e-04 tps=17527 mem=6.3G lam=None Gm=0.4363505045572917 +[g_only s0] step 2420/12000 loss=5.2225 gn=0.68 lr=5.54e-04 tps=17529 mem=6.3G lam=None Gm=0.43060302734375 +[g_only s0] step 2440/12000 loss=5.0349 gn=0.63 lr=5.53e-04 tps=17530 mem=6.3G lam=None Gm=0.4239095052083333 +[g_only s0] step 2460/12000 loss=5.0822 gn=0.69 lr=5.53e-04 tps=17532 mem=6.3G lam=None Gm=0.434173583984375 +[g_only s0] step 2480/12000 loss=5.0275 gn=0.66 lr=5.52e-04 tps=17534 mem=6.3G lam=None Gm=0.436767578125 +[g_only s0] step 2500/12000 loss=5.1960 gn=0.66 lr=5.51e-04 tps=17535 mem=6.3G lam=None Gm=0.4343058268229167 +[g_only s0] EVAL step 2500: val_loss=5.1741 val_ppl=176.63 +[g_only s0] step 2520/12000 loss=5.0969 gn=0.63 lr=5.50e-04 tps=17486 mem=6.3G lam=None Gm=0.430694580078125 +[g_only s0] step 2540/12000 loss=4.9838 gn=0.70 lr=5.49e-04 tps=17488 mem=6.3G lam=None Gm=0.4347737630208333 +[g_only s0] step 2560/12000 loss=5.2069 gn=0.69 lr=5.48e-04 tps=17489 mem=6.3G lam=None Gm=0.4235331217447917 +[g_only s0] step 2580/12000 loss=5.0807 gn=0.72 lr=5.48e-04 tps=17491 mem=6.3G lam=None Gm=0.4281209309895833 +[g_only s0] step 2600/12000 loss=4.9438 gn=0.65 lr=5.47e-04 tps=17493 mem=6.3G lam=None Gm=0.4200948079427083 +[g_only s0] step 2620/12000 loss=4.9656 gn=0.65 lr=5.46e-04 tps=17495 mem=6.3G lam=None Gm=0.4268290201822917 +[g_only s0] step 2640/12000 loss=5.1027 gn=0.64 lr=5.45e-04 tps=17497 mem=6.3G lam=None Gm=0.41314697265625 +[g_only s0] step 2660/12000 loss=5.0512 gn=0.68 lr=5.44e-04 tps=17499 mem=6.3G lam=None Gm=0.421112060546875 +[g_only s0] step 2680/12000 loss=5.0421 gn=0.67 lr=5.43e-04 tps=17501 mem=6.3G lam=None Gm=0.415191650390625 +[g_only s0] step 2700/12000 loss=5.0449 gn=0.64 lr=5.42e-04 tps=17503 mem=6.3G lam=None Gm=0.4313252766927083 +[g_only s0] step 2720/12000 loss=5.0200 gn=0.61 lr=5.41e-04 tps=17505 mem=6.3G lam=None Gm=0.4196573893229167 +[g_only s0] step 2740/12000 loss=5.0846 gn=0.68 lr=5.41e-04 tps=17507 mem=6.3G lam=None Gm=0.4194844563802083 +[g_only s0] step 2760/12000 loss=5.2395 gn=0.69 lr=5.40e-04 tps=17509 mem=6.3G lam=None Gm=0.4131571451822917 +[g_only s0] step 2780/12000 loss=5.2040 gn=0.68 lr=5.39e-04 tps=17511 mem=6.3G lam=None Gm=0.415252685546875 +[g_only s0] step 2800/12000 loss=4.9866 gn=0.64 lr=5.38e-04 tps=17513 mem=6.3G lam=None Gm=0.425384521484375 +[g_only s0] step 2820/12000 loss=5.2668 gn=0.71 lr=5.37e-04 tps=17515 mem=6.3G lam=None Gm=0.420623779296875 +[g_only s0] step 2840/12000 loss=5.0066 gn=0.69 lr=5.36e-04 tps=17517 mem=6.3G lam=None Gm=0.4202779134114583 +[g_only s0] step 2860/12000 loss=5.0308 gn=0.65 lr=5.35e-04 tps=17519 mem=6.3G lam=None Gm=0.4146931966145833 +[g_only s0] step 2880/12000 loss=5.0774 gn=0.71 lr=5.34e-04 tps=17521 mem=6.3G lam=None Gm=0.419219970703125 +[g_only s0] step 2900/12000 loss=4.8836 gn=0.63 lr=5.33e-04 tps=17523 mem=6.3G lam=None Gm=0.4025675455729167 +[g_only s0] step 2920/12000 loss=5.0361 gn=0.70 lr=5.32e-04 tps=17525 mem=6.3G lam=None Gm=0.4069112141927083 +[g_only s0] step 2940/12000 loss=5.0702 gn=0.69 lr=5.31e-04 tps=17527 mem=6.3G lam=None Gm=0.4210306803385417 +[g_only s0] step 2960/12000 loss=4.9214 gn=0.69 lr=5.30e-04 tps=17529 mem=6.3G lam=None Gm=0.4150899251302083 +[g_only s0] step 2980/12000 loss=5.0774 gn=0.66 lr=5.29e-04 tps=17530 mem=6.3G lam=None Gm=0.4201558430989583 +[g_only s0] step 3000/12000 loss=5.0681 gn=0.70 lr=5.28e-04 tps=17531 mem=6.3G lam=None Gm=0.40338134765625 +[g_only s0] EVAL step 3000: val_loss=5.0174 val_ppl=151.02 +[g_only s0] step 3020/12000 loss=5.0132 gn=0.69 lr=5.27e-04 tps=17490 mem=6.3G lam=None Gm=0.4050089518229167 +[g_only s0] step 3040/12000 loss=5.1460 gn=0.75 lr=5.26e-04 tps=17492 mem=6.3G lam=None Gm=0.4163614908854167 +[g_only s0] step 3060/12000 loss=4.8892 gn=0.62 lr=5.25e-04 tps=17493 mem=6.3G lam=None Gm=0.4075419108072917 +[g_only s0] step 3080/12000 loss=4.9639 gn=0.71 lr=5.24e-04 tps=17493 mem=6.3G lam=None Gm=0.414886474609375 +[g_only s0] step 3100/12000 loss=4.8920 gn=0.70 lr=5.23e-04 tps=17494 mem=6.3G lam=None Gm=0.4037577311197917 +[g_only s0] step 3120/12000 loss=4.8590 gn=0.67 lr=5.22e-04 tps=17494 mem=6.3G lam=None Gm=0.407501220703125 +[g_only s0] step 3140/12000 loss=5.0326 gn=0.67 lr=5.21e-04 tps=17495 mem=6.3G lam=None Gm=0.412384033203125 +[g_only s0] step 3160/12000 loss=4.8232 gn=0.65 lr=5.20e-04 tps=17495 mem=6.3G lam=None Gm=0.400726318359375 +[g_only s0] step 3180/12000 loss=4.8672 gn=0.77 lr=5.19e-04 tps=17496 mem=6.3G lam=None Gm=0.4073994954427083 +[g_only s0] step 3200/12000 loss=4.8247 gn=0.64 lr=5.18e-04 tps=17496 mem=6.3G lam=None Gm=0.38946533203125 +[g_only s0] step 3220/12000 loss=4.7886 gn=0.76 lr=5.17e-04 tps=17497 mem=6.3G lam=None Gm=0.4090067545572917 +[g_only s0] step 3240/12000 loss=4.8613 gn=0.71 lr=5.16e-04 tps=17497 mem=6.3G lam=None Gm=0.410736083984375 +[g_only s0] step 3260/12000 loss=4.9857 gn=0.71 lr=5.15e-04 tps=17498 mem=6.3G lam=None Gm=0.398529052734375 +[g_only s0] step 3280/12000 loss=4.8401 gn=0.69 lr=5.14e-04 tps=17499 mem=6.3G lam=None Gm=0.3934224446614583 +[g_only s0] step 3300/12000 loss=5.0281 gn=0.73 lr=5.13e-04 tps=17499 mem=6.3G lam=None Gm=0.4038289388020833 +[g_only s0] step 3320/12000 loss=4.8799 gn=0.71 lr=5.12e-04 tps=17500 mem=6.3G lam=None Gm=0.40118408203125 +[g_only s0] step 3340/12000 loss=4.9655 gn=0.71 lr=5.11e-04 tps=17500 mem=6.3G lam=None Gm=0.4032185872395833 +[g_only s0] step 3360/12000 loss=4.7661 gn=0.72 lr=5.10e-04 tps=17501 mem=6.3G lam=None Gm=0.4085286458333333 +[g_only s0] step 3380/12000 loss=4.9219 gn=1.13 lr=5.09e-04 tps=17502 mem=6.3G lam=None Gm=0.3979593912760417 +[g_only s0] step 3400/12000 loss=4.9601 gn=0.74 lr=5.08e-04 tps=17501 mem=6.3G lam=None Gm=0.3922017415364583 +[g_only s0] step 3420/12000 loss=4.8041 gn=0.74 lr=5.07e-04 tps=17502 mem=6.3G lam=None Gm=0.3995157877604167 +[g_only s0] step 3440/12000 loss=4.8969 gn=0.71 lr=5.06e-04 tps=17503 mem=6.3G lam=None Gm=0.3961283365885417 +[g_only s0] step 3460/12000 loss=4.8046 gn=0.71 lr=5.05e-04 tps=17503 mem=6.3G lam=None Gm=0.3955281575520833 +[g_only s0] step 3480/12000 loss=4.7783 gn=0.71 lr=5.03e-04 tps=17503 mem=6.3G lam=None Gm=0.3877054850260417 +[g_only s0] step 3500/12000 loss=4.9474 gn=0.73 lr=5.02e-04 tps=17504 mem=6.3G lam=None Gm=0.3987019856770833 +[g_only s0] EVAL step 3500: val_loss=4.9194 val_ppl=136.93 +[g_only s0] step 3520/12000 loss=4.9313 gn=0.70 lr=5.01e-04 tps=17467 mem=6.3G lam=None Gm=0.4000752766927083 +[g_only s0] step 3540/12000 loss=4.7827 gn=0.71 lr=5.00e-04 tps=17468 mem=6.3G lam=None Gm=0.3926595052083333 +[g_only s0] step 3560/12000 loss=4.8448 gn=0.72 lr=4.99e-04 tps=17468 mem=6.3G lam=None Gm=0.3927001953125 +[g_only s0] step 3580/12000 loss=4.9026 gn=0.84 lr=4.98e-04 tps=17468 mem=6.3G lam=None Gm=0.3882853190104167 +[g_only s0] step 3600/12000 loss=4.7758 gn=0.70 lr=4.97e-04 tps=17469 mem=6.3G lam=None Gm=0.3961385091145833 +[g_only s0] step 3620/12000 loss=4.8546 gn=0.73 lr=4.96e-04 tps=17469 mem=6.3G lam=None Gm=0.3955485026041667 +[g_only s0] step 3640/12000 loss=4.9126 gn=0.72 lr=4.94e-04 tps=17470 mem=6.3G lam=None Gm=0.3892822265625 +[g_only s0] step 3660/12000 loss=4.8022 gn=0.70 lr=4.93e-04 tps=17470 mem=6.3G lam=None Gm=0.3961588541666667 +[g_only s0] step 3680/12000 loss=4.9270 gn=0.74 lr=4.92e-04 tps=17471 mem=6.3G lam=None Gm=0.396026611328125 +[g_only s0] step 3700/12000 loss=4.7649 gn=0.74 lr=4.91e-04 tps=17471 mem=6.3G lam=None Gm=0.3911234537760417 +[g_only s0] step 3720/12000 loss=4.9020 gn=0.80 lr=4.90e-04 tps=17472 mem=6.3G lam=None Gm=0.3965555826822917 +[g_only s0] step 3740/12000 loss=4.8693 gn=0.75 lr=4.89e-04 tps=17472 mem=6.3G lam=None Gm=0.388153076171875 +[g_only s0] step 3760/12000 loss=4.9337 gn=0.78 lr=4.88e-04 tps=17473 mem=6.3G lam=None Gm=0.39404296875 +[g_only s0] step 3780/12000 loss=4.8962 gn=0.72 lr=4.86e-04 tps=17473 mem=6.3G lam=None Gm=0.4054158528645833 +[g_only s0] step 3800/12000 loss=4.8798 gn=0.74 lr=4.85e-04 tps=17474 mem=6.3G lam=None Gm=0.39794921875 +[g_only s0] step 3820/12000 loss=4.6627 gn=0.77 lr=4.84e-04 tps=17474 mem=6.3G lam=None Gm=0.38873291015625 +[g_only s0] step 3840/12000 loss=4.9551 gn=0.80 lr=4.83e-04 tps=17475 mem=6.3G lam=None Gm=0.388519287109375 +[g_only s0] step 3860/12000 loss=4.8363 gn=0.71 lr=4.82e-04 tps=17475 mem=6.3G lam=None Gm=0.388458251953125 +[g_only s0] step 3880/12000 loss=4.7405 gn=0.73 lr=4.80e-04 tps=17476 mem=6.3G lam=None Gm=0.39044189453125 +[g_only s0] step 3900/12000 loss=4.7199 gn=0.74 lr=4.79e-04 tps=17477 mem=6.3G lam=None Gm=0.3866984049479167 +[g_only s0] step 3920/12000 loss=4.8293 gn=0.76 lr=4.78e-04 tps=17477 mem=6.3G lam=None Gm=0.389434814453125 +[g_only s0] step 3940/12000 loss=4.6612 gn=0.73 lr=4.77e-04 tps=17478 mem=6.3G lam=None Gm=0.388763427734375 +[g_only s0] step 3960/12000 loss=4.7761 gn=0.72 lr=4.76e-04 tps=17479 mem=6.3G lam=None Gm=0.3973592122395833 +[g_only s0] step 3980/12000 loss=4.6875 gn=0.71 lr=4.74e-04 tps=17480 mem=6.3G lam=None Gm=0.3859456380208333 +[g_only s0] step 4000/12000 loss=4.7600 gn=0.71 lr=4.73e-04 tps=17480 mem=6.3G lam=None Gm=0.4009653727213542 +[g_only s0] EVAL step 4000: val_loss=4.8781 val_ppl=131.38 +[g_only s0] step 4020/12000 loss=4.9247 gn=0.73 lr=4.72e-04 tps=17448 mem=6.3G lam=None Gm=0.3859659830729167 +[g_only s0] step 4040/12000 loss=4.9006 gn=0.76 lr=4.71e-04 tps=17449 mem=6.3G lam=None Gm=0.3737284342447917 +[g_only s0] step 4060/12000 loss=4.7574 gn=0.72 lr=4.70e-04 tps=17450 mem=6.3G lam=None Gm=0.3878987630208333 +[g_only s0] step 4080/12000 loss=4.7927 gn=0.70 lr=4.68e-04 tps=17450 mem=6.3G lam=None Gm=0.3797810872395833 +[g_only s0] step 4100/12000 loss=4.6887 gn=0.75 lr=4.67e-04 tps=17451 mem=6.3G lam=None Gm=0.3900553385416667 +[g_only s0] step 4120/12000 loss=4.6250 gn=0.74 lr=4.66e-04 tps=17452 mem=6.3G lam=None Gm=0.3846842447916667 +[g_only s0] step 4140/12000 loss=4.7403 gn=0.77 lr=4.65e-04 tps=17453 mem=6.3G lam=None Gm=0.388519287109375 +[g_only s0] step 4160/12000 loss=4.7262 gn=0.77 lr=4.63e-04 tps=17453 mem=6.3G lam=None Gm=0.3886617024739583 +[g_only s0] step 4180/12000 loss=4.9169 gn=0.74 lr=4.62e-04 tps=17454 mem=6.3G lam=None Gm=0.3912404378255208 +[g_only s0] step 4200/12000 loss=4.7369 gn=0.77 lr=4.61e-04 tps=17455 mem=6.3G lam=None Gm=0.3883463541666667 +[g_only s0] step 4220/12000 loss=4.8178 gn=0.76 lr=4.60e-04 tps=17456 mem=6.3G lam=None Gm=0.3831532796223958 +[g_only s0] step 4240/12000 loss=4.8310 gn=0.72 lr=4.58e-04 tps=17457 mem=6.3G lam=None Gm=0.3846893310546875 +[g_only s0] step 4260/12000 loss=4.8384 gn=0.77 lr=4.57e-04 tps=17458 mem=6.3G lam=None Gm=0.3757883707682292 +[g_only s0] step 4280/12000 loss=4.8723 gn=0.86 lr=4.56e-04 tps=17459 mem=6.3G lam=None Gm=0.390167236328125 +[g_only s0] step 4300/12000 loss=4.8395 gn=0.75 lr=4.54e-04 tps=17460 mem=6.3G lam=None Gm=0.385894775390625 +[g_only s0] step 4320/12000 loss=4.6879 gn=0.70 lr=4.53e-04 tps=17461 mem=6.3G lam=None Gm=0.3777669270833333 +[g_only s0] step 4340/12000 loss=4.7126 gn=0.77 lr=4.52e-04 tps=17462 mem=6.3G lam=None Gm=0.3906809488932292 +[g_only s0] step 4360/12000 loss=4.8663 gn=0.75 lr=4.51e-04 tps=17463 mem=6.3G lam=None Gm=0.3986256917317708 +[g_only s0] step 4380/12000 loss=4.8447 gn=0.75 lr=4.49e-04 tps=17464 mem=6.3G lam=None Gm=0.38287353515625 +[g_only s0] step 4400/12000 loss=4.5839 gn=0.78 lr=4.48e-04 tps=17464 mem=6.3G lam=None Gm=0.3852030436197917 +[g_only s0] step 4420/12000 loss=4.8736 gn=0.74 lr=4.47e-04 tps=17465 mem=6.3G lam=None Gm=0.3882954915364583 +[g_only s0] step 4440/12000 loss=4.7160 gn=0.80 lr=4.45e-04 tps=17466 mem=6.3G lam=None Gm=0.3847859700520833 +[g_only s0] step 4460/12000 loss=4.7325 gn=0.78 lr=4.44e-04 tps=17467 mem=6.3G lam=None Gm=0.3731180826822917 +[g_only s0] step 4480/12000 loss=4.7029 gn=0.75 lr=4.43e-04 tps=17467 mem=6.3G lam=None Gm=0.3797200520833333 +[g_only s0] step 4500/12000 loss=4.7199 gn=0.81 lr=4.42e-04 tps=17468 mem=6.3G lam=None Gm=0.3925984700520833 +[g_only s0] EVAL step 4500: val_loss=4.8126 val_ppl=123.05 +[g_only s0] step 4520/12000 loss=4.7101 gn=0.75 lr=4.40e-04 tps=17440 mem=6.3G lam=None Gm=0.3822123209635417 +[g_only s0] step 4540/12000 loss=4.7683 gn=0.76 lr=4.39e-04 tps=17441 mem=6.3G lam=None Gm=0.3768259684244792 +[g_only s0] step 4560/12000 loss=4.7668 gn=0.73 lr=4.38e-04 tps=17442 mem=6.3G lam=None Gm=0.3874664306640625 +[g_only s0] step 4580/12000 loss=4.7308 gn=0.75 lr=4.36e-04 tps=17443 mem=6.3G lam=None Gm=0.382080078125 +[g_only s0] step 4600/12000 loss=4.7376 gn=0.80 lr=4.35e-04 tps=17443 mem=6.3G lam=None Gm=0.3768107096354167 +[g_only s0] step 4620/12000 loss=4.7610 gn=0.80 lr=4.34e-04 tps=17444 mem=6.3G lam=None Gm=0.3824462890625 +[g_only s0] step 4640/12000 loss=4.8049 gn=0.81 lr=4.32e-04 tps=17445 mem=6.3G lam=None Gm=0.3794657389322917 +[g_only s0] step 4660/12000 loss=4.6857 gn=0.78 lr=4.31e-04 tps=17445 mem=6.3G lam=None Gm=0.3775278727213542 +[g_only s0] step 4680/12000 loss=4.7334 gn=0.78 lr=4.30e-04 tps=17446 mem=6.3G lam=None Gm=0.3793538411458333 +[g_only s0] step 4700/12000 loss=4.6957 gn=0.79 lr=4.28e-04 tps=17447 mem=6.3G lam=None Gm=0.3828175862630208 +[g_only s0] step 4720/12000 loss=4.7321 gn=0.77 lr=4.27e-04 tps=17448 mem=6.3G lam=None Gm=0.3809000651041667 +[g_only s0] step 4740/12000 loss=4.6116 gn=0.84 lr=4.26e-04 tps=17448 mem=6.3G lam=None Gm=0.379638671875 +[g_only s0] step 4760/12000 loss=4.7081 gn=0.82 lr=4.24e-04 tps=17449 mem=6.3G lam=None Gm=0.3828175862630208 +[g_only s0] step 4780/12000 loss=4.7373 gn=0.74 lr=4.23e-04 tps=17450 mem=6.3G lam=None Gm=0.3817342122395833 +[g_only s0] step 4800/12000 loss=4.8097 gn=0.77 lr=4.22e-04 tps=17451 mem=6.3G lam=None Gm=0.3801676432291667 +[g_only s0] step 4820/12000 loss=4.7108 gn=0.79 lr=4.20e-04 tps=17452 mem=6.3G lam=None Gm=0.3775227864583333 +[g_only s0] step 4840/12000 loss=4.4690 gn=0.81 lr=4.19e-04 tps=17453 mem=6.3G lam=None Gm=0.3770650227864583 +[g_only s0] step 4860/12000 loss=4.7341 gn=0.77 lr=4.18e-04 tps=17454 mem=6.3G lam=None Gm=0.3760121663411458 +[g_only s0] step 4880/12000 loss=4.7129 gn=0.79 lr=4.16e-04 tps=17454 mem=6.3G lam=None Gm=0.37689208984375 +[g_only s0] step 4900/12000 loss=4.5292 gn=0.73 lr=4.15e-04 tps=17455 mem=6.3G lam=None Gm=0.3742116292317708 +[g_only s0] step 4920/12000 loss=4.7526 gn=0.80 lr=4.13e-04 tps=17456 mem=6.3G lam=None Gm=0.3795369466145833 +[g_only s0] step 4940/12000 loss=4.6449 gn=0.75 lr=4.12e-04 tps=17457 mem=6.3G lam=None Gm=0.3755289713541667 +[g_only s0] step 4960/12000 loss=4.7088 gn=0.74 lr=4.11e-04 tps=17457 mem=6.3G lam=None Gm=0.3860880533854167 +[g_only s0] step 4980/12000 loss=4.5575 gn=0.73 lr=4.09e-04 tps=17458 mem=6.3G lam=None Gm=0.3737131754557292 +[g_only s0] step 5000/12000 loss=4.7514 gn=0.87 lr=4.08e-04 tps=17459 mem=6.3G lam=None Gm=0.3782806396484375 +[g_only s0] EVAL step 5000: val_loss=4.7690 val_ppl=117.81 +[g_only s0] step 5020/12000 loss=4.6522 gn=0.85 lr=4.07e-04 tps=17434 mem=6.3G lam=None Gm=0.3793589274088542 +[g_only s0] step 5040/12000 loss=4.6683 gn=0.80 lr=4.05e-04 tps=17435 mem=6.3G lam=None Gm=0.3814646402994792 +[g_only s0] step 5060/12000 loss=4.5594 gn=0.82 lr=4.04e-04 tps=17435 mem=6.3G lam=None Gm=0.3766377766927083 +[g_only s0] step 5080/12000 loss=4.6669 gn=0.78 lr=4.02e-04 tps=17436 mem=6.3G lam=None Gm=0.378265380859375 +[g_only s0] step 5100/12000 loss=4.6729 gn=0.74 lr=4.01e-04 tps=17437 mem=6.3G lam=None Gm=0.3864593505859375 +[g_only s0] step 5120/12000 loss=4.5919 gn=0.76 lr=4.00e-04 tps=17438 mem=6.3G lam=None Gm=0.3751322428385417 +[g_only s0] step 5140/12000 loss=4.7002 gn=0.80 lr=3.98e-04 tps=17438 mem=6.3G lam=None Gm=0.3781382242838542 +[g_only s0] step 5160/12000 loss=4.5504 gn=0.82 lr=3.97e-04 tps=17439 mem=6.3G lam=None Gm=0.3792063395182292 +[g_only s0] step 5180/12000 loss=4.5864 gn=0.82 lr=3.95e-04 tps=17440 mem=6.3G lam=None Gm=0.3753407796223958 +[g_only s0] step 5200/12000 loss=4.5153 gn=0.81 lr=3.94e-04 tps=17441 mem=6.3G lam=None Gm=0.3815968831380208 +[g_only s0] step 5220/12000 loss=4.6232 gn=0.78 lr=3.93e-04 tps=17442 mem=6.3G lam=None Gm=0.3809661865234375 +[g_only s0] step 5240/12000 loss=4.5887 gn=0.78 lr=3.91e-04 tps=17442 mem=6.3G lam=None Gm=0.3839111328125 +[g_only s0] step 5260/12000 loss=4.5639 gn=0.73 lr=3.90e-04 tps=17443 mem=6.3G lam=None Gm=0.3776397705078125 +[g_only s0] step 5280/12000 loss=4.5948 gn=0.85 lr=3.88e-04 tps=17444 mem=6.3G lam=None Gm=0.3789571126302083 +[g_only s0] step 5300/12000 loss=4.7262 gn=0.81 lr=3.87e-04 tps=17445 mem=6.3G lam=None Gm=0.3776652018229167 +[g_only s0] step 5320/12000 loss=4.5937 gn=0.74 lr=3.86e-04 tps=17446 mem=6.3G lam=None Gm=0.3704274495442708 +[g_only s0] step 5340/12000 loss=4.6753 gn=0.80 lr=3.84e-04 tps=17447 mem=6.3G lam=None Gm=0.3780161539713542 +[g_only s0] step 5360/12000 loss=4.5193 gn=0.77 lr=3.83e-04 tps=17448 mem=6.3G lam=None Gm=0.3716684977213542 +[g_only s0] step 5380/12000 loss=4.7929 gn=0.81 lr=3.81e-04 tps=17449 mem=6.3G lam=None Gm=0.382568359375 +[g_only s0] step 5400/12000 loss=4.4962 gn=0.81 lr=3.80e-04 tps=17449 mem=6.3G lam=None Gm=0.3673553466796875 +[g_only s0] step 5420/12000 loss=4.6489 gn=0.82 lr=3.79e-04 tps=17450 mem=6.3G lam=None Gm=0.3753662109375 +[g_only s0] step 5440/12000 loss=4.5307 gn=0.78 lr=3.77e-04 tps=17451 mem=6.3G lam=None Gm=0.3731892903645833 +[g_only s0] step 5460/12000 loss=4.6567 gn=0.86 lr=3.76e-04 tps=17452 mem=6.3G lam=None Gm=0.3760223388671875 +[g_only s0] step 5480/12000 loss=4.5453 gn=0.80 lr=3.74e-04 tps=17452 mem=6.3G lam=None Gm=0.3811747233072917 +[g_only s0] step 5500/12000 loss=4.6179 gn=0.77 lr=3.73e-04 tps=17453 mem=6.3G lam=None Gm=0.3729909261067708 +[g_only s0] EVAL step 5500: val_loss=4.7001 val_ppl=109.96 +[g_only s0] step 5520/12000 loss=4.4466 gn=0.78 lr=3.72e-04 tps=17431 mem=6.3G lam=None Gm=0.3692728678385417 +[g_only s0] step 5540/12000 loss=4.6512 gn=0.84 lr=3.70e-04 tps=17431 mem=6.3G lam=None Gm=0.3766937255859375 +[g_only s0] step 5560/12000 loss=4.6952 gn=0.81 lr=3.69e-04 tps=17432 mem=6.3G lam=None Gm=0.3716888427734375 +[g_only s0] step 5580/12000 loss=4.7195 gn=0.76 lr=3.67e-04 tps=17433 mem=6.3G lam=None Gm=0.3762054443359375 +[g_only s0] step 5600/12000 loss=4.5514 gn=0.81 lr=3.66e-04 tps=17434 mem=6.3G lam=None Gm=0.3720906575520833 +[g_only s0] step 5620/12000 loss=4.6307 gn=0.81 lr=3.64e-04 tps=17434 mem=6.3G lam=None Gm=0.3732401529947917 +[g_only s0] step 5640/12000 loss=4.7421 gn=0.82 lr=3.63e-04 tps=17435 mem=6.3G lam=None Gm=0.3818105061848958 +[g_only s0] step 5660/12000 loss=4.5144 gn=0.78 lr=3.62e-04 tps=17436 mem=6.3G lam=None Gm=0.3770192464192708 +[g_only s0] step 5680/12000 loss=4.5672 gn=0.83 lr=3.60e-04 tps=17436 mem=6.3G lam=None Gm=0.3734130859375 +[g_only s0] step 5700/12000 loss=4.7192 gn=0.89 lr=3.59e-04 tps=17437 mem=6.3G lam=None Gm=0.3664194742838542 +[g_only s0] step 5720/12000 loss=4.5634 gn=0.79 lr=3.57e-04 tps=17438 mem=6.3G lam=None Gm=0.3722381591796875 +[g_only s0] step 5740/12000 loss=4.7208 gn=0.83 lr=3.56e-04 tps=17439 mem=6.3G lam=None Gm=0.37384033203125 +[g_only s0] step 5760/12000 loss=4.5761 gn=0.76 lr=3.54e-04 tps=17439 mem=6.3G lam=None Gm=0.3708445231119792 +[g_only s0] step 5780/12000 loss=4.6190 gn=0.78 lr=3.53e-04 tps=17440 mem=6.3G lam=None Gm=0.3699442545572917 +[g_only s0] step 5800/12000 loss=4.4765 gn=0.82 lr=3.52e-04 tps=17441 mem=6.3G lam=None Gm=0.3669382731119792 +[g_only s0] step 5820/12000 loss=4.7331 gn=0.77 lr=3.50e-04 tps=17442 mem=6.3G lam=None Gm=0.3724263509114583 +[g_only s0] step 5840/12000 loss=4.5336 gn=0.78 lr=3.49e-04 tps=17442 mem=6.3G lam=None Gm=0.3685709635416667 +[g_only s0] step 5860/12000 loss=4.6307 gn=0.80 lr=3.47e-04 tps=17443 mem=6.3G lam=None Gm=0.37115478515625 +[g_only s0] step 5880/12000 loss=4.6711 gn=0.82 lr=3.46e-04 tps=17444 mem=6.3G lam=None Gm=0.373016357421875 +[g_only s0] step 5900/12000 loss=4.5927 gn=0.84 lr=3.44e-04 tps=17445 mem=6.3G lam=None Gm=0.3680470784505208 +[g_only s0] step 5920/12000 loss=4.5738 gn=0.81 lr=3.43e-04 tps=17445 mem=6.3G lam=None Gm=0.3687591552734375 +[g_only s0] step 5940/12000 loss=4.4986 gn=0.79 lr=3.41e-04 tps=17446 mem=6.3G lam=None Gm=0.3685557047526042 +[g_only s0] step 5960/12000 loss=4.5447 gn=0.90 lr=3.40e-04 tps=17447 mem=6.3G lam=None Gm=0.3680674235026042 +[g_only s0] step 5980/12000 loss=4.8090 gn=0.81 lr=3.39e-04 tps=17448 mem=6.3G lam=None Gm=0.3764394124348958 +[g_only s0] step 6000/12000 loss=4.5630 gn=0.77 lr=3.37e-04 tps=17448 mem=6.3G lam=None Gm=0.3777872721354167 +[g_only s0] EVAL step 6000: val_loss=4.6554 val_ppl=105.15 +[g_only s0] step 6020/12000 loss=4.4855 gn=0.82 lr=3.36e-04 tps=17428 mem=6.3G lam=None Gm=0.36468505859375 +[g_only s0] step 6040/12000 loss=4.5117 gn=0.77 lr=3.34e-04 tps=17428 mem=6.3G lam=None Gm=0.3725128173828125 +[g_only s0] step 6060/12000 loss=4.6687 gn=0.83 lr=3.33e-04 tps=17429 mem=6.3G lam=None Gm=0.370208740234375 +[g_only s0] step 6080/12000 loss=4.6082 gn=0.80 lr=3.31e-04 tps=17430 mem=6.3G lam=None Gm=0.3683929443359375 +[g_only s0] step 6100/12000 loss=4.5237 gn=0.82 lr=3.30e-04 tps=17430 mem=6.3G lam=None Gm=0.3687744140625 +[g_only s0] step 6120/12000 loss=4.6826 gn=0.83 lr=3.29e-04 tps=17431 mem=6.3G lam=None Gm=0.3723347981770833 +[g_only s0] step 6140/12000 loss=4.5801 gn=0.84 lr=3.27e-04 tps=17432 mem=6.3G lam=None Gm=0.3765513102213542 +[g_only s0] step 6160/12000 loss=4.4775 gn=0.81 lr=3.26e-04 tps=17433 mem=6.3G lam=None Gm=0.3671366373697917 +[g_only s0] step 6180/12000 loss=4.6338 gn=0.86 lr=3.24e-04 tps=17434 mem=6.3G lam=None Gm=0.3732961018880208 +[g_only s0] step 6200/12000 loss=4.5563 gn=0.84 lr=3.23e-04 tps=17434 mem=6.3G lam=None Gm=0.3662312825520833 +[g_only s0] step 6220/12000 loss=4.5267 gn=0.87 lr=3.21e-04 tps=17435 mem=6.3G lam=None Gm=0.373443603515625 +[g_only s0] step 6240/12000 loss=4.4855 gn=0.81 lr=3.20e-04 tps=17436 mem=6.3G lam=None Gm=0.3649190266927083 +[g_only s0] step 6260/12000 loss=4.6240 gn=0.90 lr=3.19e-04 tps=17437 mem=6.3G lam=None Gm=0.3725941975911458 +[g_only s0] step 6280/12000 loss=4.5679 gn=0.78 lr=3.17e-04 tps=17437 mem=6.3G lam=None Gm=0.3739878336588542 +[g_only s0] step 6300/12000 loss=4.5081 gn=0.84 lr=3.16e-04 tps=17438 mem=6.3G lam=None Gm=0.3640492757161458 +[g_only s0] step 6320/12000 loss=4.4652 gn=0.87 lr=3.14e-04 tps=17438 mem=6.3G lam=None Gm=0.3675638834635417 +[g_only s0] step 6340/12000 loss=4.5639 gn=0.81 lr=3.13e-04 tps=17439 mem=6.3G lam=None Gm=0.3742421468098958 +[g_only s0] step 6360/12000 loss=4.5669 gn=0.84 lr=3.11e-04 tps=17439 mem=6.3G lam=None Gm=0.3691202799479167 +[g_only s0] step 6380/12000 loss=4.5760 gn=0.84 lr=3.10e-04 tps=17440 mem=6.3G lam=None Gm=0.3714141845703125 +[g_only s0] step 6400/12000 loss=4.4338 gn=0.87 lr=3.08e-04 tps=17441 mem=6.3G lam=None Gm=0.3746236165364583 +[g_only s0] step 6420/12000 loss=4.6091 gn=0.83 lr=3.07e-04 tps=17442 mem=6.3G lam=None Gm=0.3670094807942708 +[g_only s0] step 6440/12000 loss=4.6624 gn=0.91 lr=3.06e-04 tps=17442 mem=6.3G lam=None Gm=0.3597666422526042 +[g_only s0] step 6460/12000 loss=4.5086 gn=0.88 lr=3.04e-04 tps=17442 mem=6.3G lam=None Gm=0.36297607421875 +[g_only s0] step 6480/12000 loss=4.5056 gn=0.86 lr=3.03e-04 tps=17443 mem=6.3G lam=None Gm=0.3679962158203125 +[g_only s0] step 6500/12000 loss=4.5279 gn=0.84 lr=3.01e-04 tps=17444 mem=6.3G lam=None Gm=0.372222900390625 +[g_only s0] EVAL step 6500: val_loss=4.6308 val_ppl=102.60 +[g_only s0] step 6520/12000 loss=4.6789 gn=0.84 lr=3.00e-04 tps=17425 mem=6.3G lam=None Gm=0.3690846761067708 +[g_only s0] step 6540/12000 loss=4.4900 gn=0.88 lr=2.98e-04 tps=17425 mem=6.3G lam=None Gm=0.3716278076171875 +[g_only s0] step 6560/12000 loss=4.6873 gn=0.87 lr=2.97e-04 tps=17426 mem=6.3G lam=None Gm=0.3673146565755208 +[g_only s0] step 6580/12000 loss=4.6244 gn=0.88 lr=2.96e-04 tps=17426 mem=6.3G lam=None Gm=0.3666839599609375 +[g_only s0] step 6600/12000 loss=4.3844 gn=0.87 lr=2.94e-04 tps=17427 mem=6.3G lam=None Gm=0.3665262858072917 +[g_only s0] step 6620/12000 loss=4.3825 gn=0.84 lr=2.93e-04 tps=17428 mem=6.3G lam=None Gm=0.3650054931640625 +[g_only s0] step 6640/12000 loss=4.6797 gn=0.82 lr=2.91e-04 tps=17428 mem=6.3G lam=None Gm=0.3699900309244792 +[g_only s0] step 6660/12000 loss=4.6442 gn=0.80 lr=2.90e-04 tps=17429 mem=6.3G lam=None Gm=0.3717091878255208 +[g_only s0] step 6680/12000 loss=4.5277 gn=0.82 lr=2.88e-04 tps=17429 mem=6.3G lam=None Gm=0.3710886637369792 +[g_only s0] step 6700/12000 loss=4.4933 gn=0.86 lr=2.87e-04 tps=17430 mem=6.3G lam=None Gm=0.362701416015625 +[g_only s0] step 6720/12000 loss=4.5668 gn=0.89 lr=2.86e-04 tps=17431 mem=6.3G lam=None Gm=0.3676656087239583 +[g_only s0] step 6740/12000 loss=4.4762 gn=0.82 lr=2.84e-04 tps=17431 mem=6.3G lam=None Gm=0.36236572265625 +[g_only s0] step 6760/12000 loss=4.4886 gn=0.89 lr=2.83e-04 tps=17432 mem=6.3G lam=None Gm=0.3676859537760417 +[g_only s0] step 6780/12000 loss=4.4857 gn=0.88 lr=2.81e-04 tps=17433 mem=6.3G lam=None Gm=0.3684438069661458 +[g_only s0] step 6800/12000 loss=4.4420 gn=0.80 lr=2.80e-04 tps=17433 mem=6.3G lam=None Gm=0.3646901448567708 +[g_only s0] step 6820/12000 loss=4.4255 gn=0.82 lr=2.79e-04 tps=17434 mem=6.3G lam=None Gm=0.3678741455078125 +[g_only s0] step 6840/12000 loss=4.3635 gn=0.79 lr=2.77e-04 tps=17435 mem=6.3G lam=None Gm=0.3741963704427083 +[g_only s0] step 6860/12000 loss=4.5632 gn=0.88 lr=2.76e-04 tps=17436 mem=6.3G lam=None Gm=0.365386962890625 +[g_only s0] step 6880/12000 loss=4.5045 gn=0.80 lr=2.74e-04 tps=17436 mem=6.3G lam=None Gm=0.371368408203125 +[g_only s0] step 6900/12000 loss=4.6534 gn=0.86 lr=2.73e-04 tps=17437 mem=6.3G lam=None Gm=0.3704427083333333 +[g_only s0] step 6920/12000 loss=4.5683 gn=0.84 lr=2.72e-04 tps=17437 mem=6.3G lam=None Gm=0.3639984130859375 +[g_only s0] step 6940/12000 loss=4.4009 gn=0.82 lr=2.70e-04 tps=17438 mem=6.3G lam=None Gm=0.3621673583984375 +[g_only s0] step 6960/12000 loss=4.5303 gn=0.86 lr=2.69e-04 tps=17439 mem=6.3G lam=None Gm=0.361785888671875 +[g_only s0] step 6980/12000 loss=4.3487 gn=0.84 lr=2.67e-04 tps=17439 mem=6.3G lam=None Gm=0.3727823893229167 +[g_only s0] step 7000/12000 loss=4.6434 gn=0.89 lr=2.66e-04 tps=17440 mem=6.3G lam=None Gm=0.3711751302083333 +[g_only s0] EVAL step 7000: val_loss=4.5769 val_ppl=97.21 +[g_only s0] step 7020/12000 loss=4.4471 gn=0.84 lr=2.65e-04 tps=17422 mem=6.3G lam=None Gm=0.3646748860677083 +[g_only s0] step 7040/12000 loss=4.4590 gn=0.85 lr=2.63e-04 tps=17423 mem=6.3G lam=None Gm=0.36767578125 +[g_only s0] step 7060/12000 loss=4.5491 gn=0.86 lr=2.62e-04 tps=17423 mem=6.3G lam=None Gm=0.3665974934895833 +[g_only s0] step 7080/12000 loss=4.5649 gn=0.87 lr=2.60e-04 tps=17424 mem=6.3G lam=None Gm=0.367645263671875 +[g_only s0] step 7100/12000 loss=4.5187 gn=0.86 lr=2.59e-04 tps=17424 mem=6.3G lam=None Gm=0.3633524576822917 +[g_only s0] step 7120/12000 loss=4.7363 gn=0.90 lr=2.58e-04 tps=17425 mem=6.3G lam=None Gm=0.374969482421875 +[g_only s0] step 7140/12000 loss=4.5119 gn=0.86 lr=2.56e-04 tps=17425 mem=6.3G lam=None Gm=0.3676503499348958 +[g_only s0] step 7160/12000 loss=4.4502 gn=0.82 lr=2.55e-04 tps=17426 mem=6.3G lam=None Gm=0.3582000732421875 +[g_only s0] step 7180/12000 loss=4.5891 gn=0.85 lr=2.53e-04 tps=17427 mem=6.3G lam=None Gm=0.365814208984375 +[g_only s0] step 7200/12000 loss=4.6167 gn=0.93 lr=2.52e-04 tps=17427 mem=6.3G lam=None Gm=0.3671875 +[g_only s0] step 7220/12000 loss=4.5931 gn=0.84 lr=2.51e-04 tps=17427 mem=6.3G lam=None Gm=0.3732757568359375 +[g_only s0] step 7240/12000 loss=4.3946 gn=0.90 lr=2.49e-04 tps=17428 mem=6.3G lam=None Gm=0.3607279459635417 +[g_only s0] step 7260/12000 loss=4.4457 gn=0.89 lr=2.48e-04 tps=17428 mem=6.3G lam=None Gm=0.3656056722005208 +[g_only s0] step 7280/12000 loss=4.5041 gn=0.95 lr=2.47e-04 tps=17429 mem=6.3G lam=None Gm=0.3578592936197917 +[g_only s0] step 7300/12000 loss=4.3657 gn=0.83 lr=2.45e-04 tps=17430 mem=6.3G lam=None Gm=0.3644154866536458 +[g_only s0] step 7320/12000 loss=4.5959 gn=0.82 lr=2.44e-04 tps=17430 mem=6.3G lam=None Gm=0.368621826171875 +[g_only s0] step 7340/12000 loss=4.4243 gn=0.84 lr=2.42e-04 tps=17431 mem=6.3G lam=None Gm=0.3637339274088542 +[g_only s0] step 7360/12000 loss=4.3878 gn=0.82 lr=2.41e-04 tps=17432 mem=6.3G lam=None Gm=0.3563486735026042 +[g_only s0] step 7380/12000 loss=4.3474 gn=0.90 lr=2.40e-04 tps=17432 mem=6.3G lam=None Gm=0.36688232421875 +[g_only s0] step 7400/12000 loss=4.4670 gn=0.88 lr=2.38e-04 tps=17433 mem=6.3G lam=None Gm=0.3653157552083333 +[g_only s0] step 7420/12000 loss=4.5902 gn=0.91 lr=2.37e-04 tps=17434 mem=6.3G lam=None Gm=0.3709920247395833 +[g_only s0] step 7440/12000 loss=4.6146 gn=0.88 lr=2.36e-04 tps=17435 mem=6.3G lam=None Gm=0.3664093017578125 +[g_only s0] step 7460/12000 loss=4.4127 gn=0.86 lr=2.34e-04 tps=17435 mem=6.3G lam=None Gm=0.3664754231770833 +[g_only s0] step 7480/12000 loss=4.4595 gn=0.89 lr=2.33e-04 tps=17436 mem=6.3G lam=None Gm=0.3641001383463542 +[g_only s0] step 7500/12000 loss=4.5271 gn=0.87 lr=2.32e-04 tps=17436 mem=6.3G lam=None Gm=0.3530680338541667 +[g_only s0] EVAL step 7500: val_loss=4.5955 val_ppl=99.04 +[g_only s0] step 7520/12000 loss=4.5106 gn=0.87 lr=2.30e-04 tps=17420 mem=6.3G lam=None Gm=0.3646647135416667 +[g_only s0] step 7540/12000 loss=4.5885 gn=0.89 lr=2.29e-04 tps=17420 mem=6.3G lam=None Gm=0.3673146565755208 +[g_only s0] step 7560/12000 loss=4.2790 gn=0.83 lr=2.28e-04 tps=17421 mem=6.3G lam=None Gm=0.3589426676432292 +[g_only s0] step 7580/12000 loss=4.5298 gn=0.90 lr=2.26e-04 tps=17421 mem=6.3G lam=None Gm=0.3621978759765625 +[g_only s0] step 7600/12000 loss=4.3705 gn=0.85 lr=2.25e-04 tps=17422 mem=6.3G lam=None Gm=0.362823486328125 +[g_only s0] step 7620/12000 loss=4.5517 gn=0.94 lr=2.24e-04 tps=17422 mem=6.3G lam=None Gm=0.3635508219401042 +[g_only s0] step 7640/12000 loss=4.3691 gn=0.87 lr=2.22e-04 tps=17423 mem=6.3G lam=None Gm=0.3593343098958333 +[g_only s0] step 7660/12000 loss=4.5046 gn=0.86 lr=2.21e-04 tps=17423 mem=6.3G lam=None Gm=0.3573201497395833 +[g_only s0] step 7680/12000 loss=4.4871 gn=0.91 lr=2.20e-04 tps=17424 mem=6.3G lam=None Gm=0.3621063232421875 +[g_only s0] step 7700/12000 loss=4.5352 gn=0.86 lr=2.18e-04 tps=17425 mem=6.3G lam=None Gm=0.3613128662109375 +[g_only s0] step 7720/12000 loss=4.4965 gn=0.92 lr=2.17e-04 tps=17425 mem=6.3G lam=None Gm=0.3650868733723958 +[g_only s0] step 7740/12000 loss=4.3405 gn=0.84 lr=2.16e-04 tps=17426 mem=6.3G lam=None Gm=0.3590952555338542 +[g_only s0] step 7760/12000 loss=4.4722 gn=0.88 lr=2.15e-04 tps=17426 mem=6.3G lam=None Gm=0.3650054931640625 +[g_only s0] step 7780/12000 loss=4.4118 gn=0.91 lr=2.13e-04 tps=17427 mem=6.3G lam=None Gm=0.3661600748697917 +[g_only s0] step 7800/12000 loss=4.4983 gn=0.94 lr=2.12e-04 tps=17428 mem=6.3G lam=None Gm=0.3569234212239583 +[g_only s0] step 7820/12000 loss=4.5643 gn=0.94 lr=2.11e-04 tps=17428 mem=6.3G lam=None Gm=0.3673655192057292 +[g_only s0] step 7840/12000 loss=4.3218 gn=0.85 lr=2.09e-04 tps=17429 mem=6.3G lam=None Gm=0.3625996907552083 +[g_only s0] step 7860/12000 loss=4.4688 gn=0.96 lr=2.08e-04 tps=17430 mem=6.3G lam=None Gm=0.3611501057942708 +[g_only s0] step 7880/12000 loss=4.5204 gn=0.99 lr=2.07e-04 tps=17430 mem=6.3G lam=None Gm=0.3636525472005208 +[g_only s0] step 7900/12000 loss=4.2997 gn=0.91 lr=2.06e-04 tps=17431 mem=6.3G lam=None Gm=0.3564453125 +[g_only s0] step 7920/12000 loss=4.3810 gn=0.95 lr=2.04e-04 tps=17431 mem=6.3G lam=None Gm=0.36474609375 +[g_only s0] step 7940/12000 loss=4.4772 gn=0.89 lr=2.03e-04 tps=17432 mem=6.3G lam=None Gm=0.3643391927083333 +[g_only s0] step 7960/12000 loss=4.5209 gn=0.92 lr=2.02e-04 tps=17432 mem=6.3G lam=None Gm=0.3667704264322917 +[g_only s0] step 7980/12000 loss=4.4203 gn=0.88 lr=2.00e-04 tps=17433 mem=6.3G lam=None Gm=0.357940673828125 +[g_only s0] step 8000/12000 loss=4.3916 gn=0.93 lr=1.99e-04 tps=17433 mem=6.3G lam=None Gm=0.3660227457682292 +[g_only s0] EVAL step 8000: val_loss=4.5199 val_ppl=91.82 +[g_only s0] step 8020/12000 loss=4.4323 gn=0.87 lr=1.98e-04 tps=17418 mem=6.3G lam=None Gm=0.3604888916015625 +[g_only s0] step 8040/12000 loss=4.5777 gn=0.88 lr=1.97e-04 tps=17418 mem=6.3G lam=None Gm=0.3650004069010417 +[g_only s0] step 8060/12000 loss=4.5038 gn=0.86 lr=1.95e-04 tps=17419 mem=6.3G lam=None Gm=0.3595123291015625 +[g_only s0] step 8080/12000 loss=4.4024 gn=0.92 lr=1.94e-04 tps=17419 mem=6.3G lam=None Gm=0.3584696451822917 +[g_only s0] step 8100/12000 loss=4.5215 gn=0.87 lr=1.93e-04 tps=17420 mem=6.3G lam=None Gm=0.3651173909505208 +[g_only s0] step 8120/12000 loss=4.3454 gn=0.83 lr=1.92e-04 tps=17420 mem=6.3G lam=None Gm=0.3631693522135417 +[g_only s0] step 8140/12000 loss=4.5094 gn=0.95 lr=1.90e-04 tps=17421 mem=6.3G lam=None Gm=0.3565470377604167 +[g_only s0] step 8160/12000 loss=4.4921 gn=0.90 lr=1.89e-04 tps=17421 mem=6.3G lam=None Gm=0.3621317545572917 +[g_only s0] step 8180/12000 loss=4.5574 gn=0.91 lr=1.88e-04 tps=17422 mem=6.3G lam=None Gm=0.35931396484375 +[g_only s0] step 8200/12000 loss=4.2596 gn=0.92 lr=1.87e-04 tps=17422 mem=6.3G lam=None Gm=0.3680521647135417 +[g_only s0] step 8220/12000 loss=4.3980 gn=0.92 lr=1.86e-04 tps=17423 mem=6.3G lam=None Gm=0.3619893391927083 +[g_only s0] step 8240/12000 loss=4.2577 gn=0.92 lr=1.84e-04 tps=17423 mem=6.3G lam=None Gm=0.3566640218098958 +[g_only s0] step 8260/12000 loss=4.4102 gn=0.86 lr=1.83e-04 tps=17424 mem=6.3G lam=None Gm=0.3592631022135417 +[g_only s0] step 8280/12000 loss=4.3365 gn=0.88 lr=1.82e-04 tps=17425 mem=6.3G lam=None Gm=0.3590901692708333 +[g_only s0] step 8300/12000 loss=4.3764 gn=0.90 lr=1.81e-04 tps=17425 mem=6.3G lam=None Gm=0.3609771728515625 +[g_only s0] step 8320/12000 loss=4.4202 gn=0.89 lr=1.80e-04 tps=17426 mem=6.3G lam=None Gm=0.3563385009765625 +[g_only s0] step 8340/12000 loss=4.4426 gn=0.91 lr=1.78e-04 tps=17426 mem=6.3G lam=None Gm=0.3580830891927083 +[g_only s0] step 8360/12000 loss=4.4827 gn=1.01 lr=1.77e-04 tps=17427 mem=6.3G lam=None Gm=0.3618062337239583 +[g_only s0] step 8380/12000 loss=4.4377 gn=0.87 lr=1.76e-04 tps=17427 mem=6.3G lam=None Gm=0.35784912109375 +[g_only s0] step 8400/12000 loss=4.3701 gn=0.88 lr=1.75e-04 tps=17428 mem=6.3G lam=None Gm=0.35797119140625 +[g_only s0] step 8420/12000 loss=4.5214 gn=0.92 lr=1.74e-04 tps=17429 mem=6.3G lam=None Gm=0.3648223876953125 +[g_only s0] step 8440/12000 loss=4.1822 gn=0.88 lr=1.72e-04 tps=17429 mem=6.3G lam=None Gm=0.3554229736328125 +[g_only s0] step 8460/12000 loss=4.3496 gn=1.02 lr=1.71e-04 tps=17430 mem=6.3G lam=None Gm=0.358184814453125 +[g_only s0] step 8480/12000 loss=4.3018 gn=0.94 lr=1.70e-04 tps=17431 mem=6.3G lam=None Gm=0.3598276774088542 +[g_only s0] step 8500/12000 loss=4.6270 gn=0.93 lr=1.69e-04 tps=17431 mem=6.3G lam=None Gm=0.3565572102864583 +[g_only s0] EVAL step 8500: val_loss=4.5135 val_ppl=91.24 +[g_only s0] step 8520/12000 loss=4.4248 gn=0.93 lr=1.68e-04 tps=17417 mem=6.3G lam=None Gm=0.3603363037109375 +[g_only s0] step 8540/12000 loss=4.4643 gn=1.00 lr=1.67e-04 tps=17417 mem=6.3G lam=None Gm=0.3616841634114583 +[g_only s0] step 8560/12000 loss=4.5294 gn=0.89 lr=1.66e-04 tps=17417 mem=6.3G lam=None Gm=0.3626047770182292 +[g_only s0] step 8580/12000 loss=4.3674 gn=0.92 lr=1.64e-04 tps=17418 mem=6.3G lam=None Gm=0.3566741943359375 +[g_only s0] step 8600/12000 loss=4.5055 gn=0.98 lr=1.63e-04 tps=17418 mem=6.3G lam=None Gm=0.3631947835286458 +[g_only s0] step 8620/12000 loss=4.4458 gn=0.97 lr=1.62e-04 tps=17419 mem=6.3G lam=None Gm=0.3660685221354167 +[g_only s0] step 8640/12000 loss=4.4866 gn=0.91 lr=1.61e-04 tps=17419 mem=6.3G lam=None Gm=0.3589731852213542 +[g_only s0] step 8660/12000 loss=4.3137 gn=0.94 lr=1.60e-04 tps=17419 mem=6.3G lam=None Gm=0.365570068359375 +[g_only s0] step 8680/12000 loss=4.4017 gn=0.85 lr=1.59e-04 tps=17419 mem=6.3G lam=None Gm=0.359710693359375 +[g_only s0] step 8700/12000 loss=4.3789 gn=0.91 lr=1.58e-04 tps=17420 mem=6.3G lam=None Gm=0.3609670003255208 +[g_only s0] step 8720/12000 loss=4.3471 gn=0.89 lr=1.57e-04 tps=17420 mem=6.3G lam=None Gm=0.3640543619791667 +[g_only s0] step 8740/12000 loss=4.4344 gn=0.92 lr=1.55e-04 tps=17421 mem=6.3G lam=None Gm=0.3587646484375 +[g_only s0] step 8760/12000 loss=4.3305 gn=1.09 lr=1.54e-04 tps=17421 mem=6.3G lam=None Gm=0.3646494547526042 +[g_only s0] step 8780/12000 loss=4.4277 gn=0.92 lr=1.53e-04 tps=17422 mem=6.3G lam=None Gm=0.3601125081380208 +[g_only s0] step 8800/12000 loss=4.4289 gn=0.93 lr=1.52e-04 tps=17422 mem=6.3G lam=None Gm=0.3670603434244792 +[g_only s0] step 8820/12000 loss=4.4305 gn=0.98 lr=1.51e-04 tps=17422 mem=6.3G lam=None Gm=0.3646748860677083 +[g_only s0] step 8840/12000 loss=4.4323 gn=0.95 lr=1.50e-04 tps=17423 mem=6.3G lam=None Gm=0.3612213134765625 +[g_only s0] step 8860/12000 loss=4.4534 gn=0.89 lr=1.49e-04 tps=17423 mem=6.3G lam=None Gm=0.3653666178385417 +[g_only s0] step 8880/12000 loss=4.2827 gn=0.93 lr=1.48e-04 tps=17424 mem=6.3G lam=None Gm=0.3569183349609375 +[g_only s0] step 8900/12000 loss=4.3726 gn=0.90 lr=1.47e-04 tps=17425 mem=6.3G lam=None Gm=0.36083984375 +[g_only s0] step 8920/12000 loss=4.4702 gn=1.12 lr=1.46e-04 tps=17425 mem=6.3G lam=None Gm=0.3646952311197917 +[g_only s0] step 8940/12000 loss=4.2827 gn=0.93 lr=1.45e-04 tps=17426 mem=6.3G lam=None Gm=0.3631642659505208 +[g_only s0] step 8960/12000 loss=4.5207 gn=0.99 lr=1.44e-04 tps=17426 mem=6.3G lam=None Gm=0.3632456461588542 +[g_only s0] step 8980/12000 loss=4.3778 gn=0.95 lr=1.43e-04 tps=17426 mem=6.3G lam=None Gm=0.3618672688802083 +[g_only s0] step 9000/12000 loss=4.3231 gn=0.92 lr=1.42e-04 tps=17427 mem=6.3G lam=None Gm=0.3591156005859375 +[g_only s0] EVAL step 9000: val_loss=4.4778 val_ppl=88.04 +[g_only s0] step 9020/12000 loss=4.3489 gn=0.90 lr=1.41e-04 tps=17414 mem=6.3G lam=None Gm=0.3671976725260417 +[g_only s0] step 9040/12000 loss=4.3204 gn=0.88 lr=1.40e-04 tps=17414 mem=6.3G lam=None Gm=0.3591257731119792 +[g_only s0] step 9060/12000 loss=4.3290 gn=0.93 lr=1.39e-04 tps=17414 mem=6.3G lam=None Gm=0.3610687255859375 +[g_only s0] step 9080/12000 loss=4.3086 gn=0.91 lr=1.38e-04 tps=17415 mem=6.3G lam=None Gm=0.3585764567057292 +[g_only s0] step 9100/12000 loss=4.2702 gn=0.92 lr=1.37e-04 tps=17415 mem=6.3G lam=None Gm=0.3639322916666667 +[g_only s0] step 9120/12000 loss=4.3583 gn=0.92 lr=1.36e-04 tps=17415 mem=6.3G lam=None Gm=0.3607279459635417 +[g_only s0] step 9140/12000 loss=4.4679 gn=0.96 lr=1.35e-04 tps=17416 mem=6.3G lam=None Gm=0.3589579264322917 +[g_only s0] step 9160/12000 loss=4.3188 gn=0.97 lr=1.34e-04 tps=17416 mem=6.3G lam=None Gm=0.3613179524739583 +[g_only s0] step 9180/12000 loss=4.4674 gn=1.01 lr=1.33e-04 tps=17417 mem=6.3G lam=None Gm=0.367584228515625 +[g_only s0] step 9200/12000 loss=4.4271 gn=1.19 lr=1.32e-04 tps=17417 mem=6.3G lam=None Gm=0.3712107340494792 +[g_only s0] step 9220/12000 loss=4.4664 gn=1.01 lr=1.31e-04 tps=17418 mem=6.3G lam=None Gm=0.3594614664713542 +[g_only s0] step 9240/12000 loss=4.4655 gn=0.94 lr=1.30e-04 tps=17418 mem=6.3G lam=None Gm=0.3610382080078125 +[g_only s0] step 9260/12000 loss=4.4143 gn=0.97 lr=1.29e-04 tps=17419 mem=6.3G lam=None Gm=0.3596140543619792 +[g_only s0] step 9280/12000 loss=4.3467 gn=0.90 lr=1.28e-04 tps=17419 mem=6.3G lam=None Gm=0.3564402262369792 +[g_only s0] step 9300/12000 loss=4.4648 gn=1.01 lr=1.27e-04 tps=17420 mem=6.3G lam=None Gm=0.3560333251953125 +[g_only s0] step 9320/12000 loss=4.3257 gn=0.88 lr=1.26e-04 tps=17420 mem=6.3G lam=None Gm=0.3603769938151042 +[g_only s0] step 9340/12000 loss=4.4848 gn=1.02 lr=1.25e-04 tps=17421 mem=6.3G lam=None Gm=0.3589884440104167 +[g_only s0] step 9360/12000 loss=4.5629 gn=1.02 lr=1.24e-04 tps=17422 mem=6.3G lam=None Gm=0.3627878824869792 +[g_only s0] step 9380/12000 loss=4.4319 gn=0.97 lr=1.23e-04 tps=17422 mem=6.3G lam=None Gm=0.3616434733072917 +[g_only s0] step 9400/12000 loss=4.3045 gn=0.95 lr=1.22e-04 tps=17423 mem=6.3G lam=None Gm=0.3605143229166667 +[g_only s0] step 9420/12000 loss=4.5459 gn=1.10 lr=1.21e-04 tps=17423 mem=6.3G lam=None Gm=0.3676859537760417 +[g_only s0] step 9440/12000 loss=4.4031 gn=0.93 lr=1.20e-04 tps=17424 mem=6.3G lam=None Gm=0.3585611979166667 +[g_only s0] step 9460/12000 loss=4.3568 gn=0.96 lr=1.19e-04 tps=17424 mem=6.3G lam=None Gm=0.3561655680338542 +[g_only s0] step 9480/12000 loss=4.4122 gn=0.89 lr=1.19e-04 tps=17425 mem=6.3G lam=None Gm=0.3567708333333333 +[g_only s0] step 9500/12000 loss=4.3518 gn=0.98 lr=1.18e-04 tps=17425 mem=6.3G lam=None Gm=0.3572184244791667 +[g_only s0] EVAL step 9500: val_loss=4.4455 val_ppl=85.24 +[g_only s0] step 9520/12000 loss=4.2046 gn=0.89 lr=1.17e-04 tps=17413 mem=6.3G lam=None Gm=0.358001708984375 +[g_only s0] step 9540/12000 loss=4.5255 gn=1.00 lr=1.16e-04 tps=17413 mem=6.3G lam=None Gm=0.3573099772135417 +[g_only s0] step 9560/12000 loss=4.4947 gn=0.92 lr=1.15e-04 tps=17414 mem=6.3G lam=None Gm=0.3603973388671875 +[g_only s0] step 9580/12000 loss=4.2885 gn=0.97 lr=1.14e-04 tps=17414 mem=6.3G lam=None Gm=0.3542022705078125 +[g_only s0] step 9600/12000 loss=4.3515 gn=0.99 lr=1.13e-04 tps=17415 mem=6.3G lam=None Gm=0.3594970703125 +[g_only s0] step 9620/12000 loss=4.4594 gn=0.94 lr=1.12e-04 tps=17415 mem=6.3G lam=None Gm=0.3593088785807292 +[g_only s0] step 9640/12000 loss=4.3037 gn=1.02 lr=1.12e-04 tps=17415 mem=6.3G lam=None Gm=0.3560333251953125 +[g_only s0] step 9660/12000 loss=4.4281 gn=0.91 lr=1.11e-04 tps=17416 mem=6.3G lam=None Gm=0.3557790120442708 +[g_only s0] step 9680/12000 loss=4.6088 gn=1.02 lr=1.10e-04 tps=17416 mem=6.3G lam=None Gm=0.3604329427083333 +[g_only s0] step 9700/12000 loss=4.2818 gn=0.88 lr=1.09e-04 tps=17417 mem=6.3G lam=None Gm=0.3519134521484375 +[g_only s0] step 9720/12000 loss=4.2696 gn=0.92 lr=1.08e-04 tps=17417 mem=6.3G lam=None Gm=0.3573048909505208 +[g_only s0] step 9740/12000 loss=4.4429 gn=0.96 lr=1.07e-04 tps=17417 mem=6.3G lam=None Gm=0.361053466796875 +[g_only s0] step 9760/12000 loss=4.2841 gn=0.95 lr=1.07e-04 tps=17418 mem=6.3G lam=None Gm=0.3581339518229167 +[g_only s0] step 9780/12000 loss=4.3586 gn=0.93 lr=1.06e-04 tps=17418 mem=6.3G lam=None Gm=0.3564809163411458 +[g_only s0] step 9800/12000 loss=4.3301 gn=0.96 lr=1.05e-04 tps=17419 mem=6.3G lam=None Gm=0.3613382975260417 +[g_only s0] step 9820/12000 loss=4.4699 gn=1.03 lr=1.04e-04 tps=17419 mem=6.3G lam=None Gm=0.3632456461588542 +[g_only s0] step 9840/12000 loss=4.1357 gn=0.96 lr=1.03e-04 tps=17420 mem=6.3G lam=None Gm=0.3606058756510417 +[g_only s0] step 9860/12000 loss=4.4460 gn=1.00 lr=1.03e-04 tps=17420 mem=6.3G lam=None Gm=0.3594868977864583 +[g_only s0] step 9880/12000 loss=4.4969 gn=1.07 lr=1.02e-04 tps=17420 mem=6.3G lam=None Gm=0.3642832438151042 +[g_only s0] step 9900/12000 loss=4.3103 gn=0.97 lr=1.01e-04 tps=17421 mem=6.3G lam=None Gm=0.3592987060546875 +[g_only s0] step 9920/12000 loss=4.4416 gn=1.00 lr=1.00e-04 tps=17422 mem=6.3G lam=None Gm=0.355133056640625 +[g_only s0] step 9940/12000 loss=4.5675 gn=1.05 lr=9.96e-05 tps=17422 mem=6.3G lam=None Gm=0.3578135172526042 +[g_only s0] step 9960/12000 loss=4.4616 gn=0.96 lr=9.89e-05 tps=17423 mem=6.3G lam=None Gm=0.3558756510416667 +[g_only s0] step 9980/12000 loss=4.3377 gn=0.94 lr=9.81e-05 tps=17423 mem=6.3G lam=None Gm=0.3542633056640625 +[g_only s0] step 10000/12000 loss=4.4809 gn=0.99 lr=9.74e-05 tps=17423 mem=6.3G lam=None Gm=0.3570454915364583 +[g_only s0] EVAL step 10000: val_loss=4.4370 val_ppl=84.52 +[g_only s0] step 10020/12000 loss=4.2466 gn=0.93 lr=9.67e-05 tps=17411 mem=6.3G lam=None Gm=0.3589986165364583 +[g_only s0] step 10040/12000 loss=4.2370 gn=0.93 lr=9.59e-05 tps=17412 mem=6.3G lam=None Gm=0.3535308837890625 +[g_only s0] step 10060/12000 loss=4.2621 gn=0.94 lr=9.52e-05 tps=17412 mem=6.3G lam=None Gm=0.35345458984375 +[g_only s0] step 10080/12000 loss=4.6043 gn=1.21 lr=9.45e-05 tps=17413 mem=6.3G lam=None Gm=0.3556722005208333 +[g_only s0] step 10100/12000 loss=4.5516 gn=0.96 lr=9.38e-05 tps=17413 mem=6.3G lam=None Gm=0.3578999837239583 +[g_only s0] step 10120/12000 loss=4.3289 gn=0.99 lr=9.31e-05 tps=17414 mem=6.3G lam=None Gm=0.356231689453125 +[g_only s0] step 10140/12000 loss=4.3456 gn=1.02 lr=9.24e-05 tps=17414 mem=6.3G lam=None Gm=0.3543752034505208 +[g_only s0] step 10160/12000 loss=4.2942 gn=0.94 lr=9.18e-05 tps=17414 mem=6.3G lam=None Gm=0.3559417724609375 +[g_only s0] step 10180/12000 loss=4.2967 gn=0.92 lr=9.11e-05 tps=17415 mem=6.3G lam=None Gm=0.3569793701171875 +[g_only s0] step 10200/12000 loss=4.3286 gn=0.98 lr=9.04e-05 tps=17415 mem=6.3G lam=None Gm=0.3579508463541667 +[g_only s0] step 10220/12000 loss=4.3669 gn=1.06 lr=8.98e-05 tps=17415 mem=6.3G lam=None Gm=0.3594207763671875 +[g_only s0] step 10240/12000 loss=4.3504 gn=1.05 lr=8.91e-05 tps=17416 mem=6.3G lam=None Gm=0.3587595621744792 +[g_only s0] step 10260/12000 loss=4.4065 gn=1.08 lr=8.85e-05 tps=17416 mem=6.3G lam=None Gm=0.3570658365885417 +[g_only s0] step 10280/12000 loss=4.3970 gn=1.07 lr=8.78e-05 tps=17416 mem=6.3G lam=None Gm=0.3599395751953125 +[g_only s0] step 10300/12000 loss=4.2846 gn=0.95 lr=8.72e-05 tps=17417 mem=6.3G lam=None Gm=0.3535970052083333 +[g_only s0] step 10320/12000 loss=4.2473 gn=0.96 lr=8.66e-05 tps=17417 mem=6.3G lam=None Gm=0.3523966471354167 +[g_only s0] step 10340/12000 loss=4.3239 gn=1.06 lr=8.59e-05 tps=17418 mem=6.3G lam=None Gm=0.3521931966145833 +[g_only s0] step 10360/12000 loss=4.4208 gn=1.00 lr=8.53e-05 tps=17418 mem=6.3G lam=None Gm=0.3579559326171875 +[g_only s0] step 10380/12000 loss=4.2554 gn=0.98 lr=8.47e-05 tps=17418 mem=6.3G lam=None Gm=0.3529408772786458 +[g_only s0] step 10400/12000 loss=4.3561 gn=1.02 lr=8.41e-05 tps=17419 mem=6.3G lam=None Gm=0.3551890055338542 +[g_only s0] step 10420/12000 loss=4.4146 gn=1.29 lr=8.35e-05 tps=17420 mem=6.3G lam=None Gm=0.3626047770182292 +[g_only s0] step 10440/12000 loss=4.4315 gn=1.11 lr=8.30e-05 tps=17420 mem=6.3G lam=None Gm=0.3592529296875 +[g_only s0] step 10460/12000 loss=4.4656 gn=0.99 lr=8.24e-05 tps=17421 mem=6.3G lam=None Gm=0.361572265625 +[g_only s0] step 10480/12000 loss=4.3480 gn=1.05 lr=8.18e-05 tps=17421 mem=6.3G lam=None Gm=0.3616943359375 +[g_only s0] step 10500/12000 loss=4.4359 gn=0.97 lr=8.12e-05 tps=17421 mem=6.3G lam=None Gm=0.3580169677734375 +[g_only s0] EVAL step 10500: val_loss=4.4428 val_ppl=85.01 +[g_only s0] step 10520/12000 loss=4.2492 gn=0.98 lr=8.07e-05 tps=17410 mem=6.3G lam=None Gm=0.356842041015625 +[g_only s0] step 10540/12000 loss=4.4096 gn=1.00 lr=8.01e-05 tps=17410 mem=6.3G lam=None Gm=0.3544972737630208 +[g_only s0] step 10560/12000 loss=4.3669 gn=1.01 lr=7.96e-05 tps=17411 mem=6.3G lam=None Gm=0.351898193359375 +[g_only s0] step 10580/12000 loss=4.3647 gn=1.00 lr=7.91e-05 tps=17411 mem=6.3G lam=None Gm=0.3570963541666667 +[g_only s0] step 10600/12000 loss=4.3909 gn=0.94 lr=7.85e-05 tps=17412 mem=6.3G lam=None Gm=0.3606770833333333 +[g_only s0] step 10620/12000 loss=4.1999 gn=1.02 lr=7.80e-05 tps=17412 mem=6.3G lam=None Gm=0.3550618489583333 +[g_only s0] step 10640/12000 loss=4.2167 gn=0.98 lr=7.75e-05 tps=17413 mem=6.3G lam=None Gm=0.3514658610026042 +[g_only s0] step 10660/12000 loss=4.3422 gn=0.95 lr=7.70e-05 tps=17413 mem=6.3G lam=None Gm=0.3550364176432292 +[g_only s0] step 10680/12000 loss=4.4310 gn=1.18 lr=7.65e-05 tps=17413 mem=6.3G lam=None Gm=0.3586324055989583 +[g_only s0] step 10700/12000 loss=4.2650 gn=1.00 lr=7.60e-05 tps=17414 mem=6.3G lam=None Gm=0.3561350504557292 +[g_only s0] step 10720/12000 loss=4.4749 gn=1.00 lr=7.55e-05 tps=17414 mem=6.3G lam=None Gm=0.3627878824869792 +[g_only s0] step 10740/12000 loss=4.3986 gn=0.94 lr=7.50e-05 tps=17414 mem=6.3G lam=None Gm=0.3590545654296875 +[g_only s0] step 10760/12000 loss=4.3142 gn=0.97 lr=7.46e-05 tps=17415 mem=6.3G lam=None Gm=0.3566029866536458 +[g_only s0] step 10780/12000 loss=4.3783 gn=1.00 lr=7.41e-05 tps=17415 mem=6.3G lam=None Gm=0.3587849934895833 +[g_only s0] step 10800/12000 loss=4.4746 gn=1.06 lr=7.37e-05 tps=17416 mem=6.3G lam=None Gm=0.3603769938151042 +[g_only s0] step 10820/12000 loss=4.2575 gn=1.08 lr=7.32e-05 tps=17416 mem=6.3G lam=None Gm=0.359039306640625 +[g_only s0] step 10840/12000 loss=4.3657 gn=0.95 lr=7.28e-05 tps=17417 mem=6.3G lam=None Gm=0.3487497965494792 +[g_only s0] step 10860/12000 loss=4.4659 gn=1.03 lr=7.23e-05 tps=17417 mem=6.3G lam=None Gm=0.35577392578125 +[g_only s0] step 10880/12000 loss=4.2922 gn=0.99 lr=7.19e-05 tps=17418 mem=6.3G lam=None Gm=0.3505655924479167 +[g_only s0] step 10900/12000 loss=4.3883 gn=1.08 lr=7.15e-05 tps=17418 mem=6.3G lam=None Gm=0.3581492106119792 +[g_only s0] step 10920/12000 loss=4.3258 gn=0.98 lr=7.11e-05 tps=17419 mem=6.3G lam=None Gm=0.3576304117838542 +[g_only s0] step 10940/12000 loss=4.4011 gn=1.05 lr=7.07e-05 tps=17419 mem=6.3G lam=None Gm=0.3550465901692708 +[g_only s0] step 10960/12000 loss=4.4357 gn=1.10 lr=7.03e-05 tps=17419 mem=6.3G lam=None Gm=0.3572642008463542 +[g_only s0] step 10980/12000 loss=4.3296 gn=1.04 lr=6.99e-05 tps=17420 mem=6.3G lam=None Gm=0.3551228841145833 +[g_only s0] step 11000/12000 loss=4.3634 gn=0.95 lr=6.95e-05 tps=17420 mem=6.3G lam=None Gm=0.3484141031901042 +[g_only s0] EVAL step 11000: val_loss=4.4012 val_ppl=81.55 +[g_only s0] step 11020/12000 loss=4.2281 gn=0.99 lr=6.91e-05 tps=17409 mem=6.3G lam=None Gm=0.3536529541015625 +[g_only s0] step 11040/12000 loss=4.2231 gn=0.97 lr=6.88e-05 tps=17410 mem=6.3G lam=None Gm=0.3521830240885417 +[g_only s0] step 11060/12000 loss=4.4371 gn=1.03 lr=6.84e-05 tps=17410 mem=6.3G lam=None Gm=0.35595703125 +[g_only s0] step 11080/12000 loss=4.3724 gn=0.97 lr=6.81e-05 tps=17411 mem=6.3G lam=None Gm=0.35400390625 +[g_only s0] step 11100/12000 loss=4.4371 gn=1.03 lr=6.77e-05 tps=17411 mem=6.3G lam=None Gm=0.3557027180989583 +[g_only s0] step 11120/12000 loss=4.3214 gn=0.94 lr=6.74e-05 tps=17411 mem=6.3G lam=None Gm=0.3508097330729167 +[g_only s0] step 11140/12000 loss=4.2010 gn=0.94 lr=6.70e-05 tps=17412 mem=6.3G lam=None Gm=0.3573455810546875 +[g_only s0] step 11160/12000 loss=4.2842 gn=1.04 lr=6.67e-05 tps=17412 mem=6.3G lam=None Gm=0.3550567626953125 +[g_only s0] step 11180/12000 loss=4.3799 gn=0.98 lr=6.64e-05 tps=17412 mem=6.3G lam=None Gm=0.355010986328125 +[g_only s0] step 11200/12000 loss=4.3624 gn=0.97 lr=6.61e-05 tps=17413 mem=6.3G lam=None Gm=0.3535512288411458 +[g_only s0] step 11220/12000 loss=4.3585 gn=0.98 lr=6.58e-05 tps=17413 mem=6.3G lam=None Gm=0.3558349609375 +[g_only s0] step 11240/12000 loss=4.1878 gn=0.94 lr=6.55e-05 tps=17413 mem=6.3G lam=None Gm=0.3530476888020833 +[g_only s0] step 11260/12000 loss=4.3783 gn=1.03 lr=6.52e-05 tps=17414 mem=6.3G lam=None Gm=0.3552652994791667 +[g_only s0] step 11280/12000 loss=4.3894 gn=1.03 lr=6.49e-05 tps=17414 mem=6.3G lam=None Gm=0.3554636637369792 +[g_only s0] step 11300/12000 loss=4.3693 gn=0.99 lr=6.47e-05 tps=17415 mem=6.3G lam=None Gm=0.3562723795572917 +[g_only s0] step 11320/12000 loss=4.2409 gn=1.00 lr=6.44e-05 tps=17415 mem=6.3G lam=None Gm=0.35369873046875 +[g_only s0] step 11340/12000 loss=4.2923 gn=1.02 lr=6.42e-05 tps=17415 mem=6.3G lam=None Gm=0.3546040852864583 +[g_only s0] step 11360/12000 loss=4.1850 gn=1.04 lr=6.39e-05 tps=17416 mem=6.3G lam=None Gm=0.3528493245442708 +[g_only s0] step 11380/12000 loss=4.3764 gn=1.03 lr=6.37e-05 tps=17416 mem=6.3G lam=None Gm=0.357940673828125 +[g_only s0] step 11400/12000 loss=4.3025 gn=1.04 lr=6.34e-05 tps=17416 mem=6.3G lam=None Gm=0.3508555094401042 +[g_only s0] step 11420/12000 loss=4.3179 gn=1.00 lr=6.32e-05 tps=17417 mem=6.3G lam=None Gm=0.3550364176432292 +[g_only s0] step 11440/12000 loss=4.1712 gn=1.00 lr=6.30e-05 tps=17417 mem=6.3G lam=None Gm=0.352996826171875 +[g_only s0] step 11460/12000 loss=4.4032 gn=1.02 lr=6.28e-05 tps=17418 mem=6.3G lam=None Gm=0.3579915364583333 +[g_only s0] step 11480/12000 loss=4.2803 gn=1.07 lr=6.26e-05 tps=17418 mem=6.3G lam=None Gm=0.3531595865885417 +[g_only s0] step 11500/12000 loss=4.2913 gn=0.95 lr=6.24e-05 tps=17419 mem=6.3G lam=None Gm=0.349456787109375 +[g_only s0] EVAL step 11500: val_loss=4.4084 val_ppl=82.14 +[g_only s0] step 11520/12000 loss=4.3133 gn=1.04 lr=6.22e-05 tps=17408 mem=6.3G lam=None Gm=0.3519795735677083 +[g_only s0] step 11540/12000 loss=4.4386 gn=1.19 lr=6.20e-05 tps=17408 mem=6.3G lam=None Gm=0.3553517659505208 +[g_only s0] step 11560/12000 loss=4.3847 gn=1.03 lr=6.19e-05 tps=17409 mem=6.3G lam=None Gm=0.3567606608072917 +[g_only s0] step 11580/12000 loss=4.1901 gn=1.12 lr=6.17e-05 tps=17409 mem=6.3G lam=None Gm=0.3483632405598958 +[g_only s0] step 11600/12000 loss=4.3141 gn=1.10 lr=6.15e-05 tps=17409 mem=6.3G lam=None Gm=0.3612111409505208 +[g_only s0] step 11620/12000 loss=4.3808 gn=1.10 lr=6.14e-05 tps=17410 mem=6.3G lam=None Gm=0.3581593831380208 +[g_only s0] step 11640/12000 loss=4.1327 gn=0.98 lr=6.12e-05 tps=17410 mem=6.3G lam=None Gm=0.3538716634114583 +[g_only s0] step 11660/12000 loss=4.2844 gn=0.97 lr=6.11e-05 tps=17411 mem=6.3G lam=None Gm=0.3467458089192708 +[g_only s0] step 11680/12000 loss=4.2653 gn=0.94 lr=6.10e-05 tps=17411 mem=6.3G lam=None Gm=0.3534596761067708 +[g_only s0] step 11700/12000 loss=4.2847 gn=1.03 lr=6.09e-05 tps=17411 mem=6.3G lam=None Gm=0.3531951904296875 +[g_only s0] step 11720/12000 loss=4.2574 gn=1.02 lr=6.07e-05 tps=17411 mem=6.3G lam=None Gm=0.3523203531901042 +[g_only s0] step 11740/12000 loss=4.4276 gn=0.98 lr=6.06e-05 tps=17412 mem=6.3G lam=None Gm=0.3551991780598958 +[g_only s0] step 11760/12000 loss=4.3629 gn=1.16 lr=6.06e-05 tps=17412 mem=6.3G lam=None Gm=0.3559926350911458 +[g_only s0] step 11780/12000 loss=4.3637 gn=1.07 lr=6.05e-05 tps=17412 mem=6.3G lam=None Gm=0.3503316243489583 +[g_only s0] step 11800/12000 loss=4.3952 gn=0.99 lr=6.04e-05 tps=17413 mem=6.3G lam=None Gm=0.3542531331380208 +[g_only s0] step 11820/12000 loss=4.2911 gn=1.04 lr=6.03e-05 tps=17413 mem=6.3G lam=None Gm=0.3536834716796875 +[g_only s0] step 11840/12000 loss=4.3420 gn=1.05 lr=6.02e-05 tps=17414 mem=6.3G lam=None Gm=0.3522084554036458 +[g_only s0] step 11860/12000 loss=4.1418 gn=0.97 lr=6.02e-05 tps=17414 mem=6.3G lam=None Gm=0.3503316243489583 +[g_only s0] step 11880/12000 loss=4.4812 gn=1.14 lr=6.01e-05 tps=17414 mem=6.3G lam=None Gm=0.3575286865234375 +[g_only s0] step 11900/12000 loss=4.2712 gn=1.07 lr=6.01e-05 tps=17415 mem=6.3G lam=None Gm=0.3500213623046875 +[g_only s0] step 11920/12000 loss=4.4410 gn=1.05 lr=6.01e-05 tps=17415 mem=6.3G lam=None Gm=0.3537851969401042 +[g_only s0] step 11940/12000 loss=4.1319 gn=1.07 lr=6.00e-05 tps=17416 mem=6.3G lam=None Gm=0.3500620524088542 +[g_only s0] step 11960/12000 loss=4.3485 gn=1.03 lr=6.00e-05 tps=17416 mem=6.3G lam=None Gm=0.3564249674479167 +[g_only s0] step 11980/12000 loss=4.2343 gn=0.98 lr=6.00e-05 tps=17416 mem=6.3G lam=None Gm=0.3484141031901042 +[g_only s0] EVAL step 11999: val_loss=4.3595 val_ppl=78.22 +[init] variant=full seed=0 params=63.98M +[init] use_S=True use_G=True +[full s0] step 0/12000 loss=10.9028 gn=9.85 lr=3.00e-06 tps=7412 mem=5.9G lam=0.12692801654338837 Gm=0.5030857535700003 +[full s0] step 20/12000 loss=9.7835 gn=1.94 lr=6.30e-05 tps=14666 mem=6.4G lam=0.12692161525289217 Gm=0.47064803664882976 +[full s0] step 40/12000 loss=9.0604 gn=1.79 lr=1.23e-04 tps=14895 mem=6.4G lam=0.12684216164052486 Gm=0.45263293012976646 +[full s0] step 60/12000 loss=8.1133 gn=1.55 lr=1.83e-04 tps=15032 mem=6.4G lam=0.12675164019068083 Gm=0.45841004078586894 +[full s0] step 80/12000 loss=7.5705 gn=0.88 lr=2.43e-04 tps=15121 mem=6.4G lam=0.12665745243430138 Gm=0.45741988345980644 +[full s0] step 100/12000 loss=7.3037 gn=0.61 lr=3.03e-04 tps=15173 mem=6.4G lam=0.12657030237217745 Gm=0.45109697928031284 +[full s0] step 120/12000 loss=7.0787 gn=0.58 lr=3.63e-04 tps=15204 mem=6.4G lam=0.1265490793933471 Gm=0.4301280876000722 +[full s0] step 140/12000 loss=6.9724 gn=0.58 lr=4.23e-04 tps=15228 mem=6.4G lam=0.12653516170879206 Gm=0.420130950709184 +[full s0] step 160/12000 loss=6.8059 gn=0.83 lr=4.83e-04 tps=15239 mem=6.4G lam=0.12643708661198616 Gm=0.4219328748683135 +[full s0] step 180/12000 loss=6.8571 gn=0.53 lr=5.43e-04 tps=15260 mem=6.4G lam=0.12642721955974898 Gm=0.40475064516067505 +[full s0] step 200/12000 loss=6.6866 gn=0.57 lr=6.00e-04 tps=15278 mem=6.4G lam=0.12640887995560965 Gm=0.4089492931962013 +[full s0] step 220/12000 loss=6.8277 gn=0.59 lr=6.00e-04 tps=15292 mem=6.4G lam=0.12637591734528542 Gm=0.4014202232162158 +[full s0] step 240/12000 loss=6.5461 gn=0.48 lr=6.00e-04 tps=15300 mem=6.4G lam=0.1263762547944983 Gm=0.4008398416141669 +[full s0] step 260/12000 loss=6.6478 gn=0.52 lr=6.00e-04 tps=15307 mem=6.4G lam=0.12640171311795712 Gm=0.3963535465300083 +[full s0] step 280/12000 loss=6.4134 gn=0.55 lr=6.00e-04 tps=15309 mem=6.4G lam=0.12637430553634962 Gm=0.39529188101490337 +[full s0] step 300/12000 loss=6.5464 gn=0.47 lr=6.00e-04 tps=15314 mem=6.4G lam=0.12641033654411635 Gm=0.4036916022499402 +[full s0] step 320/12000 loss=6.3468 gn=0.44 lr=6.00e-04 tps=15318 mem=6.4G lam=0.12642817137142023 Gm=0.4039401262998581 +[full s0] step 340/12000 loss=6.3876 gn=0.47 lr=6.00e-04 tps=15324 mem=6.4G lam=0.12647280345360437 Gm=0.4167020283639431 +[full s0] step 360/12000 loss=6.5535 gn=0.48 lr=6.00e-04 tps=15327 mem=6.4G lam=0.12646443210542202 Gm=0.4114844873547554 +[full s0] step 380/12000 loss=6.3936 gn=0.51 lr=6.00e-04 tps=15329 mem=6.4G lam=0.12644707473615804 Gm=0.39927731330196065 +[full s0] step 400/12000 loss=6.2887 gn=0.45 lr=6.00e-04 tps=15331 mem=6.4G lam=0.1265264286970099 Gm=0.41610954081018764 +[full s0] step 420/12000 loss=6.4140 gn=0.47 lr=6.00e-04 tps=15334 mem=6.4G lam=0.1266352403908968 Gm=0.4286409330864747 +[full s0] step 440/12000 loss=6.3595 gn=0.50 lr=5.99e-04 tps=15336 mem=6.4G lam=0.12672430432091156 Gm=0.4180796990791957 +[full s0] step 460/12000 loss=6.3949 gn=0.44 lr=5.99e-04 tps=15334 mem=6.4G lam=0.12681444454938173 Gm=0.43334127217531204 +[full s0] step 480/12000 loss=6.1804 gn=0.44 lr=5.99e-04 tps=15333 mem=6.4G lam=0.12698069928834835 Gm=0.4337771063049634 +[full s0] step 500/12000 loss=6.0581 gn=0.50 lr=5.99e-04 tps=15335 mem=6.4G lam=0.12712886463850737 Gm=0.447833130757014 +[full s0] EVAL step 500: val_loss=6.2807 val_ppl=534.14 +[full s0] step 520/12000 loss=6.2456 gn=0.53 lr=5.99e-04 tps=15112 mem=6.4G lam=0.1272899117320776 Gm=0.4641081790129344 +[full s0] step 540/12000 loss=6.1780 gn=0.51 lr=5.99e-04 tps=15118 mem=6.4G lam=0.1274485563238462 Gm=0.44570186113317806 +[full s0] step 560/12000 loss=6.1401 gn=0.46 lr=5.99e-04 tps=15127 mem=6.4G lam=0.12760758120566607 Gm=0.4752626543243726 +[full s0] step 580/12000 loss=6.1977 gn=0.51 lr=5.99e-04 tps=15131 mem=6.4G lam=0.12774930521845818 Gm=0.47633228326837224 +[full s0] step 600/12000 loss=6.1417 gn=0.49 lr=5.98e-04 tps=15139 mem=6.4G lam=0.12792953196913004 Gm=0.5081705699364344 +[full s0] step 620/12000 loss=6.0375 gn=0.46 lr=5.98e-04 tps=15147 mem=6.4G lam=0.12815746354560056 Gm=0.49799548586209613 +[full s0] step 640/12000 loss=6.1392 gn=0.50 lr=5.98e-04 tps=15152 mem=6.4G lam=0.12842637362579504 Gm=0.5167213914295038 +[full s0] step 660/12000 loss=6.0097 gn=0.50 lr=5.98e-04 tps=15159 mem=6.4G lam=0.12869772935907045 Gm=0.539153183499972 +[full s0] step 680/12000 loss=5.9955 gn=0.46 lr=5.98e-04 tps=15164 mem=6.4G lam=0.12900689585755268 Gm=0.5239760689437389 +[full s0] step 700/12000 loss=5.9872 gn=0.51 lr=5.98e-04 tps=15169 mem=6.4G lam=0.12928377681722245 Gm=0.5265582477053007 +[full s0] step 720/12000 loss=6.0768 gn=0.50 lr=5.97e-04 tps=15173 mem=6.4G lam=0.12954649205009142 Gm=0.5424879031876723 +[full s0] step 740/12000 loss=5.9174 gn=0.48 lr=5.97e-04 tps=15177 mem=6.4G lam=0.12987229755769172 Gm=0.5502202585339546 +[full s0] step 760/12000 loss=5.9031 gn=0.50 lr=5.97e-04 tps=15180 mem=6.4G lam=0.13020805486788353 Gm=0.5586358122527599 +[full s0] step 780/12000 loss=5.9340 gn=0.49 lr=5.97e-04 tps=15186 mem=6.4G lam=0.13058105887224278 Gm=0.5582266089816889 +[full s0] step 800/12000 loss=5.8207 gn=0.55 lr=5.97e-04 tps=15190 mem=6.4G lam=0.13096831801036993 Gm=0.5530718813339869 +[full s0] step 820/12000 loss=5.7846 gn=0.47 lr=5.96e-04 tps=15194 mem=6.4G lam=0.13132955630620322 Gm=0.569093340386947 +[full s0] step 840/12000 loss=5.7684 gn=0.49 lr=5.96e-04 tps=15198 mem=6.4G lam=0.13163264064739147 Gm=0.5557899003227552 +[full s0] step 860/12000 loss=5.7360 gn=0.50 lr=5.96e-04 tps=15202 mem=6.4G lam=0.13197322034587464 Gm=0.5638702635963758 +[full s0] step 880/12000 loss=5.8882 gn=0.53 lr=5.96e-04 tps=15204 mem=6.4G lam=0.13233029997597137 Gm=0.5668986402451992 +[full s0] step 900/12000 loss=5.7295 gn=0.49 lr=5.95e-04 tps=15205 mem=6.4G lam=0.13276872225105762 Gm=0.5559453802804152 +[full s0] step 920/12000 loss=5.7707 gn=0.52 lr=5.95e-04 tps=15208 mem=6.4G lam=0.13312861137092113 Gm=0.5541894050935904 +[full s0] step 940/12000 loss=5.7215 gn=0.47 lr=5.95e-04 tps=15211 mem=6.4G lam=0.1335356617346406 Gm=0.5472374310096105 +[full s0] step 960/12000 loss=5.7619 gn=0.54 lr=5.94e-04 tps=15214 mem=6.4G lam=0.1339366969962915 Gm=0.549694117779533 +[full s0] step 980/12000 loss=5.8844 gn=0.55 lr=5.94e-04 tps=15217 mem=6.4G lam=0.1343954810872674 Gm=0.5470301850388447 +[full s0] step 1000/12000 loss=5.8063 gn=0.53 lr=5.94e-04 tps=15221 mem=6.4G lam=0.13477198841671148 Gm=0.5581433965514103 +[full s0] EVAL step 1000: val_loss=5.7979 val_ppl=329.61 +[full s0] step 1020/12000 loss=5.6112 gn=0.52 lr=5.94e-04 tps=15112 mem=6.4G lam=0.13516638800501823 Gm=0.5565496012568474 +[full s0] step 1040/12000 loss=5.6752 gn=0.47 lr=5.93e-04 tps=15115 mem=6.4G lam=0.1355804561947783 Gm=0.5382961761206388 +[full s0] step 1060/12000 loss=5.7272 gn=0.48 lr=5.93e-04 tps=15120 mem=6.4G lam=0.13602483086287975 Gm=0.5388183612376451 +[full s0] step 1080/12000 loss=5.7057 gn=0.49 lr=5.93e-04 tps=15125 mem=6.4G lam=0.13646685549368462 Gm=0.5334495715796947 +[full s0] step 1100/12000 loss=5.6051 gn=0.50 lr=5.92e-04 tps=15130 mem=6.4G lam=0.13685248543818793 Gm=0.5363288577646017 +[full s0] step 1120/12000 loss=5.6983 gn=0.52 lr=5.92e-04 tps=15135 mem=6.4G lam=0.1372672182818254 Gm=0.5416510012000799 +[full s0] step 1140/12000 loss=5.5237 gn=0.50 lr=5.92e-04 tps=15139 mem=6.4G lam=0.13769453205168247 Gm=0.5288531885792812 +[full s0] step 1160/12000 loss=5.5707 gn=0.49 lr=5.91e-04 tps=15143 mem=6.4G lam=0.13812631585945687 Gm=0.5419504096110662 +[full s0] step 1180/12000 loss=5.6292 gn=0.48 lr=5.91e-04 tps=15147 mem=6.4G lam=0.13858075874547163 Gm=0.5418329760432243 +[full s0] step 1200/12000 loss=5.5105 gn=0.56 lr=5.90e-04 tps=15150 mem=6.4G lam=0.13903489398459593 Gm=0.5176569943626722 +[full s0] step 1220/12000 loss=5.6837 gn=0.51 lr=5.90e-04 tps=15154 mem=6.4G lam=0.13948220138748488 Gm=0.5235080905258656 +[full s0] step 1240/12000 loss=5.6492 gn=0.52 lr=5.90e-04 tps=15157 mem=6.4G lam=0.13990986657639345 Gm=0.5182733138402303 +[full s0] step 1260/12000 loss=5.6181 gn=0.55 lr=5.89e-04 tps=15161 mem=6.4G lam=0.14034730475395918 Gm=0.5297393829872211 +[full s0] step 1280/12000 loss=5.4750 gn=0.55 lr=5.89e-04 tps=15165 mem=6.4G lam=0.14076858821014562 Gm=0.5189812332391739 +[full s0] step 1300/12000 loss=5.5636 gn=0.64 lr=5.89e-04 tps=15169 mem=6.4G lam=0.14123849043001732 Gm=0.5294869635254145 +[full s0] step 1320/12000 loss=5.5288 gn=0.53 lr=5.88e-04 tps=15172 mem=6.4G lam=0.14167054556310177 Gm=0.5145816070338091 +[full s0] step 1340/12000 loss=5.6148 gn=0.54 lr=5.88e-04 tps=15174 mem=6.4G lam=0.1420488211636742 Gm=0.5206595876564583 +[full s0] step 1360/12000 loss=5.4369 gn=0.54 lr=5.87e-04 tps=15177 mem=6.4G lam=0.1424604132771492 Gm=0.5144363678991795 +[full s0] step 1380/12000 loss=5.2705 gn=0.52 lr=5.87e-04 tps=15180 mem=6.4G lam=0.14290577142188945 Gm=0.5145300648485621 +[full s0] step 1400/12000 loss=5.5548 gn=0.55 lr=5.86e-04 tps=15182 mem=6.4G lam=0.14335099204132953 Gm=0.51185775299867 +[full s0] step 1420/12000 loss=5.5011 gn=0.55 lr=5.86e-04 tps=15184 mem=6.4G lam=0.14381762593984604 Gm=0.4992476760720213 +[full s0] step 1440/12000 loss=5.3985 gn=0.53 lr=5.85e-04 tps=15188 mem=6.4G lam=0.14428733258197704 Gm=0.4918586763863762 +[full s0] step 1460/12000 loss=5.5103 gn=0.57 lr=5.85e-04 tps=15191 mem=6.4G lam=0.14477369282394648 Gm=0.5062761837616563 +[full s0] step 1480/12000 loss=5.4019 gn=0.52 lr=5.84e-04 tps=15193 mem=6.4G lam=0.14526124050219855 Gm=0.5015654700497786 +[full s0] step 1500/12000 loss=5.6369 gn=0.57 lr=5.84e-04 tps=15196 mem=6.4G lam=0.1456847603743275 Gm=0.4963273545727134 +[full s0] EVAL step 1500: val_loss=5.5202 val_ppl=249.68 +[full s0] step 1520/12000 loss=5.4208 gn=0.56 lr=5.83e-04 tps=15124 mem=6.4G lam=0.146172607007126 Gm=0.49396850261837244 +[full s0] step 1540/12000 loss=5.2729 gn=0.51 lr=5.83e-04 tps=15126 mem=6.4G lam=0.14661350504805645 Gm=0.4788104897985856 +[full s0] step 1560/12000 loss=5.5089 gn=0.56 lr=5.82e-04 tps=15129 mem=6.4G lam=0.1470441666121284 Gm=0.5041911782075962 +[full s0] step 1580/12000 loss=5.4239 gn=0.64 lr=5.82e-04 tps=15131 mem=6.4G lam=0.1475201128050685 Gm=0.485867432008187 +[full s0] step 1600/12000 loss=5.5348 gn=0.53 lr=5.81e-04 tps=15135 mem=6.4G lam=0.1480239707355698 Gm=0.48696788617720205 +[full s0] step 1620/12000 loss=5.3318 gn=0.54 lr=5.81e-04 tps=15137 mem=6.4G lam=0.14851641623924175 Gm=0.49762762772540253 +[full s0] step 1640/12000 loss=5.3833 gn=0.57 lr=5.80e-04 tps=15140 mem=6.4G lam=0.1490536853671074 Gm=0.4872809685766697 +[full s0] step 1660/12000 loss=5.3669 gn=0.55 lr=5.80e-04 tps=15143 mem=6.4G lam=0.14951497440536818 Gm=0.4886004456008474 +[full s0] step 1680/12000 loss=5.4630 gn=0.53 lr=5.79e-04 tps=15145 mem=6.4G lam=0.14999949683745703 Gm=0.47575207458188135 +[full s0] step 1700/12000 loss=5.3841 gn=0.54 lr=5.79e-04 tps=15145 mem=6.4G lam=0.15052947433044514 Gm=0.47670603916049004 +[full s0] step 1720/12000 loss=5.3359 gn=0.61 lr=5.78e-04 tps=15147 mem=6.4G lam=0.15104490083952746 Gm=0.47610893038411933 +[full s0] step 1740/12000 loss=5.4070 gn=0.67 lr=5.78e-04 tps=15150 mem=6.4G lam=0.15157404635101557 Gm=0.4794648407648007 +[full s0] step 1760/12000 loss=5.3728 gn=0.53 lr=5.77e-04 tps=15152 mem=6.4G lam=0.152010602876544 Gm=0.47699382280309993 +[full s0] step 1780/12000 loss=5.3204 gn=0.52 lr=5.76e-04 tps=15153 mem=6.4G lam=0.15248788303385177 Gm=0.4691106112052997 +[full s0] step 1800/12000 loss=5.3583 gn=0.63 lr=5.76e-04 tps=15155 mem=6.4G lam=0.1530351536348462 Gm=0.45547197572886944 +[full s0] step 1820/12000 loss=5.2699 gn=0.60 lr=5.75e-04 tps=15157 mem=6.4G lam=0.15351038333028555 Gm=0.47635356554140645 +[full s0] step 1840/12000 loss=5.3923 gn=0.61 lr=5.75e-04 tps=15159 mem=6.4G lam=0.15402550778041282 Gm=0.47356026091923314 +[full s0] step 1860/12000 loss=5.3603 gn=0.57 lr=5.74e-04 tps=15161 mem=6.4G lam=0.15452732456227145 Gm=0.4700409785533945 +[full s0] step 1880/12000 loss=5.2979 gn=0.57 lr=5.73e-04 tps=15164 mem=6.4G lam=0.15499083107958236 Gm=0.46564727276563644 +[full s0] step 1900/12000 loss=5.2457 gn=0.60 lr=5.73e-04 tps=15166 mem=6.4G lam=0.1555120162665844 Gm=0.4621336354563634 +[full s0] step 1920/12000 loss=5.1950 gn=0.59 lr=5.72e-04 tps=15168 mem=6.4G lam=0.15611659766485295 Gm=0.46651650965213776 +[full s0] step 1940/12000 loss=5.1756 gn=0.58 lr=5.72e-04 tps=15170 mem=6.4G lam=0.15667632004866996 Gm=0.46511891204863787 +[full s0] step 1960/12000 loss=5.1710 gn=0.65 lr=5.71e-04 tps=15171 mem=6.4G lam=0.15717450808733702 Gm=0.46314529422670603 +[full s0] step 1980/12000 loss=5.0468 gn=0.57 lr=5.70e-04 tps=15173 mem=6.4G lam=0.1577411104614536 Gm=0.45130133753021556 +[full s0] step 2000/12000 loss=5.2174 gn=0.60 lr=5.70e-04 tps=15175 mem=6.4G lam=0.15821693216760954 Gm=0.46244405892988044 +[full s0] EVAL step 2000: val_loss=5.3212 val_ppl=204.64 +[full s0] step 2020/12000 loss=5.1101 gn=0.59 lr=5.69e-04 tps=15120 mem=6.4G lam=0.1586812355866035 Gm=0.4563881183664004 +[full s0] step 2040/12000 loss=5.3111 gn=1.00 lr=5.68e-04 tps=15122 mem=6.4G lam=0.15919070908178887 Gm=0.4711304021378358 +[full s0] step 2060/12000 loss=5.2923 gn=0.55 lr=5.68e-04 tps=15124 mem=6.4G lam=0.15968584828078747 Gm=0.4572604785983761 +[full s0] step 2080/12000 loss=5.2215 gn=0.67 lr=5.67e-04 tps=15127 mem=6.4G lam=0.1601607445627451 Gm=0.4637175689761837 +[full s0] step 2100/12000 loss=5.2389 gn=0.60 lr=5.66e-04 tps=15129 mem=6.4G lam=0.1607115234558781 Gm=0.45194955278808874 +[full s0] step 2120/12000 loss=5.2037 gn=0.67 lr=5.65e-04 tps=15131 mem=6.4G lam=0.1612076967333754 Gm=0.44715134520083666 +[full s0] step 2140/12000 loss=5.2527 gn=0.61 lr=5.65e-04 tps=15133 mem=6.4G lam=0.1617035949602723 Gm=0.4465855322778225 +[full s0] step 2160/12000 loss=5.3042 gn=0.58 lr=5.64e-04 tps=15136 mem=6.4G lam=0.16223273985087872 Gm=0.4485787176527083 +[full s0] step 2180/12000 loss=5.2645 gn=0.59 lr=5.63e-04 tps=15138 mem=6.4G lam=0.16272384083519378 Gm=0.4528226237744093 +[full s0] step 2200/12000 loss=5.1953 gn=0.57 lr=5.63e-04 tps=15140 mem=6.4G lam=0.16315928722421327 Gm=0.43795497932781774 +[full s0] step 2220/12000 loss=5.1914 gn=0.71 lr=5.62e-04 tps=15142 mem=6.4G lam=0.16367512413611016 Gm=0.43534837616607547 +[full s0] step 2240/12000 loss=5.0956 gn=0.61 lr=5.61e-04 tps=15144 mem=6.4G lam=0.16415310899416605 Gm=0.4575654047851761 +[full s0] step 2260/12000 loss=5.2034 gn=0.64 lr=5.60e-04 tps=15144 mem=6.4G lam=0.16468209338684878 Gm=0.4340684798856576 +[full s0] step 2280/12000 loss=5.1173 gn=0.75 lr=5.60e-04 tps=15146 mem=6.4G lam=0.1652182461693883 Gm=0.44931269545728963 +[full s0] step 2300/12000 loss=5.1441 gn=0.69 lr=5.59e-04 tps=15148 mem=6.4G lam=0.16571071526656547 Gm=0.44450906788309413 +[full s0] step 2320/12000 loss=5.1289 gn=0.64 lr=5.58e-04 tps=15150 mem=6.4G lam=0.16620047638813654 Gm=0.4379417252105971 +[full s0] step 2340/12000 loss=5.0598 gn=0.62 lr=5.57e-04 tps=15151 mem=6.4G lam=0.16663923393934965 Gm=0.4344549171316127 +[full s0] step 2360/12000 loss=5.1620 gn=0.61 lr=5.57e-04 tps=15153 mem=6.4G lam=0.16714676345388094 Gm=0.4417942327757676 +[full s0] step 2380/12000 loss=5.2219 gn=0.64 lr=5.56e-04 tps=15155 mem=6.4G lam=0.16766617447137833 Gm=0.4409579737111926 +[full s0] step 2400/12000 loss=5.1242 gn=0.65 lr=5.55e-04 tps=15157 mem=6.4G lam=0.1681990623474121 Gm=0.44166227321450907 +[full s0] step 2420/12000 loss=5.2252 gn=0.64 lr=5.54e-04 tps=15159 mem=6.4G lam=0.16869600613911948 Gm=0.43691842444241047 +[full s0] step 2440/12000 loss=5.0230 gn=0.65 lr=5.53e-04 tps=15160 mem=6.4G lam=0.16922400705516338 Gm=0.43753723163778585 +[full s0] step 2460/12000 loss=5.0776 gn=0.67 lr=5.53e-04 tps=15162 mem=6.4G lam=0.16961592870453993 Gm=0.42989698130016524 +[full s0] step 2480/12000 loss=5.0470 gn=0.67 lr=5.52e-04 tps=15164 mem=6.4G lam=0.17004887852817774 Gm=0.43911324922616285 +[full s0] step 2500/12000 loss=5.1824 gn=0.61 lr=5.51e-04 tps=15165 mem=6.4G lam=0.17053523318221173 Gm=0.43972835317254066 +[full s0] EVAL step 2500: val_loss=5.1586 val_ppl=173.92 +[full s0] step 2520/12000 loss=5.0930 gn=0.60 lr=5.50e-04 tps=15121 mem=6.4G lam=0.1710941499720017 Gm=0.432137871781985 +[full s0] step 2540/12000 loss=4.9583 gn=0.63 lr=5.49e-04 tps=15122 mem=6.4G lam=0.171570540095369 Gm=0.4367065044740836 +[full s0] step 2560/12000 loss=5.2204 gn=0.67 lr=5.48e-04 tps=15124 mem=6.4G lam=0.17201251784960428 Gm=0.4352534149462978 +[full s0] step 2580/12000 loss=5.0818 gn=0.65 lr=5.48e-04 tps=15126 mem=6.4G lam=0.17251587938517332 Gm=0.42544189825033146 +[full s0] step 2600/12000 loss=4.9320 gn=0.60 lr=5.47e-04 tps=15128 mem=6.4G lam=0.17299400735646486 Gm=0.41622836142778397 +[full s0] step 2620/12000 loss=4.9813 gn=0.68 lr=5.46e-04 tps=15130 mem=6.4G lam=0.17351485136896372 Gm=0.4240449314626555 +[full s0] step 2640/12000 loss=5.0929 gn=0.65 lr=5.45e-04 tps=15132 mem=6.4G lam=0.17392214015126228 Gm=0.42283995635807514 +[full s0] step 2660/12000 loss=5.0502 gn=0.78 lr=5.44e-04 tps=15133 mem=6.4G lam=0.17444972228258848 Gm=0.4279710765307148 +[full s0] step 2680/12000 loss=5.0416 gn=0.69 lr=5.43e-04 tps=15135 mem=6.4G lam=0.17501085375746092 Gm=0.4265733954186241 +[full s0] step 2700/12000 loss=5.0449 gn=0.63 lr=5.42e-04 tps=15136 mem=6.4G lam=0.17550581693649292 Gm=0.433244073142608 +[full s0] step 2720/12000 loss=4.9943 gn=0.65 lr=5.41e-04 tps=15138 mem=6.4G lam=0.17601683735847473 Gm=0.4283275920897722 +[full s0] step 2740/12000 loss=5.0751 gn=0.66 lr=5.41e-04 tps=15139 mem=6.4G lam=0.17651626529792944 Gm=0.42746219877153635 +[full s0] step 2760/12000 loss=5.2394 gn=0.67 lr=5.40e-04 tps=15141 mem=6.4G lam=0.17702119580159584 Gm=0.42482205961520475 +[full s0] step 2780/12000 loss=5.2076 gn=0.65 lr=5.39e-04 tps=15142 mem=6.4G lam=0.1775593419248859 Gm=0.427764184307307 +[full s0] step 2800/12000 loss=4.9843 gn=0.63 lr=5.38e-04 tps=15144 mem=6.4G lam=0.1780392462387681 Gm=0.42343819607049227 +[full s0] step 2820/12000 loss=5.2667 gn=0.71 lr=5.37e-04 tps=15145 mem=6.4G lam=0.1785952222223083 Gm=0.42792007777219015 +[full s0] step 2840/12000 loss=4.9997 gn=0.66 lr=5.36e-04 tps=15147 mem=6.4G lam=0.17908049188554287 Gm=0.4321679063141346 +[full s0] step 2860/12000 loss=5.0200 gn=0.63 lr=5.35e-04 tps=15148 mem=6.4G lam=0.17951612391819557 Gm=0.41727534495294094 +[full s0] step 2880/12000 loss=5.0660 gn=0.67 lr=5.34e-04 tps=15149 mem=6.4G lam=0.1800707345828414 Gm=0.42635943042114377 +[full s0] step 2900/12000 loss=4.8910 gn=0.65 lr=5.33e-04 tps=15151 mem=6.4G lam=0.1804889620592197 Gm=0.40841255864749354 +[full s0] step 2920/12000 loss=5.0241 gn=0.66 lr=5.32e-04 tps=15152 mem=6.4G lam=0.18106321121255556 Gm=0.41869119290883344 +[full s0] step 2940/12000 loss=5.0699 gn=0.72 lr=5.31e-04 tps=15154 mem=6.4G lam=0.18157555597523847 Gm=0.42433939781039953 +[full s0] step 2960/12000 loss=4.9180 gn=0.76 lr=5.30e-04 tps=15155 mem=6.4G lam=0.18208395907034478 Gm=0.42504422754670185 +[full s0] step 2980/12000 loss=5.0691 gn=0.72 lr=5.29e-04 tps=15157 mem=6.4G lam=0.18259849213063717 Gm=0.4200627530614535 +[full s0] step 3000/12000 loss=5.0501 gn=0.69 lr=5.28e-04 tps=15158 mem=6.4G lam=0.18303113151341677 Gm=0.4237479887281855 +[full s0] EVAL step 3000: val_loss=5.0175 val_ppl=151.03 +[full s0] step 3020/12000 loss=4.9994 gn=0.67 lr=5.27e-04 tps=15121 mem=6.4G lam=0.18354421481490135 Gm=0.4150480581447482 +[full s0] step 3040/12000 loss=5.1510 gn=0.66 lr=5.26e-04 tps=15123 mem=6.4G lam=0.18405207029233375 Gm=0.41946570575237274 +[full s0] step 3060/12000 loss=4.8888 gn=0.61 lr=5.25e-04 tps=15125 mem=6.4G lam=0.18454770868023238 Gm=0.42124345370878774 +[full s0] step 3080/12000 loss=4.9621 gn=0.67 lr=5.24e-04 tps=15126 mem=6.4G lam=0.18507327201465765 Gm=0.41507322403291863 +[full s0] step 3100/12000 loss=4.8779 gn=0.72 lr=5.23e-04 tps=15128 mem=6.4G lam=0.18557003823419413 Gm=0.4135791154888769 +[full s0] step 3120/12000 loss=4.8583 gn=0.68 lr=5.22e-04 tps=15129 mem=6.4G lam=0.18606651636461416 Gm=0.4133033159499367 +[full s0] step 3140/12000 loss=5.0407 gn=0.76 lr=5.21e-04 tps=15130 mem=6.4G lam=0.18653381088127693 Gm=0.4283616701141 +[full s0] step 3160/12000 loss=4.8284 gn=0.65 lr=5.20e-04 tps=15132 mem=6.4G lam=0.18695409906407198 Gm=0.4077521103123824 +[full s0] step 3180/12000 loss=4.8634 gn=0.73 lr=5.19e-04 tps=15133 mem=6.4G lam=0.18743332754820585 Gm=0.4180694765721758 +[full s0] step 3200/12000 loss=4.8057 gn=0.69 lr=5.18e-04 tps=15135 mem=6.4G lam=0.1878522913902998 Gm=0.4036712725646794 +[full s0] step 3220/12000 loss=4.7788 gn=0.69 lr=5.17e-04 tps=15137 mem=6.4G lam=0.18832485750317574 Gm=0.42147670841465396 +[full s0] step 3240/12000 loss=4.8345 gn=0.68 lr=5.16e-04 tps=15138 mem=6.4G lam=0.18879811962445578 Gm=0.41517681054150063 +[full s0] step 3260/12000 loss=4.9796 gn=0.66 lr=5.15e-04 tps=15140 mem=6.4G lam=0.18929200495282808 Gm=0.40772299530605477 +[full s0] step 3280/12000 loss=4.8247 gn=0.64 lr=5.14e-04 tps=15142 mem=6.4G lam=0.18980584914485613 Gm=0.4084823538238804 +[full s0] step 3300/12000 loss=5.0114 gn=0.69 lr=5.13e-04 tps=15142 mem=6.4G lam=0.1903259518245856 Gm=0.4178314986638725 +[full s0] step 3320/12000 loss=4.8810 gn=0.70 lr=5.12e-04 tps=15144 mem=6.4G lam=0.19079325441271067 Gm=0.41009761827687424 +[full s0] step 3340/12000 loss=4.9716 gn=0.69 lr=5.11e-04 tps=15145 mem=6.4G lam=0.1912977018704017 Gm=0.4167470975468556 +[full s0] step 3360/12000 loss=4.7663 gn=0.67 lr=5.10e-04 tps=15147 mem=6.4G lam=0.19171746416638294 Gm=0.408527256300052 +[full s0] step 3380/12000 loss=4.9367 gn=1.12 lr=5.09e-04 tps=15148 mem=6.4G lam=0.1921984221165379 Gm=0.40314778049166006 +[full s0] step 3400/12000 loss=4.9519 gn=0.68 lr=5.08e-04 tps=15150 mem=6.4G lam=0.19265196782847246 Gm=0.40912951560070115 +[full s0] step 3420/12000 loss=4.8115 gn=0.70 lr=5.07e-04 tps=15151 mem=6.4G lam=0.19314223621040583 Gm=0.40629174855227274 +[full s0] step 3440/12000 loss=4.8960 gn=0.69 lr=5.06e-04 tps=15152 mem=6.4G lam=0.19360573372493187 Gm=0.41025739163160324 +[full s0] step 3460/12000 loss=4.7952 gn=0.67 lr=5.05e-04 tps=15154 mem=6.4G lam=0.19409532907108465 Gm=0.3998872651718557 +[full s0] step 3480/12000 loss=4.7831 gn=0.67 lr=5.03e-04 tps=15155 mem=6.4G lam=0.19469708142181238 Gm=0.39722216641530395 +[full s0] step 3500/12000 loss=4.9400 gn=0.72 lr=5.02e-04 tps=15156 mem=6.4G lam=0.19519802369177341 Gm=0.4075687707712253 +[full s0] EVAL step 3500: val_loss=4.9113 val_ppl=135.81 +[full s0] step 3520/12000 loss=4.9210 gn=0.67 lr=5.01e-04 tps=15125 mem=6.4G lam=0.19566717185080051 Gm=0.40233205414066714 +[full s0] step 3540/12000 loss=4.7882 gn=0.73 lr=5.00e-04 tps=15126 mem=6.4G lam=0.196157518774271 Gm=0.39936579174051684 +[full s0] step 3560/12000 loss=4.8386 gn=0.68 lr=4.99e-04 tps=15127 mem=6.4G lam=0.19666343523810306 Gm=0.402777759047846 +[full s0] step 3580/12000 loss=4.9060 gn=0.72 lr=4.98e-04 tps=15129 mem=6.4G lam=0.19719347295661768 Gm=0.40737946378067136 +[full s0] step 3600/12000 loss=4.7693 gn=0.69 lr=4.97e-04 tps=15130 mem=6.4G lam=0.19768324059744677 Gm=0.4009760059416294 +[full s0] step 3620/12000 loss=4.8484 gn=0.67 lr=4.96e-04 tps=15131 mem=6.4G lam=0.19816474057734013 Gm=0.4116689378085236 +[full s0] step 3640/12000 loss=4.9033 gn=0.68 lr=4.94e-04 tps=15133 mem=6.4G lam=0.19863907744487128 Gm=0.3990051302437981 +[full s0] step 3660/12000 loss=4.7991 gn=0.72 lr=4.93e-04 tps=15134 mem=6.4G lam=0.19915147932867208 Gm=0.4010490065750976 +[full s0] step 3680/12000 loss=4.9154 gn=0.66 lr=4.92e-04 tps=15136 mem=6.4G lam=0.1995557484527429 Gm=0.4044267051698019 +[full s0] step 3700/12000 loss=4.7775 gn=0.78 lr=4.91e-04 tps=15137 mem=6.4G lam=0.1999612689639131 Gm=0.3995776614174247 +[full s0] step 3720/12000 loss=4.9063 gn=0.78 lr=4.90e-04 tps=15138 mem=6.4G lam=0.2005290056889256 Gm=0.4103377099769811 +[full s0] step 3740/12000 loss=4.8579 gn=0.68 lr=4.89e-04 tps=15140 mem=6.4G lam=0.20101405618091425 Gm=0.3964865958938996 +[full s0] step 3760/12000 loss=4.9184 gn=0.67 lr=4.88e-04 tps=15141 mem=6.4G lam=0.20150733211388192 Gm=0.40290993157153326 +[full s0] step 3780/12000 loss=4.9096 gn=0.72 lr=4.86e-04 tps=15142 mem=6.4G lam=0.2019740374137958 Gm=0.4141649509159227 +[full s0] step 3800/12000 loss=4.8836 gn=0.73 lr=4.85e-04 tps=15143 mem=6.4G lam=0.202532474262019 Gm=0.4054215303622186 +[full s0] step 3820/12000 loss=4.6448 gn=0.70 lr=4.84e-04 tps=15145 mem=6.4G lam=0.2030342317496737 Gm=0.3970128642395139 +[full s0] step 3840/12000 loss=4.9472 gn=0.70 lr=4.83e-04 tps=15146 mem=6.4G lam=0.2035044301301241 Gm=0.3974209391356756 +[full s0] step 3860/12000 loss=4.8329 gn=0.75 lr=4.82e-04 tps=15147 mem=6.4G lam=0.20403934394319853 Gm=0.40113243569309515 +[full s0] step 3880/12000 loss=4.7470 gn=0.66 lr=4.80e-04 tps=15149 mem=6.4G lam=0.20445159325997034 Gm=0.4012607814123233 +[full s0] step 3900/12000 loss=4.7204 gn=0.70 lr=4.79e-04 tps=15150 mem=6.4G lam=0.2049207870538036 Gm=0.39952827865878743 +[full s0] step 3920/12000 loss=4.8328 gn=0.78 lr=4.78e-04 tps=15151 mem=6.4G lam=0.20536515768617392 Gm=0.39485114300623536 +[full s0] step 3940/12000 loss=4.6526 gn=0.70 lr=4.77e-04 tps=15152 mem=6.4G lam=0.20595746984084448 Gm=0.3957008776875834 +[full s0] step 3960/12000 loss=4.7690 gn=0.73 lr=4.76e-04 tps=15153 mem=6.4G lam=0.206428123327593 Gm=0.4038659512686233 +[full s0] step 3980/12000 loss=4.6715 gn=0.69 lr=4.74e-04 tps=15154 mem=6.4G lam=0.20686180361857018 Gm=0.400312392351528 +[full s0] step 4000/12000 loss=4.7628 gn=0.72 lr=4.73e-04 tps=15155 mem=6.4G lam=0.20733655306200185 Gm=0.41114364052191377 +[full s0] EVAL step 4000: val_loss=4.8718 val_ppl=130.56 +[full s0] step 4020/12000 loss=4.9105 gn=0.67 lr=4.72e-04 tps=15127 mem=6.4G lam=0.20781329025824866 Gm=0.39211222203448415 +[full s0] step 4040/12000 loss=4.9056 gn=0.71 lr=4.71e-04 tps=15128 mem=6.4G lam=0.2082832089314858 Gm=0.3922128643995772 +[full s0] step 4060/12000 loss=4.7612 gn=0.70 lr=4.70e-04 tps=15129 mem=6.4G lam=0.20876962722589573 Gm=0.3957918072895457 +[full s0] step 4080/12000 loss=4.7931 gn=0.69 lr=4.68e-04 tps=15130 mem=6.4G lam=0.2092914922783772 Gm=0.3895923637319356 +[full s0] step 4100/12000 loss=4.6894 gn=0.73 lr=4.67e-04 tps=15131 mem=6.4G lam=0.20976976677775383 Gm=0.39156696464245516 +[full s0] step 4120/12000 loss=4.6260 gn=0.81 lr=4.66e-04 tps=15133 mem=6.4G lam=0.2101848175128301 Gm=0.38494530017487705 +[full s0] step 4140/12000 loss=4.7342 gn=0.74 lr=4.65e-04 tps=15134 mem=6.4G lam=0.21068514324724674 Gm=0.40033878044535715 +[full s0] step 4160/12000 loss=4.7409 gn=0.79 lr=4.63e-04 tps=15135 mem=6.4G lam=0.21110688957075277 Gm=0.39206655983192223 +[full s0] step 4180/12000 loss=4.9083 gn=0.71 lr=4.62e-04 tps=15136 mem=6.4G lam=0.21161337538311878 Gm=0.39505979341144365 +[full s0] step 4200/12000 loss=4.7281 gn=0.76 lr=4.61e-04 tps=15137 mem=6.4G lam=0.21214932451645532 Gm=0.3897402674580614 +[full s0] step 4220/12000 loss=4.8047 gn=0.67 lr=4.60e-04 tps=15139 mem=6.4G lam=0.21262734942138195 Gm=0.39211626444011927 +[full s0] step 4240/12000 loss=4.8020 gn=0.68 lr=4.58e-04 tps=15140 mem=6.4G lam=0.21304297198851904 Gm=0.38949206626663607 +[full s0] step 4260/12000 loss=4.8546 gn=0.72 lr=4.57e-04 tps=15141 mem=6.4G lam=0.21359918235490719 Gm=0.39041900425218046 +[full s0] step 4280/12000 loss=4.8642 gn=0.80 lr=4.56e-04 tps=15142 mem=6.4G lam=0.21408760144064823 Gm=0.3972075354928772 +[full s0] step 4300/12000 loss=4.8404 gn=0.75 lr=4.54e-04 tps=15143 mem=6.4G lam=0.21456307700524727 Gm=0.393576090844969 +[full s0] step 4320/12000 loss=4.6860 gn=0.74 lr=4.53e-04 tps=15144 mem=6.4G lam=0.21499274546901384 Gm=0.39231724400694173 +[full s0] step 4340/12000 loss=4.6906 gn=0.76 lr=4.52e-04 tps=15145 mem=6.4G lam=0.21551166443775097 Gm=0.3928618535088996 +[full s0] step 4360/12000 loss=4.8527 gn=0.68 lr=4.51e-04 tps=15146 mem=6.4G lam=0.2159822260340055 Gm=0.40105347249967355 +[full s0] step 4380/12000 loss=4.8320 gn=0.71 lr=4.49e-04 tps=15147 mem=6.4G lam=0.21636081269631782 Gm=0.38982328516431153 +[full s0] step 4400/12000 loss=4.5892 gn=0.74 lr=4.48e-04 tps=15148 mem=6.4G lam=0.2168724319587151 Gm=0.3918915918717782 +[full s0] step 4420/12000 loss=4.8546 gn=0.69 lr=4.47e-04 tps=15149 mem=6.4G lam=0.21732835285365582 Gm=0.39792427280917764 +[full s0] step 4440/12000 loss=4.7146 gn=0.82 lr=4.45e-04 tps=15150 mem=6.4G lam=0.21781370198975006 Gm=0.3909165835939348 +[full s0] step 4460/12000 loss=4.7260 gn=0.77 lr=4.44e-04 tps=15151 mem=6.4G lam=0.21830343548208475 Gm=0.3847840049614509 +[full s0] step 4480/12000 loss=4.6941 gn=0.69 lr=4.43e-04 tps=15152 mem=6.4G lam=0.21881148715813956 Gm=0.38561362369606894 +[full s0] step 4500/12000 loss=4.7189 gn=0.79 lr=4.42e-04 tps=15153 mem=6.4G lam=0.21923744088659683 Gm=0.3949823022509615 +[full s0] EVAL step 4500: val_loss=4.8099 val_ppl=122.72 +[full s0] step 4520/12000 loss=4.7058 gn=0.69 lr=4.40e-04 tps=15126 mem=6.4G lam=0.21971843546877304 Gm=0.3846162559154133 +[full s0] step 4540/12000 loss=4.7459 gn=0.74 lr=4.39e-04 tps=15125 mem=6.4G lam=0.220151098134617 Gm=0.38585143520807225 +[full s0] step 4560/12000 loss=4.7622 gn=0.74 lr=4.38e-04 tps=15124 mem=6.4G lam=0.22062792349606752 Gm=0.3876468523715933 +[full s0] step 4580/12000 loss=4.7210 gn=0.74 lr=4.36e-04 tps=15122 mem=6.4G lam=0.22103441537668309 Gm=0.388102900236845 +[full s0] step 4600/12000 loss=4.7170 gn=0.73 lr=4.35e-04 tps=15121 mem=6.4G lam=0.22153022326529026 Gm=0.3885766194822888 +[full s0] step 4620/12000 loss=4.7400 gn=0.77 lr=4.34e-04 tps=15119 mem=6.4G lam=0.22199827153235674 Gm=0.3859506705775857 +[full s0] step 4640/12000 loss=4.7983 gn=0.77 lr=4.32e-04 tps=15115 mem=6.4G lam=0.2224260043973724 Gm=0.3819904289363573 +[full s0] step 4660/12000 loss=4.6769 gn=0.76 lr=4.31e-04 tps=15112 mem=6.4G lam=0.22282474146535 Gm=0.3854291300910215 +[full s0] step 4680/12000 loss=4.7215 gn=0.75 lr=4.30e-04 tps=15110 mem=6.4G lam=0.22328661661595106 Gm=0.3910363026273747 +[full s0] step 4700/12000 loss=4.6889 gn=0.79 lr=4.28e-04 tps=15108 mem=6.4G lam=0.22377826614926258 Gm=0.3876813513537248 +[full s0] step 4720/12000 loss=4.7240 gn=0.76 lr=4.27e-04 tps=15105 mem=6.4G lam=0.22424745808045068 Gm=0.39499434852041304 +[full s0] step 4740/12000 loss=4.6011 gn=0.79 lr=4.26e-04 tps=15107 mem=6.4G lam=0.2247697881733378 Gm=0.3901003492064774 +[full s0] step 4760/12000 loss=4.6895 gn=0.79 lr=4.24e-04 tps=15108 mem=6.4G lam=0.2252369367827972 Gm=0.3910440008621663 +[full s0] step 4780/12000 loss=4.7345 gn=0.73 lr=4.23e-04 tps=15109 mem=6.4G lam=0.22570862962553898 Gm=0.3882999330914269 +[full s0] step 4800/12000 loss=4.7939 gn=0.74 lr=4.22e-04 tps=15110 mem=6.4G lam=0.2262173273290197 Gm=0.38989774106691283 +[full s0] step 4820/12000 loss=4.6905 gn=0.72 lr=4.20e-04 tps=15111 mem=6.4G lam=0.2266270158191522 Gm=0.38377432866642874 +[full s0] step 4840/12000 loss=4.4645 gn=0.74 lr=4.19e-04 tps=15112 mem=6.4G lam=0.2270338467011849 Gm=0.3813349664366494 +[full s0] step 4860/12000 loss=4.7202 gn=0.73 lr=4.18e-04 tps=15113 mem=6.4G lam=0.22748661134392023 Gm=0.3844110432546586 +[full s0] step 4880/12000 loss=4.7185 gn=0.75 lr=4.16e-04 tps=15113 mem=6.4G lam=0.22796694530795017 Gm=0.3827349589361499 +[full s0] step 4900/12000 loss=4.5317 gn=0.68 lr=4.15e-04 tps=15114 mem=6.4G lam=0.22844892212500176 Gm=0.3793092481791973 +[full s0] step 4920/12000 loss=4.7661 gn=0.80 lr=4.13e-04 tps=15115 mem=6.4G lam=0.2289095583061377 Gm=0.3826396669416378 +[full s0] step 4940/12000 loss=4.6248 gn=0.72 lr=4.12e-04 tps=15116 mem=6.4G lam=0.2292969487607479 Gm=0.37789292509357136 +[full s0] step 4960/12000 loss=4.7043 gn=0.73 lr=4.11e-04 tps=15117 mem=6.4G lam=0.22976280159006515 Gm=0.3848522547632456 +[full s0] step 4980/12000 loss=4.5650 gn=0.71 lr=4.09e-04 tps=15117 mem=6.4G lam=0.23024764439711967 Gm=0.37631673847014707 +[full s0] step 5000/12000 loss=4.7549 gn=0.87 lr=4.08e-04 tps=15118 mem=6.4G lam=0.2307071409498652 Gm=0.3863384883540372 +[full s0] EVAL step 5000: val_loss=4.7663 val_ppl=117.49 +[full s0] step 5020/12000 loss=4.6419 gn=0.81 lr=4.07e-04 tps=15096 mem=6.4G lam=0.23115596547722816 Gm=0.3886050156628092 +[full s0] step 5040/12000 loss=4.6542 gn=0.79 lr=4.05e-04 tps=15097 mem=6.4G lam=0.2317021976535519 Gm=0.3856355821869026 +[full s0] step 5060/12000 loss=4.5516 gn=0.82 lr=4.04e-04 tps=15094 mem=6.4G lam=0.2321969916423162 Gm=0.3778656650489817 +[full s0] step 5080/12000 loss=4.6659 gn=0.79 lr=4.02e-04 tps=15089 mem=6.4G lam=0.23262640802810589 Gm=0.3746630970854312 +[full s0] step 5100/12000 loss=4.6790 gn=0.73 lr=4.01e-04 tps=15084 mem=6.4G lam=0.2330037116383513 Gm=0.3910587728799631 +[full s0] step 5120/12000 loss=4.5758 gn=0.78 lr=4.00e-04 tps=15079 mem=6.4G lam=0.23348270170390606 Gm=0.37818511319346726 +[full s0] step 5140/12000 loss=4.7053 gn=0.82 lr=3.98e-04 tps=15073 mem=6.4G lam=0.23393933412929377 Gm=0.3811164586804807 +[full s0] step 5160/12000 loss=4.5404 gn=0.76 lr=3.97e-04 tps=15069 mem=6.4G lam=0.23435907593617836 Gm=0.3838896327652037 +[full s0] step 5180/12000 loss=4.5687 gn=0.74 lr=3.95e-04 tps=15068 mem=6.4G lam=0.23470297579964003 Gm=0.3831328412828346 +[full s0] step 5200/12000 loss=4.5032 gn=0.78 lr=3.94e-04 tps=15066 mem=6.4G lam=0.23531275211522976 Gm=0.38004589344685274 +[full s0] step 5220/12000 loss=4.6035 gn=0.74 lr=3.93e-04 tps=15067 mem=6.4G lam=0.23572298822303614 Gm=0.38306743279099464 +[full s0] step 5240/12000 loss=4.5931 gn=0.80 lr=3.91e-04 tps=15067 mem=6.4G lam=0.23611155586938062 Gm=0.388134543163081 +[full s0] step 5260/12000 loss=4.5437 gn=0.72 lr=3.90e-04 tps=15068 mem=6.4G lam=0.23657434557874998 Gm=0.3789380687909822 +[full s0] step 5280/12000 loss=4.5919 gn=0.79 lr=3.88e-04 tps=15069 mem=6.4G lam=0.23702862082670131 Gm=0.38504793277631205 +[full s0] step 5300/12000 loss=4.7179 gn=0.75 lr=3.87e-04 tps=15069 mem=6.4G lam=0.23744185672452053 Gm=0.38119179964996874 +[full s0] step 5320/12000 loss=4.5814 gn=0.72 lr=3.86e-04 tps=15069 mem=6.4G lam=0.2378607395415505 Gm=0.374936076036344 +[full s0] step 5340/12000 loss=4.6638 gn=0.75 lr=3.84e-04 tps=15070 mem=6.4G lam=0.23828900636484227 Gm=0.3817270491272211 +[full s0] step 5360/12000 loss=4.5140 gn=0.75 lr=3.83e-04 tps=15070 mem=6.4G lam=0.23877200484275818 Gm=0.38166411872953176 +[full s0] step 5380/12000 loss=4.7715 gn=0.73 lr=3.81e-04 tps=15070 mem=6.4G lam=0.23924288774530092 Gm=0.3784514629902939 +[full s0] step 5400/12000 loss=4.4737 gn=0.75 lr=3.80e-04 tps=15071 mem=6.4G lam=0.23971633551021418 Gm=0.3741914709874739 +[full s0] step 5420/12000 loss=4.6374 gn=0.72 lr=3.79e-04 tps=15072 mem=6.4G lam=0.24014966345081726 Gm=0.37872939463704824 +[full s0] step 5440/12000 loss=4.5129 gn=0.76 lr=3.77e-04 tps=15072 mem=6.4G lam=0.2405006717890501 Gm=0.3832904367397229 +[full s0] step 5460/12000 loss=4.6330 gn=0.77 lr=3.76e-04 tps=15073 mem=6.4G lam=0.24091452515373626 Gm=0.38085759912307066 +[full s0] step 5480/12000 loss=4.5419 gn=0.78 lr=3.74e-04 tps=15075 mem=6.4G lam=0.24141437280923128 Gm=0.38053329785664874 +[full s0] step 5500/12000 loss=4.6149 gn=0.79 lr=3.73e-04 tps=15076 mem=6.4G lam=0.2419064131875833 Gm=0.3722052489562581 +[full s0] EVAL step 5500: val_loss=4.6954 val_ppl=109.44 +[full s0] step 5520/12000 loss=4.4442 gn=0.77 lr=3.72e-04 tps=15055 mem=6.4G lam=0.24222877342253923 Gm=0.3731184909120202 +[full s0] step 5540/12000 loss=4.6613 gn=0.82 lr=3.70e-04 tps=15056 mem=6.4G lam=0.24264594167470932 Gm=0.3793259069013099 +[full s0] step 5560/12000 loss=4.6883 gn=0.77 lr=3.69e-04 tps=15055 mem=6.4G lam=0.24303429325421652 Gm=0.3756120529336234 +[full s0] step 5580/12000 loss=4.7245 gn=0.79 lr=3.67e-04 tps=15055 mem=6.4G lam=0.2433958643426498 Gm=0.3789449748583138 +[full s0] step 5600/12000 loss=4.5524 gn=0.73 lr=3.66e-04 tps=15055 mem=6.4G lam=0.24383553645263115 Gm=0.37716613709926605 +[full s0] step 5620/12000 loss=4.6287 gn=0.82 lr=3.64e-04 tps=15055 mem=6.4G lam=0.24424950095514456 Gm=0.3736687171428154 +[full s0] step 5640/12000 loss=4.7295 gn=0.78 lr=3.63e-04 tps=15055 mem=6.4G lam=0.2446556119248271 Gm=0.3797593166430791 +[full s0] step 5660/12000 loss=4.4944 gn=0.74 lr=3.62e-04 tps=15055 mem=6.4G lam=0.24506564593563476 Gm=0.3771767970950653 +[full s0] step 5680/12000 loss=4.5589 gn=0.80 lr=3.60e-04 tps=15055 mem=6.4G lam=0.24549279424051443 Gm=0.37535428046248853 +[full s0] step 5700/12000 loss=4.6911 gn=0.86 lr=3.59e-04 tps=15055 mem=6.4G lam=0.2459304928779602 Gm=0.3709365854350229 +[full s0] step 5720/12000 loss=4.5643 gn=0.75 lr=3.57e-04 tps=15055 mem=6.4G lam=0.2463596553231279 Gm=0.3763433409233888 +[full s0] step 5740/12000 loss=4.7305 gn=0.84 lr=3.56e-04 tps=15055 mem=6.4G lam=0.24675203238924345 Gm=0.3767194544586043 +[full s0] step 5760/12000 loss=4.5889 gn=0.73 lr=3.54e-04 tps=15055 mem=6.4G lam=0.24713907421876988 Gm=0.372845664465179 +[full s0] step 5780/12000 loss=4.6360 gn=0.78 lr=3.53e-04 tps=15055 mem=6.4G lam=0.2474533642331759 Gm=0.37353368538121384 +[full s0] step 5800/12000 loss=4.4844 gn=0.79 lr=3.52e-04 tps=15055 mem=6.4G lam=0.24780782560507456 Gm=0.37253940012305975 +[full s0] step 5820/12000 loss=4.7158 gn=0.75 lr=3.50e-04 tps=15055 mem=6.4G lam=0.24827480999131998 Gm=0.3712646110604207 +[full s0] step 5840/12000 loss=4.5314 gn=0.76 lr=3.49e-04 tps=15055 mem=6.4G lam=0.24856825297077498 Gm=0.37440690437021357 +[full s0] step 5860/12000 loss=4.6324 gn=0.77 lr=3.47e-04 tps=15055 mem=6.4G lam=0.24896474741399288 Gm=0.3694636778285106 +[full s0] step 5880/12000 loss=4.6528 gn=0.79 lr=3.46e-04 tps=15056 mem=6.4G lam=0.24932707908252874 Gm=0.3732403563335538 +[full s0] step 5900/12000 loss=4.5604 gn=0.77 lr=3.44e-04 tps=15055 mem=6.4G lam=0.2497617918998003 Gm=0.36842549964785576 +[full s0] step 5920/12000 loss=4.5715 gn=0.82 lr=3.43e-04 tps=15055 mem=6.4G lam=0.2501053201655547 Gm=0.3678687899373472 +[full s0] step 5940/12000 loss=4.4879 gn=0.81 lr=3.41e-04 tps=15055 mem=6.4G lam=0.2504966442162792 Gm=0.3693230397378405 +[full s0] step 5960/12000 loss=4.5498 gn=0.81 lr=3.40e-04 tps=15055 mem=6.4G lam=0.25092022276173037 Gm=0.3673605048873772 +[full s0] step 5980/12000 loss=4.7976 gn=0.75 lr=3.39e-04 tps=15055 mem=6.4G lam=0.2513288144643108 Gm=0.37645416404120624 +[full s0] step 6000/12000 loss=4.5449 gn=0.77 lr=3.37e-04 tps=15055 mem=6.4G lam=0.2517046018814047 Gm=0.37760728869276744 +[full s0] EVAL step 6000: val_loss=4.6436 val_ppl=103.91 +[full s0] step 6020/12000 loss=4.4859 gn=0.81 lr=3.36e-04 tps=15037 mem=6.4G lam=0.2521008277932803 Gm=0.3718633310248454 +[full s0] step 6040/12000 loss=4.5088 gn=0.81 lr=3.34e-04 tps=15036 mem=6.4G lam=0.25253972349067527 Gm=0.37357604519153637 +[full s0] step 6060/12000 loss=4.6637 gn=0.81 lr=3.33e-04 tps=15036 mem=6.4G lam=0.2529549704243739 Gm=0.37255659233778715 +[full s0] step 6080/12000 loss=4.5895 gn=0.79 lr=3.31e-04 tps=15037 mem=6.4G lam=0.2533588030685981 Gm=0.37531870991612476 +[full s0] step 6100/12000 loss=4.5302 gn=0.81 lr=3.30e-04 tps=15037 mem=6.4G lam=0.25378550589084625 Gm=0.3715431815944612 +[full s0] step 6120/12000 loss=4.6564 gn=0.81 lr=3.29e-04 tps=15038 mem=6.4G lam=0.25418926992764074 Gm=0.37067493811870617 +[full s0] step 6140/12000 loss=4.5800 gn=0.78 lr=3.27e-04 tps=15038 mem=6.4G lam=0.25454604625701904 Gm=0.37490262168770033 +[full s0] step 6160/12000 loss=4.4564 gn=0.75 lr=3.26e-04 tps=15039 mem=6.4G lam=0.25490698404610157 Gm=0.37562513157414895 +[full s0] step 6180/12000 loss=4.6259 gn=0.80 lr=3.24e-04 tps=15039 mem=6.4G lam=0.255305342686673 Gm=0.3725527743032823 +[full s0] step 6200/12000 loss=4.5427 gn=0.80 lr=3.23e-04 tps=15040 mem=6.4G lam=0.2555528149629633 Gm=0.3727540757972747 +[full s0] step 6220/12000 loss=4.5278 gn=0.79 lr=3.21e-04 tps=15040 mem=6.4G lam=0.25589611660689116 Gm=0.37571656331419945 +[full s0] step 6240/12000 loss=4.4845 gn=0.79 lr=3.20e-04 tps=15040 mem=6.4G lam=0.25627377939720947 Gm=0.36686310668786365 +[full s0] step 6260/12000 loss=4.6014 gn=0.85 lr=3.19e-04 tps=15041 mem=6.4G lam=0.25662251748144627 Gm=0.37630698752279085 +[full s0] step 6280/12000 loss=4.5704 gn=0.76 lr=3.17e-04 tps=15041 mem=6.4G lam=0.2569559992601474 Gm=0.3718844644414882 +[full s0] step 6300/12000 loss=4.5064 gn=0.82 lr=3.16e-04 tps=15042 mem=6.4G lam=0.2573394713302453 Gm=0.3704437300718079 +[full s0] step 6320/12000 loss=4.4578 gn=0.82 lr=3.14e-04 tps=15042 mem=6.4G lam=0.25775218692918617 Gm=0.3724597793382903 +[full s0] step 6340/12000 loss=4.5503 gn=0.78 lr=3.13e-04 tps=15043 mem=6.4G lam=0.2580646686255932 Gm=0.3718521043968697 +[full s0] step 6360/12000 loss=4.5711 gn=0.80 lr=3.11e-04 tps=15043 mem=6.4G lam=0.25841206808884937 Gm=0.3716808574584623 +[full s0] step 6380/12000 loss=4.5748 gn=0.79 lr=3.10e-04 tps=15044 mem=6.4G lam=0.25880969657252234 Gm=0.37009835592471063 +[full s0] step 6400/12000 loss=4.4227 gn=0.84 lr=3.08e-04 tps=15045 mem=6.4G lam=0.25912587872395915 Gm=0.37573495600372553 +[full s0] step 6420/12000 loss=4.6049 gn=0.81 lr=3.07e-04 tps=15045 mem=6.4G lam=0.25949678383767605 Gm=0.3704458430875093 +[full s0] step 6440/12000 loss=4.6541 gn=0.84 lr=3.06e-04 tps=15046 mem=6.4G lam=0.25990564345071715 Gm=0.36193965445272624 +[full s0] step 6460/12000 loss=4.4922 gn=0.88 lr=3.04e-04 tps=15046 mem=6.4G lam=0.2602034506077568 Gm=0.3694150006243338 +[full s0] step 6480/12000 loss=4.4997 gn=0.80 lr=3.03e-04 tps=15046 mem=6.4G lam=0.2605235294128458 Gm=0.3696099272929132 +[full s0] step 6500/12000 loss=4.5122 gn=0.84 lr=3.01e-04 tps=15047 mem=6.4G lam=0.2608110023041566 Gm=0.3736143534382184 +[full s0] EVAL step 6500: val_loss=4.6216 val_ppl=101.65 +[full s0] step 6520/12000 loss=4.6771 gn=0.84 lr=3.00e-04 tps=15030 mem=6.4G lam=0.2610552596549193 Gm=0.37577600435664255 +[full s0] step 6540/12000 loss=4.4731 gn=0.81 lr=2.98e-04 tps=15030 mem=6.4G lam=0.2614286780978243 Gm=0.3714661021561672 +[full s0] step 6560/12000 loss=4.6753 gn=0.80 lr=2.97e-04 tps=15031 mem=6.4G lam=0.2618023694182436 Gm=0.3748769968127211 +[full s0] step 6580/12000 loss=4.6208 gn=0.83 lr=2.96e-04 tps=15031 mem=6.4G lam=0.26221567175040644 Gm=0.3707975223660469 +[full s0] step 6600/12000 loss=4.3779 gn=0.84 lr=2.94e-04 tps=15032 mem=6.4G lam=0.26258288603276014 Gm=0.3646160075441003 +[full s0] step 6620/12000 loss=4.3626 gn=0.83 lr=2.93e-04 tps=15032 mem=6.4G lam=0.2629324250544111 Gm=0.3692897035119434 +[full s0] step 6640/12000 loss=4.6807 gn=0.84 lr=2.91e-04 tps=15033 mem=6.4G lam=0.263213450089097 Gm=0.3726840321129809 +[full s0] step 6660/12000 loss=4.6568 gn=0.77 lr=2.90e-04 tps=15033 mem=6.4G lam=0.26349103543907404 Gm=0.37219421852690476 +[full s0] step 6680/12000 loss=4.5052 gn=0.78 lr=2.88e-04 tps=15034 mem=6.4G lam=0.2638085301344593 Gm=0.36736786101634306 +[full s0] step 6700/12000 loss=4.4821 gn=0.82 lr=2.87e-04 tps=15034 mem=6.4G lam=0.2640962302684784 Gm=0.36576455164079863 +[full s0] step 6720/12000 loss=4.5648 gn=0.80 lr=2.86e-04 tps=15035 mem=6.4G lam=0.26443634430567425 Gm=0.3736119088716805 +[full s0] step 6740/12000 loss=4.4625 gn=0.83 lr=2.84e-04 tps=15035 mem=6.4G lam=0.2648094616209467 Gm=0.3644495272698502 +[full s0] step 6760/12000 loss=4.4897 gn=0.83 lr=2.83e-04 tps=15036 mem=6.4G lam=0.2651348930473129 Gm=0.3687275207291047 +[full s0] step 6780/12000 loss=4.4762 gn=0.83 lr=2.81e-04 tps=15037 mem=6.4G lam=0.26548257749527693 Gm=0.36658488120883703 +[full s0] step 6800/12000 loss=4.4248 gn=0.76 lr=2.80e-04 tps=15037 mem=6.4G lam=0.2657978093872468 Gm=0.3642652676595996 +[full s0] step 6820/12000 loss=4.4274 gn=0.79 lr=2.79e-04 tps=15037 mem=6.4G lam=0.2661310142527024 Gm=0.3702369128198673 +[full s0] step 6840/12000 loss=4.3520 gn=0.82 lr=2.77e-04 tps=15038 mem=6.4G lam=0.266487338890632 Gm=0.3702721279114485 +[full s0] step 6860/12000 loss=4.5601 gn=0.86 lr=2.76e-04 tps=15038 mem=6.4G lam=0.2667763453597824 Gm=0.36677354760468006 +[full s0] step 6880/12000 loss=4.4995 gn=0.81 lr=2.74e-04 tps=15038 mem=6.4G lam=0.26709318937112886 Gm=0.3704655942662309 +[full s0] step 6900/12000 loss=4.6483 gn=0.85 lr=2.73e-04 tps=15039 mem=6.4G lam=0.26744314935058355 Gm=0.37264287021632 +[full s0] step 6920/12000 loss=4.5642 gn=0.82 lr=2.72e-04 tps=15039 mem=6.4G lam=0.2678127496813734 Gm=0.3645773116343965 +[full s0] step 6940/12000 loss=4.3994 gn=0.82 lr=2.70e-04 tps=15040 mem=6.4G lam=0.268150607123971 Gm=0.36496744444593787 +[full s0] step 6960/12000 loss=4.5304 gn=0.82 lr=2.69e-04 tps=15040 mem=6.4G lam=0.2684732728327314 Gm=0.3607904207116614 +[full s0] step 6980/12000 loss=4.3530 gn=0.78 lr=2.67e-04 tps=15041 mem=6.4G lam=0.2687076237052679 Gm=0.3735479221989711 +[full s0] step 7000/12000 loss=4.6609 gn=0.88 lr=2.66e-04 tps=15042 mem=6.4G lam=0.26895450334995985 Gm=0.3690581746244182 +[full s0] EVAL step 7000: val_loss=4.5702 val_ppl=96.56 +[full s0] step 7020/12000 loss=4.4458 gn=0.84 lr=2.65e-04 tps=15026 mem=6.4G lam=0.2691964255645871 Gm=0.36305108907011646 +[full s0] step 7040/12000 loss=4.4551 gn=0.84 lr=2.63e-04 tps=15027 mem=6.4G lam=0.2695753965526819 Gm=0.36654868923748535 +[full s0] step 7060/12000 loss=4.5455 gn=0.84 lr=2.62e-04 tps=15027 mem=6.4G lam=0.26991186446199816 Gm=0.36299153285411495 +[full s0] step 7080/12000 loss=4.5585 gn=0.83 lr=2.60e-04 tps=15028 mem=6.4G lam=0.27025072649121284 Gm=0.36864552561504144 +[full s0] step 7100/12000 loss=4.5170 gn=0.82 lr=2.59e-04 tps=15029 mem=6.4G lam=0.2705916417762637 Gm=0.35914174071513116 +[full s0] step 7120/12000 loss=4.7402 gn=0.87 lr=2.58e-04 tps=15029 mem=6.4G lam=0.27088877093046904 Gm=0.3725667907856405 +[full s0] step 7140/12000 loss=4.5136 gn=0.85 lr=2.56e-04 tps=15030 mem=6.4G lam=0.27114191092550755 Gm=0.3694961511840423 +[full s0] step 7160/12000 loss=4.4335 gn=0.84 lr=2.55e-04 tps=15031 mem=6.4G lam=0.27142058095584315 Gm=0.36239044996909797 +[full s0] step 7180/12000 loss=4.5992 gn=0.85 lr=2.53e-04 tps=15031 mem=6.4G lam=0.2716836885859569 Gm=0.36760792788118124 +[full s0] step 7200/12000 loss=4.6143 gn=0.86 lr=2.52e-04 tps=15032 mem=6.4G lam=0.27200183769067127 Gm=0.37039985810406506 +[full s0] step 7220/12000 loss=4.5827 gn=0.84 lr=2.51e-04 tps=15033 mem=6.4G lam=0.27228064090013504 Gm=0.3716807649470866 +[full s0] step 7240/12000 loss=4.3823 gn=0.88 lr=2.49e-04 tps=15033 mem=6.4G lam=0.2725673370684187 Gm=0.3643574430607259 +[full s0] step 7260/12000 loss=4.4308 gn=0.85 lr=2.48e-04 tps=15033 mem=6.4G lam=0.27285871220131713 Gm=0.36993917970297235 +[full s0] step 7280/12000 loss=4.4968 gn=0.83 lr=2.47e-04 tps=15034 mem=6.4G lam=0.2731536269808809 Gm=0.3596188734130313 +[full s0] step 7300/12000 loss=4.3658 gn=0.81 lr=2.45e-04 tps=15034 mem=6.4G lam=0.2734677130356431 Gm=0.3683599379534523 +[full s0] step 7320/12000 loss=4.5923 gn=0.81 lr=2.44e-04 tps=15035 mem=6.4G lam=0.27378449340661365 Gm=0.3656483479620268 +[full s0] step 7340/12000 loss=4.4267 gn=0.85 lr=2.42e-04 tps=15035 mem=6.4G lam=0.2741472941512863 Gm=0.367054678266868 +[full s0] step 7360/12000 loss=4.3788 gn=0.80 lr=2.41e-04 tps=15036 mem=6.4G lam=0.2744293864816427 Gm=0.3591368881364663 +[full s0] step 7380/12000 loss=4.3564 gn=0.90 lr=2.40e-04 tps=15036 mem=6.4G lam=0.2746526872118314 Gm=0.3658777526579797 +[full s0] step 7400/12000 loss=4.4742 gn=0.81 lr=2.38e-04 tps=15037 mem=6.4G lam=0.2749244002625346 Gm=0.3663593408030768 +[full s0] step 7420/12000 loss=4.5719 gn=0.83 lr=2.37e-04 tps=15037 mem=6.4G lam=0.2752235348646839 Gm=0.3729113922454417 +[full s0] step 7440/12000 loss=4.6161 gn=0.88 lr=2.36e-04 tps=15038 mem=6.4G lam=0.2755170976743102 Gm=0.3705325702515741 +[full s0] step 7460/12000 loss=4.4212 gn=0.91 lr=2.34e-04 tps=15039 mem=6.4G lam=0.27575831456730765 Gm=0.3638699528140326 +[full s0] step 7480/12000 loss=4.4558 gn=0.85 lr=2.33e-04 tps=15039 mem=6.4G lam=0.27598631009459496 Gm=0.36698949166263145 +[full s0] step 7500/12000 loss=4.5123 gn=0.83 lr=2.32e-04 tps=15040 mem=6.4G lam=0.27628016161421937 Gm=0.35959672601893544 +[full s0] EVAL step 7500: val_loss=4.5871 val_ppl=98.21 +[full s0] step 7520/12000 loss=4.4979 gn=0.87 lr=2.30e-04 tps=15025 mem=6.4G lam=0.27653876536836225 Gm=0.3611405238043517 +[full s0] step 7540/12000 loss=4.5826 gn=0.83 lr=2.29e-04 tps=15025 mem=6.4G lam=0.2767543140798807 Gm=0.3669085471580426 +[full s0] step 7560/12000 loss=4.2771 gn=0.82 lr=2.28e-04 tps=15026 mem=6.4G lam=0.27702591630319756 Gm=0.3615619655077656 +[full s0] step 7580/12000 loss=4.5010 gn=0.87 lr=2.26e-04 tps=15027 mem=6.4G lam=0.2772821479787429 Gm=0.3618208991829306 +[full s0] step 7600/12000 loss=4.3555 gn=0.87 lr=2.25e-04 tps=15027 mem=6.4G lam=0.2775251657391588 Gm=0.36202836005638045 +[full s0] step 7620/12000 loss=4.5437 gn=0.88 lr=2.24e-04 tps=15028 mem=6.4G lam=0.2777461313332121 Gm=0.3631243551305185 +[full s0] step 7640/12000 loss=4.3583 gn=0.84 lr=2.22e-04 tps=15029 mem=6.4G lam=0.2780178893978397 Gm=0.36482729964579147 +[full s0] step 7660/12000 loss=4.5074 gn=0.86 lr=2.21e-04 tps=15029 mem=6.4G lam=0.27825140797843534 Gm=0.36049557306493324 +[full s0] step 7680/12000 loss=4.4795 gn=0.87 lr=2.20e-04 tps=15030 mem=6.4G lam=0.27846843966593343 Gm=0.36374363225574297 +[full s0] step 7700/12000 loss=4.5335 gn=0.82 lr=2.18e-04 tps=15030 mem=6.4G lam=0.27875682059675455 Gm=0.3610304932420452 +[full s0] step 7720/12000 loss=4.4901 gn=0.90 lr=2.17e-04 tps=15031 mem=6.4G lam=0.2790155593926708 Gm=0.36043354294573265 +[full s0] step 7740/12000 loss=4.3281 gn=0.86 lr=2.16e-04 tps=15032 mem=6.4G lam=0.2792706002170841 Gm=0.3593555052454273 +[full s0] step 7760/12000 loss=4.4626 gn=0.85 lr=2.15e-04 tps=15032 mem=6.4G lam=0.27950433890024823 Gm=0.36844494519755244 +[full s0] step 7780/12000 loss=4.4021 gn=0.86 lr=2.13e-04 tps=15033 mem=6.4G lam=0.279755847218136 Gm=0.3665320348615448 +[full s0] step 7800/12000 loss=4.4933 gn=0.86 lr=2.12e-04 tps=15033 mem=6.4G lam=0.28003670182079077 Gm=0.3566385632536064 +[full s0] step 7820/12000 loss=4.5508 gn=0.93 lr=2.11e-04 tps=15034 mem=6.4G lam=0.2803347948938608 Gm=0.368739877982686 +[full s0] step 7840/12000 loss=4.3020 gn=0.83 lr=2.09e-04 tps=15034 mem=6.4G lam=0.28050737734884024 Gm=0.36179986541780335 +[full s0] step 7860/12000 loss=4.4785 gn=0.89 lr=2.08e-04 tps=15035 mem=6.4G lam=0.28076241600016755 Gm=0.36224934404405457 +[full s0] step 7880/12000 loss=4.4995 gn=0.89 lr=2.07e-04 tps=15036 mem=6.4G lam=0.2809906577070554 Gm=0.3594335032782207 +[full s0] step 7900/12000 loss=4.2875 gn=0.86 lr=2.06e-04 tps=15037 mem=6.4G lam=0.28123165263483924 Gm=0.36183837909872335 +[full s0] step 7920/12000 loss=4.3579 gn=0.83 lr=2.04e-04 tps=15037 mem=6.4G lam=0.281511386235555 Gm=0.3657500286741803 +[full s0] step 7940/12000 loss=4.4568 gn=0.84 lr=2.03e-04 tps=15038 mem=6.4G lam=0.2817483637481928 Gm=0.364165772140647 +[full s0] step 7960/12000 loss=4.5151 gn=0.92 lr=2.02e-04 tps=15038 mem=6.4G lam=0.28194482903927565 Gm=0.36385091316575807 +[full s0] step 7980/12000 loss=4.4236 gn=0.87 lr=2.00e-04 tps=15039 mem=6.4G lam=0.2822293716793259 Gm=0.3599027187253038 +[full s0] step 8000/12000 loss=4.3778 gn=0.91 lr=1.99e-04 tps=15040 mem=6.4G lam=0.2825062225262324 Gm=0.36515758908353746 +[full s0] EVAL step 8000: val_loss=4.5158 val_ppl=91.45 +[full s0] step 8020/12000 loss=4.4228 gn=0.82 lr=1.98e-04 tps=15026 mem=6.4G lam=0.28274272723744315 Gm=0.3592962365752707 +[full s0] step 8040/12000 loss=4.5740 gn=0.86 lr=1.97e-04 tps=15026 mem=6.4G lam=0.2829401275763909 Gm=0.3604607152907799 +[full s0] step 8060/12000 loss=4.4966 gn=0.85 lr=1.95e-04 tps=15027 mem=6.4G lam=0.28316085909803707 Gm=0.3568619737246384 +[full s0] step 8080/12000 loss=4.3811 gn=0.87 lr=1.94e-04 tps=15028 mem=6.4G lam=0.28331817282984656 Gm=0.3570008672152956 +[full s0] step 8100/12000 loss=4.5145 gn=0.88 lr=1.93e-04 tps=15028 mem=6.4G lam=0.2835194831714034 Gm=0.35911922878585756 +[full s0] step 8120/12000 loss=4.3450 gn=0.82 lr=1.92e-04 tps=15029 mem=6.4G lam=0.28374885333081085 Gm=0.35938746194976073 +[full s0] step 8140/12000 loss=4.5035 gn=0.94 lr=1.90e-04 tps=15030 mem=6.4G lam=0.283959973603487 Gm=0.3583690511683623 +[full s0] step 8160/12000 loss=4.4741 gn=0.89 lr=1.89e-04 tps=15030 mem=6.4G lam=0.2841803689176838 Gm=0.3603085884824395 +[full s0] step 8180/12000 loss=4.5566 gn=0.88 lr=1.88e-04 tps=15031 mem=6.4G lam=0.28441902343183756 Gm=0.3599741633515805 +[full s0] step 8200/12000 loss=4.2181 gn=0.86 lr=1.87e-04 tps=15032 mem=6.4G lam=0.28463245648890734 Gm=0.3628523667963843 +[full s0] step 8220/12000 loss=4.3862 gn=0.86 lr=1.86e-04 tps=15032 mem=6.4G lam=0.28481648800273734 Gm=0.3613696300114195 +[full s0] step 8240/12000 loss=4.2606 gn=0.85 lr=1.84e-04 tps=15033 mem=6.4G lam=0.28503039944916964 Gm=0.3557949614090224 +[full s0] step 8260/12000 loss=4.4015 gn=0.85 lr=1.83e-04 tps=15033 mem=6.4G lam=0.28520850899318856 Gm=0.3636594043734173 +[full s0] step 8280/12000 loss=4.3351 gn=0.88 lr=1.82e-04 tps=15034 mem=6.4G lam=0.2853433412189285 Gm=0.35658857855014503 +[full s0] step 8300/12000 loss=4.3665 gn=0.85 lr=1.81e-04 tps=15034 mem=6.4G lam=0.28553036755571765 Gm=0.36088929045945406 +[full s0] step 8320/12000 loss=4.4105 gn=0.91 lr=1.80e-04 tps=15035 mem=6.4G lam=0.2856839296097557 Gm=0.35626769554801285 +[full s0] step 8340/12000 loss=4.4386 gn=0.86 lr=1.78e-04 tps=15035 mem=6.4G lam=0.2859235570455591 Gm=0.35951902306017774 +[full s0] step 8360/12000 loss=4.4759 gn=0.90 lr=1.77e-04 tps=15036 mem=6.4G lam=0.2861397524053852 Gm=0.3596027287033697 +[full s0] step 8380/12000 loss=4.4281 gn=0.83 lr=1.76e-04 tps=15036 mem=6.4G lam=0.2863622574756543 Gm=0.35720649687573314 +[full s0] step 8400/12000 loss=4.3725 gn=0.85 lr=1.75e-04 tps=15037 mem=6.4G lam=0.2865758442009489 Gm=0.3613565871492028 +[full s0] step 8420/12000 loss=4.5130 gn=0.88 lr=1.74e-04 tps=15037 mem=6.4G lam=0.2867719891170661 Gm=0.36507285906312364 +[full s0] step 8440/12000 loss=4.1814 gn=0.87 lr=1.72e-04 tps=15038 mem=6.4G lam=0.2869762113938729 Gm=0.3540540363173932 +[full s0] step 8460/12000 loss=4.3558 gn=0.92 lr=1.71e-04 tps=15038 mem=6.4G lam=0.28715901263058186 Gm=0.356534885475412 +[full s0] step 8480/12000 loss=4.3043 gn=0.85 lr=1.70e-04 tps=15039 mem=6.4G lam=0.2873095801721017 Gm=0.36014279816299677 +[full s0] step 8500/12000 loss=4.5970 gn=0.93 lr=1.69e-04 tps=15040 mem=6.4G lam=0.28749642614275217 Gm=0.361760484908397 +[full s0] EVAL step 8500: val_loss=4.5069 val_ppl=90.64 +[full s0] step 8520/12000 loss=4.4134 gn=0.89 lr=1.68e-04 tps=15027 mem=6.4G lam=0.28769484131286543 Gm=0.36184446707678336 +[full s0] step 8540/12000 loss=4.4638 gn=0.96 lr=1.67e-04 tps=15027 mem=6.4G lam=0.2878760118037462 Gm=0.3581450345615546 +[full s0] step 8560/12000 loss=4.5068 gn=0.88 lr=1.66e-04 tps=15027 mem=6.4G lam=0.2880452141786615 Gm=0.3618041845038533 +[full s0] step 8580/12000 loss=4.3614 gn=0.85 lr=1.64e-04 tps=15028 mem=6.4G lam=0.2882512103145321 Gm=0.3550414452329278 +[full s0] step 8600/12000 loss=4.4874 gn=1.06 lr=1.63e-04 tps=15029 mem=6.4G lam=0.288453777320683 Gm=0.36168521526269615 +[full s0] step 8620/12000 loss=4.4381 gn=0.89 lr=1.62e-04 tps=15030 mem=6.4G lam=0.2886202170824011 Gm=0.3659763087828954 +[full s0] step 8640/12000 loss=4.4817 gn=0.92 lr=1.61e-04 tps=15030 mem=6.4G lam=0.2888142690062523 Gm=0.3601409282224874 +[full s0] step 8660/12000 loss=4.3261 gn=0.93 lr=1.60e-04 tps=15030 mem=6.4G lam=0.2889861998458703 Gm=0.3637861223736157 +[full s0] step 8680/12000 loss=4.3965 gn=0.84 lr=1.59e-04 tps=15031 mem=6.4G lam=0.2891568535317977 Gm=0.3605387373827398 +[full s0] step 8700/12000 loss=4.3670 gn=0.84 lr=1.58e-04 tps=15031 mem=6.4G lam=0.2893655312558015 Gm=0.35853359134246904 +[full s0] step 8720/12000 loss=4.3537 gn=0.90 lr=1.57e-04 tps=15031 mem=6.4G lam=0.28954537988950807 Gm=0.3610219975622992 +[full s0] step 8740/12000 loss=4.4243 gn=0.90 lr=1.55e-04 tps=15032 mem=6.4G lam=0.2897492793078224 Gm=0.3538465742021799 +[full s0] step 8760/12000 loss=4.3076 gn=0.96 lr=1.54e-04 tps=15032 mem=6.4G lam=0.28994202831139165 Gm=0.3604841170211633 +[full s0] step 8780/12000 loss=4.4112 gn=0.86 lr=1.53e-04 tps=15032 mem=6.4G lam=0.29014147880176705 Gm=0.3596707660083969 +[full s0] step 8800/12000 loss=4.4354 gn=0.94 lr=1.52e-04 tps=15033 mem=6.4G lam=0.29028414624432725 Gm=0.36567126167938113 +[full s0] step 8820/12000 loss=4.4267 gn=0.95 lr=1.51e-04 tps=15033 mem=6.4G lam=0.29045009923477966 Gm=0.36128956149332225 +[full s0] step 8840/12000 loss=4.4248 gn=0.90 lr=1.50e-04 tps=15034 mem=6.4G lam=0.2906138049438596 Gm=0.35865468842287856 +[full s0] step 8860/12000 loss=4.4539 gn=0.86 lr=1.49e-04 tps=15034 mem=6.4G lam=0.29082162771373987 Gm=0.3615119362560411 +[full s0] step 8880/12000 loss=4.2567 gn=0.94 lr=1.48e-04 tps=15034 mem=6.4G lam=0.2910139604161183 Gm=0.3585327711577217 +[full s0] step 8900/12000 loss=4.3863 gn=0.92 lr=1.47e-04 tps=15034 mem=6.4G lam=0.29117806411037844 Gm=0.3592370899083714 +[full s0] step 8920/12000 loss=4.4433 gn=0.96 lr=1.46e-04 tps=15035 mem=6.4G lam=0.2913525427381198 Gm=0.362059460952878 +[full s0] step 8940/12000 loss=4.2639 gn=0.92 lr=1.45e-04 tps=15035 mem=6.4G lam=0.29155348520725965 Gm=0.3615978870075196 +[full s0] step 8960/12000 loss=4.5148 gn=0.95 lr=1.44e-04 tps=15035 mem=6.4G lam=0.2917197570204735 Gm=0.359662250848487 +[full s0] step 8980/12000 loss=4.3733 gn=0.90 lr=1.43e-04 tps=15036 mem=6.4G lam=0.2919054962694645 Gm=0.36268803601463634 +[full s0] step 9000/12000 loss=4.3176 gn=0.89 lr=1.42e-04 tps=15036 mem=6.4G lam=0.29205132741481066 Gm=0.35789333873738843 +[full s0] EVAL step 9000: val_loss=4.4709 val_ppl=87.44 +[full s0] step 9020/12000 loss=4.3404 gn=0.88 lr=1.41e-04 tps=15024 mem=6.4G lam=0.2922489109138648 Gm=0.3656360963359475 +[full s0] step 9040/12000 loss=4.3045 gn=0.88 lr=1.40e-04 tps=15025 mem=6.4G lam=0.292378350161016 Gm=0.35682266919563216 +[full s0] step 9060/12000 loss=4.3297 gn=0.88 lr=1.39e-04 tps=15025 mem=6.4G lam=0.29250552225857973 Gm=0.35918089048936963 +[full s0] step 9080/12000 loss=4.3088 gn=0.88 lr=1.38e-04 tps=15026 mem=6.4G lam=0.2926728830983241 Gm=0.360056731539468 +[full s0] step 9100/12000 loss=4.2640 gn=0.87 lr=1.37e-04 tps=15026 mem=6.4G lam=0.2928345936040084 Gm=0.36117946713541943 +[full s0] step 9120/12000 loss=4.3515 gn=0.91 lr=1.36e-04 tps=15027 mem=6.4G lam=0.29298064950853586 Gm=0.36127024757054943 +[full s0] step 9140/12000 loss=4.4725 gn=0.89 lr=1.35e-04 tps=15027 mem=6.4G lam=0.29312839824706316 Gm=0.3602388611373802 +[full s0] step 9160/12000 loss=4.3088 gn=0.96 lr=1.34e-04 tps=15027 mem=6.4G lam=0.29322462684164446 Gm=0.3603007663041353 +[full s0] step 9180/12000 loss=4.4702 gn=0.94 lr=1.33e-04 tps=15028 mem=6.4G lam=0.29340921094020206 Gm=0.3631959219152729 +[full s0] step 9200/12000 loss=4.4143 gn=0.98 lr=1.32e-04 tps=15028 mem=6.4G lam=0.29357157461345196 Gm=0.3681852699567874 +[full s0] step 9220/12000 loss=4.4608 gn=0.91 lr=1.31e-04 tps=15029 mem=6.4G lam=0.2937112453704079 Gm=0.3586834479744236 +[full s0] step 9240/12000 loss=4.4569 gn=0.91 lr=1.30e-04 tps=15029 mem=6.4G lam=0.29386542613307637 Gm=0.3578078344774743 +[full s0] step 9260/12000 loss=4.3927 gn=0.91 lr=1.29e-04 tps=15029 mem=6.4G lam=0.2940109533568223 Gm=0.3576989093174537 +[full s0] step 9280/12000 loss=4.3364 gn=0.86 lr=1.28e-04 tps=15030 mem=6.4G lam=0.2941441560784976 Gm=0.35499814540768665 +[full s0] step 9300/12000 loss=4.4400 gn=0.90 lr=1.27e-04 tps=15030 mem=6.4G lam=0.2942757671698928 Gm=0.35635567060671747 +[full s0] step 9320/12000 loss=4.3231 gn=0.88 lr=1.26e-04 tps=15030 mem=6.4G lam=0.29442432802170515 Gm=0.35952125846718747 +[full s0] step 9340/12000 loss=4.4702 gn=0.90 lr=1.25e-04 tps=15031 mem=6.4G lam=0.29459185991436243 Gm=0.35549192999800044 +[full s0] step 9360/12000 loss=4.5357 gn=1.01 lr=1.24e-04 tps=15031 mem=6.4G lam=0.29475019592791796 Gm=0.36132036180545885 +[full s0] step 9380/12000 loss=4.4238 gn=0.94 lr=1.23e-04 tps=15032 mem=6.4G lam=0.2949247695505619 Gm=0.35978238695922 +[full s0] step 9400/12000 loss=4.2905 gn=0.89 lr=1.22e-04 tps=15032 mem=6.4G lam=0.2950837602838874 Gm=0.3607391412369907 +[full s0] step 9420/12000 loss=4.5255 gn=0.98 lr=1.21e-04 tps=15033 mem=6.4G lam=0.2952130176126957 Gm=0.36994058825075626 +[full s0] step 9440/12000 loss=4.4170 gn=0.89 lr=1.20e-04 tps=15033 mem=6.4G lam=0.2953497739508748 Gm=0.35707308794371784 +[full s0] step 9460/12000 loss=4.3392 gn=0.90 lr=1.19e-04 tps=15034 mem=6.4G lam=0.2954567385216554 Gm=0.3551462067601581 +[full s0] step 9480/12000 loss=4.4075 gn=0.88 lr=1.19e-04 tps=15034 mem=6.4G lam=0.2956008951490124 Gm=0.35745541223635274 +[full s0] step 9500/12000 loss=4.3464 gn=0.93 lr=1.18e-04 tps=15035 mem=6.4G lam=0.29575240239501 Gm=0.3559929217832784 +[full s0] EVAL step 9500: val_loss=4.4395 val_ppl=84.73 +[full s0] step 9520/12000 loss=4.2105 gn=0.94 lr=1.17e-04 tps=15023 mem=6.4G lam=0.2959083517392476 Gm=0.35803080404487747 +[full s0] step 9540/12000 loss=4.5182 gn=1.00 lr=1.16e-04 tps=15024 mem=6.4G lam=0.2960332526514928 Gm=0.3582208335089187 +[full s0] step 9560/12000 loss=4.4700 gn=0.90 lr=1.15e-04 tps=15024 mem=6.4G lam=0.29619730326036614 Gm=0.3564637800057729 +[full s0] step 9580/12000 loss=4.2814 gn=0.96 lr=1.14e-04 tps=15024 mem=6.4G lam=0.2963089033340414 Gm=0.3531519764413436 +[full s0] step 9600/12000 loss=4.3505 gn=0.93 lr=1.13e-04 tps=15024 mem=6.4G lam=0.29642096534371376 Gm=0.3597104190848768 +[full s0] step 9620/12000 loss=4.4456 gn=0.98 lr=1.12e-04 tps=15025 mem=6.4G lam=0.2965585341056188 Gm=0.35844819985019666 +[full s0] step 9640/12000 loss=4.2992 gn=0.99 lr=1.12e-04 tps=15025 mem=6.4G lam=0.29669858049601316 Gm=0.3540510962096353 +[full s0] step 9660/12000 loss=4.4338 gn=0.87 lr=1.11e-04 tps=15026 mem=6.4G lam=0.2968192957341671 Gm=0.3594433416922887 +[full s0] step 9680/12000 loss=4.5979 gn=0.92 lr=1.10e-04 tps=15026 mem=6.4G lam=0.2969347055380543 Gm=0.3582506523622821 +[full s0] step 9700/12000 loss=4.2676 gn=0.94 lr=1.09e-04 tps=15027 mem=6.4G lam=0.29709628007064265 Gm=0.34946609071145457 +[full s0] step 9720/12000 loss=4.2603 gn=0.91 lr=1.08e-04 tps=15027 mem=6.4G lam=0.29720732538650435 Gm=0.35389701354627806 +[full s0] step 9740/12000 loss=4.4314 gn=1.20 lr=1.07e-04 tps=15027 mem=6.4G lam=0.29730000377943117 Gm=0.3581996407204618 +[full s0] step 9760/12000 loss=4.2802 gn=0.91 lr=1.07e-04 tps=15028 mem=6.4G lam=0.29742907267063856 Gm=0.356913054206719 +[full s0] step 9780/12000 loss=4.3648 gn=0.92 lr=1.06e-04 tps=15028 mem=6.4G lam=0.29753911557296914 Gm=0.35329023050144315 +[full s0] step 9800/12000 loss=4.3229 gn=0.94 lr=1.05e-04 tps=15029 mem=6.4G lam=0.29766348532090586 Gm=0.359042527154088 +[full s0] step 9820/12000 loss=4.4660 gn=0.94 lr=1.04e-04 tps=15029 mem=6.4G lam=0.29777155971775454 Gm=0.35748573547850054 +[full s0] step 9840/12000 loss=4.1220 gn=0.96 lr=1.03e-04 tps=15030 mem=6.4G lam=0.29788480264445144 Gm=0.3607317948093017 +[full s0] step 9860/12000 loss=4.4565 gn=0.98 lr=1.03e-04 tps=15030 mem=6.4G lam=0.29804216232150793 Gm=0.3568486788620551 +[full s0] step 9880/12000 loss=4.4989 gn=1.05 lr=1.02e-04 tps=15031 mem=6.4G lam=0.29817223610977334 Gm=0.36374200973659754 +[full s0] step 9900/12000 loss=4.3117 gn=0.95 lr=1.01e-04 tps=15031 mem=6.4G lam=0.29827761308600503 Gm=0.35590190161019564 +[full s0] step 9920/12000 loss=4.4288 gn=0.91 lr=1.00e-04 tps=15031 mem=6.4G lam=0.29838708105186623 Gm=0.35463118460029364 +[full s0] step 9940/12000 loss=4.5714 gn=1.01 lr=9.96e-05 tps=15032 mem=6.4G lam=0.29851334790388745 Gm=0.35825271997600794 +[full s0] step 9960/12000 loss=4.4721 gn=0.90 lr=9.89e-05 tps=15032 mem=6.4G lam=0.2986322057743867 Gm=0.3540138112536321 +[full s0] step 9980/12000 loss=4.3464 gn=0.87 lr=9.81e-05 tps=15033 mem=6.4G lam=0.2987455374871691 Gm=0.35476406291127205 +[full s0] step 10000/12000 loss=4.4690 gn=0.97 lr=9.74e-05 tps=15033 mem=6.4G lam=0.2988184057176113 Gm=0.3558954269780467 +[full s0] EVAL step 10000: val_loss=4.4306 val_ppl=83.98 +[full s0] step 10020/12000 loss=4.2446 gn=0.90 lr=9.67e-05 tps=15023 mem=6.4G lam=0.29891528841108084 Gm=0.3568601736333221 +[full s0] step 10040/12000 loss=4.2397 gn=0.89 lr=9.59e-05 tps=15023 mem=6.4G lam=0.2990020752574007 Gm=0.35231720842421055 +[full s0] step 10060/12000 loss=4.2559 gn=0.91 lr=9.52e-05 tps=15024 mem=6.4G lam=0.2991238233322899 Gm=0.3519400929411252 +[full s0] step 10080/12000 loss=4.5952 gn=0.97 lr=9.45e-05 tps=15024 mem=6.4G lam=0.2992284058903654 Gm=0.35400338113928836 +[full s0] step 10100/12000 loss=4.5736 gn=0.96 lr=9.38e-05 tps=15024 mem=6.4G lam=0.2993472069501877 Gm=0.3586220351668696 +[full s0] step 10120/12000 loss=4.3268 gn=0.96 lr=9.31e-05 tps=15025 mem=6.4G lam=0.29943193929890793 Gm=0.35663331719115376 +[full s0] step 10140/12000 loss=4.3493 gn=1.01 lr=9.24e-05 tps=15025 mem=6.4G lam=0.2995311391229431 Gm=0.3545027143942813 +[full s0] step 10160/12000 loss=4.2979 gn=0.94 lr=9.18e-05 tps=15026 mem=6.4G lam=0.2996478605394562 Gm=0.35680585998731357 +[full s0] step 10180/12000 loss=4.2925 gn=0.94 lr=9.11e-05 tps=15026 mem=6.4G lam=0.29974141499648493 Gm=0.35732950766881305 +[full s0] step 10200/12000 loss=4.3327 gn=0.93 lr=9.04e-05 tps=15026 mem=6.4G lam=0.29983059347917634 Gm=0.36047388481286663 +[full s0] step 10220/12000 loss=4.3515 gn=1.01 lr=8.98e-05 tps=15027 mem=6.4G lam=0.2999269347637892 Gm=0.3570485101081431 +[full s0] step 10240/12000 loss=4.3338 gn=0.95 lr=8.91e-05 tps=15027 mem=6.4G lam=0.3000488355755806 Gm=0.35683103433499735 +[full s0] step 10260/12000 loss=4.4055 gn=1.09 lr=8.85e-05 tps=15027 mem=6.4G lam=0.3001634751756986 Gm=0.35455567156895995 +[full s0] step 10280/12000 loss=4.3807 gn=0.99 lr=8.78e-05 tps=15027 mem=6.4G lam=0.3002860962102811 Gm=0.3569905702024698 +[full s0] step 10300/12000 loss=4.2820 gn=0.92 lr=8.72e-05 tps=15028 mem=6.4G lam=0.3003728125865261 Gm=0.3525042791540424 +[full s0] step 10320/12000 loss=4.2292 gn=0.89 lr=8.66e-05 tps=15028 mem=6.4G lam=0.3004772945617636 Gm=0.3497640168449531 +[full s0] step 10340/12000 loss=4.3175 gn=0.99 lr=8.59e-05 tps=15028 mem=6.4G lam=0.3005873554696639 Gm=0.3537204213595639 +[full s0] step 10360/12000 loss=4.4039 gn=1.00 lr=8.53e-05 tps=15029 mem=6.4G lam=0.3006585069621603 Gm=0.3563681039959192 +[full s0] step 10380/12000 loss=4.2411 gn=0.91 lr=8.47e-05 tps=15029 mem=6.4G lam=0.30072542900840443 Gm=0.35499131423421204 +[full s0] step 10400/12000 loss=4.3632 gn=0.98 lr=8.41e-05 tps=15029 mem=6.4G lam=0.3008216901992758 Gm=0.35223177867010236 +[full s0] step 10420/12000 loss=4.3888 gn=1.01 lr=8.35e-05 tps=15030 mem=6.4G lam=0.30091601020346087 Gm=0.3604333245505889 +[full s0] step 10440/12000 loss=4.4242 gn=1.02 lr=8.30e-05 tps=15030 mem=6.4G lam=0.30101171880960464 Gm=0.35854766246241826 +[full s0] step 10460/12000 loss=4.4676 gn=1.01 lr=8.24e-05 tps=15030 mem=6.4G lam=0.30109395862867433 Gm=0.36066112147333723 +[full s0] step 10480/12000 loss=4.3377 gn=1.01 lr=8.18e-05 tps=15030 mem=6.4G lam=0.3011544942855835 Gm=0.36058713821694255 +[full s0] step 10500/12000 loss=4.4270 gn=0.96 lr=8.12e-05 tps=15031 mem=6.4G lam=0.3012528655429681 Gm=0.3568641215873261 +[full s0] EVAL step 10500: val_loss=4.4352 val_ppl=84.36 +[full s0] step 10520/12000 loss=4.2305 gn=0.97 lr=8.07e-05 tps=15021 mem=6.4G lam=0.3013236640642087 Gm=0.3546206353542705 +[full s0] step 10540/12000 loss=4.4120 gn=0.93 lr=8.01e-05 tps=15021 mem=6.4G lam=0.3014307947208484 Gm=0.35363122339670855 +[full s0] step 10560/12000 loss=4.3548 gn=0.99 lr=7.96e-05 tps=15021 mem=6.4G lam=0.30151529765377444 Gm=0.3494523138118287 +[full s0] step 10580/12000 loss=4.3665 gn=0.95 lr=7.91e-05 tps=15021 mem=6.4G lam=0.30161114037036896 Gm=0.3549492438323796 +[full s0] step 10600/12000 loss=4.3779 gn=0.90 lr=7.85e-05 tps=15021 mem=6.4G lam=0.3017227003971736 Gm=0.3549186848104 +[full s0] step 10620/12000 loss=4.1942 gn=0.92 lr=7.80e-05 tps=15022 mem=6.4G lam=0.30181511274228495 Gm=0.3555798502638936 +[full s0] step 10640/12000 loss=4.2026 gn=0.95 lr=7.75e-05 tps=15022 mem=6.4G lam=0.30191439017653465 Gm=0.35305707110092044 +[full s0] step 10660/12000 loss=4.3487 gn=0.97 lr=7.70e-05 tps=15022 mem=6.4G lam=0.3020153675849239 Gm=0.3549336750681202 +[full s0] step 10680/12000 loss=4.4296 gn=0.91 lr=7.65e-05 tps=15023 mem=6.4G lam=0.3020895943045616 Gm=0.35832432843744755 +[full s0] step 10700/12000 loss=4.2411 gn=0.98 lr=7.60e-05 tps=15023 mem=6.4G lam=0.3021532253672679 Gm=0.35331386234611273 +[full s0] step 10720/12000 loss=4.4645 gn=1.02 lr=7.55e-05 tps=15024 mem=6.4G lam=0.3022256375600894 Gm=0.36003056991224486 +[full s0] step 10740/12000 loss=4.4051 gn=0.96 lr=7.50e-05 tps=15024 mem=6.4G lam=0.30232095004369813 Gm=0.35763002016271156 +[full s0] step 10760/12000 loss=4.2963 gn=0.92 lr=7.46e-05 tps=15025 mem=6.4G lam=0.3024042571584384 Gm=0.35448452721660334 +[full s0] step 10780/12000 loss=4.3868 gn=0.98 lr=7.41e-05 tps=15025 mem=6.4G lam=0.30249367157618207 Gm=0.35735314649840194 +[full s0] step 10800/12000 loss=4.4720 gn=1.05 lr=7.37e-05 tps=15026 mem=6.4G lam=0.30258614011108875 Gm=0.3596738283522427 +[full s0] step 10820/12000 loss=4.2562 gn=1.06 lr=7.32e-05 tps=15026 mem=6.4G lam=0.302665904785196 Gm=0.3585772806157668 +[full s0] step 10840/12000 loss=4.3682 gn=1.00 lr=7.28e-05 tps=15027 mem=6.4G lam=0.3027230082079768 Gm=0.3486389470441888 +[full s0] step 10860/12000 loss=4.4627 gn=1.05 lr=7.23e-05 tps=15027 mem=6.4G lam=0.30280512167761725 Gm=0.3532739697645108 +[full s0] step 10880/12000 loss=4.2785 gn=0.95 lr=7.19e-05 tps=15028 mem=6.4G lam=0.30287775211036205 Gm=0.3460147036239505 +[full s0] step 10900/12000 loss=4.4000 gn=0.98 lr=7.15e-05 tps=15028 mem=6.4G lam=0.3029637100795905 Gm=0.3560061762109399 +[full s0] step 10920/12000 loss=4.3101 gn=0.95 lr=7.11e-05 tps=15028 mem=6.4G lam=0.3030237536877394 Gm=0.35558117328522104 +[full s0] step 10940/12000 loss=4.3905 gn=0.98 lr=7.07e-05 tps=15029 mem=6.4G lam=0.3030867511406541 Gm=0.3543637775971244 +[full s0] step 10960/12000 loss=4.4324 gn=1.03 lr=7.03e-05 tps=15029 mem=6.4G lam=0.30318172307064134 Gm=0.3544179388942818 +[full s0] step 10980/12000 loss=4.3412 gn=1.00 lr=6.99e-05 tps=15030 mem=6.4G lam=0.30324514272312325 Gm=0.3545960403668384 +[full s0] step 11000/12000 loss=4.3568 gn=0.96 lr=6.95e-05 tps=15030 mem=6.4G lam=0.303302401676774 Gm=0.34809166674191755 +[full s0] EVAL step 11000: val_loss=4.3940 val_ppl=80.97 +[full s0] step 11020/12000 loss=4.2265 gn=0.95 lr=6.91e-05 tps=15020 mem=6.4G lam=0.30338037976374227 Gm=0.35431842335189384 +[full s0] step 11040/12000 loss=4.2158 gn=0.97 lr=6.88e-05 tps=15021 mem=6.4G lam=0.30344449697683257 Gm=0.3516675451149543 +[full s0] step 11060/12000 loss=4.4249 gn=1.07 lr=6.84e-05 tps=15021 mem=6.4G lam=0.30352770909667015 Gm=0.35496364037195843 +[full s0] step 11080/12000 loss=4.3553 gn=1.01 lr=6.81e-05 tps=15022 mem=6.4G lam=0.3036169521510601 Gm=0.35138007610415417 +[full s0] step 11100/12000 loss=4.4282 gn=0.99 lr=6.77e-05 tps=15022 mem=6.4G lam=0.30369426403194666 Gm=0.35393421406236786 +[full s0] step 11120/12000 loss=4.3024 gn=0.97 lr=6.74e-05 tps=15022 mem=6.4G lam=0.3037645888204376 Gm=0.34984520620976883 +[full s0] step 11140/12000 loss=4.1832 gn=0.93 lr=6.70e-05 tps=15023 mem=6.4G lam=0.303836852312088 Gm=0.3570408138912171 +[full s0] step 11160/12000 loss=4.2794 gn=1.03 lr=6.67e-05 tps=15023 mem=6.4G lam=0.30390045667688054 Gm=0.35645185426498455 +[full s0] step 11180/12000 loss=4.3601 gn=0.96 lr=6.64e-05 tps=15024 mem=6.4G lam=0.3040075519432624 Gm=0.35261975058044 +[full s0] step 11200/12000 loss=4.3587 gn=0.94 lr=6.61e-05 tps=15024 mem=6.4G lam=0.304081619096299 Gm=0.35336496580081683 +[full s0] step 11220/12000 loss=4.3532 gn=0.93 lr=6.58e-05 tps=15024 mem=6.4G lam=0.30416804800430935 Gm=0.3550302661024034 +[full s0] step 11240/12000 loss=4.1762 gn=0.92 lr=6.55e-05 tps=15025 mem=6.4G lam=0.3042526834954818 Gm=0.35358641560499865 +[full s0] step 11260/12000 loss=4.3781 gn=0.99 lr=6.52e-05 tps=15025 mem=6.4G lam=0.30432180563608807 Gm=0.3543307015982767 +[full s0] step 11280/12000 loss=4.3839 gn=1.16 lr=6.49e-05 tps=15026 mem=6.4G lam=0.3044009543955326 Gm=0.3566186165747543 +[full s0] step 11300/12000 loss=4.3664 gn=1.00 lr=6.47e-05 tps=15026 mem=6.4G lam=0.3044759665305416 Gm=0.3548339807117979 +[full s0] step 11320/12000 loss=4.2278 gn=0.97 lr=6.44e-05 tps=15026 mem=6.4G lam=0.30453793052583933 Gm=0.35302414395846426 +[full s0] step 11340/12000 loss=4.3067 gn=1.03 lr=6.42e-05 tps=15027 mem=6.4G lam=0.3046170463785529 Gm=0.35333169748385745 +[full s0] step 11360/12000 loss=4.1833 gn=0.98 lr=6.39e-05 tps=15027 mem=6.4G lam=0.3047155017654101 Gm=0.3526973278882603 +[full s0] step 11380/12000 loss=4.3793 gn=0.97 lr=6.37e-05 tps=15028 mem=6.4G lam=0.3047927661488454 Gm=0.3552943426184356 +[full s0] step 11400/12000 loss=4.2883 gn=0.99 lr=6.34e-05 tps=15028 mem=6.4G lam=0.30486341565847397 Gm=0.35069042979739606 +[full s0] step 11420/12000 loss=4.3079 gn=1.03 lr=6.32e-05 tps=15029 mem=6.4G lam=0.3049508572245638 Gm=0.35548072036666173 +[full s0] step 11440/12000 loss=4.1648 gn=0.98 lr=6.30e-05 tps=15029 mem=6.4G lam=0.30502159986644983 Gm=0.35188748586612445 +[full s0] step 11460/12000 loss=4.3986 gn=0.97 lr=6.28e-05 tps=15029 mem=6.4G lam=0.3051067106425762 Gm=0.3561136273977657 +[full s0] step 11480/12000 loss=4.2707 gn=1.02 lr=6.26e-05 tps=15030 mem=6.4G lam=0.30518741967777413 Gm=0.3505121430692573 +[full s0] step 11500/12000 loss=4.2875 gn=0.98 lr=6.24e-05 tps=15030 mem=6.4G lam=0.3052515930806597 Gm=0.34763204368452233 +[full s0] EVAL step 11500: val_loss=4.4039 val_ppl=81.77 +[full s0] step 11520/12000 loss=4.3041 gn=0.99 lr=6.22e-05 tps=15021 mem=6.4G lam=0.305330343854924 Gm=0.3500541648051391 +[full s0] step 11540/12000 loss=4.4261 gn=1.02 lr=6.20e-05 tps=15021 mem=6.4G lam=0.3053871961310506 Gm=0.3533576432770739 +[full s0] step 11560/12000 loss=4.3737 gn=1.04 lr=6.19e-05 tps=15022 mem=6.4G lam=0.3054633044327299 Gm=0.35542486468330026 +[full s0] step 11580/12000 loss=4.1796 gn=0.97 lr=6.17e-05 tps=15022 mem=6.4G lam=0.3055233849833409 Gm=0.3486354884225875 +[full s0] step 11600/12000 loss=4.3103 gn=0.98 lr=6.15e-05 tps=15023 mem=6.4G lam=0.3055758196860552 Gm=0.3596073631973316 +[full s0] step 11620/12000 loss=4.3755 gn=0.99 lr=6.14e-05 tps=15023 mem=6.4G lam=0.3056767834350467 Gm=0.3549321588749687 +[full s0] step 11640/12000 loss=4.1414 gn=1.00 lr=6.12e-05 tps=15024 mem=6.4G lam=0.30575469974428415 Gm=0.35379307996481657 +[full s0] step 11660/12000 loss=4.2807 gn=0.98 lr=6.11e-05 tps=15024 mem=6.4G lam=0.30581848820050556 Gm=0.3464067072297136 +[full s0] step 11680/12000 loss=4.2598 gn=0.93 lr=6.10e-05 tps=15024 mem=6.4G lam=0.30589383747428656 Gm=0.3537711181367437 +[full s0] step 11700/12000 loss=4.2682 gn=1.03 lr=6.09e-05 tps=15025 mem=6.4G lam=0.30599048392226297 Gm=0.35530768893659115 +[full s0] step 11720/12000 loss=4.2455 gn=1.03 lr=6.07e-05 tps=15025 mem=6.4G lam=0.30607374105602503 Gm=0.35104059559913975 +[full s0] step 11740/12000 loss=4.4240 gn=1.01 lr=6.06e-05 tps=15026 mem=6.4G lam=0.30613534307728213 Gm=0.3536324218536417 +[full s0] step 11760/12000 loss=4.3551 gn=1.04 lr=6.06e-05 tps=15026 mem=6.4G lam=0.306194850243628 Gm=0.3544285321452965 +[full s0] step 11780/12000 loss=4.3585 gn=1.01 lr=6.05e-05 tps=15026 mem=6.4G lam=0.3062613724420468 Gm=0.34824644681066275 +[full s0] step 11800/12000 loss=4.3861 gn=1.00 lr=6.04e-05 tps=15027 mem=6.4G lam=0.3063259053354462 Gm=0.35595441741558415 +[full s0] step 11820/12000 loss=4.2861 gn=1.00 lr=6.03e-05 tps=15027 mem=6.4G lam=0.3063966992000739 Gm=0.3542495178990066 +[full s0] step 11840/12000 loss=4.3292 gn=0.96 lr=6.02e-05 tps=15028 mem=6.4G lam=0.30647587186346453 Gm=0.3518916078998397 +[full s0] step 11860/12000 loss=4.1296 gn=0.96 lr=6.02e-05 tps=15028 mem=6.4G lam=0.30654072389006615 Gm=0.34820271912030876 +[full s0] step 11880/12000 loss=4.4762 gn=1.10 lr=6.01e-05 tps=15028 mem=6.4G lam=0.3066139652704199 Gm=0.35586218621271354 +[full s0] step 11900/12000 loss=4.2781 gn=1.06 lr=6.01e-05 tps=15029 mem=6.4G lam=0.3066550896813472 Gm=0.34946974491079646 +[full s0] step 11920/12000 loss=4.4473 gn=1.07 lr=6.01e-05 tps=15029 mem=6.4G lam=0.3067334496105711 Gm=0.3539520923513919 +[full s0] step 11940/12000 loss=4.1412 gn=1.03 lr=6.00e-05 tps=15030 mem=6.4G lam=0.3068108210961024 Gm=0.34945884401289123 +[full s0] step 11960/12000 loss=4.3300 gn=1.03 lr=6.00e-05 tps=15030 mem=6.4G lam=0.30687362390259904 Gm=0.35756496984201175 +[full s0] step 11980/12000 loss=4.2250 gn=1.02 lr=6.00e-05 tps=15030 mem=6.4G lam=0.3069125845407446 Gm=0.3505655957075457 +[full s0] EVAL step 11999: val_loss=4.3550 val_ppl=77.86 +[init] variant=baseline seed=1 params=63.38M +[init] use_S=False use_G=False +[baseline s1] step 0/12000 loss=10.8942 gn=12.84 lr=3.00e-06 tps=5038 mem=5.5G lam=None Gm=None +[baseline s1] step 20/12000 loss=9.8216 gn=2.19 lr=6.30e-05 tps=16730 mem=6.0G lam=None Gm=None +[baseline s1] step 40/12000 loss=9.1776 gn=1.69 lr=1.23e-04 tps=17681 mem=6.0G lam=None Gm=None +[baseline s1] step 60/12000 loss=8.2794 gn=1.45 lr=1.83e-04 tps=18040 mem=6.0G lam=None Gm=None +[baseline s1] step 80/12000 loss=7.4224 gn=0.95 lr=2.43e-04 tps=18202 mem=6.0G lam=None Gm=None +[baseline s1] step 100/12000 loss=7.3052 gn=0.66 lr=3.03e-04 tps=18300 mem=6.0G lam=None Gm=None +[baseline s1] step 120/12000 loss=7.0891 gn=0.62 lr=3.63e-04 tps=18353 mem=6.0G lam=None Gm=None +[baseline s1] step 140/12000 loss=6.8202 gn=0.62 lr=4.23e-04 tps=18404 mem=6.0G lam=None Gm=None +[baseline s1] step 160/12000 loss=6.7918 gn=0.63 lr=4.83e-04 tps=18442 mem=6.0G lam=None Gm=None +[baseline s1] step 180/12000 loss=6.7707 gn=0.68 lr=5.43e-04 tps=18472 mem=6.0G lam=None Gm=None +[baseline s1] step 200/12000 loss=6.6163 gn=0.61 lr=6.00e-04 tps=18495 mem=6.0G lam=None Gm=None +[baseline s1] step 220/12000 loss=6.6114 gn=0.63 lr=6.00e-04 tps=18516 mem=6.0G lam=None Gm=None +[baseline s1] step 240/12000 loss=6.6257 gn=0.50 lr=6.00e-04 tps=18529 mem=6.0G lam=None Gm=None +[baseline s1] step 260/12000 loss=6.6284 gn=0.51 lr=6.00e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s1] step 280/12000 loss=6.7107 gn=0.54 lr=6.00e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s1] step 300/12000 loss=6.5453 gn=0.55 lr=6.00e-04 tps=18563 mem=6.0G lam=None Gm=None +[baseline s1] step 320/12000 loss=6.4755 gn=0.57 lr=6.00e-04 tps=18570 mem=6.0G lam=None Gm=None +[baseline s1] step 340/12000 loss=6.5023 gn=0.57 lr=6.00e-04 tps=18578 mem=6.0G lam=None Gm=None +[baseline s1] step 360/12000 loss=6.2481 gn=0.64 lr=6.00e-04 tps=18584 mem=6.0G lam=None Gm=None +[baseline s1] step 380/12000 loss=6.3090 gn=0.49 lr=6.00e-04 tps=18591 mem=6.0G lam=None Gm=None +[baseline s1] step 400/12000 loss=6.2735 gn=0.49 lr=6.00e-04 tps=18596 mem=6.0G lam=None Gm=None +[baseline s1] step 420/12000 loss=6.3714 gn=0.51 lr=6.00e-04 tps=18604 mem=6.0G lam=None Gm=None +[baseline s1] step 440/12000 loss=6.2732 gn=0.62 lr=5.99e-04 tps=18608 mem=6.0G lam=None Gm=None +[baseline s1] step 460/12000 loss=6.3454 gn=0.53 lr=5.99e-04 tps=18611 mem=6.0G lam=None Gm=None +[baseline s1] step 480/12000 loss=6.2030 gn=0.50 lr=5.99e-04 tps=18614 mem=6.0G lam=None Gm=None +[baseline s1] step 500/12000 loss=6.3393 gn=0.58 lr=5.99e-04 tps=18617 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 500: val_loss=6.2618 val_ppl=524.19 +[baseline s1] step 520/12000 loss=6.1817 gn=0.55 lr=5.99e-04 tps=18364 mem=6.0G lam=None Gm=None +[baseline s1] step 540/12000 loss=6.2739 gn=0.56 lr=5.99e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 560/12000 loss=6.1630 gn=0.51 lr=5.99e-04 tps=18386 mem=6.0G lam=None Gm=None +[baseline s1] step 580/12000 loss=6.1042 gn=0.55 lr=5.99e-04 tps=18395 mem=6.0G lam=None Gm=None +[baseline s1] step 600/12000 loss=6.1552 gn=0.60 lr=5.98e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 620/12000 loss=6.2666 gn=0.49 lr=5.98e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 640/12000 loss=6.2632 gn=0.58 lr=5.98e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 660/12000 loss=5.9867 gn=0.51 lr=5.98e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 680/12000 loss=6.0133 gn=0.54 lr=5.98e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 700/12000 loss=5.9909 gn=0.51 lr=5.98e-04 tps=18440 mem=6.0G lam=None Gm=None +[baseline s1] step 720/12000 loss=6.0741 gn=0.51 lr=5.97e-04 tps=18446 mem=6.0G lam=None Gm=None +[baseline s1] step 740/12000 loss=5.9970 gn=0.54 lr=5.97e-04 tps=18451 mem=6.0G lam=None Gm=None +[baseline s1] step 760/12000 loss=6.0543 gn=0.51 lr=5.97e-04 tps=18457 mem=6.0G lam=None Gm=None +[baseline s1] step 780/12000 loss=6.0313 gn=0.51 lr=5.97e-04 tps=18461 mem=6.0G lam=None Gm=None +[baseline s1] step 800/12000 loss=5.9516 gn=0.56 lr=5.97e-04 tps=18466 mem=6.0G lam=None Gm=None +[baseline s1] step 820/12000 loss=5.8576 gn=0.54 lr=5.96e-04 tps=18470 mem=6.0G lam=None Gm=None +[baseline s1] step 840/12000 loss=5.7900 gn=0.55 lr=5.96e-04 tps=18475 mem=6.0G lam=None Gm=None +[baseline s1] step 860/12000 loss=5.6962 gn=0.51 lr=5.96e-04 tps=18479 mem=6.0G lam=None Gm=None +[baseline s1] step 880/12000 loss=5.7735 gn=0.55 lr=5.96e-04 tps=18482 mem=6.0G lam=None Gm=None +[baseline s1] step 900/12000 loss=5.6976 gn=0.55 lr=5.95e-04 tps=18487 mem=6.0G lam=None Gm=None +[baseline s1] step 920/12000 loss=5.8904 gn=0.56 lr=5.95e-04 tps=18491 mem=6.0G lam=None Gm=None +[baseline s1] step 940/12000 loss=5.7963 gn=0.53 lr=5.95e-04 tps=18495 mem=6.0G lam=None Gm=None +[baseline s1] step 960/12000 loss=5.9370 gn=0.53 lr=5.94e-04 tps=18498 mem=6.0G lam=None Gm=None +[baseline s1] step 980/12000 loss=5.6383 gn=0.51 lr=5.94e-04 tps=18501 mem=6.0G lam=None Gm=None +[baseline s1] step 1000/12000 loss=5.7745 gn=0.54 lr=5.94e-04 tps=18503 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 1000: val_loss=5.7938 val_ppl=328.26 +[baseline s1] step 1020/12000 loss=5.6848 gn=0.57 lr=5.94e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 1040/12000 loss=5.7374 gn=0.57 lr=5.93e-04 tps=18380 mem=6.0G lam=None Gm=None +[baseline s1] step 1060/12000 loss=5.7957 gn=0.55 lr=5.93e-04 tps=18384 mem=6.0G lam=None Gm=None +[baseline s1] step 1080/12000 loss=5.5819 gn=0.51 lr=5.93e-04 tps=18389 mem=6.0G lam=None Gm=None +[baseline s1] step 1100/12000 loss=5.8031 gn=0.56 lr=5.92e-04 tps=18394 mem=6.0G lam=None Gm=None +[baseline s1] step 1120/12000 loss=5.6612 gn=0.58 lr=5.92e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 1140/12000 loss=5.6952 gn=0.49 lr=5.92e-04 tps=18403 mem=6.0G lam=None Gm=None +[baseline s1] step 1160/12000 loss=5.6915 gn=0.58 lr=5.91e-04 tps=18406 mem=6.0G lam=None Gm=None +[baseline s1] step 1180/12000 loss=5.7245 gn=0.59 lr=5.91e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 1200/12000 loss=5.6623 gn=0.56 lr=5.90e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 1220/12000 loss=5.8445 gn=0.64 lr=5.90e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 1240/12000 loss=5.6054 gn=0.58 lr=5.90e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 1260/12000 loss=5.6093 gn=0.63 lr=5.89e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 1280/12000 loss=5.4945 gn=0.53 lr=5.89e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 1300/12000 loss=5.6336 gn=0.58 lr=5.89e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 1320/12000 loss=5.4952 gn=0.59 lr=5.88e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 1340/12000 loss=5.7443 gn=0.62 lr=5.88e-04 tps=18439 mem=6.0G lam=None Gm=None +[baseline s1] step 1360/12000 loss=5.5660 gn=0.55 lr=5.87e-04 tps=18442 mem=6.0G lam=None Gm=None +[baseline s1] step 1380/12000 loss=5.4896 gn=0.57 lr=5.87e-04 tps=18444 mem=6.0G lam=None Gm=None +[baseline s1] step 1400/12000 loss=5.7026 gn=0.55 lr=5.86e-04 tps=18447 mem=6.0G lam=None Gm=None +[baseline s1] step 1420/12000 loss=5.4585 gn=0.53 lr=5.86e-04 tps=18449 mem=6.0G lam=None Gm=None +[baseline s1] step 1440/12000 loss=5.5970 gn=0.55 lr=5.85e-04 tps=18452 mem=6.0G lam=None Gm=None +[baseline s1] step 1460/12000 loss=5.6174 gn=0.65 lr=5.85e-04 tps=18454 mem=6.0G lam=None Gm=None +[baseline s1] step 1480/12000 loss=5.5049 gn=0.54 lr=5.84e-04 tps=18457 mem=6.0G lam=None Gm=None +[baseline s1] step 1500/12000 loss=5.3207 gn=0.65 lr=5.84e-04 tps=18460 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 1500: val_loss=5.5179 val_ppl=249.10 +[baseline s1] step 1520/12000 loss=5.5032 gn=0.57 lr=5.83e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 1540/12000 loss=5.5093 gn=0.64 lr=5.83e-04 tps=18378 mem=6.0G lam=None Gm=None +[baseline s1] step 1560/12000 loss=5.4795 gn=0.59 lr=5.82e-04 tps=18381 mem=6.0G lam=None Gm=None +[baseline s1] step 1580/12000 loss=5.1629 gn=0.55 lr=5.82e-04 tps=18384 mem=6.0G lam=None Gm=None +[baseline s1] step 1600/12000 loss=5.2554 gn=0.65 lr=5.81e-04 tps=18387 mem=6.0G lam=None Gm=None +[baseline s1] step 1620/12000 loss=5.2679 gn=0.58 lr=5.81e-04 tps=18390 mem=6.0G lam=None Gm=None +[baseline s1] step 1640/12000 loss=5.2696 gn=0.55 lr=5.80e-04 tps=18392 mem=6.0G lam=None Gm=None +[baseline s1] step 1660/12000 loss=5.4071 gn=0.64 lr=5.80e-04 tps=18395 mem=6.0G lam=None Gm=None +[baseline s1] step 1680/12000 loss=5.2069 gn=0.58 lr=5.79e-04 tps=18397 mem=6.0G lam=None Gm=None +[baseline s1] step 1700/12000 loss=5.2907 gn=0.55 lr=5.79e-04 tps=18400 mem=6.0G lam=None Gm=None +[baseline s1] step 1720/12000 loss=5.3633 gn=0.59 lr=5.78e-04 tps=18403 mem=6.0G lam=None Gm=None +[baseline s1] step 1740/12000 loss=5.3033 gn=0.56 lr=5.78e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 1760/12000 loss=5.3570 gn=0.60 lr=5.77e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] step 1780/12000 loss=5.3478 gn=0.58 lr=5.76e-04 tps=18409 mem=6.0G lam=None Gm=None +[baseline s1] step 1800/12000 loss=5.4557 gn=0.63 lr=5.76e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 1820/12000 loss=5.4416 gn=0.59 lr=5.75e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 1840/12000 loss=5.2222 gn=0.61 lr=5.75e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 1860/12000 loss=5.3025 gn=0.65 lr=5.74e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 1880/12000 loss=5.1262 gn=0.58 lr=5.73e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 1900/12000 loss=5.3645 gn=0.63 lr=5.73e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 1920/12000 loss=5.3889 gn=0.62 lr=5.72e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 1940/12000 loss=5.5049 gn=0.59 lr=5.72e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 1960/12000 loss=5.1523 gn=0.59 lr=5.71e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 1980/12000 loss=5.2678 gn=0.63 lr=5.70e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 2000/12000 loss=5.3214 gn=0.64 lr=5.70e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 2000: val_loss=5.3677 val_ppl=214.37 +[baseline s1] step 2020/12000 loss=5.2378 gn=0.63 lr=5.69e-04 tps=18368 mem=6.0G lam=None Gm=None +[baseline s1] step 2040/12000 loss=5.3092 gn=0.65 lr=5.68e-04 tps=18371 mem=6.0G lam=None Gm=None +[baseline s1] step 2060/12000 loss=5.2569 gn=0.59 lr=5.68e-04 tps=18373 mem=6.0G lam=None Gm=None +[baseline s1] step 2080/12000 loss=5.2958 gn=0.63 lr=5.67e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 2100/12000 loss=5.3091 gn=0.57 lr=5.66e-04 tps=18378 mem=6.0G lam=None Gm=None +[baseline s1] step 2120/12000 loss=5.1899 gn=0.63 lr=5.65e-04 tps=18380 mem=6.0G lam=None Gm=None +[baseline s1] step 2140/12000 loss=5.1618 gn=0.64 lr=5.65e-04 tps=18383 mem=6.0G lam=None Gm=None +[baseline s1] step 2160/12000 loss=5.2060 gn=0.62 lr=5.64e-04 tps=18385 mem=6.0G lam=None Gm=None +[baseline s1] step 2180/12000 loss=5.2176 gn=0.63 lr=5.63e-04 tps=18387 mem=6.0G lam=None Gm=None +[baseline s1] step 2200/12000 loss=5.1752 gn=0.59 lr=5.63e-04 tps=18389 mem=6.0G lam=None Gm=None +[baseline s1] step 2220/12000 loss=5.3097 gn=0.67 lr=5.62e-04 tps=18391 mem=6.0G lam=None Gm=None +[baseline s1] step 2240/12000 loss=5.1129 gn=0.59 lr=5.61e-04 tps=18393 mem=6.0G lam=None Gm=None +[baseline s1] step 2260/12000 loss=5.0872 gn=0.61 lr=5.60e-04 tps=18396 mem=6.0G lam=None Gm=None +[baseline s1] step 2280/12000 loss=5.2048 gn=0.64 lr=5.60e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 2300/12000 loss=5.1340 gn=0.63 lr=5.59e-04 tps=18401 mem=6.0G lam=None Gm=None +[baseline s1] step 2320/12000 loss=5.0246 gn=0.62 lr=5.58e-04 tps=18402 mem=6.0G lam=None Gm=None +[baseline s1] step 2340/12000 loss=5.1899 gn=0.60 lr=5.57e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 2360/12000 loss=5.2621 gn=0.64 lr=5.57e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 2380/12000 loss=5.2203 gn=0.67 lr=5.56e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] step 2400/12000 loss=5.1400 gn=0.73 lr=5.55e-04 tps=18410 mem=6.0G lam=None Gm=None +[baseline s1] step 2420/12000 loss=5.1597 gn=0.61 lr=5.54e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 2440/12000 loss=5.1495 gn=0.67 lr=5.53e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 2460/12000 loss=5.1852 gn=0.67 lr=5.53e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 2480/12000 loss=5.1600 gn=0.64 lr=5.52e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 2500/12000 loss=5.1778 gn=0.68 lr=5.51e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 2500: val_loss=5.2045 val_ppl=182.08 +[baseline s1] step 2520/12000 loss=5.2728 gn=0.81 lr=5.50e-04 tps=18371 mem=6.0G lam=None Gm=None +[baseline s1] step 2540/12000 loss=5.2142 gn=0.62 lr=5.49e-04 tps=18374 mem=6.0G lam=None Gm=None +[baseline s1] step 2560/12000 loss=5.1838 gn=0.68 lr=5.48e-04 tps=18376 mem=6.0G lam=None Gm=None +[baseline s1] step 2580/12000 loss=5.0281 gn=0.65 lr=5.48e-04 tps=18378 mem=6.0G lam=None Gm=None +[baseline s1] step 2600/12000 loss=5.0963 gn=0.63 lr=5.47e-04 tps=18380 mem=6.0G lam=None Gm=None +[baseline s1] step 2620/12000 loss=5.0234 gn=0.70 lr=5.46e-04 tps=18382 mem=6.0G lam=None Gm=None +[baseline s1] step 2640/12000 loss=5.2165 gn=0.74 lr=5.45e-04 tps=18385 mem=6.0G lam=None Gm=None +[baseline s1] step 2660/12000 loss=5.0493 gn=0.65 lr=5.44e-04 tps=18387 mem=6.0G lam=None Gm=None +[baseline s1] step 2680/12000 loss=5.0622 gn=0.66 lr=5.43e-04 tps=18388 mem=6.0G lam=None Gm=None +[baseline s1] step 2700/12000 loss=5.1995 gn=0.71 lr=5.42e-04 tps=18391 mem=6.0G lam=None Gm=None +[baseline s1] step 2720/12000 loss=5.0524 gn=0.71 lr=5.41e-04 tps=18393 mem=6.0G lam=None Gm=None +[baseline s1] step 2740/12000 loss=5.2126 gn=0.73 lr=5.41e-04 tps=18395 mem=6.0G lam=None Gm=None +[baseline s1] step 2760/12000 loss=5.0344 gn=0.70 lr=5.40e-04 tps=18397 mem=6.0G lam=None Gm=None +[baseline s1] step 2780/12000 loss=5.1576 gn=0.68 lr=5.39e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 2800/12000 loss=4.8997 gn=0.69 lr=5.38e-04 tps=18397 mem=6.0G lam=None Gm=None +[baseline s1] step 2820/12000 loss=5.2207 gn=0.67 lr=5.37e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 2840/12000 loss=5.0774 gn=0.71 lr=5.36e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 2860/12000 loss=5.0120 gn=0.66 lr=5.35e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 2880/12000 loss=5.0840 gn=0.70 lr=5.34e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 2900/12000 loss=5.0623 gn=0.70 lr=5.33e-04 tps=18401 mem=6.0G lam=None Gm=None +[baseline s1] step 2920/12000 loss=5.0105 gn=0.73 lr=5.32e-04 tps=18400 mem=6.0G lam=None Gm=None +[baseline s1] step 2940/12000 loss=5.0966 gn=0.70 lr=5.31e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 2960/12000 loss=4.8802 gn=0.70 lr=5.30e-04 tps=18402 mem=6.0G lam=None Gm=None +[baseline s1] step 2980/12000 loss=5.0023 gn=0.67 lr=5.29e-04 tps=18404 mem=6.0G lam=None Gm=None +[baseline s1] step 3000/12000 loss=4.9826 gn=0.72 lr=5.28e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 3000: val_loss=5.0635 val_ppl=158.15 +[baseline s1] step 3020/12000 loss=5.0254 gn=0.73 lr=5.27e-04 tps=18363 mem=6.0G lam=None Gm=None +[baseline s1] step 3040/12000 loss=4.8631 gn=0.68 lr=5.26e-04 tps=18365 mem=6.0G lam=None Gm=None +[baseline s1] step 3060/12000 loss=5.0071 gn=0.69 lr=5.25e-04 tps=18367 mem=6.0G lam=None Gm=None +[baseline s1] step 3080/12000 loss=4.8681 gn=0.68 lr=5.24e-04 tps=18369 mem=6.0G lam=None Gm=None +[baseline s1] step 3100/12000 loss=5.0220 gn=0.71 lr=5.23e-04 tps=18371 mem=6.0G lam=None Gm=None +[baseline s1] step 3120/12000 loss=5.0237 gn=0.71 lr=5.22e-04 tps=18373 mem=6.0G lam=None Gm=None +[baseline s1] step 3140/12000 loss=4.9762 gn=0.75 lr=5.21e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 3160/12000 loss=4.9826 gn=0.73 lr=5.20e-04 tps=18377 mem=6.0G lam=None Gm=None +[baseline s1] step 3180/12000 loss=5.0306 gn=0.77 lr=5.19e-04 tps=18379 mem=6.0G lam=None Gm=None +[baseline s1] step 3200/12000 loss=4.8005 gn=0.68 lr=5.18e-04 tps=18381 mem=6.0G lam=None Gm=None +[baseline s1] step 3220/12000 loss=5.0144 gn=0.74 lr=5.17e-04 tps=18383 mem=6.0G lam=None Gm=None +[baseline s1] step 3240/12000 loss=5.0394 gn=0.71 lr=5.16e-04 tps=18385 mem=6.0G lam=None Gm=None +[baseline s1] step 3260/12000 loss=4.9292 gn=0.74 lr=5.15e-04 tps=18387 mem=6.0G lam=None Gm=None +[baseline s1] step 3280/12000 loss=4.8513 gn=0.73 lr=5.14e-04 tps=18389 mem=6.0G lam=None Gm=None +[baseline s1] step 3300/12000 loss=5.0374 gn=0.71 lr=5.13e-04 tps=18390 mem=6.0G lam=None Gm=None +[baseline s1] step 3320/12000 loss=5.0594 gn=0.73 lr=5.12e-04 tps=18392 mem=6.0G lam=None Gm=None +[baseline s1] step 3340/12000 loss=4.9045 gn=0.69 lr=5.11e-04 tps=18394 mem=6.0G lam=None Gm=None +[baseline s1] step 3360/12000 loss=4.8175 gn=0.76 lr=5.10e-04 tps=18396 mem=6.0G lam=None Gm=None +[baseline s1] step 3380/12000 loss=5.0970 gn=0.76 lr=5.09e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 3400/12000 loss=5.1734 gn=0.70 lr=5.08e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 3420/12000 loss=5.0692 gn=0.72 lr=5.07e-04 tps=18401 mem=6.0G lam=None Gm=None +[baseline s1] step 3440/12000 loss=4.8756 gn=0.73 lr=5.06e-04 tps=18403 mem=6.0G lam=None Gm=None +[baseline s1] step 3460/12000 loss=5.0044 gn=0.76 lr=5.05e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 3480/12000 loss=4.8811 gn=0.76 lr=5.03e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 3500/12000 loss=4.9095 gn=0.82 lr=5.02e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 3500: val_loss=5.0082 val_ppl=149.64 +[baseline s1] step 3520/12000 loss=4.8630 gn=0.78 lr=5.01e-04 tps=18373 mem=6.0G lam=None Gm=None +[baseline s1] step 3540/12000 loss=4.9723 gn=0.81 lr=5.00e-04 tps=18375 mem=6.0G lam=None Gm=None +[baseline s1] step 3560/12000 loss=4.9496 gn=0.78 lr=4.99e-04 tps=18377 mem=6.0G lam=None Gm=None +[baseline s1] step 3580/12000 loss=4.9998 gn=0.74 lr=4.98e-04 tps=18379 mem=6.0G lam=None Gm=None +[baseline s1] step 3600/12000 loss=5.0711 gn=0.80 lr=4.97e-04 tps=18381 mem=6.0G lam=None Gm=None +[baseline s1] step 3620/12000 loss=4.9352 gn=0.74 lr=4.96e-04 tps=18383 mem=6.0G lam=None Gm=None +[baseline s1] step 3640/12000 loss=4.7936 gn=0.72 lr=4.94e-04 tps=18384 mem=6.0G lam=None Gm=None +[baseline s1] step 3660/12000 loss=4.7497 gn=0.83 lr=4.93e-04 tps=18386 mem=6.0G lam=None Gm=None +[baseline s1] step 3680/12000 loss=4.7491 gn=0.73 lr=4.92e-04 tps=18388 mem=6.0G lam=None Gm=None +[baseline s1] step 3700/12000 loss=4.9640 gn=0.72 lr=4.91e-04 tps=18390 mem=6.0G lam=None Gm=None +[baseline s1] step 3720/12000 loss=4.9147 gn=0.77 lr=4.90e-04 tps=18392 mem=6.0G lam=None Gm=None +[baseline s1] step 3740/12000 loss=4.9336 gn=0.70 lr=4.89e-04 tps=18393 mem=6.0G lam=None Gm=None +[baseline s1] step 3760/12000 loss=4.8907 gn=0.76 lr=4.88e-04 tps=18395 mem=6.0G lam=None Gm=None +[baseline s1] step 3780/12000 loss=4.7951 gn=0.77 lr=4.86e-04 tps=18397 mem=6.0G lam=None Gm=None +[baseline s1] step 3800/12000 loss=4.7972 gn=0.77 lr=4.85e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 3820/12000 loss=4.8673 gn=0.70 lr=4.84e-04 tps=18400 mem=6.0G lam=None Gm=None +[baseline s1] step 3840/12000 loss=4.9440 gn=0.71 lr=4.83e-04 tps=18402 mem=6.0G lam=None Gm=None +[baseline s1] step 3860/12000 loss=4.8352 gn=0.81 lr=4.82e-04 tps=18404 mem=6.0G lam=None Gm=None +[baseline s1] step 3880/12000 loss=4.6936 gn=0.67 lr=4.80e-04 tps=18406 mem=6.0G lam=None Gm=None +[baseline s1] step 3900/12000 loss=4.8088 gn=0.79 lr=4.79e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 3920/12000 loss=4.9179 gn=0.86 lr=4.78e-04 tps=18409 mem=6.0G lam=None Gm=None +[baseline s1] step 3940/12000 loss=4.7513 gn=0.78 lr=4.77e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 3960/12000 loss=4.9858 gn=0.70 lr=4.76e-04 tps=18412 mem=6.0G lam=None Gm=None +[baseline s1] step 3980/12000 loss=5.0405 gn=0.78 lr=4.74e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 4000/12000 loss=4.9492 gn=0.78 lr=4.73e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 4000: val_loss=4.8806 val_ppl=131.71 +[baseline s1] step 4020/12000 loss=4.5856 gn=0.73 lr=4.72e-04 tps=18384 mem=6.0G lam=None Gm=None +[baseline s1] step 4040/12000 loss=4.8740 gn=0.81 lr=4.71e-04 tps=18386 mem=6.0G lam=None Gm=None +[baseline s1] step 4060/12000 loss=4.9241 gn=0.75 lr=4.70e-04 tps=18388 mem=6.0G lam=None Gm=None +[baseline s1] step 4080/12000 loss=4.7094 gn=0.76 lr=4.68e-04 tps=18389 mem=6.0G lam=None Gm=None +[baseline s1] step 4100/12000 loss=4.8523 gn=0.84 lr=4.67e-04 tps=18391 mem=6.0G lam=None Gm=None +[baseline s1] step 4120/12000 loss=4.8482 gn=0.79 lr=4.66e-04 tps=18393 mem=6.0G lam=None Gm=None +[baseline s1] step 4140/12000 loss=4.8982 gn=0.74 lr=4.65e-04 tps=18394 mem=6.0G lam=None Gm=None +[baseline s1] step 4160/12000 loss=4.7360 gn=0.75 lr=4.63e-04 tps=18396 mem=6.0G lam=None Gm=None +[baseline s1] step 4180/12000 loss=5.0098 gn=0.84 lr=4.62e-04 tps=18398 mem=6.0G lam=None Gm=None +[baseline s1] step 4200/12000 loss=4.6877 gn=0.77 lr=4.61e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 4220/12000 loss=4.7586 gn=0.82 lr=4.60e-04 tps=18400 mem=6.0G lam=None Gm=None +[baseline s1] step 4240/12000 loss=4.7160 gn=0.75 lr=4.58e-04 tps=18402 mem=6.0G lam=None Gm=None +[baseline s1] step 4260/12000 loss=4.7698 gn=0.75 lr=4.57e-04 tps=18403 mem=6.0G lam=None Gm=None +[baseline s1] step 4280/12000 loss=4.8895 gn=0.83 lr=4.56e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 4300/12000 loss=4.9280 gn=0.74 lr=4.54e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 4320/12000 loss=4.7246 gn=0.78 lr=4.53e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] step 4340/12000 loss=4.7741 gn=0.79 lr=4.52e-04 tps=18410 mem=6.0G lam=None Gm=None +[baseline s1] step 4360/12000 loss=4.8671 gn=0.80 lr=4.51e-04 tps=18412 mem=6.0G lam=None Gm=None +[baseline s1] step 4380/12000 loss=4.7392 gn=0.77 lr=4.49e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 4400/12000 loss=4.8354 gn=0.77 lr=4.48e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 4420/12000 loss=4.8169 gn=0.85 lr=4.47e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 4440/12000 loss=4.7308 gn=0.81 lr=4.45e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 4460/12000 loss=4.5458 gn=0.80 lr=4.44e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 4480/12000 loss=4.6964 gn=0.79 lr=4.43e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 4500/12000 loss=4.9732 gn=0.76 lr=4.42e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 4500: val_loss=4.7788 val_ppl=118.96 +[baseline s1] step 4520/12000 loss=4.8236 gn=0.78 lr=4.40e-04 tps=18394 mem=6.0G lam=None Gm=None +[baseline s1] step 4540/12000 loss=4.8043 gn=0.75 lr=4.39e-04 tps=18396 mem=6.0G lam=None Gm=None +[baseline s1] step 4560/12000 loss=4.6766 gn=0.81 lr=4.38e-04 tps=18397 mem=6.0G lam=None Gm=None +[baseline s1] step 4580/12000 loss=4.8493 gn=0.81 lr=4.36e-04 tps=18399 mem=6.0G lam=None Gm=None +[baseline s1] step 4600/12000 loss=4.6799 gn=0.78 lr=4.35e-04 tps=18401 mem=6.0G lam=None Gm=None +[baseline s1] step 4620/12000 loss=4.6813 gn=0.82 lr=4.34e-04 tps=18402 mem=6.0G lam=None Gm=None +[baseline s1] step 4640/12000 loss=4.8433 gn=0.83 lr=4.32e-04 tps=18404 mem=6.0G lam=None Gm=None +[baseline s1] step 4660/12000 loss=4.7256 gn=0.81 lr=4.31e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 4680/12000 loss=4.8237 gn=0.78 lr=4.30e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 4700/12000 loss=4.7894 gn=0.82 lr=4.28e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] step 4720/12000 loss=4.7998 gn=0.86 lr=4.27e-04 tps=18409 mem=6.0G lam=None Gm=None +[baseline s1] step 4740/12000 loss=4.7556 gn=0.85 lr=4.26e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 4760/12000 loss=4.8265 gn=0.78 lr=4.24e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 4780/12000 loss=4.6334 gn=0.88 lr=4.23e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 4800/12000 loss=4.6049 gn=0.82 lr=4.22e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 4820/12000 loss=4.7682 gn=0.76 lr=4.20e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 4840/12000 loss=4.7663 gn=0.84 lr=4.19e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 4860/12000 loss=4.8485 gn=0.73 lr=4.18e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 4880/12000 loss=4.6181 gn=0.84 lr=4.16e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 4900/12000 loss=4.7425 gn=0.79 lr=4.15e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 4920/12000 loss=4.7868 gn=0.84 lr=4.13e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 4940/12000 loss=4.5380 gn=0.80 lr=4.12e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 4960/12000 loss=4.7404 gn=0.84 lr=4.11e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 4980/12000 loss=4.8395 gn=0.78 lr=4.09e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 5000/12000 loss=4.7404 gn=0.89 lr=4.08e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 5000: val_loss=4.7538 val_ppl=116.02 +[baseline s1] step 5020/12000 loss=4.6892 gn=0.82 lr=4.07e-04 tps=18404 mem=6.0G lam=None Gm=None +[baseline s1] step 5040/12000 loss=4.6248 gn=0.82 lr=4.05e-04 tps=18405 mem=6.0G lam=None Gm=None +[baseline s1] step 5060/12000 loss=4.7636 gn=0.82 lr=4.04e-04 tps=18406 mem=6.0G lam=None Gm=None +[baseline s1] step 5080/12000 loss=4.7832 gn=0.84 lr=4.02e-04 tps=18407 mem=6.0G lam=None Gm=None +[baseline s1] step 5100/12000 loss=4.6449 gn=0.84 lr=4.01e-04 tps=18408 mem=6.0G lam=None Gm=None +[baseline s1] step 5120/12000 loss=4.7724 gn=0.78 lr=4.00e-04 tps=18410 mem=6.0G lam=None Gm=None +[baseline s1] step 5140/12000 loss=4.7211 gn=0.78 lr=3.98e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 5160/12000 loss=4.7481 gn=0.90 lr=3.97e-04 tps=18412 mem=6.0G lam=None Gm=None +[baseline s1] step 5180/12000 loss=4.7993 gn=0.97 lr=3.95e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 5200/12000 loss=4.7623 gn=0.87 lr=3.94e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 5220/12000 loss=4.5704 gn=0.90 lr=3.93e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 5240/12000 loss=4.8896 gn=0.91 lr=3.91e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 5260/12000 loss=4.6528 gn=0.81 lr=3.90e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 5280/12000 loss=4.5570 gn=0.80 lr=3.88e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 5300/12000 loss=4.6711 gn=0.86 lr=3.87e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 5320/12000 loss=4.6723 gn=0.87 lr=3.86e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 5340/12000 loss=4.7462 gn=0.86 lr=3.84e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 5360/12000 loss=4.5036 gn=0.80 lr=3.83e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 5380/12000 loss=4.5441 gn=0.91 lr=3.81e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 5400/12000 loss=4.5632 gn=0.81 lr=3.80e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 5420/12000 loss=4.6425 gn=0.82 lr=3.79e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 5440/12000 loss=4.7967 gn=0.91 lr=3.77e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 5460/12000 loss=4.6687 gn=0.84 lr=3.76e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 5480/12000 loss=4.6868 gn=0.87 lr=3.74e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 5500/12000 loss=4.6621 gn=0.86 lr=3.73e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 5500: val_loss=4.6941 val_ppl=109.30 +[baseline s1] step 5520/12000 loss=4.4939 gn=0.78 lr=3.72e-04 tps=18409 mem=6.0G lam=None Gm=None +[baseline s1] step 5540/12000 loss=4.6331 gn=0.90 lr=3.70e-04 tps=18410 mem=6.0G lam=None Gm=None +[baseline s1] step 5560/12000 loss=4.6536 gn=0.82 lr=3.69e-04 tps=18411 mem=6.0G lam=None Gm=None +[baseline s1] step 5580/12000 loss=4.5012 gn=0.80 lr=3.67e-04 tps=18412 mem=6.0G lam=None Gm=None +[baseline s1] step 5600/12000 loss=4.7213 gn=0.89 lr=3.66e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 5620/12000 loss=4.6998 gn=0.85 lr=3.64e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 5640/12000 loss=4.6486 gn=0.82 lr=3.63e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 5660/12000 loss=4.7555 gn=0.87 lr=3.62e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 5680/12000 loss=4.7762 gn=0.99 lr=3.60e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 5700/12000 loss=4.6229 gn=0.81 lr=3.59e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 5720/12000 loss=4.5099 gn=0.80 lr=3.57e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 5740/12000 loss=4.6484 gn=0.87 lr=3.56e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 5760/12000 loss=4.6843 gn=0.81 lr=3.54e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 5780/12000 loss=4.6247 gn=0.80 lr=3.53e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 5800/12000 loss=4.5962 gn=0.88 lr=3.52e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 5820/12000 loss=4.5805 gn=0.78 lr=3.50e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 5840/12000 loss=4.5685 gn=0.81 lr=3.49e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 5860/12000 loss=4.5714 gn=0.83 lr=3.47e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 5880/12000 loss=4.5495 gn=0.88 lr=3.46e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 5900/12000 loss=4.5322 gn=0.85 lr=3.44e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 5920/12000 loss=4.6424 gn=0.87 lr=3.43e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 5940/12000 loss=4.6042 gn=0.87 lr=3.41e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 5960/12000 loss=4.7566 gn=0.92 lr=3.40e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 5980/12000 loss=4.5366 gn=0.79 lr=3.39e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 6000/12000 loss=4.5820 gn=0.89 lr=3.37e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 6000: val_loss=4.7056 val_ppl=110.56 +[baseline s1] step 6020/12000 loss=4.6408 gn=0.88 lr=3.36e-04 tps=18413 mem=6.0G lam=None Gm=None +[baseline s1] step 6040/12000 loss=4.5280 gn=0.84 lr=3.34e-04 tps=18414 mem=6.0G lam=None Gm=None +[baseline s1] step 6060/12000 loss=4.4498 gn=0.81 lr=3.33e-04 tps=18415 mem=6.0G lam=None Gm=None +[baseline s1] step 6080/12000 loss=4.4741 gn=0.89 lr=3.31e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 6100/12000 loss=4.4737 gn=0.87 lr=3.30e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 6120/12000 loss=4.5782 gn=0.79 lr=3.29e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 6140/12000 loss=4.5056 gn=0.88 lr=3.27e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 6160/12000 loss=4.5039 gn=0.86 lr=3.26e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 6180/12000 loss=4.6282 gn=0.92 lr=3.24e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 6200/12000 loss=4.5559 gn=0.90 lr=3.23e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 6220/12000 loss=4.5917 gn=0.88 lr=3.21e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 6240/12000 loss=4.5389 gn=0.94 lr=3.20e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 6260/12000 loss=4.5628 gn=1.02 lr=3.19e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 6280/12000 loss=4.6161 gn=0.85 lr=3.17e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 6300/12000 loss=4.4911 gn=0.87 lr=3.16e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 6320/12000 loss=4.6335 gn=0.88 lr=3.14e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 6340/12000 loss=4.4839 gn=0.88 lr=3.13e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 6360/12000 loss=4.5510 gn=0.96 lr=3.11e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 6380/12000 loss=4.5991 gn=0.94 lr=3.10e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 6400/12000 loss=4.5308 gn=0.99 lr=3.08e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 6420/12000 loss=4.5993 gn=0.93 lr=3.07e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 6440/12000 loss=4.5498 gn=0.84 lr=3.06e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 6460/12000 loss=4.6349 gn=0.88 lr=3.04e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 6480/12000 loss=4.4166 gn=0.87 lr=3.03e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 6500/12000 loss=4.5336 gn=0.90 lr=3.01e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 6500: val_loss=4.6431 val_ppl=103.86 +[baseline s1] step 6520/12000 loss=4.4875 gn=0.81 lr=3.00e-04 tps=18416 mem=6.0G lam=None Gm=None +[baseline s1] step 6540/12000 loss=4.6413 gn=0.98 lr=2.98e-04 tps=18417 mem=6.0G lam=None Gm=None +[baseline s1] step 6560/12000 loss=4.6242 gn=0.92 lr=2.97e-04 tps=18418 mem=6.0G lam=None Gm=None +[baseline s1] step 6580/12000 loss=4.5357 gn=0.89 lr=2.96e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 6600/12000 loss=4.3417 gn=0.90 lr=2.94e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 6620/12000 loss=4.7605 gn=0.94 lr=2.93e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 6640/12000 loss=4.5656 gn=0.94 lr=2.91e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 6660/12000 loss=4.5155 gn=0.90 lr=2.90e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 6680/12000 loss=4.5043 gn=0.84 lr=2.88e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 6700/12000 loss=4.5149 gn=0.92 lr=2.87e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 6720/12000 loss=4.5247 gn=0.91 lr=2.86e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 6740/12000 loss=4.5085 gn=0.90 lr=2.84e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 6760/12000 loss=4.5714 gn=0.85 lr=2.83e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 6780/12000 loss=4.5654 gn=0.88 lr=2.81e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 6800/12000 loss=4.5203 gn=1.03 lr=2.80e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 6820/12000 loss=4.6795 gn=1.05 lr=2.79e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 6840/12000 loss=4.5743 gn=0.90 lr=2.77e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 6860/12000 loss=4.4899 gn=0.88 lr=2.76e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 6880/12000 loss=4.5218 gn=0.94 lr=2.74e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 6900/12000 loss=4.5056 gn=0.98 lr=2.73e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 6920/12000 loss=4.4429 gn=0.96 lr=2.72e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 6940/12000 loss=4.4340 gn=0.88 lr=2.70e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 6960/12000 loss=4.5079 gn=0.87 lr=2.69e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 6980/12000 loss=4.3642 gn=0.90 lr=2.67e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 7000/12000 loss=4.3672 gn=0.90 lr=2.66e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 7000: val_loss=4.5818 val_ppl=97.69 +[baseline s1] step 7020/12000 loss=4.5377 gn=0.94 lr=2.65e-04 tps=18419 mem=6.0G lam=None Gm=None +[baseline s1] step 7040/12000 loss=4.6253 gn=0.92 lr=2.63e-04 tps=18420 mem=6.0G lam=None Gm=None +[baseline s1] step 7060/12000 loss=4.5321 gn=0.96 lr=2.62e-04 tps=18421 mem=6.0G lam=None Gm=None +[baseline s1] step 7080/12000 loss=4.4324 gn=0.87 lr=2.60e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 7100/12000 loss=4.5660 gn=0.94 lr=2.59e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 7120/12000 loss=4.4968 gn=1.14 lr=2.58e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 7140/12000 loss=4.4403 gn=0.92 lr=2.56e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 7160/12000 loss=4.3843 gn=0.97 lr=2.55e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 7180/12000 loss=4.5960 gn=0.97 lr=2.53e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 7200/12000 loss=4.4775 gn=0.91 lr=2.52e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 7220/12000 loss=4.5675 gn=0.93 lr=2.51e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 7240/12000 loss=4.4793 gn=1.05 lr=2.49e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 7260/12000 loss=4.5236 gn=0.98 lr=2.48e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 7280/12000 loss=4.6083 gn=0.88 lr=2.47e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 7300/12000 loss=4.4318 gn=0.94 lr=2.45e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 7320/12000 loss=4.4956 gn=0.86 lr=2.44e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 7340/12000 loss=4.5247 gn=0.97 lr=2.42e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 7360/12000 loss=4.5341 gn=0.89 lr=2.41e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 7380/12000 loss=4.3777 gn=0.93 lr=2.40e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 7400/12000 loss=4.3794 gn=0.87 lr=2.38e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 7420/12000 loss=4.5619 gn=0.94 lr=2.37e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 7440/12000 loss=4.5978 gn=0.98 lr=2.36e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 7460/12000 loss=4.4458 gn=1.03 lr=2.34e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] step 7480/12000 loss=4.6889 gn=1.00 lr=2.33e-04 tps=18438 mem=6.0G lam=None Gm=None +[baseline s1] step 7500/12000 loss=4.3211 gn=0.91 lr=2.32e-04 tps=18439 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 7500: val_loss=4.5603 val_ppl=95.61 +[baseline s1] step 7520/12000 loss=4.5505 gn=0.90 lr=2.30e-04 tps=18422 mem=6.0G lam=None Gm=None +[baseline s1] step 7540/12000 loss=4.4226 gn=0.94 lr=2.29e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 7560/12000 loss=4.4327 gn=0.96 lr=2.28e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 7580/12000 loss=4.3823 gn=0.86 lr=2.26e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 7600/12000 loss=4.4051 gn=0.97 lr=2.25e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 7620/12000 loss=4.3883 gn=0.92 lr=2.24e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 7640/12000 loss=4.4345 gn=0.98 lr=2.22e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 7660/12000 loss=4.5072 gn=1.04 lr=2.21e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 7680/12000 loss=4.3994 gn=1.05 lr=2.20e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 7700/12000 loss=4.4944 gn=0.96 lr=2.18e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 7720/12000 loss=4.5457 gn=1.22 lr=2.17e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 7740/12000 loss=4.3692 gn=0.92 lr=2.16e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 7760/12000 loss=4.5953 gn=1.02 lr=2.15e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 7780/12000 loss=4.4551 gn=0.93 lr=2.13e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 7800/12000 loss=4.5012 gn=1.05 lr=2.12e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 7820/12000 loss=4.3599 gn=1.01 lr=2.11e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 7840/12000 loss=4.5130 gn=0.99 lr=2.09e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 7860/12000 loss=4.4570 gn=1.07 lr=2.08e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 7880/12000 loss=4.4097 gn=1.08 lr=2.07e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 7900/12000 loss=4.5817 gn=1.01 lr=2.06e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 7920/12000 loss=4.4542 gn=0.99 lr=2.04e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 7940/12000 loss=4.4669 gn=0.97 lr=2.03e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] step 7960/12000 loss=4.5377 gn=1.02 lr=2.02e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] step 7980/12000 loss=4.2623 gn=0.96 lr=2.00e-04 tps=18438 mem=6.0G lam=None Gm=None +[baseline s1] step 8000/12000 loss=4.3629 gn=1.06 lr=1.99e-04 tps=18439 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 8000: val_loss=4.5506 val_ppl=94.68 +[baseline s1] step 8020/12000 loss=4.6304 gn=0.96 lr=1.98e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 8040/12000 loss=4.4215 gn=0.99 lr=1.97e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 8060/12000 loss=4.3141 gn=0.90 lr=1.95e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 8080/12000 loss=4.4044 gn=0.91 lr=1.94e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 8100/12000 loss=4.3996 gn=0.95 lr=1.93e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 8120/12000 loss=4.4419 gn=0.90 lr=1.92e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 8140/12000 loss=4.4163 gn=0.95 lr=1.90e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 8160/12000 loss=4.5199 gn=1.19 lr=1.89e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 8180/12000 loss=4.4431 gn=0.95 lr=1.88e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 8200/12000 loss=4.5016 gn=1.11 lr=1.87e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 8220/12000 loss=4.3937 gn=1.05 lr=1.86e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 8240/12000 loss=4.4447 gn=1.03 lr=1.84e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 8260/12000 loss=4.3438 gn=0.98 lr=1.83e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 8280/12000 loss=4.4215 gn=0.94 lr=1.82e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 8300/12000 loss=4.5922 gn=1.11 lr=1.81e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 8320/12000 loss=4.2331 gn=0.92 lr=1.80e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 8340/12000 loss=4.4177 gn=1.06 lr=1.78e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 8360/12000 loss=4.4145 gn=0.98 lr=1.77e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 8380/12000 loss=4.3719 gn=0.98 lr=1.76e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 8400/12000 loss=4.5594 gn=0.95 lr=1.75e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 8420/12000 loss=4.3761 gn=0.99 lr=1.74e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 8440/12000 loss=4.3946 gn=0.92 lr=1.72e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 8460/12000 loss=4.4169 gn=0.94 lr=1.71e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] step 8480/12000 loss=4.5407 gn=0.98 lr=1.70e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] step 8500/12000 loss=4.4970 gn=1.11 lr=1.69e-04 tps=18438 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 8500: val_loss=4.4988 val_ppl=89.91 +[baseline s1] step 8520/12000 loss=4.4349 gn=1.06 lr=1.68e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 8540/12000 loss=4.3767 gn=1.05 lr=1.67e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 8560/12000 loss=4.3671 gn=1.18 lr=1.66e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 8580/12000 loss=4.4947 gn=1.04 lr=1.64e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 8600/12000 loss=4.5463 gn=1.08 lr=1.63e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 8620/12000 loss=4.5071 gn=0.96 lr=1.62e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 8640/12000 loss=4.4157 gn=1.01 lr=1.61e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 8660/12000 loss=4.4469 gn=1.02 lr=1.60e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 8680/12000 loss=4.3779 gn=1.07 lr=1.59e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 8700/12000 loss=4.3632 gn=1.04 lr=1.58e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 8720/12000 loss=4.4515 gn=0.94 lr=1.57e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 8740/12000 loss=4.5819 gn=1.06 lr=1.55e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 8760/12000 loss=4.4088 gn=1.07 lr=1.54e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 8780/12000 loss=4.1668 gn=0.98 lr=1.53e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 8800/12000 loss=4.4184 gn=1.05 lr=1.52e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 8820/12000 loss=4.3961 gn=1.06 lr=1.51e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 8840/12000 loss=4.2472 gn=1.05 lr=1.50e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 8860/12000 loss=4.3924 gn=1.07 lr=1.49e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 8880/12000 loss=4.5065 gn=1.00 lr=1.48e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 8900/12000 loss=4.5684 gn=1.02 lr=1.47e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 8920/12000 loss=4.3842 gn=1.37 lr=1.46e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 8940/12000 loss=4.4030 gn=1.00 lr=1.45e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 8960/12000 loss=4.4125 gn=1.11 lr=1.44e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 8980/12000 loss=4.4460 gn=1.00 lr=1.43e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 9000/12000 loss=4.4310 gn=1.03 lr=1.42e-04 tps=18437 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 9000: val_loss=4.4682 val_ppl=87.20 +[baseline s1] step 9020/12000 loss=4.4577 gn=1.01 lr=1.41e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 9040/12000 loss=4.4054 gn=1.09 lr=1.40e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 9060/12000 loss=4.4429 gn=0.95 lr=1.39e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 9080/12000 loss=4.4544 gn=1.11 lr=1.38e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 9100/12000 loss=4.3750 gn=1.11 lr=1.37e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 9120/12000 loss=4.2426 gn=1.00 lr=1.36e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 9140/12000 loss=4.4814 gn=1.04 lr=1.35e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 9160/12000 loss=4.5378 gn=1.02 lr=1.34e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 9180/12000 loss=4.3442 gn=1.03 lr=1.33e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 9200/12000 loss=4.4563 gn=1.06 lr=1.32e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 9220/12000 loss=4.3607 gn=0.94 lr=1.31e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 9240/12000 loss=4.3787 gn=1.18 lr=1.30e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 9260/12000 loss=4.4731 gn=0.98 lr=1.29e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 9280/12000 loss=4.4653 gn=1.31 lr=1.28e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 9300/12000 loss=4.5457 gn=1.13 lr=1.27e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 9320/12000 loss=4.3060 gn=1.10 lr=1.26e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 9340/12000 loss=4.2943 gn=1.09 lr=1.25e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 9360/12000 loss=4.4849 gn=1.18 lr=1.24e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 9380/12000 loss=4.3974 gn=1.02 lr=1.23e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 9400/12000 loss=4.3438 gn=1.08 lr=1.22e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 9420/12000 loss=4.2457 gn=1.05 lr=1.21e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 9440/12000 loss=4.3401 gn=1.05 lr=1.20e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 9460/12000 loss=4.3251 gn=1.09 lr=1.19e-04 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 9480/12000 loss=4.4010 gn=1.00 lr=1.19e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 9500/12000 loss=4.3694 gn=1.09 lr=1.18e-04 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 9500: val_loss=4.4614 val_ppl=86.61 +[baseline s1] step 9520/12000 loss=4.4205 gn=1.09 lr=1.17e-04 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 9540/12000 loss=4.2887 gn=1.00 lr=1.16e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 9560/12000 loss=4.4135 gn=1.17 lr=1.15e-04 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 9580/12000 loss=4.4614 gn=1.26 lr=1.14e-04 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 9600/12000 loss=4.3536 gn=1.07 lr=1.13e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 9620/12000 loss=4.3793 gn=1.09 lr=1.12e-04 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 9640/12000 loss=4.2970 gn=1.07 lr=1.12e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 9660/12000 loss=4.5353 gn=1.09 lr=1.11e-04 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 9680/12000 loss=4.3365 gn=0.96 lr=1.10e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 9700/12000 loss=4.4184 gn=1.08 lr=1.09e-04 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 9720/12000 loss=4.5080 gn=1.09 lr=1.08e-04 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 9740/12000 loss=4.2414 gn=1.02 lr=1.07e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 9760/12000 loss=4.4584 gn=1.04 lr=1.07e-04 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 9780/12000 loss=4.3582 gn=1.03 lr=1.06e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 9800/12000 loss=4.4585 gn=1.01 lr=1.05e-04 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 9820/12000 loss=4.2971 gn=1.03 lr=1.04e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 9840/12000 loss=4.4989 gn=1.09 lr=1.03e-04 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 9860/12000 loss=4.5410 gn=1.12 lr=1.03e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 9880/12000 loss=4.3592 gn=1.05 lr=1.02e-04 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 9900/12000 loss=4.3332 gn=1.00 lr=1.01e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 9920/12000 loss=4.3157 gn=1.11 lr=1.00e-04 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 9940/12000 loss=4.2746 gn=1.04 lr=9.96e-05 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 9960/12000 loss=4.3426 gn=1.16 lr=9.89e-05 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 9980/12000 loss=4.4115 gn=0.98 lr=9.81e-05 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] step 10000/12000 loss=4.3415 gn=1.11 lr=9.74e-05 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 10000: val_loss=4.4268 val_ppl=83.66 +[baseline s1] step 10020/12000 loss=4.6535 gn=1.16 lr=9.67e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 10040/12000 loss=4.2519 gn=1.13 lr=9.59e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 10060/12000 loss=4.4899 gn=1.13 lr=9.52e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 10080/12000 loss=4.2401 gn=1.08 lr=9.45e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 10100/12000 loss=4.3120 gn=1.08 lr=9.38e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 10120/12000 loss=4.3431 gn=1.08 lr=9.31e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 10140/12000 loss=4.2732 gn=1.18 lr=9.24e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 10160/12000 loss=4.3828 gn=1.03 lr=9.18e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 10180/12000 loss=4.2786 gn=1.03 lr=9.11e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 10200/12000 loss=4.4759 gn=1.13 lr=9.04e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 10220/12000 loss=4.4018 gn=1.08 lr=8.98e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 10240/12000 loss=4.3370 gn=1.12 lr=8.91e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 10260/12000 loss=4.4513 gn=0.99 lr=8.85e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 10280/12000 loss=4.3546 gn=1.12 lr=8.78e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 10300/12000 loss=4.2743 gn=1.13 lr=8.72e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 10320/12000 loss=4.3157 gn=1.06 lr=8.66e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 10340/12000 loss=4.4732 gn=1.13 lr=8.59e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 10360/12000 loss=4.3550 gn=1.00 lr=8.53e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 10380/12000 loss=4.3686 gn=1.10 lr=8.47e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 10400/12000 loss=4.4664 gn=1.21 lr=8.41e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 10420/12000 loss=4.4142 gn=1.27 lr=8.35e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 10440/12000 loss=4.4370 gn=1.07 lr=8.30e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 10460/12000 loss=4.2530 gn=1.13 lr=8.24e-05 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 10480/12000 loss=4.3505 gn=1.16 lr=8.18e-05 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] step 10500/12000 loss=4.5247 gn=1.55 lr=8.12e-05 tps=18436 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 10500: val_loss=4.4197 val_ppl=83.07 +[baseline s1] step 10520/12000 loss=4.3662 gn=1.26 lr=8.07e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 10540/12000 loss=4.4589 gn=1.07 lr=8.01e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 10560/12000 loss=4.4280 gn=1.13 lr=7.96e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 10580/12000 loss=4.3070 gn=1.06 lr=7.91e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 10600/12000 loss=4.2547 gn=1.21 lr=7.85e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 10620/12000 loss=4.3578 gn=1.11 lr=7.80e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 10640/12000 loss=4.4616 gn=1.13 lr=7.75e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 10660/12000 loss=4.2728 gn=1.08 lr=7.70e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 10680/12000 loss=4.2902 gn=1.15 lr=7.65e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 10700/12000 loss=4.3403 gn=1.03 lr=7.60e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 10720/12000 loss=4.3569 gn=1.23 lr=7.55e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 10740/12000 loss=4.4283 gn=1.20 lr=7.50e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 10760/12000 loss=4.3169 gn=1.12 lr=7.46e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 10780/12000 loss=4.3548 gn=1.05 lr=7.41e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 10800/12000 loss=4.4195 gn=1.13 lr=7.37e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 10820/12000 loss=4.5033 gn=1.17 lr=7.32e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 10840/12000 loss=4.2096 gn=1.05 lr=7.28e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 10860/12000 loss=4.3964 gn=1.06 lr=7.23e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 10880/12000 loss=4.2285 gn=1.21 lr=7.19e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 10900/12000 loss=4.3472 gn=1.06 lr=7.15e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 10920/12000 loss=4.3534 gn=1.15 lr=7.11e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 10940/12000 loss=4.3001 gn=1.13 lr=7.07e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 10960/12000 loss=4.3193 gn=1.22 lr=7.03e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 10980/12000 loss=4.5111 gn=1.19 lr=6.99e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 11000/12000 loss=4.3061 gn=1.06 lr=6.95e-05 tps=18435 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 11000: val_loss=4.4054 val_ppl=81.89 +[baseline s1] step 11020/12000 loss=4.1244 gn=1.10 lr=6.91e-05 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 11040/12000 loss=4.3274 gn=1.30 lr=6.88e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 11060/12000 loss=4.3958 gn=1.19 lr=6.84e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 11080/12000 loss=4.3954 gn=1.14 lr=6.81e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 11100/12000 loss=4.4151 gn=1.14 lr=6.77e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 11120/12000 loss=4.3972 gn=1.09 lr=6.74e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 11140/12000 loss=4.2544 gn=1.11 lr=6.70e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 11160/12000 loss=4.2885 gn=1.19 lr=6.67e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 11180/12000 loss=4.3165 gn=1.13 lr=6.64e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 11200/12000 loss=4.5276 gn=1.67 lr=6.61e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 11220/12000 loss=4.3708 gn=1.25 lr=6.58e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 11240/12000 loss=4.2863 gn=1.10 lr=6.55e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 11260/12000 loss=4.3003 gn=1.17 lr=6.52e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 11280/12000 loss=4.1908 gn=1.26 lr=6.49e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 11300/12000 loss=4.3983 gn=1.08 lr=6.47e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 11320/12000 loss=4.2142 gn=1.11 lr=6.44e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 11340/12000 loss=4.3979 gn=1.11 lr=6.42e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 11360/12000 loss=4.2898 gn=1.13 lr=6.39e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 11380/12000 loss=4.3408 gn=1.22 lr=6.37e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 11400/12000 loss=4.3889 gn=1.20 lr=6.34e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 11420/12000 loss=4.0944 gn=1.13 lr=6.32e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 11440/12000 loss=4.2008 gn=1.05 lr=6.30e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 11460/12000 loss=4.3280 gn=1.24 lr=6.28e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 11480/12000 loss=4.3799 gn=1.03 lr=6.26e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] step 11500/12000 loss=4.5152 gn=1.32 lr=6.24e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 11500: val_loss=4.3979 val_ppl=81.28 +[baseline s1] step 11520/12000 loss=4.2977 gn=1.06 lr=6.22e-05 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 11540/12000 loss=4.2003 gn=1.05 lr=6.20e-05 tps=18423 mem=6.0G lam=None Gm=None +[baseline s1] step 11560/12000 loss=4.3402 gn=1.14 lr=6.19e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 11580/12000 loss=4.3809 gn=1.15 lr=6.17e-05 tps=18424 mem=6.0G lam=None Gm=None +[baseline s1] step 11600/12000 loss=4.1878 gn=1.21 lr=6.15e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 11620/12000 loss=4.4372 gn=1.23 lr=6.14e-05 tps=18425 mem=6.0G lam=None Gm=None +[baseline s1] step 11640/12000 loss=4.3807 gn=1.19 lr=6.12e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 11660/12000 loss=4.3717 gn=1.17 lr=6.11e-05 tps=18426 mem=6.0G lam=None Gm=None +[baseline s1] step 11680/12000 loss=4.2896 gn=1.17 lr=6.10e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 11700/12000 loss=4.3142 gn=1.07 lr=6.09e-05 tps=18427 mem=6.0G lam=None Gm=None +[baseline s1] step 11720/12000 loss=4.1651 gn=1.11 lr=6.07e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 11740/12000 loss=4.3996 gn=1.16 lr=6.06e-05 tps=18428 mem=6.0G lam=None Gm=None +[baseline s1] step 11760/12000 loss=4.2782 gn=1.16 lr=6.06e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 11780/12000 loss=4.2668 gn=1.14 lr=6.05e-05 tps=18429 mem=6.0G lam=None Gm=None +[baseline s1] step 11800/12000 loss=4.2469 gn=1.11 lr=6.04e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 11820/12000 loss=4.3317 gn=1.10 lr=6.03e-05 tps=18430 mem=6.0G lam=None Gm=None +[baseline s1] step 11840/12000 loss=4.2990 gn=1.13 lr=6.02e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 11860/12000 loss=4.3106 gn=1.14 lr=6.02e-05 tps=18431 mem=6.0G lam=None Gm=None +[baseline s1] step 11880/12000 loss=4.2720 gn=1.12 lr=6.01e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 11900/12000 loss=4.4107 gn=1.26 lr=6.01e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 11920/12000 loss=4.2431 gn=1.11 lr=6.01e-05 tps=18432 mem=6.0G lam=None Gm=None +[baseline s1] step 11940/12000 loss=4.3208 gn=1.20 lr=6.00e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 11960/12000 loss=4.2274 gn=1.16 lr=6.00e-05 tps=18433 mem=6.0G lam=None Gm=None +[baseline s1] step 11980/12000 loss=4.3223 gn=1.18 lr=6.00e-05 tps=18434 mem=6.0G lam=None Gm=None +[baseline s1] EVAL step 11999: val_loss=4.3936 val_ppl=80.93 +[init] variant=s_only seed=1 params=63.97M +[init] use_S=True use_G=False +[s_only s1] step 0/12000 loss=10.9138 gn=14.33 lr=3.00e-06 tps=7773 mem=5.6G lam=0.12692801654338837 Gm=None +[s_only s1] step 20/12000 loss=9.8083 gn=1.98 lr=6.30e-05 tps=15763 mem=6.2G lam=0.1269337609410286 Gm=None +[s_only s1] step 40/12000 loss=9.1595 gn=1.65 lr=1.23e-04 tps=16174 mem=6.2G lam=0.1268890624245008 Gm=None +[s_only s1] step 60/12000 loss=8.2635 gn=1.44 lr=1.83e-04 tps=16312 mem=6.2G lam=0.12680185213685036 Gm=None +[s_only s1] step 80/12000 loss=7.4172 gn=0.91 lr=2.43e-04 tps=16374 mem=6.2G lam=0.1266710323592027 Gm=None +[s_only s1] step 100/12000 loss=7.3038 gn=0.81 lr=3.03e-04 tps=16388 mem=6.2G lam=0.126587375998497 Gm=None +[s_only s1] step 120/12000 loss=7.0751 gn=0.69 lr=3.63e-04 tps=16419 mem=6.2G lam=0.1264726873487234 Gm=None +[s_only s1] step 140/12000 loss=6.8135 gn=0.71 lr=4.23e-04 tps=16440 mem=6.2G lam=0.12643363699316978 Gm=None +[s_only s1] step 160/12000 loss=6.7891 gn=0.68 lr=4.83e-04 tps=16454 mem=6.2G lam=0.12643427463869253 Gm=None +[s_only s1] step 180/12000 loss=6.7430 gn=0.59 lr=5.43e-04 tps=16474 mem=6.2G lam=0.12639994733035564 Gm=None +[s_only s1] step 200/12000 loss=6.6010 gn=0.62 lr=6.00e-04 tps=16484 mem=6.2G lam=0.12635636826356253 Gm=None +[s_only s1] step 220/12000 loss=6.6032 gn=0.63 lr=6.00e-04 tps=16490 mem=6.2G lam=0.12631579457471767 Gm=None +[s_only s1] step 240/12000 loss=6.6112 gn=0.48 lr=6.00e-04 tps=16497 mem=6.2G lam=0.12623944816490015 Gm=None +[s_only s1] step 260/12000 loss=6.6394 gn=0.62 lr=6.00e-04 tps=16504 mem=6.2G lam=0.12624299277861914 Gm=None +[s_only s1] step 280/12000 loss=6.7016 gn=0.56 lr=6.00e-04 tps=16516 mem=6.2G lam=0.12625163421034813 Gm=None +[s_only s1] step 300/12000 loss=6.5404 gn=0.54 lr=6.00e-04 tps=16520 mem=6.2G lam=0.12629619799554348 Gm=None +[s_only s1] step 320/12000 loss=6.4789 gn=0.57 lr=6.00e-04 tps=16524 mem=6.2G lam=0.12625167922427258 Gm=None +[s_only s1] step 340/12000 loss=6.4816 gn=0.55 lr=6.00e-04 tps=16526 mem=6.2G lam=0.12621581585456929 Gm=None +[s_only s1] step 360/12000 loss=6.2320 gn=0.51 lr=6.00e-04 tps=16525 mem=6.2G lam=0.12625759281218052 Gm=None +[s_only s1] step 380/12000 loss=6.3064 gn=0.56 lr=6.00e-04 tps=16530 mem=6.2G lam=0.12635301953802505 Gm=None +[s_only s1] step 400/12000 loss=6.2726 gn=0.50 lr=6.00e-04 tps=16532 mem=6.2G lam=0.12638964938620725 Gm=None +[s_only s1] step 420/12000 loss=6.3667 gn=0.55 lr=6.00e-04 tps=16532 mem=6.2G lam=0.1264471293737491 Gm=None +[s_only s1] step 440/12000 loss=6.2563 gn=0.51 lr=5.99e-04 tps=16530 mem=6.2G lam=0.1265504832069079 Gm=None +[s_only s1] step 460/12000 loss=6.3407 gn=0.52 lr=5.99e-04 tps=16530 mem=6.2G lam=0.12668250656376281 Gm=None +[s_only s1] step 480/12000 loss=6.2073 gn=0.58 lr=5.99e-04 tps=16533 mem=6.2G lam=0.12679455522447824 Gm=None +[s_only s1] step 500/12000 loss=6.3373 gn=0.54 lr=5.99e-04 tps=16534 mem=6.2G lam=0.1268880981951952 Gm=None +[s_only s1] EVAL step 500: val_loss=6.2600 val_ppl=523.21 +[s_only s1] step 520/12000 loss=6.1760 gn=0.54 lr=5.99e-04 tps=16302 mem=6.2G lam=0.1270150557781259 Gm=None +[s_only s1] step 540/12000 loss=6.2579 gn=0.53 lr=5.99e-04 tps=16311 mem=6.2G lam=0.1271467193340262 Gm=None +[s_only s1] step 560/12000 loss=6.1517 gn=0.48 lr=5.99e-04 tps=16315 mem=6.2G lam=0.12724667911728224 Gm=None +[s_only s1] step 580/12000 loss=6.1054 gn=0.63 lr=5.99e-04 tps=16324 mem=6.2G lam=0.12735473550856113 Gm=None +[s_only s1] step 600/12000 loss=6.1537 gn=0.58 lr=5.98e-04 tps=16331 mem=6.2G lam=0.1275418313841025 Gm=None +[s_only s1] step 620/12000 loss=6.2587 gn=0.49 lr=5.98e-04 tps=16339 mem=6.2G lam=0.1277033037816485 Gm=None +[s_only s1] step 640/12000 loss=6.2521 gn=0.57 lr=5.98e-04 tps=16345 mem=6.2G lam=0.12790542840957642 Gm=None +[s_only s1] step 660/12000 loss=5.9787 gn=0.54 lr=5.98e-04 tps=16352 mem=6.2G lam=0.12812327376256386 Gm=None +[s_only s1] step 680/12000 loss=6.0112 gn=0.53 lr=5.98e-04 tps=16359 mem=6.2G lam=0.12831780749062696 Gm=None +[s_only s1] step 700/12000 loss=5.9847 gn=0.53 lr=5.98e-04 tps=16366 mem=6.2G lam=0.12856514068941274 Gm=None +[s_only s1] step 720/12000 loss=6.0853 gn=0.53 lr=5.97e-04 tps=16372 mem=6.2G lam=0.12878051586449146 Gm=None +[s_only s1] step 740/12000 loss=5.9937 gn=0.51 lr=5.97e-04 tps=16378 mem=6.2G lam=0.1290573781977097 Gm=None +[s_only s1] step 760/12000 loss=6.0607 gn=0.52 lr=5.97e-04 tps=16384 mem=6.2G lam=0.12932050041854382 Gm=None +[s_only s1] step 780/12000 loss=6.0263 gn=0.51 lr=5.97e-04 tps=16389 mem=6.2G lam=0.1295907891665896 Gm=None +[s_only s1] step 800/12000 loss=5.9474 gn=0.54 lr=5.97e-04 tps=16392 mem=6.2G lam=0.1299926262969772 Gm=None +[s_only s1] step 820/12000 loss=5.8420 gn=0.52 lr=5.96e-04 tps=16396 mem=6.2G lam=0.13038730093588433 Gm=None +[s_only s1] step 840/12000 loss=5.7945 gn=0.56 lr=5.96e-04 tps=16400 mem=6.2G lam=0.1307868113120397 Gm=None +[s_only s1] step 860/12000 loss=5.6875 gn=0.55 lr=5.96e-04 tps=16403 mem=6.2G lam=0.13115752022713423 Gm=None +[s_only s1] step 880/12000 loss=5.7762 gn=0.59 lr=5.96e-04 tps=16407 mem=6.2G lam=0.13149215218921503 Gm=None +[s_only s1] step 900/12000 loss=5.6953 gn=0.51 lr=5.95e-04 tps=16410 mem=6.2G lam=0.1318753423790137 Gm=None +[s_only s1] step 920/12000 loss=5.8785 gn=0.54 lr=5.95e-04 tps=16413 mem=6.2G lam=0.13227269115547338 Gm=None +[s_only s1] step 940/12000 loss=5.8010 gn=0.53 lr=5.95e-04 tps=16416 mem=6.2G lam=0.13269481404374042 Gm=None +[s_only s1] step 960/12000 loss=5.9304 gn=0.53 lr=5.94e-04 tps=16418 mem=6.2G lam=0.1330983421454827 Gm=None +[s_only s1] step 980/12000 loss=5.6390 gn=0.51 lr=5.94e-04 tps=16422 mem=6.2G lam=0.1335068643093109 Gm=None +[s_only s1] step 1000/12000 loss=5.7672 gn=0.52 lr=5.94e-04 tps=16425 mem=6.2G lam=0.1339150465403994 Gm=None +[s_only s1] EVAL step 1000: val_loss=5.7917 val_ppl=327.59 +[s_only s1] step 1020/12000 loss=5.6844 gn=0.58 lr=5.94e-04 tps=16309 mem=6.2G lam=0.13436406229933104 Gm=None +[s_only s1] step 1040/12000 loss=5.7338 gn=0.58 lr=5.93e-04 tps=16313 mem=6.2G lam=0.1347834775224328 Gm=None +[s_only s1] step 1060/12000 loss=5.7809 gn=0.53 lr=5.93e-04 tps=16317 mem=6.2G lam=0.1352554593856136 Gm=None +[s_only s1] step 1080/12000 loss=5.6063 gn=0.55 lr=5.93e-04 tps=16320 mem=6.2G lam=0.13574631170680126 Gm=None +[s_only s1] step 1100/12000 loss=5.7902 gn=0.52 lr=5.92e-04 tps=16324 mem=6.2G lam=0.13624282553792 Gm=None +[s_only s1] step 1120/12000 loss=5.6525 gn=0.58 lr=5.92e-04 tps=16328 mem=6.2G lam=0.13670004003991684 Gm=None +[s_only s1] step 1140/12000 loss=5.7052 gn=0.52 lr=5.92e-04 tps=16332 mem=6.2G lam=0.1371044429639975 Gm=None +[s_only s1] step 1160/12000 loss=5.6833 gn=0.52 lr=5.91e-04 tps=16335 mem=6.2G lam=0.13758571663250527 Gm=None +[s_only s1] step 1180/12000 loss=5.7231 gn=0.54 lr=5.91e-04 tps=16340 mem=6.2G lam=0.13802034252633652 Gm=None +[s_only s1] step 1200/12000 loss=5.6520 gn=0.55 lr=5.90e-04 tps=16344 mem=6.2G lam=0.13843956217169762 Gm=None +[s_only s1] step 1220/12000 loss=5.8461 gn=0.57 lr=5.90e-04 tps=16349 mem=6.2G lam=0.1389069234331449 Gm=None +[s_only s1] step 1240/12000 loss=5.6091 gn=0.54 lr=5.90e-04 tps=16354 mem=6.2G lam=0.13948270430167517 Gm=None +[s_only s1] step 1260/12000 loss=5.5989 gn=0.61 lr=5.89e-04 tps=16357 mem=6.2G lam=0.13999891777833304 Gm=None +[s_only s1] step 1280/12000 loss=5.4953 gn=0.54 lr=5.89e-04 tps=16359 mem=6.2G lam=0.1404899936169386 Gm=None +[s_only s1] step 1300/12000 loss=5.6312 gn=0.53 lr=5.89e-04 tps=16361 mem=6.2G lam=0.14096317067742348 Gm=None +[s_only s1] step 1320/12000 loss=5.4846 gn=0.51 lr=5.88e-04 tps=16364 mem=6.2G lam=0.14142730024953684 Gm=None +[s_only s1] step 1340/12000 loss=5.7495 gn=0.59 lr=5.88e-04 tps=16366 mem=6.2G lam=0.1419028239324689 Gm=None +[s_only s1] step 1360/12000 loss=5.5737 gn=0.60 lr=5.87e-04 tps=16368 mem=6.2G lam=0.14241558189193407 Gm=None +[s_only s1] step 1380/12000 loss=5.4946 gn=0.59 lr=5.87e-04 tps=16370 mem=6.2G lam=0.14287282464404902 Gm=None +[s_only s1] step 1400/12000 loss=5.6923 gn=0.54 lr=5.86e-04 tps=16372 mem=6.2G lam=0.1434031662841638 Gm=None +[s_only s1] step 1420/12000 loss=5.4647 gn=0.55 lr=5.86e-04 tps=16375 mem=6.2G lam=0.1438761449729403 Gm=None +[s_only s1] step 1440/12000 loss=5.5949 gn=0.59 lr=5.85e-04 tps=16378 mem=6.2G lam=0.14437459564457336 Gm=None +[s_only s1] step 1460/12000 loss=5.6263 gn=0.66 lr=5.85e-04 tps=16380 mem=6.2G lam=0.14482611448814472 Gm=None +[s_only s1] step 1480/12000 loss=5.4944 gn=0.56 lr=5.84e-04 tps=16382 mem=6.2G lam=0.14530750301977 Gm=None +[s_only s1] step 1500/12000 loss=5.3239 gn=0.66 lr=5.84e-04 tps=16384 mem=6.2G lam=0.1457845807696382 Gm=None +[s_only s1] EVAL step 1500: val_loss=5.5236 val_ppl=250.52 +[s_only s1] step 1520/12000 loss=5.5054 gn=0.57 lr=5.83e-04 tps=16306 mem=6.2G lam=0.14629698637872934 Gm=None +[s_only s1] step 1540/12000 loss=5.5262 gn=0.66 lr=5.83e-04 tps=16308 mem=6.2G lam=0.1468135400985678 Gm=None +[s_only s1] step 1560/12000 loss=5.4769 gn=0.58 lr=5.82e-04 tps=16310 mem=6.2G lam=0.1472929505010446 Gm=None +[s_only s1] step 1580/12000 loss=5.1674 gn=0.55 lr=5.82e-04 tps=16313 mem=6.2G lam=0.1477924439435204 Gm=None +[s_only s1] step 1600/12000 loss=5.2524 gn=0.61 lr=5.81e-04 tps=16316 mem=6.2G lam=0.14826941831658283 Gm=None +[s_only s1] step 1620/12000 loss=5.2591 gn=0.59 lr=5.81e-04 tps=16319 mem=6.2G lam=0.14885611552745104 Gm=None +[s_only s1] step 1640/12000 loss=5.2684 gn=0.55 lr=5.80e-04 tps=16322 mem=6.2G lam=0.1493890251343449 Gm=None +[s_only s1] step 1660/12000 loss=5.4059 gn=0.62 lr=5.80e-04 tps=16325 mem=6.2G lam=0.14986131712794304 Gm=None +[s_only s1] step 1680/12000 loss=5.2008 gn=0.57 lr=5.79e-04 tps=16328 mem=6.2G lam=0.150368538685143 Gm=None +[s_only s1] step 1700/12000 loss=5.2903 gn=0.56 lr=5.79e-04 tps=16331 mem=6.2G lam=0.15082499819497266 Gm=None +[s_only s1] step 1720/12000 loss=5.3582 gn=0.60 lr=5.78e-04 tps=16334 mem=6.2G lam=0.1512852944433689 Gm=None +[s_only s1] step 1740/12000 loss=5.2952 gn=0.57 lr=5.78e-04 tps=16337 mem=6.2G lam=0.151852834969759 Gm=None +[s_only s1] step 1760/12000 loss=5.3256 gn=0.60 lr=5.77e-04 tps=16339 mem=6.2G lam=0.15237282123416662 Gm=None +[s_only s1] step 1780/12000 loss=5.3401 gn=0.57 lr=5.76e-04 tps=16342 mem=6.2G lam=0.1528942659497261 Gm=None +[s_only s1] step 1800/12000 loss=5.4556 gn=0.60 lr=5.76e-04 tps=16345 mem=6.2G lam=0.15338196170826754 Gm=None +[s_only s1] step 1820/12000 loss=5.4394 gn=0.58 lr=5.75e-04 tps=16347 mem=6.2G lam=0.153898353377978 Gm=None +[s_only s1] step 1840/12000 loss=5.2026 gn=0.58 lr=5.75e-04 tps=16349 mem=6.2G lam=0.15443131203452745 Gm=None +[s_only s1] step 1860/12000 loss=5.3042 gn=0.61 lr=5.74e-04 tps=16351 mem=6.2G lam=0.15492957613120475 Gm=None +[s_only s1] step 1880/12000 loss=5.1300 gn=0.61 lr=5.73e-04 tps=16353 mem=6.2G lam=0.15545474427441755 Gm=None +[s_only s1] step 1900/12000 loss=5.3719 gn=0.63 lr=5.73e-04 tps=16355 mem=6.2G lam=0.15595518549283346 Gm=None +[s_only s1] step 1920/12000 loss=5.4032 gn=0.68 lr=5.72e-04 tps=16357 mem=6.2G lam=0.1564826499670744 Gm=None +[s_only s1] step 1940/12000 loss=5.4913 gn=0.63 lr=5.72e-04 tps=16359 mem=6.2G lam=0.15702887500325838 Gm=None +[s_only s1] step 1960/12000 loss=5.1443 gn=0.59 lr=5.71e-04 tps=16361 mem=6.2G lam=0.1576035919909676 Gm=None +[s_only s1] step 1980/12000 loss=5.2561 gn=0.62 lr=5.70e-04 tps=16363 mem=6.2G lam=0.15813736959050098 Gm=None +[s_only s1] step 2000/12000 loss=5.3350 gn=0.60 lr=5.70e-04 tps=16364 mem=6.2G lam=0.15865843867262205 Gm=None +[s_only s1] EVAL step 2000: val_loss=5.3609 val_ppl=212.91 +[s_only s1] step 2020/12000 loss=5.2255 gn=0.60 lr=5.69e-04 tps=16305 mem=6.2G lam=0.15921892349918684 Gm=None +[s_only s1] step 2040/12000 loss=5.3135 gn=0.63 lr=5.68e-04 tps=16307 mem=6.2G lam=0.15979720714191595 Gm=None +[s_only s1] step 2060/12000 loss=5.2643 gn=0.60 lr=5.68e-04 tps=16309 mem=6.2G lam=0.16033665463328362 Gm=None +[s_only s1] step 2080/12000 loss=5.2812 gn=0.64 lr=5.67e-04 tps=16311 mem=6.2G lam=0.16084065133084854 Gm=None +[s_only s1] step 2100/12000 loss=5.3098 gn=0.60 lr=5.66e-04 tps=16313 mem=6.2G lam=0.16127128433436155 Gm=None +[s_only s1] step 2120/12000 loss=5.1718 gn=0.61 lr=5.65e-04 tps=16316 mem=6.2G lam=0.16179857278863588 Gm=None +[s_only s1] step 2140/12000 loss=5.1545 gn=0.66 lr=5.65e-04 tps=16318 mem=6.2G lam=0.16237957713504633 Gm=None +[s_only s1] step 2160/12000 loss=5.1979 gn=0.63 lr=5.64e-04 tps=16320 mem=6.2G lam=0.16286179361244044 Gm=None +[s_only s1] step 2180/12000 loss=5.2275 gn=0.63 lr=5.63e-04 tps=16321 mem=6.2G lam=0.16340992165108523 Gm=None +[s_only s1] step 2200/12000 loss=5.1612 gn=0.60 lr=5.63e-04 tps=16323 mem=6.2G lam=0.1639583039407929 Gm=None +[s_only s1] step 2220/12000 loss=5.3056 gn=0.62 lr=5.62e-04 tps=16321 mem=6.2G lam=0.16446124607076248 Gm=None +[s_only s1] step 2240/12000 loss=5.0953 gn=0.61 lr=5.61e-04 tps=16318 mem=6.2G lam=0.16502089177568754 Gm=None +[s_only s1] step 2260/12000 loss=5.0766 gn=0.65 lr=5.60e-04 tps=16320 mem=6.2G lam=0.16543854027986526 Gm=None +[s_only s1] step 2280/12000 loss=5.1987 gn=0.64 lr=5.60e-04 tps=16320 mem=6.2G lam=0.16603718946377435 Gm=None +[s_only s1] step 2300/12000 loss=5.1303 gn=0.63 lr=5.59e-04 tps=16317 mem=6.2G lam=0.1665330103908976 Gm=None +[s_only s1] step 2320/12000 loss=5.0140 gn=0.63 lr=5.58e-04 tps=16316 mem=6.2G lam=0.16711237064252296 Gm=None +[s_only s1] step 2340/12000 loss=5.1916 gn=0.62 lr=5.57e-04 tps=16315 mem=6.2G lam=0.16765353952844939 Gm=None +[s_only s1] step 2360/12000 loss=5.2579 gn=0.71 lr=5.57e-04 tps=16314 mem=6.2G lam=0.16824979707598686 Gm=None +[s_only s1] step 2380/12000 loss=5.2061 gn=0.67 lr=5.56e-04 tps=16315 mem=6.2G lam=0.16881747792164484 Gm=None +[s_only s1] step 2400/12000 loss=5.1287 gn=0.70 lr=5.55e-04 tps=16316 mem=6.2G lam=0.16938115749508142 Gm=None +[s_only s1] step 2420/12000 loss=5.1588 gn=0.59 lr=5.54e-04 tps=16318 mem=6.2G lam=0.16997747868299484 Gm=None +[s_only s1] step 2440/12000 loss=5.1443 gn=0.68 lr=5.53e-04 tps=16319 mem=6.2G lam=0.17054113652557135 Gm=None +[s_only s1] step 2460/12000 loss=5.1782 gn=0.66 lr=5.53e-04 tps=16320 mem=6.2G lam=0.17114587097118297 Gm=None +[s_only s1] step 2480/12000 loss=5.1523 gn=0.66 lr=5.52e-04 tps=16319 mem=6.2G lam=0.17173184982190529 Gm=None +[s_only s1] step 2500/12000 loss=5.1685 gn=0.64 lr=5.51e-04 tps=16319 mem=6.2G lam=0.17231567110866308 Gm=None +[s_only s1] EVAL step 2500: val_loss=5.2007 val_ppl=181.40 +[s_only s1] step 2520/12000 loss=5.2574 gn=0.72 lr=5.50e-04 tps=16272 mem=6.2G lam=0.172867512019972 Gm=None +[s_only s1] step 2540/12000 loss=5.2123 gn=0.64 lr=5.49e-04 tps=16273 mem=6.2G lam=0.17346866025278965 Gm=None +[s_only s1] step 2560/12000 loss=5.1843 gn=0.66 lr=5.48e-04 tps=16272 mem=6.2G lam=0.17405668056259552 Gm=None +[s_only s1] step 2580/12000 loss=5.0134 gn=0.63 lr=5.48e-04 tps=16271 mem=6.2G lam=0.17460820513466993 Gm=None +[s_only s1] step 2600/12000 loss=5.1165 gn=0.69 lr=5.47e-04 tps=16271 mem=6.2G lam=0.175164177082479 Gm=None +[s_only s1] step 2620/12000 loss=5.0169 gn=0.70 lr=5.46e-04 tps=16269 mem=6.2G lam=0.17576510272920132 Gm=None +[s_only s1] step 2640/12000 loss=5.2141 gn=0.70 lr=5.45e-04 tps=16269 mem=6.2G lam=0.1763387325530251 Gm=None +[s_only s1] step 2660/12000 loss=5.0438 gn=0.65 lr=5.44e-04 tps=16267 mem=6.2G lam=0.17688615278651318 Gm=None +[s_only s1] step 2680/12000 loss=5.0352 gn=0.69 lr=5.43e-04 tps=16266 mem=6.2G lam=0.17741507881631455 Gm=None +[s_only s1] step 2700/12000 loss=5.1916 gn=0.73 lr=5.42e-04 tps=16263 mem=6.2G lam=0.1779287482301394 Gm=None +[s_only s1] step 2720/12000 loss=5.0429 gn=0.69 lr=5.41e-04 tps=16261 mem=6.2G lam=0.1784988402699431 Gm=None +[s_only s1] step 2740/12000 loss=5.1965 gn=0.75 lr=5.41e-04 tps=16260 mem=6.2G lam=0.17905204960455498 Gm=None +[s_only s1] step 2760/12000 loss=5.0314 gn=0.68 lr=5.40e-04 tps=16259 mem=6.2G lam=0.17957370976607004 Gm=None +[s_only s1] step 2780/12000 loss=5.1443 gn=0.67 lr=5.39e-04 tps=16259 mem=6.2G lam=0.18005858516941467 Gm=None +[s_only s1] step 2800/12000 loss=4.9148 gn=0.68 lr=5.38e-04 tps=16260 mem=6.2G lam=0.18056016570578018 Gm=None +[s_only s1] step 2820/12000 loss=5.2103 gn=0.67 lr=5.37e-04 tps=16262 mem=6.2G lam=0.18117344876130423 Gm=None +[s_only s1] step 2840/12000 loss=5.0685 gn=0.68 lr=5.36e-04 tps=16263 mem=6.2G lam=0.1817519503335158 Gm=None +[s_only s1] step 2860/12000 loss=4.9971 gn=0.66 lr=5.35e-04 tps=16265 mem=6.2G lam=0.1822829246520996 Gm=None +[s_only s1] step 2880/12000 loss=5.0624 gn=0.72 lr=5.34e-04 tps=16266 mem=6.2G lam=0.18283796838174263 Gm=None +[s_only s1] step 2900/12000 loss=5.0510 gn=0.65 lr=5.33e-04 tps=16268 mem=6.2G lam=0.18338641648491225 Gm=None +[s_only s1] step 2920/12000 loss=5.0224 gn=0.73 lr=5.32e-04 tps=16269 mem=6.2G lam=0.18394001200795174 Gm=None +[s_only s1] step 2940/12000 loss=5.0989 gn=0.74 lr=5.31e-04 tps=16271 mem=6.2G lam=0.18452770542353392 Gm=None +[s_only s1] step 2960/12000 loss=4.8663 gn=0.66 lr=5.30e-04 tps=16272 mem=6.2G lam=0.18501598419000706 Gm=None +[s_only s1] step 2980/12000 loss=4.9882 gn=0.67 lr=5.29e-04 tps=16274 mem=6.2G lam=0.185632831727465 Gm=None +[s_only s1] step 3000/12000 loss=4.9812 gn=0.70 lr=5.28e-04 tps=16276 mem=6.2G lam=0.18626184264818826 Gm=None +[s_only s1] EVAL step 3000: val_loss=5.0551 val_ppl=156.82 +[s_only s1] step 3020/12000 loss=5.0230 gn=0.68 lr=5.27e-04 tps=16238 mem=6.2G lam=0.1868814124415318 Gm=None +[s_only s1] step 3040/12000 loss=4.8433 gn=0.64 lr=5.26e-04 tps=16240 mem=6.2G lam=0.18740633316338062 Gm=None +[s_only s1] step 3060/12000 loss=5.0058 gn=0.72 lr=5.25e-04 tps=16241 mem=6.2G lam=0.18796476597587267 Gm=None +[s_only s1] step 3080/12000 loss=4.8633 gn=0.68 lr=5.24e-04 tps=16243 mem=6.2G lam=0.18859058059751987 Gm=None +[s_only s1] step 3100/12000 loss=5.0337 gn=0.74 lr=5.23e-04 tps=16246 mem=6.2G lam=0.18914883149166903 Gm=None +[s_only s1] step 3120/12000 loss=4.9910 gn=0.68 lr=5.22e-04 tps=16248 mem=6.2G lam=0.1896604880069693 Gm=None +[s_only s1] step 3140/12000 loss=4.9562 gn=0.74 lr=5.21e-04 tps=16250 mem=6.2G lam=0.19020988481740156 Gm=None +[s_only s1] step 3160/12000 loss=4.9772 gn=0.70 lr=5.20e-04 tps=16252 mem=6.2G lam=0.19080132991075516 Gm=None +[s_only s1] step 3180/12000 loss=5.0105 gn=0.75 lr=5.19e-04 tps=16253 mem=6.2G lam=0.19126997329294682 Gm=None +[s_only s1] step 3200/12000 loss=4.7925 gn=0.68 lr=5.18e-04 tps=16254 mem=6.2G lam=0.19178635037193695 Gm=None +[s_only s1] step 3220/12000 loss=5.0055 gn=0.81 lr=5.17e-04 tps=16256 mem=6.2G lam=0.1923125380029281 Gm=None +[s_only s1] step 3240/12000 loss=5.0382 gn=0.71 lr=5.16e-04 tps=16258 mem=6.2G lam=0.1928144202878078 Gm=None +[s_only s1] step 3260/12000 loss=4.9082 gn=0.72 lr=5.15e-04 tps=16259 mem=6.2G lam=0.19340320397168398 Gm=None +[s_only s1] step 3280/12000 loss=4.8452 gn=0.68 lr=5.14e-04 tps=16261 mem=6.2G lam=0.19391346598664919 Gm=None +[s_only s1] step 3300/12000 loss=5.0328 gn=0.79 lr=5.13e-04 tps=16263 mem=6.2G lam=0.19451883838822445 Gm=None +[s_only s1] step 3320/12000 loss=5.0610 gn=0.79 lr=5.12e-04 tps=16265 mem=6.2G lam=0.19497454476853213 Gm=None +[s_only s1] step 3340/12000 loss=4.8940 gn=0.67 lr=5.11e-04 tps=16266 mem=6.2G lam=0.19549941613028446 Gm=None +[s_only s1] step 3360/12000 loss=4.8054 gn=0.69 lr=5.10e-04 tps=16268 mem=6.2G lam=0.19609533064067364 Gm=None +[s_only s1] step 3380/12000 loss=5.0814 gn=0.66 lr=5.09e-04 tps=16270 mem=6.2G lam=0.1965920760606726 Gm=None +[s_only s1] step 3400/12000 loss=5.1662 gn=0.70 lr=5.08e-04 tps=16271 mem=6.2G lam=0.19719922076910734 Gm=None +[s_only s1] step 3420/12000 loss=5.0512 gn=0.73 lr=5.07e-04 tps=16273 mem=6.2G lam=0.19779709385087094 Gm=None +[s_only s1] step 3440/12000 loss=4.8687 gn=0.74 lr=5.06e-04 tps=16274 mem=6.2G lam=0.19838039732227722 Gm=None +[s_only s1] step 3460/12000 loss=4.9895 gn=0.73 lr=5.05e-04 tps=16275 mem=6.2G lam=0.19891245880474648 Gm=None +[s_only s1] step 3480/12000 loss=4.8902 gn=0.73 lr=5.03e-04 tps=16277 mem=6.2G lam=0.19942132538805404 Gm=None +[s_only s1] step 3500/12000 loss=4.8951 gn=0.73 lr=5.02e-04 tps=16278 mem=6.2G lam=0.1999991238117218 Gm=None +[s_only s1] EVAL step 3500: val_loss=4.9964 val_ppl=147.88 +[s_only s1] step 3520/12000 loss=4.8404 gn=0.70 lr=5.01e-04 tps=16246 mem=6.2G lam=0.20054473169147968 Gm=None +[s_only s1] step 3540/12000 loss=4.9551 gn=0.73 lr=5.00e-04 tps=16247 mem=6.2G lam=0.20109170333792767 Gm=None +[s_only s1] step 3560/12000 loss=4.9349 gn=0.76 lr=4.99e-04 tps=16247 mem=6.2G lam=0.20170255781461796 Gm=None +[s_only s1] step 3580/12000 loss=4.9839 gn=0.73 lr=4.98e-04 tps=16248 mem=6.2G lam=0.20238710846751928 Gm=None +[s_only s1] step 3600/12000 loss=5.0498 gn=0.78 lr=4.97e-04 tps=16249 mem=6.2G lam=0.20292103787263235 Gm=None +[s_only s1] step 3620/12000 loss=4.9221 gn=0.72 lr=4.96e-04 tps=16250 mem=6.2G lam=0.20331392840792736 Gm=None +[s_only s1] step 3640/12000 loss=4.7991 gn=0.72 lr=4.94e-04 tps=16250 mem=6.2G lam=0.20394288407017788 Gm=None +[s_only s1] step 3660/12000 loss=4.7452 gn=0.72 lr=4.93e-04 tps=16251 mem=6.2G lam=0.20456965640187263 Gm=None +[s_only s1] step 3680/12000 loss=4.7519 gn=0.78 lr=4.92e-04 tps=16251 mem=6.2G lam=0.20514997405310473 Gm=None +[s_only s1] step 3700/12000 loss=4.9660 gn=0.73 lr=4.91e-04 tps=16252 mem=6.2G lam=0.20566142257303 Gm=None +[s_only s1] step 3720/12000 loss=4.9160 gn=0.82 lr=4.90e-04 tps=16252 mem=6.2G lam=0.20612394716590643 Gm=None +[s_only s1] step 3740/12000 loss=4.9027 gn=0.73 lr=4.89e-04 tps=16253 mem=6.2G lam=0.20660605188459158 Gm=None +[s_only s1] step 3760/12000 loss=4.8831 gn=0.77 lr=4.88e-04 tps=16253 mem=6.2G lam=0.20713374329109988 Gm=None +[s_only s1] step 3780/12000 loss=4.7952 gn=0.75 lr=4.86e-04 tps=16254 mem=6.2G lam=0.2076346476872762 Gm=None +[s_only s1] step 3800/12000 loss=4.7942 gn=0.75 lr=4.85e-04 tps=16254 mem=6.2G lam=0.20815570993969837 Gm=None +[s_only s1] step 3820/12000 loss=4.8684 gn=0.69 lr=4.84e-04 tps=16255 mem=6.2G lam=0.2086173960318168 Gm=None +[s_only s1] step 3840/12000 loss=4.9448 gn=0.73 lr=4.83e-04 tps=16256 mem=6.2G lam=0.20917892704407373 Gm=None +[s_only s1] step 3860/12000 loss=4.8272 gn=0.77 lr=4.82e-04 tps=16255 mem=6.2G lam=0.20966031743834415 Gm=None +[s_only s1] step 3880/12000 loss=4.6796 gn=0.70 lr=4.80e-04 tps=16256 mem=6.2G lam=0.21018888366719088 Gm=None +[s_only s1] step 3900/12000 loss=4.8031 gn=0.82 lr=4.79e-04 tps=16257 mem=6.2G lam=0.21072336720923582 Gm=None +[s_only s1] step 3920/12000 loss=4.9224 gn=0.85 lr=4.78e-04 tps=16257 mem=6.2G lam=0.2112676159789165 Gm=None +[s_only s1] step 3940/12000 loss=4.7551 gn=0.73 lr=4.77e-04 tps=16257 mem=6.2G lam=0.21177825517952442 Gm=None +[s_only s1] step 3960/12000 loss=4.9855 gn=0.75 lr=4.76e-04 tps=16257 mem=6.2G lam=0.21231603622436523 Gm=None +[s_only s1] step 3980/12000 loss=5.0297 gn=0.78 lr=4.74e-04 tps=16257 mem=6.2G lam=0.21281759782383838 Gm=None +[s_only s1] step 4000/12000 loss=4.9292 gn=0.76 lr=4.73e-04 tps=16258 mem=6.2G lam=0.2132228616004189 Gm=None +[s_only s1] EVAL step 4000: val_loss=4.8644 val_ppl=129.59 +[s_only s1] step 4020/12000 loss=4.5777 gn=0.69 lr=4.72e-04 tps=16228 mem=6.2G lam=0.2137232068926096 Gm=None +[s_only s1] step 4040/12000 loss=4.8574 gn=0.79 lr=4.71e-04 tps=16229 mem=6.2G lam=0.21418657122800747 Gm=None +[s_only s1] step 4060/12000 loss=4.9170 gn=0.78 lr=4.70e-04 tps=16230 mem=6.2G lam=0.21473546574513117 Gm=None +[s_only s1] step 4080/12000 loss=4.7203 gn=0.77 lr=4.68e-04 tps=16231 mem=6.2G lam=0.2152970445652803 Gm=None +[s_only s1] step 4100/12000 loss=4.8417 gn=0.78 lr=4.67e-04 tps=16232 mem=6.2G lam=0.21576747857034206 Gm=None +[s_only s1] step 4120/12000 loss=4.8391 gn=0.79 lr=4.66e-04 tps=16233 mem=6.2G lam=0.2162958082432548 Gm=None +[s_only s1] step 4140/12000 loss=4.9117 gn=0.75 lr=4.65e-04 tps=16233 mem=6.2G lam=0.21682898017267385 Gm=None +[s_only s1] step 4160/12000 loss=4.7417 gn=0.75 lr=4.63e-04 tps=16233 mem=6.2G lam=0.21729009101788202 Gm=None +[s_only s1] step 4180/12000 loss=4.9840 gn=0.79 lr=4.62e-04 tps=16234 mem=6.2G lam=0.21780867657313743 Gm=None +[s_only s1] step 4200/12000 loss=4.6923 gn=0.77 lr=4.61e-04 tps=16235 mem=6.2G lam=0.21833004957685867 Gm=None +[s_only s1] step 4220/12000 loss=4.7590 gn=0.75 lr=4.60e-04 tps=16235 mem=6.2G lam=0.21881539188325405 Gm=None +[s_only s1] step 4240/12000 loss=4.7196 gn=0.72 lr=4.58e-04 tps=16236 mem=6.2G lam=0.21940650790929794 Gm=None +[s_only s1] step 4260/12000 loss=4.7439 gn=0.73 lr=4.57e-04 tps=16237 mem=6.2G lam=0.21987265317390361 Gm=None +[s_only s1] step 4280/12000 loss=4.8704 gn=0.81 lr=4.56e-04 tps=16238 mem=6.2G lam=0.2204120603079597 Gm=None +[s_only s1] step 4300/12000 loss=4.9375 gn=0.74 lr=4.54e-04 tps=16238 mem=6.2G lam=0.22104367365439734 Gm=None +[s_only s1] step 4320/12000 loss=4.7194 gn=0.73 lr=4.53e-04 tps=16239 mem=6.2G lam=0.221507434733212 Gm=None +[s_only s1] step 4340/12000 loss=4.7748 gn=0.75 lr=4.52e-04 tps=16239 mem=6.2G lam=0.22193144106616577 Gm=None +[s_only s1] step 4360/12000 loss=4.8517 gn=0.78 lr=4.51e-04 tps=16240 mem=6.2G lam=0.22243090563764176 Gm=None +[s_only s1] step 4380/12000 loss=4.7132 gn=0.76 lr=4.49e-04 tps=16240 mem=6.2G lam=0.22293827403336763 Gm=None +[s_only s1] step 4400/12000 loss=4.8309 gn=0.77 lr=4.48e-04 tps=16240 mem=6.2G lam=0.2234069760888815 Gm=None +[s_only s1] step 4420/12000 loss=4.8139 gn=0.80 lr=4.47e-04 tps=16241 mem=6.2G lam=0.22393617716928324 Gm=None +[s_only s1] step 4440/12000 loss=4.7198 gn=0.79 lr=4.45e-04 tps=16242 mem=6.2G lam=0.22445128268251816 Gm=None +[s_only s1] step 4460/12000 loss=4.5363 gn=0.76 lr=4.44e-04 tps=16243 mem=6.2G lam=0.22499836919208369 Gm=None +[s_only s1] step 4480/12000 loss=4.6838 gn=0.76 lr=4.43e-04 tps=16243 mem=6.2G lam=0.22555278396854797 Gm=None +[s_only s1] step 4500/12000 loss=4.9712 gn=0.77 lr=4.42e-04 tps=16244 mem=6.2G lam=0.22603931122769913 Gm=None +[s_only s1] EVAL step 4500: val_loss=4.7724 val_ppl=118.21 +[s_only s1] step 4520/12000 loss=4.8329 gn=0.75 lr=4.40e-04 tps=16218 mem=6.2G lam=0.22653679456561804 Gm=None +[s_only s1] step 4540/12000 loss=4.8028 gn=0.76 lr=4.39e-04 tps=16219 mem=6.2G lam=0.22691284585744143 Gm=None +[s_only s1] step 4560/12000 loss=4.6710 gn=0.77 lr=4.38e-04 tps=16219 mem=6.2G lam=0.22744173867007098 Gm=None +[s_only s1] step 4580/12000 loss=4.8364 gn=0.76 lr=4.36e-04 tps=16220 mem=6.2G lam=0.22795684480418762 Gm=None +[s_only s1] step 4600/12000 loss=4.6813 gn=0.81 lr=4.35e-04 tps=16220 mem=6.2G lam=0.22850086353719234 Gm=None +[s_only s1] step 4620/12000 loss=4.6851 gn=0.92 lr=4.34e-04 tps=16221 mem=6.2G lam=0.22903884605815014 Gm=None +[s_only s1] step 4640/12000 loss=4.8456 gn=0.76 lr=4.32e-04 tps=16222 mem=6.2G lam=0.22954097048689923 Gm=None +[s_only s1] step 4660/12000 loss=4.7249 gn=0.77 lr=4.31e-04 tps=16222 mem=6.2G lam=0.2300376029064258 Gm=None +[s_only s1] step 4680/12000 loss=4.8267 gn=0.78 lr=4.30e-04 tps=16223 mem=6.2G lam=0.23048847634345293 Gm=None +[s_only s1] step 4700/12000 loss=4.7745 gn=0.85 lr=4.28e-04 tps=16223 mem=6.2G lam=0.2309194632495443 Gm=None +[s_only s1] step 4720/12000 loss=4.7959 gn=0.89 lr=4.27e-04 tps=16224 mem=6.2G lam=0.23139036322633424 Gm=None +[s_only s1] step 4740/12000 loss=4.7494 gn=0.84 lr=4.26e-04 tps=16224 mem=6.2G lam=0.23179769857476154 Gm=None +[s_only s1] step 4760/12000 loss=4.8359 gn=0.80 lr=4.24e-04 tps=16225 mem=6.2G lam=0.23223973655452332 Gm=None +[s_only s1] step 4780/12000 loss=4.6215 gn=0.90 lr=4.23e-04 tps=16226 mem=6.2G lam=0.23278148161868253 Gm=None +[s_only s1] step 4800/12000 loss=4.5888 gn=0.77 lr=4.22e-04 tps=16227 mem=6.2G lam=0.23327693218986192 Gm=None +[s_only s1] step 4820/12000 loss=4.7603 gn=0.74 lr=4.20e-04 tps=16227 mem=6.2G lam=0.23378719513614973 Gm=None +[s_only s1] step 4840/12000 loss=4.7747 gn=0.84 lr=4.19e-04 tps=16228 mem=6.2G lam=0.2342853986968597 Gm=None +[s_only s1] step 4860/12000 loss=4.8483 gn=0.76 lr=4.18e-04 tps=16228 mem=6.2G lam=0.2348183145125707 Gm=None +[s_only s1] step 4880/12000 loss=4.6160 gn=0.80 lr=4.16e-04 tps=16228 mem=6.2G lam=0.23521187342703342 Gm=None +[s_only s1] step 4900/12000 loss=4.7345 gn=0.77 lr=4.15e-04 tps=16228 mem=6.2G lam=0.23574173512558141 Gm=None +[s_only s1] step 4920/12000 loss=4.7616 gn=0.80 lr=4.13e-04 tps=16229 mem=6.2G lam=0.23624510938922563 Gm=None +[s_only s1] step 4940/12000 loss=4.5235 gn=0.83 lr=4.12e-04 tps=16229 mem=6.2G lam=0.236639233926932 Gm=None +[s_only s1] step 4960/12000 loss=4.7173 gn=0.92 lr=4.11e-04 tps=16230 mem=6.2G lam=0.23703490725407997 Gm=None +[s_only s1] step 4980/12000 loss=4.8506 gn=0.83 lr=4.09e-04 tps=16231 mem=6.2G lam=0.23747386752317348 Gm=None +[s_only s1] step 5000/12000 loss=4.7302 gn=0.82 lr=4.08e-04 tps=16232 mem=6.2G lam=0.237977827899158 Gm=None +[s_only s1] EVAL step 5000: val_loss=4.7448 val_ppl=114.99 +[s_only s1] step 5020/12000 loss=4.6815 gn=0.92 lr=4.07e-04 tps=16208 mem=6.2G lam=0.23852366767823696 Gm=None +[s_only s1] step 5040/12000 loss=4.6233 gn=0.79 lr=4.05e-04 tps=16209 mem=6.2G lam=0.23905244500686726 Gm=None +[s_only s1] step 5060/12000 loss=4.7608 gn=0.77 lr=4.04e-04 tps=16210 mem=6.2G lam=0.2395200508957108 Gm=None +[s_only s1] step 5080/12000 loss=4.7529 gn=0.81 lr=4.02e-04 tps=16210 mem=6.2G lam=0.2399194591368238 Gm=None +[s_only s1] step 5100/12000 loss=4.6451 gn=0.85 lr=4.01e-04 tps=16211 mem=6.2G lam=0.24033786139140526 Gm=None +[s_only s1] step 5120/12000 loss=4.7651 gn=0.81 lr=4.00e-04 tps=16212 mem=6.2G lam=0.24083295185118914 Gm=None +[s_only s1] step 5140/12000 loss=4.7322 gn=0.80 lr=3.98e-04 tps=16212 mem=6.2G lam=0.24127883557230234 Gm=None +[s_only s1] step 5160/12000 loss=4.7390 gn=0.86 lr=3.97e-04 tps=16213 mem=6.2G lam=0.24180774359653393 Gm=None +[s_only s1] step 5180/12000 loss=4.7991 gn=0.95 lr=3.95e-04 tps=16213 mem=6.2G lam=0.24235093065847954 Gm=None +[s_only s1] step 5200/12000 loss=4.7378 gn=0.82 lr=3.94e-04 tps=16214 mem=6.2G lam=0.24274292991807064 Gm=None +[s_only s1] step 5220/12000 loss=4.5515 gn=0.81 lr=3.93e-04 tps=16215 mem=6.2G lam=0.24322882946580648 Gm=None +[s_only s1] step 5240/12000 loss=4.8880 gn=0.88 lr=3.91e-04 tps=16215 mem=6.2G lam=0.24372390968104204 Gm=None +[s_only s1] step 5260/12000 loss=4.6336 gn=0.80 lr=3.90e-04 tps=16216 mem=6.2G lam=0.24423549976199865 Gm=None +[s_only s1] step 5280/12000 loss=4.5322 gn=0.80 lr=3.88e-04 tps=16217 mem=6.2G lam=0.24468839230636755 Gm=None +[s_only s1] step 5300/12000 loss=4.6660 gn=0.82 lr=3.87e-04 tps=16217 mem=6.2G lam=0.24515169827888408 Gm=None +[s_only s1] step 5320/12000 loss=4.6419 gn=0.85 lr=3.86e-04 tps=16218 mem=6.2G lam=0.2455571498721838 Gm=None +[s_only s1] step 5340/12000 loss=4.7444 gn=0.83 lr=3.84e-04 tps=16218 mem=6.2G lam=0.2460312486315767 Gm=None +[s_only s1] step 5360/12000 loss=4.5006 gn=0.77 lr=3.83e-04 tps=16219 mem=6.2G lam=0.24649443663656712 Gm=None +[s_only s1] step 5380/12000 loss=4.5476 gn=0.88 lr=3.81e-04 tps=16219 mem=6.2G lam=0.2468827199190855 Gm=None +[s_only s1] step 5400/12000 loss=4.5644 gn=0.79 lr=3.80e-04 tps=16220 mem=6.2G lam=0.24734110354135433 Gm=None +[s_only s1] step 5420/12000 loss=4.6420 gn=0.78 lr=3.79e-04 tps=16221 mem=6.2G lam=0.24781132334222397 Gm=None +[s_only s1] step 5440/12000 loss=4.7810 gn=0.83 lr=3.77e-04 tps=16222 mem=6.2G lam=0.24836311272035042 Gm=None +[s_only s1] step 5460/12000 loss=4.6637 gn=0.84 lr=3.76e-04 tps=16222 mem=6.2G lam=0.24892379343509674 Gm=None +[s_only s1] step 5480/12000 loss=4.6675 gn=0.81 lr=3.74e-04 tps=16223 mem=6.2G lam=0.2494250281403462 Gm=None +[s_only s1] step 5500/12000 loss=4.6465 gn=0.84 lr=3.73e-04 tps=16223 mem=6.2G lam=0.24991634860634804 Gm=None +[s_only s1] EVAL step 5500: val_loss=4.6839 val_ppl=108.19 +[s_only s1] step 5520/12000 loss=4.5056 gn=0.84 lr=3.72e-04 tps=16202 mem=6.2G lam=0.25030915811657906 Gm=None +[s_only s1] step 5540/12000 loss=4.6566 gn=0.87 lr=3.70e-04 tps=16203 mem=6.2G lam=0.25082471687346697 Gm=None +[s_only s1] step 5560/12000 loss=4.6428 gn=0.81 lr=3.69e-04 tps=16203 mem=6.2G lam=0.2511963254461686 Gm=None +[s_only s1] step 5580/12000 loss=4.5032 gn=0.84 lr=3.67e-04 tps=16204 mem=6.2G lam=0.2516166179751356 Gm=None +[s_only s1] step 5600/12000 loss=4.7097 gn=0.88 lr=3.66e-04 tps=16204 mem=6.2G lam=0.2521317082767685 Gm=None +[s_only s1] step 5620/12000 loss=4.7062 gn=0.86 lr=3.64e-04 tps=16205 mem=6.2G lam=0.25256990641355515 Gm=None +[s_only s1] step 5640/12000 loss=4.6473 gn=0.81 lr=3.63e-04 tps=16205 mem=6.2G lam=0.25302438189586 Gm=None +[s_only s1] step 5660/12000 loss=4.7584 gn=0.88 lr=3.62e-04 tps=16206 mem=6.2G lam=0.2534272037446499 Gm=None +[s_only s1] step 5680/12000 loss=4.7789 gn=0.87 lr=3.60e-04 tps=16207 mem=6.2G lam=0.2538827977453669 Gm=None +[s_only s1] step 5700/12000 loss=4.6151 gn=0.78 lr=3.59e-04 tps=16207 mem=6.2G lam=0.25425380654633045 Gm=None +[s_only s1] step 5720/12000 loss=4.5079 gn=0.86 lr=3.57e-04 tps=16208 mem=6.2G lam=0.254712109764417 Gm=None +[s_only s1] step 5740/12000 loss=4.6173 gn=0.83 lr=3.56e-04 tps=16208 mem=6.2G lam=0.255151907602946 Gm=None +[s_only s1] step 5760/12000 loss=4.6787 gn=0.80 lr=3.54e-04 tps=16208 mem=6.2G lam=0.2556258620073398 Gm=None +[s_only s1] step 5780/12000 loss=4.6094 gn=0.85 lr=3.53e-04 tps=16209 mem=6.2G lam=0.2559850973387559 Gm=None +[s_only s1] step 5800/12000 loss=4.5926 gn=0.89 lr=3.52e-04 tps=16209 mem=6.2G lam=0.25638340413570404 Gm=None +[s_only s1] step 5820/12000 loss=4.5718 gn=0.81 lr=3.50e-04 tps=16210 mem=6.2G lam=0.25676216805974644 Gm=None +[s_only s1] step 5840/12000 loss=4.5583 gn=0.81 lr=3.49e-04 tps=16210 mem=6.2G lam=0.2571030988668402 Gm=None +[s_only s1] step 5860/12000 loss=4.5656 gn=0.81 lr=3.47e-04 tps=16211 mem=6.2G lam=0.25747053666661185 Gm=None +[s_only s1] step 5880/12000 loss=4.5412 gn=0.84 lr=3.46e-04 tps=16212 mem=6.2G lam=0.25787291023880243 Gm=None +[s_only s1] step 5900/12000 loss=4.5295 gn=0.82 lr=3.44e-04 tps=16212 mem=6.2G lam=0.2583777088051041 Gm=None +[s_only s1] step 5920/12000 loss=4.6413 gn=0.84 lr=3.43e-04 tps=16213 mem=6.2G lam=0.25876234006136656 Gm=None +[s_only s1] step 5940/12000 loss=4.5995 gn=0.89 lr=3.41e-04 tps=16214 mem=6.2G lam=0.2591073689982295 Gm=None +[s_only s1] step 5960/12000 loss=4.7520 gn=0.85 lr=3.40e-04 tps=16215 mem=6.2G lam=0.25953002894918126 Gm=None +[s_only s1] step 5980/12000 loss=4.5354 gn=0.78 lr=3.39e-04 tps=16215 mem=6.2G lam=0.25990053980300826 Gm=None +[s_only s1] step 6000/12000 loss=4.5790 gn=0.85 lr=3.37e-04 tps=16216 mem=6.2G lam=0.2603245973587036 Gm=None +[s_only s1] EVAL step 6000: val_loss=4.6978 val_ppl=109.71 +[s_only s1] step 6020/12000 loss=4.6426 gn=0.80 lr=3.36e-04 tps=16196 mem=6.2G lam=0.26074495911598206 Gm=None +[s_only s1] step 6040/12000 loss=4.5099 gn=0.83 lr=3.34e-04 tps=16197 mem=6.2G lam=0.2611030812064807 Gm=None +[s_only s1] step 6060/12000 loss=4.4261 gn=0.77 lr=3.33e-04 tps=16198 mem=6.2G lam=0.26152166817337275 Gm=None +[s_only s1] step 6080/12000 loss=4.4628 gn=0.87 lr=3.31e-04 tps=16198 mem=6.2G lam=0.26188795051227015 Gm=None +[s_only s1] step 6100/12000 loss=4.4726 gn=0.81 lr=3.30e-04 tps=16198 mem=6.2G lam=0.2622650337095062 Gm=None +[s_only s1] step 6120/12000 loss=4.5783 gn=0.82 lr=3.29e-04 tps=16199 mem=6.2G lam=0.2626294791698456 Gm=None +[s_only s1] step 6140/12000 loss=4.4890 gn=0.94 lr=3.27e-04 tps=16199 mem=6.2G lam=0.26305302884429693 Gm=None +[s_only s1] step 6160/12000 loss=4.4964 gn=0.89 lr=3.26e-04 tps=16200 mem=6.2G lam=0.26349415195484954 Gm=None +[s_only s1] step 6180/12000 loss=4.6249 gn=0.89 lr=3.24e-04 tps=16201 mem=6.2G lam=0.26389257827152807 Gm=None +[s_only s1] step 6200/12000 loss=4.5612 gn=0.89 lr=3.23e-04 tps=16201 mem=6.2G lam=0.2642594479645292 Gm=None +[s_only s1] step 6220/12000 loss=4.5892 gn=0.98 lr=3.21e-04 tps=16202 mem=6.2G lam=0.26466826753069955 Gm=None +[s_only s1] step 6240/12000 loss=4.5299 gn=0.92 lr=3.20e-04 tps=16202 mem=6.2G lam=0.26514975478251773 Gm=None +[s_only s1] step 6260/12000 loss=4.5662 gn=0.97 lr=3.19e-04 tps=16202 mem=6.2G lam=0.26555513062824804 Gm=None +[s_only s1] step 6280/12000 loss=4.6135 gn=0.87 lr=3.17e-04 tps=16202 mem=6.2G lam=0.26597001248349744 Gm=None +[s_only s1] step 6300/12000 loss=4.4636 gn=0.83 lr=3.16e-04 tps=16203 mem=6.2G lam=0.2663515542323391 Gm=None +[s_only s1] step 6320/12000 loss=4.6453 gn=0.89 lr=3.14e-04 tps=16203 mem=6.2G lam=0.26670451493312913 Gm=None +[s_only s1] step 6340/12000 loss=4.4820 gn=0.87 lr=3.13e-04 tps=16204 mem=6.2G lam=0.26708447095006704 Gm=None +[s_only s1] step 6360/12000 loss=4.5483 gn=0.90 lr=3.11e-04 tps=16204 mem=6.2G lam=0.2673929035663605 Gm=None +[s_only s1] step 6380/12000 loss=4.5871 gn=0.84 lr=3.10e-04 tps=16205 mem=6.2G lam=0.26778790323684615 Gm=None +[s_only s1] step 6400/12000 loss=4.5170 gn=0.95 lr=3.08e-04 tps=16206 mem=6.2G lam=0.2682243551438053 Gm=None +[s_only s1] step 6420/12000 loss=4.5933 gn=0.90 lr=3.07e-04 tps=16206 mem=6.2G lam=0.26853790258367854 Gm=None +[s_only s1] step 6440/12000 loss=4.5440 gn=0.86 lr=3.06e-04 tps=16207 mem=6.2G lam=0.26889505749568343 Gm=None +[s_only s1] step 6460/12000 loss=4.6263 gn=0.92 lr=3.04e-04 tps=16207 mem=6.2G lam=0.26922307815402746 Gm=None +[s_only s1] step 6480/12000 loss=4.4226 gn=0.88 lr=3.03e-04 tps=16208 mem=6.2G lam=0.2696030301352342 Gm=None +[s_only s1] step 6500/12000 loss=4.5131 gn=0.87 lr=3.01e-04 tps=16208 mem=6.2G lam=0.2699735634960234 Gm=None +[s_only s1] EVAL step 6500: val_loss=4.6381 val_ppl=103.35 +[s_only s1] step 6520/12000 loss=4.4886 gn=0.89 lr=3.00e-04 tps=16190 mem=6.2G lam=0.27029684589554864 Gm=None +[s_only s1] step 6540/12000 loss=4.6378 gn=0.92 lr=2.98e-04 tps=16191 mem=6.2G lam=0.2706656969773273 Gm=None +[s_only s1] step 6560/12000 loss=4.5889 gn=0.85 lr=2.97e-04 tps=16192 mem=6.2G lam=0.2710023927502334 Gm=None +[s_only s1] step 6580/12000 loss=4.5405 gn=0.86 lr=2.96e-04 tps=16192 mem=6.2G lam=0.2714395858347416 Gm=None +[s_only s1] step 6600/12000 loss=4.3470 gn=0.91 lr=2.94e-04 tps=16193 mem=6.2G lam=0.2717702404285471 Gm=None +[s_only s1] step 6620/12000 loss=4.7611 gn=0.91 lr=2.93e-04 tps=16193 mem=6.2G lam=0.27213627596696216 Gm=None +[s_only s1] step 6640/12000 loss=4.5550 gn=0.92 lr=2.91e-04 tps=16194 mem=6.2G lam=0.2725620820807914 Gm=None +[s_only s1] step 6660/12000 loss=4.5124 gn=0.90 lr=2.90e-04 tps=16194 mem=6.2G lam=0.2729447220141689 Gm=None +[s_only s1] step 6680/12000 loss=4.4943 gn=0.85 lr=2.88e-04 tps=16194 mem=6.2G lam=0.2733023804612458 Gm=None +[s_only s1] step 6700/12000 loss=4.5163 gn=0.90 lr=2.87e-04 tps=16195 mem=6.2G lam=0.27375338536997634 Gm=None +[s_only s1] step 6720/12000 loss=4.5143 gn=0.89 lr=2.86e-04 tps=16195 mem=6.2G lam=0.2741064333046476 Gm=None +[s_only s1] step 6740/12000 loss=4.5131 gn=0.85 lr=2.84e-04 tps=16195 mem=6.2G lam=0.27435077851017314 Gm=None +[s_only s1] step 6760/12000 loss=4.5687 gn=0.90 lr=2.83e-04 tps=16196 mem=6.2G lam=0.2746713110245764 Gm=None +[s_only s1] step 6780/12000 loss=4.5655 gn=0.88 lr=2.81e-04 tps=16196 mem=6.2G lam=0.2749913861043751 Gm=None +[s_only s1] step 6800/12000 loss=4.5284 gn=1.00 lr=2.80e-04 tps=16197 mem=6.2G lam=0.27527797035872936 Gm=None +[s_only s1] step 6820/12000 loss=4.6767 gn=0.96 lr=2.79e-04 tps=16197 mem=6.2G lam=0.2756367122444014 Gm=None +[s_only s1] step 6840/12000 loss=4.5713 gn=0.85 lr=2.77e-04 tps=16197 mem=6.2G lam=0.2760385259365042 Gm=None +[s_only s1] step 6860/12000 loss=4.4641 gn=0.88 lr=2.76e-04 tps=16198 mem=6.2G lam=0.27639530630161363 Gm=None +[s_only s1] step 6880/12000 loss=4.5133 gn=0.87 lr=2.74e-04 tps=16199 mem=6.2G lam=0.27673674483473104 Gm=None +[s_only s1] step 6900/12000 loss=4.4966 gn=0.98 lr=2.73e-04 tps=16199 mem=6.2G lam=0.27704178014149267 Gm=None +[s_only s1] step 6920/12000 loss=4.4332 gn=0.88 lr=2.72e-04 tps=16200 mem=6.2G lam=0.2773617124184966 Gm=None +[s_only s1] step 6940/12000 loss=4.4361 gn=0.88 lr=2.70e-04 tps=16200 mem=6.2G lam=0.2776341244267921 Gm=None +[s_only s1] step 6960/12000 loss=4.4973 gn=0.88 lr=2.69e-04 tps=16201 mem=6.2G lam=0.27790067018941045 Gm=None +[s_only s1] step 6980/12000 loss=4.3628 gn=0.86 lr=2.67e-04 tps=16201 mem=6.2G lam=0.27827820073192316 Gm=None +[s_only s1] step 7000/12000 loss=4.3637 gn=0.88 lr=2.66e-04 tps=16201 mem=6.2G lam=0.2786052088874082 Gm=None +[s_only s1] EVAL step 7000: val_loss=4.5790 val_ppl=97.42 +[s_only s1] step 7020/12000 loss=4.5281 gn=0.94 lr=2.65e-04 tps=16185 mem=6.2G lam=0.27888333471491933 Gm=None +[s_only s1] step 7040/12000 loss=4.6259 gn=0.88 lr=2.63e-04 tps=16185 mem=6.2G lam=0.2791189434938133 Gm=None +[s_only s1] step 7060/12000 loss=4.5331 gn=0.95 lr=2.62e-04 tps=16185 mem=6.2G lam=0.27952013180280727 Gm=None +[s_only s1] step 7080/12000 loss=4.4301 gn=0.91 lr=2.60e-04 tps=16186 mem=6.2G lam=0.2798736188560724 Gm=None +[s_only s1] step 7100/12000 loss=4.5773 gn=1.00 lr=2.59e-04 tps=16186 mem=6.2G lam=0.2802042147765557 Gm=None +[s_only s1] step 7120/12000 loss=4.4904 gn=0.94 lr=2.58e-04 tps=16187 mem=6.2G lam=0.2805185269874831 Gm=None +[s_only s1] step 7140/12000 loss=4.4290 gn=0.93 lr=2.56e-04 tps=16188 mem=6.2G lam=0.28072876498724025 Gm=None +[s_only s1] step 7160/12000 loss=4.3796 gn=1.00 lr=2.55e-04 tps=16188 mem=6.2G lam=0.2810599375516176 Gm=None +[s_only s1] step 7180/12000 loss=4.5904 gn=0.90 lr=2.53e-04 tps=16189 mem=6.2G lam=0.28139493831743795 Gm=None +[s_only s1] step 7200/12000 loss=4.4847 gn=0.91 lr=2.52e-04 tps=16189 mem=6.2G lam=0.28168125186736387 Gm=None +[s_only s1] step 7220/12000 loss=4.5702 gn=0.98 lr=2.51e-04 tps=16189 mem=6.2G lam=0.28198602857689065 Gm=None +[s_only s1] step 7240/12000 loss=4.4875 gn=1.00 lr=2.49e-04 tps=16190 mem=6.2G lam=0.28232393444826204 Gm=None +[s_only s1] step 7260/12000 loss=4.5132 gn=0.88 lr=2.48e-04 tps=16190 mem=6.2G lam=0.28259801616271335 Gm=None +[s_only s1] step 7280/12000 loss=4.5983 gn=0.89 lr=2.47e-04 tps=16191 mem=6.2G lam=0.28293947301184136 Gm=None +[s_only s1] step 7300/12000 loss=4.4318 gn=0.92 lr=2.45e-04 tps=16192 mem=6.2G lam=0.28323067740226787 Gm=None +[s_only s1] step 7320/12000 loss=4.4899 gn=0.87 lr=2.44e-04 tps=16192 mem=6.2G lam=0.28351221652701497 Gm=None +[s_only s1] step 7340/12000 loss=4.4970 gn=0.88 lr=2.42e-04 tps=16193 mem=6.2G lam=0.2837839404431482 Gm=None +[s_only s1] step 7360/12000 loss=4.5273 gn=0.88 lr=2.41e-04 tps=16193 mem=6.2G lam=0.2840099895062546 Gm=None +[s_only s1] step 7380/12000 loss=4.3642 gn=0.90 lr=2.40e-04 tps=16194 mem=6.2G lam=0.2843011942071219 Gm=None +[s_only s1] step 7400/12000 loss=4.3892 gn=0.86 lr=2.38e-04 tps=16194 mem=6.2G lam=0.28454958569879335 Gm=None +[s_only s1] step 7420/12000 loss=4.5467 gn=0.91 lr=2.37e-04 tps=16195 mem=6.2G lam=0.284821090909342 Gm=None +[s_only s1] step 7440/12000 loss=4.5750 gn=0.98 lr=2.36e-04 tps=16195 mem=6.2G lam=0.2850601420116921 Gm=None +[s_only s1] step 7460/12000 loss=4.4480 gn=0.86 lr=2.34e-04 tps=16196 mem=6.2G lam=0.28534231210748356 Gm=None +[s_only s1] step 7480/12000 loss=4.6729 gn=0.93 lr=2.33e-04 tps=16196 mem=6.2G lam=0.28561265968407196 Gm=None +[s_only s1] step 7500/12000 loss=4.3031 gn=0.89 lr=2.32e-04 tps=16197 mem=6.2G lam=0.2858827984891832 Gm=None +[s_only s1] EVAL step 7500: val_loss=4.5562 val_ppl=95.22 +[s_only s1] step 7520/12000 loss=4.5489 gn=0.89 lr=2.30e-04 tps=16181 mem=6.2G lam=0.2861437917066117 Gm=None +[s_only s1] step 7540/12000 loss=4.4049 gn=0.89 lr=2.29e-04 tps=16181 mem=6.2G lam=0.28641958596805733 Gm=None +[s_only s1] step 7560/12000 loss=4.4227 gn=0.94 lr=2.28e-04 tps=16182 mem=6.2G lam=0.2867658364896973 Gm=None +[s_only s1] step 7580/12000 loss=4.3909 gn=0.91 lr=2.26e-04 tps=16182 mem=6.2G lam=0.28706618460516137 Gm=None +[s_only s1] step 7600/12000 loss=4.4006 gn=0.95 lr=2.25e-04 tps=16183 mem=6.2G lam=0.28733425370107096 Gm=None +[s_only s1] step 7620/12000 loss=4.3881 gn=0.92 lr=2.24e-04 tps=16183 mem=6.2G lam=0.2875923849642277 Gm=None +[s_only s1] step 7640/12000 loss=4.4256 gn=0.89 lr=2.22e-04 tps=16184 mem=6.2G lam=0.28787522250786424 Gm=None +[s_only s1] step 7660/12000 loss=4.5180 gn=1.11 lr=2.21e-04 tps=16184 mem=6.2G lam=0.2881760687256853 Gm=None +[s_only s1] step 7680/12000 loss=4.3804 gn=1.09 lr=2.20e-04 tps=16184 mem=6.2G lam=0.2884117648936808 Gm=None +[s_only s1] step 7700/12000 loss=4.4922 gn=0.97 lr=2.18e-04 tps=16184 mem=6.2G lam=0.28867070640747744 Gm=None +[s_only s1] step 7720/12000 loss=4.5335 gn=0.98 lr=2.17e-04 tps=16184 mem=6.2G lam=0.28893172089010477 Gm=None +[s_only s1] step 7740/12000 loss=4.3698 gn=0.99 lr=2.16e-04 tps=16184 mem=6.2G lam=0.2891350717594226 Gm=None +[s_only s1] step 7760/12000 loss=4.5838 gn=0.95 lr=2.15e-04 tps=16185 mem=6.2G lam=0.2893577837385237 Gm=None +[s_only s1] step 7780/12000 loss=4.4459 gn=0.93 lr=2.13e-04 tps=16185 mem=6.2G lam=0.2896332307718694 Gm=None +[s_only s1] step 7800/12000 loss=4.4886 gn=0.94 lr=2.12e-04 tps=16185 mem=6.2G lam=0.28990340682988364 Gm=None +[s_only s1] step 7820/12000 loss=4.3816 gn=0.97 lr=2.11e-04 tps=16185 mem=6.2G lam=0.29018339949349564 Gm=None +[s_only s1] step 7840/12000 loss=4.5020 gn=0.99 lr=2.09e-04 tps=16186 mem=6.2G lam=0.290459793061018 Gm=None +[s_only s1] step 7860/12000 loss=4.4494 gn=0.95 lr=2.08e-04 tps=16186 mem=6.2G lam=0.29071293693656725 Gm=None +[s_only s1] step 7880/12000 loss=4.4036 gn=1.05 lr=2.07e-04 tps=16186 mem=6.2G lam=0.290951491954426 Gm=None +[s_only s1] step 7900/12000 loss=4.5785 gn=1.07 lr=2.06e-04 tps=16186 mem=6.2G lam=0.2912028799764812 Gm=None +[s_only s1] step 7920/12000 loss=4.4272 gn=1.00 lr=2.04e-04 tps=16186 mem=6.2G lam=0.2914314012353619 Gm=None +[s_only s1] step 7940/12000 loss=4.4753 gn=0.95 lr=2.03e-04 tps=16186 mem=6.2G lam=0.2916813792350392 Gm=None +[s_only s1] step 7960/12000 loss=4.5353 gn=0.99 lr=2.02e-04 tps=16186 mem=6.2G lam=0.29196102793018025 Gm=None +[s_only s1] step 7980/12000 loss=4.2700 gn=0.95 lr=2.00e-04 tps=16186 mem=6.2G lam=0.29219992536430556 Gm=None +[s_only s1] step 8000/12000 loss=4.3450 gn=0.94 lr=1.99e-04 tps=16186 mem=6.2G lam=0.2924469136632979 Gm=None +[s_only s1] EVAL step 8000: val_loss=4.5496 val_ppl=94.60 +[s_only s1] step 8020/12000 loss=4.6320 gn=1.07 lr=1.98e-04 tps=16171 mem=6.2G lam=0.29267588490620255 Gm=None +[s_only s1] step 8040/12000 loss=4.4192 gn=0.91 lr=1.97e-04 tps=16171 mem=6.2G lam=0.29289949949209887 Gm=None +[s_only s1] step 8060/12000 loss=4.3069 gn=0.90 lr=1.95e-04 tps=16171 mem=6.2G lam=0.29312398346761864 Gm=None +[s_only s1] step 8080/12000 loss=4.4068 gn=0.97 lr=1.94e-04 tps=16171 mem=6.2G lam=0.29332011255125207 Gm=None +[s_only s1] step 8100/12000 loss=4.4132 gn=0.99 lr=1.93e-04 tps=16172 mem=6.2G lam=0.2935580853372812 Gm=None +[s_only s1] step 8120/12000 loss=4.4489 gn=0.90 lr=1.92e-04 tps=16172 mem=6.2G lam=0.2937704309200247 Gm=None +[s_only s1] step 8140/12000 loss=4.4058 gn=0.93 lr=1.90e-04 tps=16172 mem=6.2G lam=0.2940021494093041 Gm=None +[s_only s1] step 8160/12000 loss=4.5192 gn=1.13 lr=1.89e-04 tps=16172 mem=6.2G lam=0.29422547575086355 Gm=None +[s_only s1] step 8180/12000 loss=4.4482 gn=0.90 lr=1.88e-04 tps=16173 mem=6.2G lam=0.2944705636861424 Gm=None +[s_only s1] step 8200/12000 loss=4.5251 gn=1.04 lr=1.87e-04 tps=16173 mem=6.2G lam=0.2946924924229582 Gm=None +[s_only s1] step 8220/12000 loss=4.3746 gn=0.91 lr=1.86e-04 tps=16173 mem=6.2G lam=0.2949130347309013 Gm=None +[s_only s1] step 8240/12000 loss=4.4491 gn=1.02 lr=1.84e-04 tps=16174 mem=6.2G lam=0.29514892228568596 Gm=None +[s_only s1] step 8260/12000 loss=4.3428 gn=0.94 lr=1.83e-04 tps=16175 mem=6.2G lam=0.2953907987102866 Gm=None +[s_only s1] step 8280/12000 loss=4.4227 gn=0.93 lr=1.82e-04 tps=16175 mem=6.2G lam=0.2956513344931106 Gm=None +[s_only s1] step 8300/12000 loss=4.5936 gn=1.06 lr=1.81e-04 tps=16176 mem=6.2G lam=0.2958928888353209 Gm=None +[s_only s1] step 8320/12000 loss=4.2272 gn=0.90 lr=1.80e-04 tps=16176 mem=6.2G lam=0.29607979394495487 Gm=None +[s_only s1] step 8340/12000 loss=4.4248 gn=0.98 lr=1.78e-04 tps=16177 mem=6.2G lam=0.2962655096004407 Gm=None +[s_only s1] step 8360/12000 loss=4.4205 gn=0.94 lr=1.77e-04 tps=16177 mem=6.2G lam=0.2964733244540791 Gm=None +[s_only s1] step 8380/12000 loss=4.3637 gn=0.91 lr=1.76e-04 tps=16177 mem=6.2G lam=0.2966975133555631 Gm=None +[s_only s1] step 8400/12000 loss=4.5586 gn=0.94 lr=1.75e-04 tps=16178 mem=6.2G lam=0.29683464315409464 Gm=None +[s_only s1] step 8420/12000 loss=4.3722 gn=0.90 lr=1.74e-04 tps=16178 mem=6.2G lam=0.2970579563019176 Gm=None +[s_only s1] step 8440/12000 loss=4.3764 gn=0.90 lr=1.72e-04 tps=16178 mem=6.2G lam=0.2972819013521075 Gm=None +[s_only s1] step 8460/12000 loss=4.4108 gn=0.90 lr=1.71e-04 tps=16179 mem=6.2G lam=0.2975311658034722 Gm=None +[s_only s1] step 8480/12000 loss=4.5313 gn=1.02 lr=1.70e-04 tps=16179 mem=6.2G lam=0.29772764382263023 Gm=None +[s_only s1] step 8500/12000 loss=4.5020 gn=0.97 lr=1.69e-04 tps=16179 mem=6.2G lam=0.29791661196698743 Gm=None +[s_only s1] EVAL step 8500: val_loss=4.4939 val_ppl=89.47 +[s_only s1] step 8520/12000 loss=4.4215 gn=1.07 lr=1.68e-04 tps=16166 mem=6.2G lam=0.29813544374580186 Gm=None +[s_only s1] step 8540/12000 loss=4.3799 gn=0.98 lr=1.67e-04 tps=16166 mem=6.2G lam=0.29832247536008555 Gm=None +[s_only s1] step 8560/12000 loss=4.3662 gn=1.04 lr=1.66e-04 tps=16167 mem=6.2G lam=0.29848232166841626 Gm=None +[s_only s1] step 8580/12000 loss=4.4920 gn=0.94 lr=1.64e-04 tps=16167 mem=6.2G lam=0.298711233617117 Gm=None +[s_only s1] step 8600/12000 loss=4.5366 gn=0.96 lr=1.63e-04 tps=16168 mem=6.2G lam=0.2989866834444304 Gm=None +[s_only s1] step 8620/12000 loss=4.5002 gn=0.94 lr=1.62e-04 tps=16168 mem=6.2G lam=0.29916213965043426 Gm=None +[s_only s1] step 8640/12000 loss=4.3989 gn=0.99 lr=1.61e-04 tps=16168 mem=6.2G lam=0.2993601846198241 Gm=None +[s_only s1] step 8660/12000 loss=4.4395 gn=1.00 lr=1.60e-04 tps=16169 mem=6.2G lam=0.29952491509417695 Gm=None +[s_only s1] step 8680/12000 loss=4.3805 gn=0.96 lr=1.59e-04 tps=16169 mem=6.2G lam=0.29966902174055576 Gm=None +[s_only s1] step 8700/12000 loss=4.3594 gn=1.04 lr=1.58e-04 tps=16170 mem=6.2G lam=0.2998958514072001 Gm=None +[s_only s1] step 8720/12000 loss=4.4612 gn=0.91 lr=1.57e-04 tps=16170 mem=6.2G lam=0.30011184234172106 Gm=None +[s_only s1] step 8740/12000 loss=4.5801 gn=0.93 lr=1.55e-04 tps=16171 mem=6.2G lam=0.30031712834412855 Gm=None +[s_only s1] step 8760/12000 loss=4.4164 gn=1.09 lr=1.54e-04 tps=16171 mem=6.2G lam=0.30051958793774247 Gm=None +[s_only s1] step 8780/12000 loss=4.1453 gn=0.97 lr=1.53e-04 tps=16172 mem=6.2G lam=0.3007090220538278 Gm=None +[s_only s1] step 8800/12000 loss=4.4175 gn=1.07 lr=1.52e-04 tps=16172 mem=6.2G lam=0.30085172643885016 Gm=None +[s_only s1] step 8820/12000 loss=4.3888 gn=1.00 lr=1.51e-04 tps=16172 mem=6.2G lam=0.30100381762410205 Gm=None +[s_only s1] step 8840/12000 loss=4.2432 gn=0.99 lr=1.50e-04 tps=16173 mem=6.2G lam=0.301206838650008 Gm=None +[s_only s1] step 8860/12000 loss=4.3952 gn=0.98 lr=1.49e-04 tps=16173 mem=6.2G lam=0.30141823552548885 Gm=None +[s_only s1] step 8880/12000 loss=4.5092 gn=1.04 lr=1.48e-04 tps=16174 mem=6.2G lam=0.3015864448000987 Gm=None +[s_only s1] step 8900/12000 loss=4.5600 gn=0.94 lr=1.47e-04 tps=16174 mem=6.2G lam=0.3017045119777322 Gm=None +[s_only s1] step 8920/12000 loss=4.3838 gn=0.95 lr=1.46e-04 tps=16174 mem=6.2G lam=0.3019007472321391 Gm=None +[s_only s1] step 8940/12000 loss=4.4080 gn=1.01 lr=1.45e-04 tps=16175 mem=6.2G lam=0.30209623742848635 Gm=None +[s_only s1] step 8960/12000 loss=4.4084 gn=0.97 lr=1.44e-04 tps=16175 mem=6.2G lam=0.3022719824997087 Gm=None +[s_only s1] step 8980/12000 loss=4.4460 gn=1.00 lr=1.43e-04 tps=16175 mem=6.2G lam=0.302461097172151 Gm=None +[s_only s1] step 9000/12000 loss=4.4278 gn=0.96 lr=1.42e-04 tps=16175 mem=6.2G lam=0.30265433123956126 Gm=None +[s_only s1] EVAL step 9000: val_loss=4.4646 val_ppl=86.89 +[s_only s1] step 9020/12000 loss=4.4445 gn=0.97 lr=1.41e-04 tps=16163 mem=6.2G lam=0.3028355611798664 Gm=None +[s_only s1] step 9040/12000 loss=4.4096 gn=1.11 lr=1.40e-04 tps=16163 mem=6.2G lam=0.30303765755767625 Gm=None +[s_only s1] step 9060/12000 loss=4.4454 gn=1.03 lr=1.39e-04 tps=16164 mem=6.2G lam=0.3032073308713734 Gm=None +[s_only s1] step 9080/12000 loss=4.4312 gn=1.06 lr=1.38e-04 tps=16164 mem=6.2G lam=0.3033461334804694 Gm=None +[s_only s1] step 9100/12000 loss=4.3486 gn=1.05 lr=1.37e-04 tps=16164 mem=6.2G lam=0.3035028548911214 Gm=None +[s_only s1] step 9120/12000 loss=4.2333 gn=0.99 lr=1.36e-04 tps=16165 mem=6.2G lam=0.30368954051906866 Gm=None +[s_only s1] step 9140/12000 loss=4.4805 gn=1.00 lr=1.35e-04 tps=16165 mem=6.2G lam=0.30388032101715606 Gm=None +[s_only s1] step 9160/12000 loss=4.5517 gn=0.93 lr=1.34e-04 tps=16165 mem=6.2G lam=0.30407132146259147 Gm=None +[s_only s1] step 9180/12000 loss=4.3443 gn=1.05 lr=1.33e-04 tps=16166 mem=6.2G lam=0.304265855345875 Gm=None +[s_only s1] step 9200/12000 loss=4.4500 gn=1.08 lr=1.32e-04 tps=16166 mem=6.2G lam=0.3043803380181392 Gm=None +[s_only s1] step 9220/12000 loss=4.3483 gn=0.98 lr=1.31e-04 tps=16167 mem=6.2G lam=0.3045242913067341 Gm=None +[s_only s1] step 9240/12000 loss=4.3740 gn=1.00 lr=1.30e-04 tps=16167 mem=6.2G lam=0.30468938561777276 Gm=None +[s_only s1] step 9260/12000 loss=4.4634 gn=1.01 lr=1.29e-04 tps=16168 mem=6.2G lam=0.3048354503698647 Gm=None +[s_only s1] step 9280/12000 loss=4.4736 gn=1.07 lr=1.28e-04 tps=16168 mem=6.2G lam=0.30499903702487546 Gm=None +[s_only s1] step 9300/12000 loss=4.5369 gn=1.03 lr=1.27e-04 tps=16169 mem=6.2G lam=0.30516315003236133 Gm=None +[s_only s1] step 9320/12000 loss=4.2866 gn=0.96 lr=1.26e-04 tps=16169 mem=6.2G lam=0.3053437997587025 Gm=None +[s_only s1] step 9340/12000 loss=4.2981 gn=1.02 lr=1.25e-04 tps=16169 mem=6.2G lam=0.30546039699887234 Gm=None +[s_only s1] step 9360/12000 loss=4.4795 gn=1.08 lr=1.24e-04 tps=16170 mem=6.2G lam=0.3055997472256422 Gm=None +[s_only s1] step 9380/12000 loss=4.3767 gn=1.00 lr=1.23e-04 tps=16170 mem=6.2G lam=0.3057355503551662 Gm=None +[s_only s1] step 9400/12000 loss=4.3505 gn=1.07 lr=1.22e-04 tps=16170 mem=6.2G lam=0.30588876688852906 Gm=None +[s_only s1] step 9420/12000 loss=4.2351 gn=1.01 lr=1.21e-04 tps=16171 mem=6.2G lam=0.30604788412650424 Gm=None +[s_only s1] step 9440/12000 loss=4.3333 gn=0.97 lr=1.20e-04 tps=16171 mem=6.2G lam=0.30621867704515654 Gm=None +[s_only s1] step 9460/12000 loss=4.3156 gn=1.19 lr=1.19e-04 tps=16172 mem=6.2G lam=0.306339383435746 Gm=None +[s_only s1] step 9480/12000 loss=4.3997 gn=0.96 lr=1.19e-04 tps=16172 mem=6.2G lam=0.3064597112437089 Gm=None +[s_only s1] step 9500/12000 loss=4.3685 gn=1.00 lr=1.18e-04 tps=16173 mem=6.2G lam=0.30658378048489493 Gm=None +[s_only s1] EVAL step 9500: val_loss=4.4568 val_ppl=86.21 +[s_only s1] step 9520/12000 loss=4.4193 gn=1.08 lr=1.17e-04 tps=16160 mem=6.2G lam=0.30672946432605386 Gm=None +[s_only s1] step 9540/12000 loss=4.2982 gn=1.02 lr=1.16e-04 tps=16161 mem=6.2G lam=0.30691562484328944 Gm=None +[s_only s1] step 9560/12000 loss=4.4171 gn=1.20 lr=1.15e-04 tps=16161 mem=6.2G lam=0.30705101881176233 Gm=None +[s_only s1] step 9580/12000 loss=4.4429 gn=1.06 lr=1.14e-04 tps=16161 mem=6.2G lam=0.30719937989488244 Gm=None +[s_only s1] step 9600/12000 loss=4.3417 gn=1.02 lr=1.13e-04 tps=16162 mem=6.2G lam=0.30734188699473936 Gm=None +[s_only s1] step 9620/12000 loss=4.3658 gn=1.03 lr=1.12e-04 tps=16162 mem=6.2G lam=0.30748422909528017 Gm=None +[s_only s1] step 9640/12000 loss=4.2851 gn=0.99 lr=1.12e-04 tps=16163 mem=6.2G lam=0.30764527525752783 Gm=None +[s_only s1] step 9660/12000 loss=4.5468 gn=1.01 lr=1.11e-04 tps=16163 mem=6.2G lam=0.30776598552862805 Gm=None +[s_only s1] step 9680/12000 loss=4.3373 gn=0.99 lr=1.10e-04 tps=16163 mem=6.2G lam=0.30789166238779825 Gm=None +[s_only s1] step 9700/12000 loss=4.4328 gn=0.98 lr=1.09e-04 tps=16164 mem=6.2G lam=0.3079889953757326 Gm=None +[s_only s1] step 9720/12000 loss=4.4984 gn=1.22 lr=1.08e-04 tps=16165 mem=6.2G lam=0.30813327291980386 Gm=None +[s_only s1] step 9740/12000 loss=4.2326 gn=1.00 lr=1.07e-04 tps=16165 mem=6.2G lam=0.30829782349367935 Gm=None +[s_only s1] step 9760/12000 loss=4.4566 gn=0.98 lr=1.07e-04 tps=16166 mem=6.2G lam=0.3084360828312735 Gm=None +[s_only s1] step 9780/12000 loss=4.3684 gn=1.00 lr=1.06e-04 tps=16166 mem=6.2G lam=0.3085886087889473 Gm=None +[s_only s1] step 9800/12000 loss=4.4423 gn=0.97 lr=1.05e-04 tps=16166 mem=6.2G lam=0.30873542713622254 Gm=None +[s_only s1] step 9820/12000 loss=4.2852 gn=1.03 lr=1.04e-04 tps=16167 mem=6.2G lam=0.30887045664712787 Gm=None +[s_only s1] step 9840/12000 loss=4.4939 gn=1.04 lr=1.03e-04 tps=16167 mem=6.2G lam=0.3090095343068242 Gm=None +[s_only s1] step 9860/12000 loss=4.5284 gn=1.09 lr=1.03e-04 tps=16167 mem=6.2G lam=0.3091390375047922 Gm=None +[s_only s1] step 9880/12000 loss=4.3648 gn=1.03 lr=1.02e-04 tps=16167 mem=6.2G lam=0.3092653672210872 Gm=None +[s_only s1] step 9900/12000 loss=4.3368 gn=1.00 lr=1.01e-04 tps=16168 mem=6.2G lam=0.3093430567532778 Gm=None +[s_only s1] step 9920/12000 loss=4.3032 gn=1.07 lr=1.00e-04 tps=16168 mem=6.2G lam=0.3094260774863263 Gm=None +[s_only s1] step 9940/12000 loss=4.2798 gn=1.06 lr=9.96e-05 tps=16169 mem=6.2G lam=0.3095025531947613 Gm=None +[s_only s1] step 9960/12000 loss=4.3437 gn=1.05 lr=9.89e-05 tps=16169 mem=6.2G lam=0.3096251782650749 Gm=None +[s_only s1] step 9980/12000 loss=4.3944 gn=0.98 lr=9.81e-05 tps=16169 mem=6.2G lam=0.309730507278194 Gm=None +[s_only s1] step 10000/12000 loss=4.3346 gn=1.04 lr=9.74e-05 tps=16169 mem=6.2G lam=0.3098534542756776 Gm=None +[s_only s1] EVAL step 10000: val_loss=4.4237 val_ppl=83.41 +[s_only s1] step 10020/12000 loss=4.6528 gn=1.09 lr=9.67e-05 tps=16158 mem=6.2G lam=0.30996352480724454 Gm=None +[s_only s1] step 10040/12000 loss=4.2564 gn=1.10 lr=9.59e-05 tps=16158 mem=6.2G lam=0.3100249149526159 Gm=None +[s_only s1] step 10060/12000 loss=4.4776 gn=1.06 lr=9.52e-05 tps=16158 mem=6.2G lam=0.31014164723455906 Gm=None +[s_only s1] step 10080/12000 loss=4.2597 gn=1.06 lr=9.45e-05 tps=16159 mem=6.2G lam=0.3102500283469756 Gm=None +[s_only s1] step 10100/12000 loss=4.3071 gn=1.10 lr=9.38e-05 tps=16159 mem=6.2G lam=0.31035940255969763 Gm=None +[s_only s1] step 10120/12000 loss=4.3273 gn=1.03 lr=9.31e-05 tps=16160 mem=6.2G lam=0.31048025159786147 Gm=None +[s_only s1] step 10140/12000 loss=4.2682 gn=1.20 lr=9.24e-05 tps=16160 mem=6.2G lam=0.31059078375498456 Gm=None +[s_only s1] step 10160/12000 loss=4.3764 gn=1.09 lr=9.18e-05 tps=16161 mem=6.2G lam=0.31069645363216597 Gm=None +[s_only s1] step 10180/12000 loss=4.2788 gn=0.96 lr=9.11e-05 tps=16161 mem=6.2G lam=0.3107927044232686 Gm=None +[s_only s1] step 10200/12000 loss=4.4699 gn=1.08 lr=9.04e-05 tps=16161 mem=6.2G lam=0.31091234227642417 Gm=None +[s_only s1] step 10220/12000 loss=4.3973 gn=0.98 lr=8.98e-05 tps=16162 mem=6.2G lam=0.31099399241308373 Gm=None +[s_only s1] step 10240/12000 loss=4.3390 gn=1.09 lr=8.91e-05 tps=16162 mem=6.2G lam=0.31114252827440697 Gm=None +[s_only s1] step 10260/12000 loss=4.4422 gn=0.98 lr=8.85e-05 tps=16163 mem=6.2G lam=0.311273121740669 Gm=None +[s_only s1] step 10280/12000 loss=4.3464 gn=0.99 lr=8.78e-05 tps=16163 mem=6.2G lam=0.31137941405177116 Gm=None +[s_only s1] step 10300/12000 loss=4.2606 gn=1.11 lr=8.72e-05 tps=16163 mem=6.2G lam=0.3114518163104852 Gm=None +[s_only s1] step 10320/12000 loss=4.3123 gn=1.03 lr=8.66e-05 tps=16164 mem=6.2G lam=0.3115803723533948 Gm=None +[s_only s1] step 10340/12000 loss=4.4627 gn=1.14 lr=8.59e-05 tps=16164 mem=6.2G lam=0.3117086680916448 Gm=None +[s_only s1] step 10360/12000 loss=4.3554 gn=0.99 lr=8.53e-05 tps=16164 mem=6.2G lam=0.3118272772990167 Gm=None +[s_only s1] step 10380/12000 loss=4.3600 gn=1.05 lr=8.47e-05 tps=16165 mem=6.2G lam=0.31192218729605276 Gm=None +[s_only s1] step 10400/12000 loss=4.4743 gn=1.26 lr=8.41e-05 tps=16165 mem=6.2G lam=0.3119936046811442 Gm=None +[s_only s1] step 10420/12000 loss=4.4109 gn=1.18 lr=8.35e-05 tps=16165 mem=6.2G lam=0.31210906136160094 Gm=None +[s_only s1] step 10440/12000 loss=4.4242 gn=1.04 lr=8.30e-05 tps=16166 mem=6.2G lam=0.3121997270112236 Gm=None +[s_only s1] step 10460/12000 loss=4.2530 gn=1.07 lr=8.24e-05 tps=16166 mem=6.2G lam=0.31231579557061195 Gm=None +[s_only s1] step 10480/12000 loss=4.3512 gn=1.17 lr=8.18e-05 tps=16167 mem=6.2G lam=0.3124179888206224 Gm=None +[s_only s1] step 10500/12000 loss=4.5399 gn=1.36 lr=8.12e-05 tps=16167 mem=6.2G lam=0.3125395557532708 Gm=None +[s_only s1] EVAL step 10500: val_loss=4.4165 val_ppl=82.81 +[s_only s1] step 10520/12000 loss=4.3655 gn=1.07 lr=8.07e-05 tps=16156 mem=6.2G lam=0.312620484114935 Gm=None +[s_only s1] step 10540/12000 loss=4.4482 gn=1.19 lr=8.01e-05 tps=16157 mem=6.2G lam=0.31271016225218773 Gm=None +[s_only s1] step 10560/12000 loss=4.4332 gn=1.17 lr=7.96e-05 tps=16157 mem=6.2G lam=0.31277095278104144 Gm=None +[s_only s1] step 10580/12000 loss=4.3283 gn=1.06 lr=7.91e-05 tps=16158 mem=6.2G lam=0.3128591288502018 Gm=None +[s_only s1] step 10600/12000 loss=4.2424 gn=1.12 lr=7.85e-05 tps=16158 mem=6.2G lam=0.31294214849670726 Gm=None +[s_only s1] step 10620/12000 loss=4.3485 gn=1.17 lr=7.80e-05 tps=16158 mem=6.2G lam=0.3130654306150973 Gm=None +[s_only s1] step 10640/12000 loss=4.4581 gn=1.04 lr=7.75e-05 tps=16159 mem=6.2G lam=0.31318632746115327 Gm=None +[s_only s1] step 10660/12000 loss=4.2741 gn=1.04 lr=7.70e-05 tps=16159 mem=6.2G lam=0.31331028742715716 Gm=None +[s_only s1] step 10680/12000 loss=4.2864 gn=1.09 lr=7.65e-05 tps=16160 mem=6.2G lam=0.3134273610388239 Gm=None +[s_only s1] step 10700/12000 loss=4.3320 gn=1.01 lr=7.60e-05 tps=16161 mem=6.2G lam=0.3135062408012648 Gm=None +[s_only s1] step 10720/12000 loss=4.3547 gn=1.19 lr=7.55e-05 tps=16161 mem=6.2G lam=0.31357066659256816 Gm=None +[s_only s1] step 10740/12000 loss=4.4195 gn=1.11 lr=7.50e-05 tps=16161 mem=6.2G lam=0.3136758793455859 Gm=None +[s_only s1] step 10760/12000 loss=4.3105 gn=1.05 lr=7.46e-05 tps=16162 mem=6.2G lam=0.31378814407313865 Gm=None +[s_only s1] step 10780/12000 loss=4.3427 gn=1.06 lr=7.41e-05 tps=16162 mem=6.2G lam=0.313869867939502 Gm=None +[s_only s1] step 10800/12000 loss=4.4190 gn=1.16 lr=7.37e-05 tps=16163 mem=6.2G lam=0.31393232801929116 Gm=None +[s_only s1] step 10820/12000 loss=4.5065 gn=1.06 lr=7.32e-05 tps=16163 mem=6.2G lam=0.31401774551098544 Gm=None +[s_only s1] step 10840/12000 loss=4.1948 gn=1.04 lr=7.28e-05 tps=16163 mem=6.2G lam=0.31410947705929476 Gm=None +[s_only s1] step 10860/12000 loss=4.3720 gn=1.03 lr=7.23e-05 tps=16164 mem=6.2G lam=0.3141969363205135 Gm=None +[s_only s1] step 10880/12000 loss=4.2007 gn=1.14 lr=7.19e-05 tps=16164 mem=6.2G lam=0.3142947376084824 Gm=None +[s_only s1] step 10900/12000 loss=4.3436 gn=1.04 lr=7.15e-05 tps=16164 mem=6.2G lam=0.31438156589865685 Gm=None +[s_only s1] step 10920/12000 loss=4.3518 gn=1.08 lr=7.11e-05 tps=16165 mem=6.2G lam=0.31448253119985264 Gm=None +[s_only s1] step 10940/12000 loss=4.2882 gn=1.05 lr=7.07e-05 tps=16165 mem=6.2G lam=0.31458225365107256 Gm=None +[s_only s1] step 10960/12000 loss=4.3067 gn=1.02 lr=7.03e-05 tps=16166 mem=6.2G lam=0.31466511357575655 Gm=None +[s_only s1] step 10980/12000 loss=4.5007 gn=1.22 lr=6.99e-05 tps=16166 mem=6.2G lam=0.3147580255754292 Gm=None +[s_only s1] step 11000/12000 loss=4.3024 gn=1.04 lr=6.95e-05 tps=16166 mem=6.2G lam=0.31484835284451645 Gm=None +[s_only s1] EVAL step 11000: val_loss=4.3996 val_ppl=81.42 +[s_only s1] step 11020/12000 loss=4.1213 gn=1.08 lr=6.91e-05 tps=16156 mem=6.2G lam=0.3149232864379883 Gm=None +[s_only s1] step 11040/12000 loss=4.3402 gn=1.09 lr=6.88e-05 tps=16156 mem=6.2G lam=0.3150128290678064 Gm=None +[s_only s1] step 11060/12000 loss=4.3816 gn=1.19 lr=6.84e-05 tps=16157 mem=6.2G lam=0.31513188205038506 Gm=None +[s_only s1] step 11080/12000 loss=4.3981 gn=1.09 lr=6.81e-05 tps=16157 mem=6.2G lam=0.3152274911602338 Gm=None +[s_only s1] step 11100/12000 loss=4.4048 gn=1.11 lr=6.77e-05 tps=16157 mem=6.2G lam=0.3153082470720013 Gm=None +[s_only s1] step 11120/12000 loss=4.3931 gn=1.06 lr=6.74e-05 tps=16158 mem=6.2G lam=0.31538375342885655 Gm=None +[s_only s1] step 11140/12000 loss=4.2590 gn=1.11 lr=6.70e-05 tps=16158 mem=6.2G lam=0.3154756515286863 Gm=None +[s_only s1] step 11160/12000 loss=4.2895 gn=1.07 lr=6.67e-05 tps=16158 mem=6.2G lam=0.31555634643882513 Gm=None +[s_only s1] step 11180/12000 loss=4.3262 gn=1.02 lr=6.64e-05 tps=16159 mem=6.2G lam=0.3156367408422132 Gm=None +[s_only s1] step 11200/12000 loss=4.4916 gn=1.21 lr=6.61e-05 tps=16159 mem=6.2G lam=0.31571987783536315 Gm=None +[s_only s1] step 11220/12000 loss=4.3504 gn=1.13 lr=6.58e-05 tps=16160 mem=6.2G lam=0.31580875456954044 Gm=None +[s_only s1] step 11240/12000 loss=4.2690 gn=1.12 lr=6.55e-05 tps=16160 mem=6.2G lam=0.3159041058582564 Gm=None +[s_only s1] step 11260/12000 loss=4.2953 gn=1.04 lr=6.52e-05 tps=16161 mem=6.2G lam=0.31598934189726907 Gm=None +[s_only s1] step 11280/12000 loss=4.1952 gn=1.21 lr=6.49e-05 tps=16161 mem=6.2G lam=0.3160843026513855 Gm=None +[s_only s1] step 11300/12000 loss=4.3901 gn=1.09 lr=6.47e-05 tps=16161 mem=6.2G lam=0.31617185349265736 Gm=None +[s_only s1] step 11320/12000 loss=4.2246 gn=1.13 lr=6.44e-05 tps=16162 mem=6.2G lam=0.3162553041862945 Gm=None +[s_only s1] step 11340/12000 loss=4.4007 gn=1.16 lr=6.42e-05 tps=16162 mem=6.2G lam=0.3163252801944812 Gm=None +[s_only s1] step 11360/12000 loss=4.2702 gn=1.18 lr=6.39e-05 tps=16163 mem=6.2G lam=0.3164070590088765 Gm=None +[s_only s1] step 11380/12000 loss=4.3383 gn=1.24 lr=6.37e-05 tps=16163 mem=6.2G lam=0.31648378695050877 Gm=None +[s_only s1] step 11400/12000 loss=4.4019 gn=1.12 lr=6.34e-05 tps=16163 mem=6.2G lam=0.31654646232103306 Gm=None +[s_only s1] step 11420/12000 loss=4.0740 gn=1.05 lr=6.32e-05 tps=16164 mem=6.2G lam=0.3166309637017548 Gm=None +[s_only s1] step 11440/12000 loss=4.1887 gn=1.13 lr=6.30e-05 tps=16164 mem=6.2G lam=0.3167249468776087 Gm=None +[s_only s1] step 11460/12000 loss=4.3222 gn=1.15 lr=6.28e-05 tps=16164 mem=6.2G lam=0.31683856016024947 Gm=None +[s_only s1] step 11480/12000 loss=4.3734 gn=1.11 lr=6.26e-05 tps=16165 mem=6.2G lam=0.31691893469542265 Gm=None +[s_only s1] step 11500/12000 loss=4.5225 gn=1.45 lr=6.24e-05 tps=16165 mem=6.2G lam=0.31699860421940684 Gm=None +[s_only s1] EVAL step 11500: val_loss=4.3972 val_ppl=81.22 +[s_only s1] step 11520/12000 loss=4.2918 gn=1.08 lr=6.22e-05 tps=16155 mem=6.2G lam=0.3170652633222441 Gm=None +[s_only s1] step 11540/12000 loss=4.2003 gn=1.16 lr=6.20e-05 tps=16155 mem=6.2G lam=0.317131075852861 Gm=None +[s_only s1] step 11560/12000 loss=4.3293 gn=1.10 lr=6.19e-05 tps=16156 mem=6.2G lam=0.31719852921863395 Gm=None +[s_only s1] step 11580/12000 loss=4.3581 gn=1.10 lr=6.17e-05 tps=16156 mem=6.2G lam=0.3172711847970883 Gm=None +[s_only s1] step 11600/12000 loss=4.1821 gn=1.10 lr=6.15e-05 tps=16156 mem=6.2G lam=0.3173346972713868 Gm=None +[s_only s1] step 11620/12000 loss=4.4416 gn=1.12 lr=6.14e-05 tps=16157 mem=6.2G lam=0.3174167543960114 Gm=None +[s_only s1] step 11640/12000 loss=4.3761 gn=1.21 lr=6.12e-05 tps=16157 mem=6.2G lam=0.31748952344059944 Gm=None +[s_only s1] step 11660/12000 loss=4.3682 gn=1.04 lr=6.11e-05 tps=16158 mem=6.2G lam=0.31755445928623277 Gm=None +[s_only s1] step 11680/12000 loss=4.2741 gn=1.04 lr=6.10e-05 tps=16158 mem=6.2G lam=0.31763995019719005 Gm=None +[s_only s1] step 11700/12000 loss=4.3037 gn=1.06 lr=6.09e-05 tps=16159 mem=6.2G lam=0.3177475438763698 Gm=None +[s_only s1] step 11720/12000 loss=4.1671 gn=1.05 lr=6.07e-05 tps=16159 mem=6.2G lam=0.31783172860741615 Gm=None +[s_only s1] step 11740/12000 loss=4.3945 gn=1.15 lr=6.06e-05 tps=16159 mem=6.2G lam=0.3178861439228058 Gm=None +[s_only s1] step 11760/12000 loss=4.2698 gn=1.19 lr=6.06e-05 tps=16160 mem=6.2G lam=0.31793792685493827 Gm=None +[s_only s1] step 11780/12000 loss=4.2575 gn=1.15 lr=6.05e-05 tps=16160 mem=6.2G lam=0.3180032406623165 Gm=None +[s_only s1] step 11800/12000 loss=4.2508 gn=1.13 lr=6.04e-05 tps=16161 mem=6.2G lam=0.31810388589898747 Gm=None +[s_only s1] step 11820/12000 loss=4.3273 gn=1.08 lr=6.03e-05 tps=16161 mem=6.2G lam=0.31819315332298476 Gm=None +[s_only s1] step 11840/12000 loss=4.2997 gn=1.07 lr=6.02e-05 tps=16161 mem=6.2G lam=0.31826623653372127 Gm=None +[s_only s1] step 11860/12000 loss=4.3077 gn=1.15 lr=6.02e-05 tps=16162 mem=6.2G lam=0.31833786983042955 Gm=None +[s_only s1] step 11880/12000 loss=4.2761 gn=1.12 lr=6.01e-05 tps=16162 mem=6.2G lam=0.3184195230714977 Gm=None +[s_only s1] step 11900/12000 loss=4.3952 gn=1.21 lr=6.01e-05 tps=16162 mem=6.2G lam=0.3184930654242635 Gm=None +[s_only s1] step 11920/12000 loss=4.2454 gn=1.21 lr=6.01e-05 tps=16163 mem=6.2G lam=0.3185646793184181 Gm=None +[s_only s1] step 11940/12000 loss=4.3119 gn=1.09 lr=6.00e-05 tps=16163 mem=6.2G lam=0.31862772318224114 Gm=None +[s_only s1] step 11960/12000 loss=4.2169 gn=1.12 lr=6.00e-05 tps=16164 mem=6.2G lam=0.3186955444204311 Gm=None +[s_only s1] step 11980/12000 loss=4.3138 gn=1.14 lr=6.00e-05 tps=16164 mem=6.2G lam=0.3187686977908015 Gm=None +[s_only s1] EVAL step 11999: val_loss=4.3928 val_ppl=80.86 +[init] variant=g_only seed=1 params=63.39M +[init] use_S=False use_G=True +[g_only s1] step 0/12000 loss=10.9250 gn=9.60 lr=3.00e-06 tps=8306 mem=5.8G lam=None Gm=0.5144856770833334 +[g_only s1] step 20/12000 loss=9.7823 gn=2.08 lr=6.30e-05 tps=16894 mem=6.3G lam=None Gm=0.462158203125 +[g_only s1] step 40/12000 loss=9.1641 gn=1.70 lr=1.23e-04 tps=17299 mem=6.3G lam=None Gm=0.4429524739583333 +[g_only s1] step 60/12000 loss=8.2672 gn=1.52 lr=1.83e-04 tps=17445 mem=6.3G lam=None Gm=0.438232421875 +[g_only s1] step 80/12000 loss=7.4100 gn=0.88 lr=2.43e-04 tps=17523 mem=6.3G lam=None Gm=0.44091796875 +[g_only s1] step 100/12000 loss=7.2856 gn=0.65 lr=3.03e-04 tps=17570 mem=6.3G lam=None Gm=0.4335123697916667 +[g_only s1] step 120/12000 loss=7.0711 gn=0.77 lr=3.63e-04 tps=17609 mem=6.3G lam=None Gm=0.425537109375 +[g_only s1] step 140/12000 loss=6.8017 gn=0.60 lr=4.23e-04 tps=17634 mem=6.3G lam=None Gm=0.412353515625 +[g_only s1] step 160/12000 loss=6.7860 gn=0.53 lr=4.83e-04 tps=17653 mem=6.3G lam=None Gm=0.418212890625 +[g_only s1] step 180/12000 loss=6.7595 gn=0.59 lr=5.43e-04 tps=17666 mem=6.3G lam=None Gm=0.3993326822916667 +[g_only s1] step 200/12000 loss=6.6058 gn=0.59 lr=6.00e-04 tps=17677 mem=6.3G lam=None Gm=0.3927408854166667 +[g_only s1] step 220/12000 loss=6.6006 gn=0.53 lr=6.00e-04 tps=17684 mem=6.3G lam=None Gm=0.3961588541666667 +[g_only s1] step 240/12000 loss=6.6277 gn=0.50 lr=6.00e-04 tps=17689 mem=6.3G lam=None Gm=0.3956705729166667 +[g_only s1] step 260/12000 loss=6.6279 gn=0.54 lr=6.00e-04 tps=17697 mem=6.3G lam=None Gm=0.3970540364583333 +[g_only s1] step 280/12000 loss=6.6911 gn=0.53 lr=6.00e-04 tps=17704 mem=6.3G lam=None Gm=0.4051920572916667 +[g_only s1] step 300/12000 loss=6.5490 gn=0.52 lr=6.00e-04 tps=17711 mem=6.3G lam=None Gm=0.4017740885416667 +[g_only s1] step 320/12000 loss=6.4643 gn=0.45 lr=6.00e-04 tps=17714 mem=6.3G lam=None Gm=0.4063313802083333 +[g_only s1] step 340/12000 loss=6.4810 gn=0.51 lr=6.00e-04 tps=17719 mem=6.3G lam=None Gm=0.4210611979166667 +[g_only s1] step 360/12000 loss=6.2241 gn=0.47 lr=6.00e-04 tps=17716 mem=6.3G lam=None Gm=0.40087890625 +[g_only s1] step 380/12000 loss=6.2944 gn=0.43 lr=6.00e-04 tps=17718 mem=6.3G lam=None Gm=0.409912109375 +[g_only s1] step 400/12000 loss=6.2616 gn=0.45 lr=6.00e-04 tps=17722 mem=6.3G lam=None Gm=0.4324544270833333 +[g_only s1] step 420/12000 loss=6.3508 gn=0.45 lr=6.00e-04 tps=17724 mem=6.3G lam=None Gm=0.4244791666666667 +[g_only s1] step 440/12000 loss=6.2575 gn=0.48 lr=5.99e-04 tps=17726 mem=6.3G lam=None Gm=0.4252522786458333 +[g_only s1] step 460/12000 loss=6.3201 gn=0.51 lr=5.99e-04 tps=17728 mem=6.3G lam=None Gm=0.437744140625 +[g_only s1] step 480/12000 loss=6.1983 gn=0.49 lr=5.99e-04 tps=17731 mem=6.3G lam=None Gm=0.4337565104166667 +[g_only s1] step 500/12000 loss=6.3213 gn=0.49 lr=5.99e-04 tps=17733 mem=6.3G lam=None Gm=0.4768880208333333 +[g_only s1] EVAL step 500: val_loss=6.2448 val_ppl=515.35 +[g_only s1] step 520/12000 loss=6.1609 gn=0.52 lr=5.99e-04 tps=17482 mem=6.3G lam=None Gm=0.4537760416666667 +[g_only s1] step 540/12000 loss=6.2233 gn=0.45 lr=5.99e-04 tps=17491 mem=6.3G lam=None Gm=0.4691569010416667 +[g_only s1] step 560/12000 loss=6.1353 gn=0.44 lr=5.99e-04 tps=17499 mem=6.3G lam=None Gm=0.5189615885416666 +[g_only s1] step 580/12000 loss=6.0861 gn=0.53 lr=5.99e-04 tps=17507 mem=6.3G lam=None Gm=0.470458984375 +[g_only s1] step 600/12000 loss=6.1251 gn=0.50 lr=5.98e-04 tps=17515 mem=6.3G lam=None Gm=0.5110677083333334 +[g_only s1] step 620/12000 loss=6.2376 gn=0.45 lr=5.98e-04 tps=17522 mem=6.3G lam=None Gm=0.5538736979166666 +[g_only s1] step 640/12000 loss=6.2369 gn=0.58 lr=5.98e-04 tps=17529 mem=6.3G lam=None Gm=0.560791015625 +[g_only s1] step 660/12000 loss=5.9627 gn=0.47 lr=5.98e-04 tps=17536 mem=6.3G lam=None Gm=0.5425618489583334 +[g_only s1] step 680/12000 loss=5.9822 gn=0.51 lr=5.98e-04 tps=17544 mem=6.3G lam=None Gm=0.5274251302083334 +[g_only s1] step 700/12000 loss=5.9762 gn=0.46 lr=5.98e-04 tps=17550 mem=6.3G lam=None Gm=0.5130208333333334 +[g_only s1] step 720/12000 loss=6.0532 gn=0.47 lr=5.97e-04 tps=17556 mem=6.3G lam=None Gm=0.5751953125 +[g_only s1] step 740/12000 loss=5.9972 gn=0.55 lr=5.97e-04 tps=17561 mem=6.3G lam=None Gm=0.56884765625 +[g_only s1] step 760/12000 loss=6.0233 gn=0.47 lr=5.97e-04 tps=17566 mem=6.3G lam=None Gm=0.5531412760416666 +[g_only s1] step 780/12000 loss=6.0063 gn=0.47 lr=5.97e-04 tps=17572 mem=6.3G lam=None Gm=0.5902506510416666 +[g_only s1] step 800/12000 loss=5.9273 gn=0.51 lr=5.97e-04 tps=17577 mem=6.3G lam=None Gm=0.586669921875 +[g_only s1] step 820/12000 loss=5.8405 gn=0.52 lr=5.96e-04 tps=17583 mem=6.3G lam=None Gm=0.5594075520833334 +[g_only s1] step 840/12000 loss=5.7600 gn=0.49 lr=5.96e-04 tps=17588 mem=6.3G lam=None Gm=0.559814453125 +[g_only s1] step 860/12000 loss=5.6698 gn=0.50 lr=5.96e-04 tps=17592 mem=6.3G lam=None Gm=0.5826009114583334 +[g_only s1] step 880/12000 loss=5.7631 gn=0.59 lr=5.96e-04 tps=17598 mem=6.3G lam=None Gm=0.5708821614583334 +[g_only s1] step 900/12000 loss=5.6853 gn=0.50 lr=5.95e-04 tps=17602 mem=6.3G lam=None Gm=0.5380045572916666 +[g_only s1] step 920/12000 loss=5.8657 gn=0.51 lr=5.95e-04 tps=17606 mem=6.3G lam=None Gm=0.5624186197916666 +[g_only s1] step 940/12000 loss=5.7703 gn=0.47 lr=5.95e-04 tps=17610 mem=6.3G lam=None Gm=0.566162109375 +[g_only s1] step 960/12000 loss=5.9165 gn=0.50 lr=5.94e-04 tps=17612 mem=6.3G lam=None Gm=0.5484619140625 +[g_only s1] step 980/12000 loss=5.6262 gn=0.51 lr=5.94e-04 tps=17616 mem=6.3G lam=None Gm=0.5350341796875 +[g_only s1] step 1000/12000 loss=5.7452 gn=0.49 lr=5.94e-04 tps=17619 mem=6.3G lam=None Gm=0.5567220052083334 +[g_only s1] EVAL step 1000: val_loss=5.7746 val_ppl=322.02 +[g_only s1] step 1020/12000 loss=5.6612 gn=0.53 lr=5.94e-04 tps=17495 mem=6.3G lam=None Gm=0.5634765625 +[g_only s1] step 1040/12000 loss=5.7239 gn=0.52 lr=5.93e-04 tps=17499 mem=6.3G lam=None Gm=0.5581868489583334 +[g_only s1] step 1060/12000 loss=5.7635 gn=0.53 lr=5.93e-04 tps=17502 mem=6.3G lam=None Gm=0.5459798177083334 +[g_only s1] step 1080/12000 loss=5.5737 gn=0.49 lr=5.93e-04 tps=17507 mem=6.3G lam=None Gm=0.5531819661458334 +[g_only s1] step 1100/12000 loss=5.7832 gn=0.53 lr=5.92e-04 tps=17510 mem=6.3G lam=None Gm=0.5497639973958334 +[g_only s1] step 1120/12000 loss=5.6307 gn=0.50 lr=5.92e-04 tps=17514 mem=6.3G lam=None Gm=0.5359700520833334 +[g_only s1] step 1140/12000 loss=5.6763 gn=0.48 lr=5.92e-04 tps=17517 mem=6.3G lam=None Gm=0.5467122395833334 +[g_only s1] step 1160/12000 loss=5.6697 gn=0.55 lr=5.91e-04 tps=17521 mem=6.3G lam=None Gm=0.5389811197916666 +[g_only s1] step 1180/12000 loss=5.7141 gn=0.57 lr=5.91e-04 tps=17524 mem=6.3G lam=None Gm=0.5594482421875 +[g_only s1] step 1200/12000 loss=5.6442 gn=0.57 lr=5.90e-04 tps=17528 mem=6.3G lam=None Gm=0.5287272135416666 +[g_only s1] step 1220/12000 loss=5.8126 gn=0.52 lr=5.90e-04 tps=17531 mem=6.3G lam=None Gm=0.5346272786458334 +[g_only s1] step 1240/12000 loss=5.5930 gn=0.53 lr=5.90e-04 tps=17534 mem=6.3G lam=None Gm=0.5206705729166666 +[g_only s1] step 1260/12000 loss=5.5711 gn=0.55 lr=5.89e-04 tps=17536 mem=6.3G lam=None Gm=0.5228271484375 +[g_only s1] step 1280/12000 loss=5.4935 gn=0.56 lr=5.89e-04 tps=17537 mem=6.3G lam=None Gm=0.5172119140625 +[g_only s1] step 1300/12000 loss=5.6105 gn=0.54 lr=5.89e-04 tps=17539 mem=6.3G lam=None Gm=0.5262044270833334 +[g_only s1] step 1320/12000 loss=5.4675 gn=0.52 lr=5.88e-04 tps=17537 mem=6.3G lam=None Gm=0.5109049479166666 +[g_only s1] step 1340/12000 loss=5.7245 gn=0.56 lr=5.88e-04 tps=17537 mem=6.3G lam=None Gm=0.5124104817708334 +[g_only s1] step 1360/12000 loss=5.5482 gn=0.56 lr=5.87e-04 tps=17535 mem=6.3G lam=None Gm=0.5126953125 +[g_only s1] step 1380/12000 loss=5.4525 gn=0.53 lr=5.87e-04 tps=17532 mem=6.3G lam=None Gm=0.5108235677083334 +[g_only s1] step 1400/12000 loss=5.6623 gn=0.51 lr=5.86e-04 tps=17530 mem=6.3G lam=None Gm=0.5013834635416666 +[g_only s1] step 1420/12000 loss=5.4324 gn=0.49 lr=5.86e-04 tps=17524 mem=6.3G lam=None Gm=0.4960530598958333 +[g_only s1] step 1440/12000 loss=5.5741 gn=0.55 lr=5.85e-04 tps=17520 mem=6.3G lam=None Gm=0.5017903645833334 +[g_only s1] step 1460/12000 loss=5.5879 gn=0.59 lr=5.85e-04 tps=17518 mem=6.3G lam=None Gm=0.5019938151041666 +[g_only s1] step 1480/12000 loss=5.4717 gn=0.51 lr=5.84e-04 tps=17516 mem=6.3G lam=None Gm=0.5011189778645834 +[g_only s1] step 1500/12000 loss=5.2913 gn=0.56 lr=5.84e-04 tps=17514 mem=6.3G lam=None Gm=0.5101725260416666 +[g_only s1] EVAL step 1500: val_loss=5.4958 val_ppl=243.67 +[g_only s1] step 1520/12000 loss=5.4862 gn=0.55 lr=5.83e-04 tps=17432 mem=6.3G lam=None Gm=0.5014851888020834 +[g_only s1] step 1540/12000 loss=5.4912 gn=0.60 lr=5.83e-04 tps=17435 mem=6.3G lam=None Gm=0.4924519856770833 +[g_only s1] step 1560/12000 loss=5.4581 gn=0.53 lr=5.82e-04 tps=17439 mem=6.3G lam=None Gm=0.4734903971354167 +[g_only s1] step 1580/12000 loss=5.1449 gn=0.51 lr=5.82e-04 tps=17442 mem=6.3G lam=None Gm=0.48455810546875 +[g_only s1] step 1600/12000 loss=5.2275 gn=0.57 lr=5.81e-04 tps=17445 mem=6.3G lam=None Gm=0.47247314453125 +[g_only s1] step 1620/12000 loss=5.2471 gn=0.55 lr=5.81e-04 tps=17449 mem=6.3G lam=None Gm=0.4798380533854167 +[g_only s1] step 1640/12000 loss=5.2394 gn=0.53 lr=5.80e-04 tps=17451 mem=6.3G lam=None Gm=0.4689127604166667 +[g_only s1] step 1660/12000 loss=5.3718 gn=0.54 lr=5.80e-04 tps=17454 mem=6.3G lam=None Gm=0.4813232421875 +[g_only s1] step 1680/12000 loss=5.1802 gn=0.56 lr=5.79e-04 tps=17458 mem=6.3G lam=None Gm=0.4839884440104167 +[g_only s1] step 1700/12000 loss=5.2555 gn=0.52 lr=5.79e-04 tps=17461 mem=6.3G lam=None Gm=0.4751383463541667 +[g_only s1] step 1720/12000 loss=5.3372 gn=0.56 lr=5.78e-04 tps=17464 mem=6.3G lam=None Gm=0.47662353515625 +[g_only s1] step 1740/12000 loss=5.2705 gn=0.53 lr=5.78e-04 tps=17467 mem=6.3G lam=None Gm=0.4689737955729167 +[g_only s1] step 1760/12000 loss=5.3088 gn=0.55 lr=5.77e-04 tps=17470 mem=6.3G lam=None Gm=0.4785359700520833 +[g_only s1] step 1780/12000 loss=5.3200 gn=0.56 lr=5.76e-04 tps=17474 mem=6.3G lam=None Gm=0.4659830729166667 +[g_only s1] step 1800/12000 loss=5.4155 gn=0.58 lr=5.76e-04 tps=17477 mem=6.3G lam=None Gm=0.4661051432291667 +[g_only s1] step 1820/12000 loss=5.4147 gn=0.55 lr=5.75e-04 tps=17480 mem=6.3G lam=None Gm=0.4720052083333333 +[g_only s1] step 1840/12000 loss=5.1878 gn=0.58 lr=5.75e-04 tps=17483 mem=6.3G lam=None Gm=0.4651896158854167 +[g_only s1] step 1860/12000 loss=5.2876 gn=0.61 lr=5.74e-04 tps=17486 mem=6.3G lam=None Gm=0.47296142578125 +[g_only s1] step 1880/12000 loss=5.1112 gn=0.58 lr=5.73e-04 tps=17488 mem=6.3G lam=None Gm=0.4586385091145833 +[g_only s1] step 1900/12000 loss=5.3322 gn=0.56 lr=5.73e-04 tps=17489 mem=6.3G lam=None Gm=0.4619954427083333 +[g_only s1] step 1920/12000 loss=5.3682 gn=0.55 lr=5.72e-04 tps=17491 mem=6.3G lam=None Gm=0.4544474283854167 +[g_only s1] step 1940/12000 loss=5.4947 gn=0.65 lr=5.72e-04 tps=17494 mem=6.3G lam=None Gm=0.4643758138020833 +[g_only s1] step 1960/12000 loss=5.1227 gn=0.63 lr=5.71e-04 tps=17497 mem=6.3G lam=None Gm=0.43365478515625 +[g_only s1] step 1980/12000 loss=5.2502 gn=0.59 lr=5.70e-04 tps=17499 mem=6.3G lam=None Gm=0.4436238606770833 +[g_only s1] step 2000/12000 loss=5.3033 gn=0.58 lr=5.70e-04 tps=17502 mem=6.3G lam=None Gm=0.4567057291666667 +[g_only s1] EVAL step 2000: val_loss=5.3342 val_ppl=207.30 +[g_only s1] step 2020/12000 loss=5.2059 gn=0.62 lr=5.69e-04 tps=17441 mem=6.3G lam=None Gm=0.4540812174479167 +[g_only s1] step 2040/12000 loss=5.2920 gn=0.60 lr=5.68e-04 tps=17444 mem=6.3G lam=None Gm=0.4493001302083333 +[g_only s1] step 2060/12000 loss=5.2231 gn=0.55 lr=5.68e-04 tps=17446 mem=6.3G lam=None Gm=0.4518229166666667 +[g_only s1] step 2080/12000 loss=5.2627 gn=0.63 lr=5.67e-04 tps=17449 mem=6.3G lam=None Gm=0.4603271484375 +[g_only s1] step 2100/12000 loss=5.2872 gn=0.58 lr=5.66e-04 tps=17452 mem=6.3G lam=None Gm=0.4472249348958333 +[g_only s1] step 2120/12000 loss=5.1534 gn=0.60 lr=5.65e-04 tps=17454 mem=6.3G lam=None Gm=0.4482014973958333 +[g_only s1] step 2140/12000 loss=5.1374 gn=0.65 lr=5.65e-04 tps=17457 mem=6.3G lam=None Gm=0.4360860188802083 +[g_only s1] step 2160/12000 loss=5.1858 gn=0.57 lr=5.64e-04 tps=17459 mem=6.3G lam=None Gm=0.4343363444010417 +[g_only s1] step 2180/12000 loss=5.2034 gn=0.60 lr=5.63e-04 tps=17462 mem=6.3G lam=None Gm=0.433197021484375 +[g_only s1] step 2200/12000 loss=5.1500 gn=0.60 lr=5.63e-04 tps=17464 mem=6.3G lam=None Gm=0.4318033854166667 +[g_only s1] step 2220/12000 loss=5.2906 gn=0.59 lr=5.62e-04 tps=17466 mem=6.3G lam=None Gm=0.4376118977864583 +[g_only s1] step 2240/12000 loss=5.0748 gn=0.60 lr=5.61e-04 tps=17469 mem=6.3G lam=None Gm=0.4405517578125 +[g_only s1] step 2260/12000 loss=5.0489 gn=0.60 lr=5.60e-04 tps=17471 mem=6.3G lam=None Gm=0.4287007649739583 +[g_only s1] step 2280/12000 loss=5.1508 gn=0.58 lr=5.60e-04 tps=17474 mem=6.3G lam=None Gm=0.4318949381510417 +[g_only s1] step 2300/12000 loss=5.0861 gn=0.59 lr=5.59e-04 tps=17476 mem=6.3G lam=None Gm=0.4255777994791667 +[g_only s1] step 2320/12000 loss=4.9651 gn=0.60 lr=5.58e-04 tps=17479 mem=6.3G lam=None Gm=0.4287109375 +[g_only s1] step 2340/12000 loss=5.1456 gn=0.58 lr=5.57e-04 tps=17481 mem=6.3G lam=None Gm=0.43035888671875 +[g_only s1] step 2360/12000 loss=5.2054 gn=0.60 lr=5.57e-04 tps=17482 mem=6.3G lam=None Gm=0.4256795247395833 +[g_only s1] step 2380/12000 loss=5.1884 gn=0.66 lr=5.56e-04 tps=17482 mem=6.3G lam=None Gm=0.432769775390625 +[g_only s1] step 2400/12000 loss=5.0963 gn=0.63 lr=5.55e-04 tps=17483 mem=6.3G lam=None Gm=0.440673828125 +[g_only s1] step 2420/12000 loss=5.1290 gn=0.62 lr=5.54e-04 tps=17483 mem=6.3G lam=None Gm=0.43072509765625 +[g_only s1] step 2440/12000 loss=5.1168 gn=0.63 lr=5.53e-04 tps=17482 mem=6.3G lam=None Gm=0.4240926106770833 +[g_only s1] step 2460/12000 loss=5.1411 gn=0.61 lr=5.53e-04 tps=17482 mem=6.3G lam=None Gm=0.4329020182291667 +[g_only s1] step 2480/12000 loss=5.1171 gn=0.66 lr=5.52e-04 tps=17481 mem=6.3G lam=None Gm=0.4241943359375 +[g_only s1] step 2500/12000 loss=5.1424 gn=0.64 lr=5.51e-04 tps=17479 mem=6.3G lam=None Gm=0.42279052734375 +[g_only s1] EVAL step 2500: val_loss=5.1718 val_ppl=176.24 +[g_only s1] step 2520/12000 loss=5.2243 gn=0.67 lr=5.50e-04 tps=17427 mem=6.3G lam=None Gm=0.4182942708333333 +[g_only s1] step 2540/12000 loss=5.1923 gn=0.67 lr=5.49e-04 tps=17426 mem=6.3G lam=None Gm=0.4267171223958333 +[g_only s1] step 2560/12000 loss=5.1589 gn=0.64 lr=5.48e-04 tps=17427 mem=6.3G lam=None Gm=0.424591064453125 +[g_only s1] step 2580/12000 loss=4.9864 gn=0.61 lr=5.48e-04 tps=17426 mem=6.3G lam=None Gm=0.408050537109375 +[g_only s1] step 2600/12000 loss=5.0671 gn=0.64 lr=5.47e-04 tps=17427 mem=6.3G lam=None Gm=0.41888427734375 +[g_only s1] step 2620/12000 loss=4.9870 gn=0.61 lr=5.46e-04 tps=17428 mem=6.3G lam=None Gm=0.4148457845052083 +[g_only s1] step 2640/12000 loss=5.1590 gn=0.64 lr=5.45e-04 tps=17430 mem=6.3G lam=None Gm=0.4182332356770833 +[g_only s1] step 2660/12000 loss=5.0170 gn=0.64 lr=5.44e-04 tps=17430 mem=6.3G lam=None Gm=0.4131876627604167 +[g_only s1] step 2680/12000 loss=5.0034 gn=0.62 lr=5.43e-04 tps=17432 mem=6.3G lam=None Gm=0.416534423828125 +[g_only s1] step 2700/12000 loss=5.1606 gn=0.64 lr=5.42e-04 tps=17434 mem=6.3G lam=None Gm=0.4107564290364583 +[g_only s1] step 2720/12000 loss=5.0008 gn=0.62 lr=5.41e-04 tps=17436 mem=6.3G lam=None Gm=0.4187723795572917 +[g_only s1] step 2740/12000 loss=5.1684 gn=0.70 lr=5.41e-04 tps=17438 mem=6.3G lam=None Gm=0.4188130696614583 +[g_only s1] step 2760/12000 loss=4.9872 gn=0.76 lr=5.40e-04 tps=17439 mem=6.3G lam=None Gm=0.417083740234375 +[g_only s1] step 2780/12000 loss=5.1211 gn=0.68 lr=5.39e-04 tps=17441 mem=6.3G lam=None Gm=0.405181884765625 +[g_only s1] step 2800/12000 loss=4.8778 gn=0.64 lr=5.38e-04 tps=17444 mem=6.3G lam=None Gm=0.3957417805989583 +[g_only s1] step 2820/12000 loss=5.1693 gn=0.62 lr=5.37e-04 tps=17446 mem=6.3G lam=None Gm=0.4107462565104167 +[g_only s1] step 2840/12000 loss=5.0537 gn=0.68 lr=5.36e-04 tps=17448 mem=6.3G lam=None Gm=0.4073588053385417 +[g_only s1] step 2860/12000 loss=4.9958 gn=0.69 lr=5.35e-04 tps=17449 mem=6.3G lam=None Gm=0.4192403157552083 +[g_only s1] step 2880/12000 loss=5.0392 gn=0.71 lr=5.34e-04 tps=17451 mem=6.3G lam=None Gm=0.4008585611979167 +[g_only s1] step 2900/12000 loss=5.0287 gn=0.63 lr=5.33e-04 tps=17453 mem=6.3G lam=None Gm=0.4063720703125 +[g_only s1] step 2920/12000 loss=4.9863 gn=0.71 lr=5.32e-04 tps=17455 mem=6.3G lam=None Gm=0.4036966959635417 +[g_only s1] step 2940/12000 loss=5.0662 gn=0.63 lr=5.31e-04 tps=17456 mem=6.3G lam=None Gm=0.40277099609375 +[g_only s1] step 2960/12000 loss=4.8499 gn=0.66 lr=5.30e-04 tps=17458 mem=6.3G lam=None Gm=0.3961893717447917 +[g_only s1] step 2980/12000 loss=4.9477 gn=0.61 lr=5.29e-04 tps=17460 mem=6.3G lam=None Gm=0.3992106119791667 +[g_only s1] step 3000/12000 loss=4.9370 gn=0.65 lr=5.28e-04 tps=17462 mem=6.3G lam=None Gm=0.4038798014322917 +[g_only s1] EVAL step 3000: val_loss=5.0280 val_ppl=152.63 +[g_only s1] step 3020/12000 loss=4.9838 gn=0.69 lr=5.27e-04 tps=17422 mem=6.3G lam=None Gm=0.4064229329427083 +[g_only s1] step 3040/12000 loss=4.8255 gn=0.63 lr=5.26e-04 tps=17423 mem=6.3G lam=None Gm=0.3988545735677083 +[g_only s1] step 3060/12000 loss=4.9766 gn=0.68 lr=5.25e-04 tps=17425 mem=6.3G lam=None Gm=0.3974812825520833 +[g_only s1] step 3080/12000 loss=4.8309 gn=0.67 lr=5.24e-04 tps=17427 mem=6.3G lam=None Gm=0.3967793782552083 +[g_only s1] step 3100/12000 loss=4.9876 gn=0.69 lr=5.23e-04 tps=17429 mem=6.3G lam=None Gm=0.3948262532552083 +[g_only s1] step 3120/12000 loss=4.9535 gn=0.65 lr=5.22e-04 tps=17431 mem=6.3G lam=None Gm=0.4093221028645833 +[g_only s1] step 3140/12000 loss=4.9247 gn=0.65 lr=5.21e-04 tps=17432 mem=6.3G lam=None Gm=0.4039408365885417 +[g_only s1] step 3160/12000 loss=4.9405 gn=0.65 lr=5.20e-04 tps=17434 mem=6.3G lam=None Gm=0.3929341634114583 +[g_only s1] step 3180/12000 loss=4.9870 gn=0.70 lr=5.19e-04 tps=17436 mem=6.3G lam=None Gm=0.3896280924479167 +[g_only s1] step 3200/12000 loss=4.7676 gn=0.65 lr=5.18e-04 tps=17437 mem=6.3G lam=None Gm=0.3953043619791667 +[g_only s1] step 3220/12000 loss=4.9594 gn=0.80 lr=5.17e-04 tps=17439 mem=6.3G lam=None Gm=0.40289306640625 +[g_only s1] step 3240/12000 loss=5.0038 gn=0.66 lr=5.16e-04 tps=17441 mem=6.3G lam=None Gm=0.39788818359375 +[g_only s1] step 3260/12000 loss=4.8818 gn=0.70 lr=5.15e-04 tps=17443 mem=6.3G lam=None Gm=0.396728515625 +[g_only s1] step 3280/12000 loss=4.8294 gn=0.69 lr=5.14e-04 tps=17445 mem=6.3G lam=None Gm=0.3914286295572917 +[g_only s1] step 3300/12000 loss=5.0099 gn=0.67 lr=5.13e-04 tps=17446 mem=6.3G lam=None Gm=0.3991800944010417 +[g_only s1] step 3320/12000 loss=5.0287 gn=0.65 lr=5.12e-04 tps=17448 mem=6.3G lam=None Gm=0.397796630859375 +[g_only s1] step 3340/12000 loss=4.8631 gn=0.61 lr=5.11e-04 tps=17450 mem=6.3G lam=None Gm=0.3956197102864583 +[g_only s1] step 3360/12000 loss=4.7950 gn=0.65 lr=5.10e-04 tps=17452 mem=6.3G lam=None Gm=0.394775390625 +[g_only s1] step 3380/12000 loss=5.0629 gn=0.65 lr=5.09e-04 tps=17454 mem=6.3G lam=None Gm=0.395782470703125 +[g_only s1] step 3400/12000 loss=5.1412 gn=0.67 lr=5.08e-04 tps=17456 mem=6.3G lam=None Gm=0.395660400390625 +[g_only s1] step 3420/12000 loss=5.0328 gn=0.70 lr=5.07e-04 tps=17458 mem=6.3G lam=None Gm=0.3959554036458333 +[g_only s1] step 3440/12000 loss=4.8408 gn=0.71 lr=5.06e-04 tps=17460 mem=6.3G lam=None Gm=0.3919779459635417 +[g_only s1] step 3460/12000 loss=4.9705 gn=0.73 lr=5.05e-04 tps=17462 mem=6.3G lam=None Gm=0.3860575358072917 +[g_only s1] step 3480/12000 loss=4.8525 gn=0.71 lr=5.03e-04 tps=17464 mem=6.3G lam=None Gm=0.3939615885416667 +[g_only s1] step 3500/12000 loss=4.8812 gn=0.70 lr=5.02e-04 tps=17466 mem=6.3G lam=None Gm=0.3855438232421875 +[g_only s1] EVAL step 3500: val_loss=4.9812 val_ppl=145.65 +[g_only s1] step 3520/12000 loss=4.8354 gn=0.70 lr=5.01e-04 tps=17431 mem=6.3G lam=None Gm=0.3933207194010417 +[g_only s1] step 3540/12000 loss=4.9336 gn=0.69 lr=5.00e-04 tps=17433 mem=6.3G lam=None Gm=0.3863728841145833 +[g_only s1] step 3560/12000 loss=4.9232 gn=0.72 lr=4.99e-04 tps=17435 mem=6.3G lam=None Gm=0.388702392578125 +[g_only s1] step 3580/12000 loss=4.9799 gn=0.80 lr=4.98e-04 tps=17436 mem=6.3G lam=None Gm=0.380462646484375 +[g_only s1] step 3600/12000 loss=5.0464 gn=0.71 lr=4.97e-04 tps=17438 mem=6.3G lam=None Gm=0.3907674153645833 +[g_only s1] step 3620/12000 loss=4.8929 gn=0.70 lr=4.96e-04 tps=17440 mem=6.3G lam=None Gm=0.3847096761067708 +[g_only s1] step 3640/12000 loss=4.7874 gn=0.69 lr=4.94e-04 tps=17441 mem=6.3G lam=None Gm=0.3949076334635417 +[g_only s1] step 3660/12000 loss=4.7326 gn=0.70 lr=4.93e-04 tps=17443 mem=6.3G lam=None Gm=0.3787282307942708 +[g_only s1] step 3680/12000 loss=4.7295 gn=0.70 lr=4.92e-04 tps=17444 mem=6.3G lam=None Gm=0.3822174072265625 +[g_only s1] step 3700/12000 loss=4.9625 gn=0.71 lr=4.91e-04 tps=17446 mem=6.3G lam=None Gm=0.38720703125 +[g_only s1] step 3720/12000 loss=4.8797 gn=0.71 lr=4.90e-04 tps=17448 mem=6.3G lam=None Gm=0.386627197265625 +[g_only s1] step 3740/12000 loss=4.8868 gn=0.68 lr=4.89e-04 tps=17449 mem=6.3G lam=None Gm=0.38140869140625 +[g_only s1] step 3760/12000 loss=4.8741 gn=0.75 lr=4.88e-04 tps=17451 mem=6.3G lam=None Gm=0.3845570882161458 +[g_only s1] step 3780/12000 loss=4.7642 gn=0.69 lr=4.86e-04 tps=17452 mem=6.3G lam=None Gm=0.385467529296875 +[g_only s1] step 3800/12000 loss=4.7744 gn=0.70 lr=4.85e-04 tps=17454 mem=6.3G lam=None Gm=0.3767293294270833 +[g_only s1] step 3820/12000 loss=4.8379 gn=0.67 lr=4.84e-04 tps=17455 mem=6.3G lam=None Gm=0.3859761555989583 +[g_only s1] step 3840/12000 loss=4.9107 gn=0.67 lr=4.83e-04 tps=17457 mem=6.3G lam=None Gm=0.3854573567708333 +[g_only s1] step 3860/12000 loss=4.7947 gn=0.71 lr=4.82e-04 tps=17458 mem=6.3G lam=None Gm=0.3789316813151042 +[g_only s1] step 3880/12000 loss=4.6660 gn=0.67 lr=4.80e-04 tps=17460 mem=6.3G lam=None Gm=0.3786671956380208 +[g_only s1] step 3900/12000 loss=4.7875 gn=0.73 lr=4.79e-04 tps=17462 mem=6.3G lam=None Gm=0.3799591064453125 +[g_only s1] step 3920/12000 loss=4.8976 gn=0.71 lr=4.78e-04 tps=17463 mem=6.3G lam=None Gm=0.3781331380208333 +[g_only s1] step 3940/12000 loss=4.7318 gn=0.68 lr=4.77e-04 tps=17465 mem=6.3G lam=None Gm=0.3866933186848958 +[g_only s1] step 3960/12000 loss=4.9535 gn=0.69 lr=4.76e-04 tps=17467 mem=6.3G lam=None Gm=0.3754984537760417 +[g_only s1] step 3980/12000 loss=5.0184 gn=0.72 lr=4.74e-04 tps=17468 mem=6.3G lam=None Gm=0.3777720133463542 +[g_only s1] step 4000/12000 loss=4.9299 gn=0.73 lr=4.73e-04 tps=17470 mem=6.3G lam=None Gm=0.37908935546875 +[g_only s1] EVAL step 4000: val_loss=4.8522 val_ppl=128.03 +[g_only s1] step 4020/12000 loss=4.5796 gn=0.73 lr=4.72e-04 tps=17439 mem=6.3G lam=None Gm=0.3863983154296875 +[g_only s1] step 4040/12000 loss=4.8268 gn=0.69 lr=4.71e-04 tps=17441 mem=6.3G lam=None Gm=0.3796335856119792 +[g_only s1] step 4060/12000 loss=4.9099 gn=0.82 lr=4.70e-04 tps=17443 mem=6.3G lam=None Gm=0.375823974609375 +[g_only s1] step 4080/12000 loss=4.6893 gn=0.70 lr=4.68e-04 tps=17444 mem=6.3G lam=None Gm=0.3745524088541667 +[g_only s1] step 4100/12000 loss=4.8327 gn=0.77 lr=4.67e-04 tps=17446 mem=6.3G lam=None Gm=0.3698679606119792 +[g_only s1] step 4120/12000 loss=4.8211 gn=0.78 lr=4.66e-04 tps=17447 mem=6.3G lam=None Gm=0.3814036051432292 +[g_only s1] step 4140/12000 loss=4.8977 gn=0.72 lr=4.65e-04 tps=17448 mem=6.3G lam=None Gm=0.38043212890625 +[g_only s1] step 4160/12000 loss=4.7234 gn=0.72 lr=4.63e-04 tps=17450 mem=6.3G lam=None Gm=0.3803151448567708 +[g_only s1] step 4180/12000 loss=4.9575 gn=0.73 lr=4.62e-04 tps=17451 mem=6.3G lam=None Gm=0.3775583902994792 +[g_only s1] step 4200/12000 loss=4.6648 gn=0.68 lr=4.61e-04 tps=17452 mem=6.3G lam=None Gm=0.3763885498046875 +[g_only s1] step 4220/12000 loss=4.7364 gn=0.68 lr=4.60e-04 tps=17454 mem=6.3G lam=None Gm=0.3742116292317708 +[g_only s1] step 4240/12000 loss=4.6960 gn=0.72 lr=4.58e-04 tps=17454 mem=6.3G lam=None Gm=0.3782857259114583 +[g_only s1] step 4260/12000 loss=4.7349 gn=0.70 lr=4.57e-04 tps=17455 mem=6.3G lam=None Gm=0.3737030029296875 +[g_only s1] step 4280/12000 loss=4.8699 gn=0.74 lr=4.56e-04 tps=17457 mem=6.3G lam=None Gm=0.3787027994791667 +[g_only s1] step 4300/12000 loss=4.9178 gn=0.71 lr=4.54e-04 tps=17458 mem=6.3G lam=None Gm=0.3741607666015625 +[g_only s1] step 4320/12000 loss=4.7089 gn=0.71 lr=4.53e-04 tps=17460 mem=6.3G lam=None Gm=0.3753458658854167 +[g_only s1] step 4340/12000 loss=4.7715 gn=0.74 lr=4.52e-04 tps=17461 mem=6.3G lam=None Gm=0.3725229899088542 +[g_only s1] step 4360/12000 loss=4.8277 gn=0.75 lr=4.51e-04 tps=17462 mem=6.3G lam=None Gm=0.3810170491536458 +[g_only s1] step 4380/12000 loss=4.6967 gn=0.73 lr=4.49e-04 tps=17464 mem=6.3G lam=None Gm=0.3679911295572917 +[g_only s1] step 4400/12000 loss=4.8134 gn=0.71 lr=4.48e-04 tps=17465 mem=6.3G lam=None Gm=0.3666585286458333 +[g_only s1] step 4420/12000 loss=4.7916 gn=0.74 lr=4.47e-04 tps=17467 mem=6.3G lam=None Gm=0.3729044596354167 +[g_only s1] step 4440/12000 loss=4.7036 gn=0.76 lr=4.45e-04 tps=17468 mem=6.3G lam=None Gm=0.37213134765625 +[g_only s1] step 4460/12000 loss=4.5284 gn=0.71 lr=4.44e-04 tps=17469 mem=6.3G lam=None Gm=0.3770345052083333 +[g_only s1] step 4480/12000 loss=4.6711 gn=0.71 lr=4.43e-04 tps=17471 mem=6.3G lam=None Gm=0.3812459309895833 +[g_only s1] step 4500/12000 loss=4.9377 gn=0.73 lr=4.42e-04 tps=17472 mem=6.3G lam=None Gm=0.3671213785807292 +[g_only s1] EVAL step 4500: val_loss=4.7567 val_ppl=116.36 +[g_only s1] step 4520/12000 loss=4.8148 gn=0.72 lr=4.40e-04 tps=17445 mem=6.3G lam=None Gm=0.3761698404947917 +[g_only s1] step 4540/12000 loss=4.7806 gn=0.72 lr=4.39e-04 tps=17446 mem=6.3G lam=None Gm=0.372344970703125 +[g_only s1] step 4560/12000 loss=4.6561 gn=0.70 lr=4.38e-04 tps=17448 mem=6.3G lam=None Gm=0.3755340576171875 +[g_only s1] step 4580/12000 loss=4.8362 gn=0.75 lr=4.36e-04 tps=17449 mem=6.3G lam=None Gm=0.3760223388671875 +[g_only s1] step 4600/12000 loss=4.6781 gn=0.76 lr=4.35e-04 tps=17450 mem=6.3G lam=None Gm=0.3758087158203125 +[g_only s1] step 4620/12000 loss=4.6641 gn=0.73 lr=4.34e-04 tps=17451 mem=6.3G lam=None Gm=0.3749593098958333 +[g_only s1] step 4640/12000 loss=4.8244 gn=0.72 lr=4.32e-04 tps=17452 mem=6.3G lam=None Gm=0.372528076171875 +[g_only s1] step 4660/12000 loss=4.7033 gn=0.74 lr=4.31e-04 tps=17453 mem=6.3G lam=None Gm=0.3779500325520833 +[g_only s1] step 4680/12000 loss=4.7977 gn=0.73 lr=4.30e-04 tps=17455 mem=6.3G lam=None Gm=0.3743133544921875 +[g_only s1] step 4700/12000 loss=4.7487 gn=0.75 lr=4.28e-04 tps=17456 mem=6.3G lam=None Gm=0.3710988362630208 +[g_only s1] step 4720/12000 loss=4.7932 gn=0.73 lr=4.27e-04 tps=17457 mem=6.3G lam=None Gm=0.3699188232421875 +[g_only s1] step 4740/12000 loss=4.7239 gn=0.73 lr=4.26e-04 tps=17458 mem=6.3G lam=None Gm=0.3738861083984375 +[g_only s1] step 4760/12000 loss=4.8175 gn=0.70 lr=4.24e-04 tps=17459 mem=6.3G lam=None Gm=0.3666127522786458 +[g_only s1] step 4780/12000 loss=4.6124 gn=0.73 lr=4.23e-04 tps=17460 mem=6.3G lam=None Gm=0.3711954752604167 +[g_only s1] step 4800/12000 loss=4.5894 gn=0.70 lr=4.22e-04 tps=17461 mem=6.3G lam=None Gm=0.3662261962890625 +[g_only s1] step 4820/12000 loss=4.7644 gn=0.71 lr=4.20e-04 tps=17462 mem=6.3G lam=None Gm=0.3689422607421875 +[g_only s1] step 4840/12000 loss=4.7410 gn=0.75 lr=4.19e-04 tps=17464 mem=6.3G lam=None Gm=0.36505126953125 +[g_only s1] step 4860/12000 loss=4.8375 gn=0.68 lr=4.18e-04 tps=17465 mem=6.3G lam=None Gm=0.3694712320963542 +[g_only s1] step 4880/12000 loss=4.6025 gn=0.73 lr=4.16e-04 tps=17466 mem=6.3G lam=None Gm=0.3762054443359375 +[g_only s1] step 4900/12000 loss=4.7250 gn=0.70 lr=4.15e-04 tps=17467 mem=6.3G lam=None Gm=0.3689676920572917 +[g_only s1] step 4920/12000 loss=4.7525 gn=0.73 lr=4.13e-04 tps=17468 mem=6.3G lam=None Gm=0.3674570719401042 +[g_only s1] step 4940/12000 loss=4.5086 gn=0.76 lr=4.12e-04 tps=17470 mem=6.3G lam=None Gm=0.3556365966796875 +[g_only s1] step 4960/12000 loss=4.7218 gn=0.76 lr=4.11e-04 tps=17471 mem=6.3G lam=None Gm=0.3737131754557292 +[g_only s1] step 4980/12000 loss=4.8352 gn=0.75 lr=4.09e-04 tps=17472 mem=6.3G lam=None Gm=0.3632100423177083 +[g_only s1] step 5000/12000 loss=4.7083 gn=0.77 lr=4.08e-04 tps=17473 mem=6.3G lam=None Gm=0.3760325113932292 +[g_only s1] EVAL step 5000: val_loss=4.7352 val_ppl=113.89 +[g_only s1] step 5020/12000 loss=4.6610 gn=0.74 lr=4.07e-04 tps=17449 mem=6.3G lam=None Gm=0.3663889567057292 +[g_only s1] step 5040/12000 loss=4.6124 gn=0.76 lr=4.05e-04 tps=17449 mem=6.3G lam=None Gm=0.3631846110026042 +[g_only s1] step 5060/12000 loss=4.7483 gn=0.71 lr=4.04e-04 tps=17451 mem=6.3G lam=None Gm=0.3625996907552083 +[g_only s1] step 5080/12000 loss=4.7551 gn=0.78 lr=4.02e-04 tps=17452 mem=6.3G lam=None Gm=0.3662821451822917 +[g_only s1] step 5100/12000 loss=4.6228 gn=0.73 lr=4.01e-04 tps=17453 mem=6.3G lam=None Gm=0.3630472819010417 +[g_only s1] step 5120/12000 loss=4.7290 gn=0.77 lr=4.00e-04 tps=17454 mem=6.3G lam=None Gm=0.3684590657552083 +[g_only s1] step 5140/12000 loss=4.7067 gn=0.74 lr=3.98e-04 tps=17455 mem=6.3G lam=None Gm=0.3562418619791667 +[g_only s1] step 5160/12000 loss=4.7419 gn=0.79 lr=3.97e-04 tps=17456 mem=6.3G lam=None Gm=0.372283935546875 +[g_only s1] step 5180/12000 loss=4.7764 gn=0.79 lr=3.95e-04 tps=17456 mem=6.3G lam=None Gm=0.3670450846354167 +[g_only s1] step 5200/12000 loss=4.7335 gn=0.78 lr=3.94e-04 tps=17457 mem=6.3G lam=None Gm=0.367523193359375 +[g_only s1] step 5220/12000 loss=4.5492 gn=0.73 lr=3.93e-04 tps=17458 mem=6.3G lam=None Gm=0.361053466796875 +[g_only s1] step 5240/12000 loss=4.8835 gn=0.81 lr=3.91e-04 tps=17459 mem=6.3G lam=None Gm=0.3613026936848958 +[g_only s1] step 5260/12000 loss=4.6359 gn=0.75 lr=3.90e-04 tps=17460 mem=6.3G lam=None Gm=0.3608042399088542 +[g_only s1] step 5280/12000 loss=4.5187 gn=0.72 lr=3.88e-04 tps=17462 mem=6.3G lam=None Gm=0.3667551676432292 +[g_only s1] step 5300/12000 loss=4.6555 gn=0.74 lr=3.87e-04 tps=17463 mem=6.3G lam=None Gm=0.3676961263020833 +[g_only s1] step 5320/12000 loss=4.6398 gn=0.79 lr=3.86e-04 tps=17463 mem=6.3G lam=None Gm=0.3649851481119792 +[g_only s1] step 5340/12000 loss=4.7309 gn=0.75 lr=3.84e-04 tps=17464 mem=6.3G lam=None Gm=0.3657023111979167 +[g_only s1] step 5360/12000 loss=4.4851 gn=0.76 lr=3.83e-04 tps=17465 mem=6.3G lam=None Gm=0.3590443929036458 +[g_only s1] step 5380/12000 loss=4.5391 gn=0.82 lr=3.81e-04 tps=17466 mem=6.3G lam=None Gm=0.3706258138020833 +[g_only s1] step 5400/12000 loss=4.5542 gn=0.75 lr=3.80e-04 tps=17468 mem=6.3G lam=None Gm=0.3594411214192708 +[g_only s1] step 5420/12000 loss=4.6301 gn=0.70 lr=3.79e-04 tps=17469 mem=6.3G lam=None Gm=0.3555755615234375 +[g_only s1] step 5440/12000 loss=4.7573 gn=0.78 lr=3.77e-04 tps=17470 mem=6.3G lam=None Gm=0.3597005208333333 +[g_only s1] step 5460/12000 loss=4.6616 gn=0.76 lr=3.76e-04 tps=17471 mem=6.3G lam=None Gm=0.3613026936848958 +[g_only s1] step 5480/12000 loss=4.6654 gn=0.78 lr=3.74e-04 tps=17472 mem=6.3G lam=None Gm=0.3580271402994792 +[g_only s1] step 5500/12000 loss=4.6485 gn=0.81 lr=3.73e-04 tps=17473 mem=6.3G lam=None Gm=0.36041259765625 +[g_only s1] EVAL step 5500: val_loss=4.6726 val_ppl=106.97 +[g_only s1] step 5520/12000 loss=4.4886 gn=0.76 lr=3.72e-04 tps=17451 mem=6.3G lam=None Gm=0.3600362141927083 +[g_only s1] step 5540/12000 loss=4.6311 gn=0.81 lr=3.70e-04 tps=17452 mem=6.3G lam=None Gm=0.36376953125 +[g_only s1] step 5560/12000 loss=4.6313 gn=0.74 lr=3.69e-04 tps=17453 mem=6.3G lam=None Gm=0.3683878580729167 +[g_only s1] step 5580/12000 loss=4.4919 gn=0.75 lr=3.67e-04 tps=17454 mem=6.3G lam=None Gm=0.3597768147786458 +[g_only s1] step 5600/12000 loss=4.6804 gn=0.79 lr=3.66e-04 tps=17456 mem=6.3G lam=None Gm=0.361114501953125 +[g_only s1] step 5620/12000 loss=4.6923 gn=0.80 lr=3.64e-04 tps=17456 mem=6.3G lam=None Gm=0.3608500162760417 +[g_only s1] step 5640/12000 loss=4.6291 gn=0.75 lr=3.63e-04 tps=17457 mem=6.3G lam=None Gm=0.3607381184895833 +[g_only s1] step 5660/12000 loss=4.7410 gn=0.78 lr=3.62e-04 tps=17458 mem=6.3G lam=None Gm=0.3626505533854167 +[g_only s1] step 5680/12000 loss=4.7808 gn=0.81 lr=3.60e-04 tps=17459 mem=6.3G lam=None Gm=0.3603922526041667 +[g_only s1] step 5700/12000 loss=4.6007 gn=0.77 lr=3.59e-04 tps=17460 mem=6.3G lam=None Gm=0.3546142578125 +[g_only s1] step 5720/12000 loss=4.4941 gn=0.78 lr=3.57e-04 tps=17461 mem=6.3G lam=None Gm=0.3608500162760417 +[g_only s1] step 5740/12000 loss=4.6206 gn=0.75 lr=3.56e-04 tps=17462 mem=6.3G lam=None Gm=0.3602701822916667 +[g_only s1] step 5760/12000 loss=4.6728 gn=0.77 lr=3.54e-04 tps=17463 mem=6.3G lam=None Gm=0.3581136067708333 +[g_only s1] step 5780/12000 loss=4.6041 gn=0.79 lr=3.53e-04 tps=17464 mem=6.3G lam=None Gm=0.3518727620442708 +[g_only s1] step 5800/12000 loss=4.5862 gn=0.77 lr=3.52e-04 tps=17465 mem=6.3G lam=None Gm=0.3592529296875 +[g_only s1] step 5820/12000 loss=4.5817 gn=0.74 lr=3.50e-04 tps=17466 mem=6.3G lam=None Gm=0.355010986328125 +[g_only s1] step 5840/12000 loss=4.5493 gn=0.78 lr=3.49e-04 tps=17467 mem=6.3G lam=None Gm=0.3570709228515625 +[g_only s1] step 5860/12000 loss=4.5665 gn=0.78 lr=3.47e-04 tps=17468 mem=6.3G lam=None Gm=0.35968017578125 +[g_only s1] step 5880/12000 loss=4.5293 gn=0.77 lr=3.46e-04 tps=17469 mem=6.3G lam=None Gm=0.354766845703125 +[g_only s1] step 5900/12000 loss=4.5102 gn=0.75 lr=3.44e-04 tps=17470 mem=6.3G lam=None Gm=0.3643239339192708 +[g_only s1] step 5920/12000 loss=4.6192 gn=0.81 lr=3.43e-04 tps=17471 mem=6.3G lam=None Gm=0.3586680094401042 +[g_only s1] step 5940/12000 loss=4.5841 gn=0.81 lr=3.41e-04 tps=17472 mem=6.3G lam=None Gm=0.3606923421223958 +[g_only s1] step 5960/12000 loss=4.7438 gn=0.77 lr=3.40e-04 tps=17473 mem=6.3G lam=None Gm=0.3596750895182292 +[g_only s1] step 5980/12000 loss=4.5258 gn=0.75 lr=3.39e-04 tps=17474 mem=6.3G lam=None Gm=0.3533681233723958 +[g_only s1] step 6000/12000 loss=4.5599 gn=0.80 lr=3.37e-04 tps=17475 mem=6.3G lam=None Gm=0.3537038167317708 +[g_only s1] EVAL step 6000: val_loss=4.6906 val_ppl=108.91 +[g_only s1] step 6020/12000 loss=4.6416 gn=0.79 lr=3.36e-04 tps=17455 mem=6.3G lam=None Gm=0.3661041259765625 +[g_only s1] step 6040/12000 loss=4.5218 gn=0.78 lr=3.34e-04 tps=17456 mem=6.3G lam=None Gm=0.3648732503255208 +[g_only s1] step 6060/12000 loss=4.4327 gn=0.73 lr=3.33e-04 tps=17457 mem=6.3G lam=None Gm=0.3562571207682292 +[g_only s1] step 6080/12000 loss=4.4433 gn=0.78 lr=3.31e-04 tps=17458 mem=6.3G lam=None Gm=0.3553212483723958 +[g_only s1] step 6100/12000 loss=4.4608 gn=0.78 lr=3.30e-04 tps=17459 mem=6.3G lam=None Gm=0.3536580403645833 +[g_only s1] step 6120/12000 loss=4.5708 gn=0.74 lr=3.29e-04 tps=17460 mem=6.3G lam=None Gm=0.35491943359375 +[g_only s1] step 6140/12000 loss=4.4864 gn=0.78 lr=3.27e-04 tps=17461 mem=6.3G lam=None Gm=0.351837158203125 +[g_only s1] step 6160/12000 loss=4.4914 gn=0.79 lr=3.26e-04 tps=17461 mem=6.3G lam=None Gm=0.3553568522135417 +[g_only s1] step 6180/12000 loss=4.6172 gn=0.82 lr=3.24e-04 tps=17462 mem=6.3G lam=None Gm=0.35418701171875 +[g_only s1] step 6200/12000 loss=4.5416 gn=0.78 lr=3.23e-04 tps=17463 mem=6.3G lam=None Gm=0.3566029866536458 +[g_only s1] step 6220/12000 loss=4.5817 gn=0.82 lr=3.21e-04 tps=17464 mem=6.3G lam=None Gm=0.3485972086588542 +[g_only s1] step 6240/12000 loss=4.5330 gn=0.81 lr=3.20e-04 tps=17464 mem=6.3G lam=None Gm=0.3582051595052083 +[g_only s1] step 6260/12000 loss=4.5511 gn=0.81 lr=3.19e-04 tps=17465 mem=6.3G lam=None Gm=0.3601837158203125 +[g_only s1] step 6280/12000 loss=4.6138 gn=0.77 lr=3.17e-04 tps=17465 mem=6.3G lam=None Gm=0.35284423828125 +[g_only s1] step 6300/12000 loss=4.4477 gn=0.80 lr=3.16e-04 tps=17465 mem=6.3G lam=None Gm=0.3530222574869792 +[g_only s1] step 6320/12000 loss=4.6393 gn=0.78 lr=3.14e-04 tps=17466 mem=6.3G lam=None Gm=0.35357666015625 +[g_only s1] step 6340/12000 loss=4.4764 gn=0.79 lr=3.13e-04 tps=17466 mem=6.3G lam=None Gm=0.3550313313802083 +[g_only s1] step 6360/12000 loss=4.5385 gn=0.80 lr=3.11e-04 tps=17467 mem=6.3G lam=None Gm=0.3601735432942708 +[g_only s1] step 6380/12000 loss=4.5891 gn=0.79 lr=3.10e-04 tps=17468 mem=6.3G lam=None Gm=0.3587544759114583 +[g_only s1] step 6400/12000 loss=4.5246 gn=0.84 lr=3.08e-04 tps=17468 mem=6.3G lam=None Gm=0.36456298828125 +[g_only s1] step 6420/12000 loss=4.5841 gn=0.82 lr=3.07e-04 tps=17468 mem=6.3G lam=None Gm=0.3588612874348958 +[g_only s1] step 6440/12000 loss=4.5538 gn=0.79 lr=3.06e-04 tps=17469 mem=6.3G lam=None Gm=0.3623758951822917 +[g_only s1] step 6460/12000 loss=4.6183 gn=0.80 lr=3.04e-04 tps=17469 mem=6.3G lam=None Gm=0.3533681233723958 +[g_only s1] step 6480/12000 loss=4.4120 gn=0.78 lr=3.03e-04 tps=17470 mem=6.3G lam=None Gm=0.3542887369791667 +[g_only s1] step 6500/12000 loss=4.5033 gn=0.79 lr=3.01e-04 tps=17470 mem=6.3G lam=None Gm=0.3512420654296875 +[g_only s1] EVAL step 6500: val_loss=4.6314 val_ppl=102.66 +[g_only s1] step 6520/12000 loss=4.4911 gn=0.77 lr=3.00e-04 tps=17451 mem=6.3G lam=None Gm=0.3529052734375 +[g_only s1] step 6540/12000 loss=4.6209 gn=0.81 lr=2.98e-04 tps=17451 mem=6.3G lam=None Gm=0.3534444173177083 +[g_only s1] step 6560/12000 loss=4.5796 gn=0.80 lr=2.97e-04 tps=17452 mem=6.3G lam=None Gm=0.35638427734375 +[g_only s1] step 6580/12000 loss=4.5348 gn=0.77 lr=2.96e-04 tps=17452 mem=6.3G lam=None Gm=0.355010986328125 +[g_only s1] step 6600/12000 loss=4.3250 gn=0.78 lr=2.94e-04 tps=17453 mem=6.3G lam=None Gm=0.34930419921875 +[g_only s1] step 6620/12000 loss=4.7558 gn=0.80 lr=2.93e-04 tps=17453 mem=6.3G lam=None Gm=0.3445994059244792 +[g_only s1] step 6640/12000 loss=4.5436 gn=0.82 lr=2.91e-04 tps=17453 mem=6.3G lam=None Gm=0.3557027180989583 +[g_only s1] step 6660/12000 loss=4.5070 gn=0.83 lr=2.90e-04 tps=17454 mem=6.3G lam=None Gm=0.3510894775390625 +[g_only s1] step 6680/12000 loss=4.4870 gn=0.77 lr=2.88e-04 tps=17454 mem=6.3G lam=None Gm=0.3523661295572917 +[g_only s1] step 6700/12000 loss=4.5078 gn=0.78 lr=2.87e-04 tps=17454 mem=6.3G lam=None Gm=0.3486378987630208 +[g_only s1] step 6720/12000 loss=4.5128 gn=0.83 lr=2.86e-04 tps=17455 mem=6.3G lam=None Gm=0.3490498860677083 +[g_only s1] step 6740/12000 loss=4.5103 gn=0.80 lr=2.84e-04 tps=17455 mem=6.3G lam=None Gm=0.3537546793619792 +[g_only s1] step 6760/12000 loss=4.5623 gn=0.81 lr=2.83e-04 tps=17455 mem=6.3G lam=None Gm=0.3597819010416667 +[g_only s1] step 6780/12000 loss=4.5500 gn=0.81 lr=2.81e-04 tps=17456 mem=6.3G lam=None Gm=0.3502858479817708 +[g_only s1] step 6800/12000 loss=4.5062 gn=0.89 lr=2.80e-04 tps=17456 mem=6.3G lam=None Gm=0.3533884684244792 +[g_only s1] step 6820/12000 loss=4.6571 gn=0.87 lr=2.79e-04 tps=17456 mem=6.3G lam=None Gm=0.3542226155598958 +[g_only s1] step 6840/12000 loss=4.5728 gn=0.83 lr=2.77e-04 tps=17457 mem=6.3G lam=None Gm=0.3551228841145833 +[g_only s1] step 6860/12000 loss=4.4708 gn=0.82 lr=2.76e-04 tps=17457 mem=6.3G lam=None Gm=0.346466064453125 +[g_only s1] step 6880/12000 loss=4.5120 gn=0.82 lr=2.74e-04 tps=17457 mem=6.3G lam=None Gm=0.3535868326822917 +[g_only s1] step 6900/12000 loss=4.4990 gn=0.82 lr=2.73e-04 tps=17458 mem=6.3G lam=None Gm=0.3566538492838542 +[g_only s1] step 6920/12000 loss=4.4122 gn=0.81 lr=2.72e-04 tps=17458 mem=6.3G lam=None Gm=0.3524932861328125 +[g_only s1] step 6940/12000 loss=4.4316 gn=0.79 lr=2.70e-04 tps=17458 mem=6.3G lam=None Gm=0.3551025390625 +[g_only s1] step 6960/12000 loss=4.4941 gn=0.83 lr=2.69e-04 tps=17459 mem=6.3G lam=None Gm=0.3448232014973958 +[g_only s1] step 6980/12000 loss=4.3580 gn=0.80 lr=2.67e-04 tps=17459 mem=6.3G lam=None Gm=0.3525950113932292 +[g_only s1] step 7000/12000 loss=4.3722 gn=0.80 lr=2.66e-04 tps=17460 mem=6.3G lam=None Gm=0.3452301025390625 +[g_only s1] EVAL step 7000: val_loss=4.5723 val_ppl=96.76 +[g_only s1] step 7020/12000 loss=4.5216 gn=0.84 lr=2.65e-04 tps=17442 mem=6.3G lam=None Gm=0.3577372233072917 +[g_only s1] step 7040/12000 loss=4.6144 gn=0.81 lr=2.63e-04 tps=17442 mem=6.3G lam=None Gm=0.3582814534505208 +[g_only s1] step 7060/12000 loss=4.5114 gn=0.80 lr=2.62e-04 tps=17443 mem=6.3G lam=None Gm=0.3539683024088542 +[g_only s1] step 7080/12000 loss=4.4156 gn=0.79 lr=2.60e-04 tps=17443 mem=6.3G lam=None Gm=0.3539021809895833 +[g_only s1] step 7100/12000 loss=4.5662 gn=0.85 lr=2.59e-04 tps=17444 mem=6.3G lam=None Gm=0.35186767578125 +[g_only s1] step 7120/12000 loss=4.4632 gn=0.81 lr=2.58e-04 tps=17444 mem=6.3G lam=None Gm=0.3457234700520833 +[g_only s1] step 7140/12000 loss=4.4194 gn=0.85 lr=2.56e-04 tps=17445 mem=6.3G lam=None Gm=0.3402506510416667 +[g_only s1] step 7160/12000 loss=4.3620 gn=0.82 lr=2.55e-04 tps=17445 mem=6.3G lam=None Gm=0.3391316731770833 +[g_only s1] step 7180/12000 loss=4.5742 gn=0.82 lr=2.53e-04 tps=17445 mem=6.3G lam=None Gm=0.349273681640625 +[g_only s1] step 7200/12000 loss=4.4749 gn=0.83 lr=2.52e-04 tps=17446 mem=6.3G lam=None Gm=0.3492482503255208 +[g_only s1] step 7220/12000 loss=4.5561 gn=0.86 lr=2.51e-04 tps=17446 mem=6.3G lam=None Gm=0.3484649658203125 +[g_only s1] step 7240/12000 loss=4.4641 gn=0.83 lr=2.49e-04 tps=17446 mem=6.3G lam=None Gm=0.3487955729166667 +[g_only s1] step 7260/12000 loss=4.5183 gn=0.85 lr=2.48e-04 tps=17447 mem=6.3G lam=None Gm=0.3523712158203125 +[g_only s1] step 7280/12000 loss=4.5972 gn=0.82 lr=2.47e-04 tps=17447 mem=6.3G lam=None Gm=0.3527984619140625 +[g_only s1] step 7300/12000 loss=4.4203 gn=0.82 lr=2.45e-04 tps=17447 mem=6.3G lam=None Gm=0.34539794921875 +[g_only s1] step 7320/12000 loss=4.4788 gn=0.79 lr=2.44e-04 tps=17448 mem=6.3G lam=None Gm=0.35272216796875 +[g_only s1] step 7340/12000 loss=4.5019 gn=0.85 lr=2.42e-04 tps=17448 mem=6.3G lam=None Gm=0.3497823079427083 +[g_only s1] step 7360/12000 loss=4.5075 gn=0.83 lr=2.41e-04 tps=17448 mem=6.3G lam=None Gm=0.3462575276692708 +[g_only s1] step 7380/12000 loss=4.3543 gn=0.84 lr=2.40e-04 tps=17449 mem=6.3G lam=None Gm=0.3486124674479167 +[g_only s1] step 7400/12000 loss=4.3707 gn=0.86 lr=2.38e-04 tps=17449 mem=6.3G lam=None Gm=0.3422088623046875 +[g_only s1] step 7420/12000 loss=4.5448 gn=0.82 lr=2.37e-04 tps=17449 mem=6.3G lam=None Gm=0.3496500651041667 +[g_only s1] step 7440/12000 loss=4.5701 gn=0.84 lr=2.36e-04 tps=17450 mem=6.3G lam=None Gm=0.3588765462239583 +[g_only s1] step 7460/12000 loss=4.4348 gn=0.83 lr=2.34e-04 tps=17450 mem=6.3G lam=None Gm=0.34765625 +[g_only s1] step 7480/12000 loss=4.6745 gn=0.87 lr=2.33e-04 tps=17451 mem=6.3G lam=None Gm=0.3489278157552083 +[g_only s1] step 7500/12000 loss=4.3107 gn=0.83 lr=2.32e-04 tps=17451 mem=6.3G lam=None Gm=0.3455912272135417 +[g_only s1] EVAL step 7500: val_loss=4.5501 val_ppl=94.64 +[g_only s1] step 7520/12000 loss=4.5422 gn=0.85 lr=2.30e-04 tps=17435 mem=6.3G lam=None Gm=0.3443654378255208 +[g_only s1] step 7540/12000 loss=4.3941 gn=0.83 lr=2.29e-04 tps=17435 mem=6.3G lam=None Gm=0.3415374755859375 +[g_only s1] step 7560/12000 loss=4.4230 gn=0.86 lr=2.28e-04 tps=17436 mem=6.3G lam=None Gm=0.3556772867838542 +[g_only s1] step 7580/12000 loss=4.3733 gn=0.80 lr=2.26e-04 tps=17436 mem=6.3G lam=None Gm=0.351470947265625 +[g_only s1] step 7600/12000 loss=4.4062 gn=0.82 lr=2.25e-04 tps=17436 mem=6.3G lam=None Gm=0.3468373616536458 +[g_only s1] step 7620/12000 loss=4.3840 gn=0.84 lr=2.24e-04 tps=17437 mem=6.3G lam=None Gm=0.3490397135416667 +[g_only s1] step 7640/12000 loss=4.4150 gn=0.85 lr=2.22e-04 tps=17437 mem=6.3G lam=None Gm=0.353485107421875 +[g_only s1] step 7660/12000 loss=4.5016 gn=0.88 lr=2.21e-04 tps=17437 mem=6.3G lam=None Gm=0.3591054280598958 +[g_only s1] step 7680/12000 loss=4.3737 gn=0.88 lr=2.20e-04 tps=17437 mem=6.3G lam=None Gm=0.35015869140625 +[g_only s1] step 7700/12000 loss=4.4779 gn=0.88 lr=2.18e-04 tps=17438 mem=6.3G lam=None Gm=0.3482462565104167 +[g_only s1] step 7720/12000 loss=4.5342 gn=0.89 lr=2.17e-04 tps=17438 mem=6.3G lam=None Gm=0.3534393310546875 +[g_only s1] step 7740/12000 loss=4.3530 gn=0.84 lr=2.16e-04 tps=17438 mem=6.3G lam=None Gm=0.33978271484375 +[g_only s1] step 7760/12000 loss=4.5712 gn=0.84 lr=2.15e-04 tps=17439 mem=6.3G lam=None Gm=0.3506266276041667 +[g_only s1] step 7780/12000 loss=4.4305 gn=0.82 lr=2.13e-04 tps=17439 mem=6.3G lam=None Gm=0.3449910481770833 +[g_only s1] step 7800/12000 loss=4.4880 gn=0.89 lr=2.12e-04 tps=17439 mem=6.3G lam=None Gm=0.3510945638020833 +[g_only s1] step 7820/12000 loss=4.3597 gn=0.86 lr=2.11e-04 tps=17440 mem=6.3G lam=None Gm=0.3475240071614583 +[g_only s1] step 7840/12000 loss=4.4897 gn=0.86 lr=2.09e-04 tps=17440 mem=6.3G lam=None Gm=0.3510487874348958 +[g_only s1] step 7860/12000 loss=4.4413 gn=0.89 lr=2.08e-04 tps=17440 mem=6.3G lam=None Gm=0.347442626953125 +[g_only s1] step 7880/12000 loss=4.4076 gn=0.87 lr=2.07e-04 tps=17441 mem=6.3G lam=None Gm=0.3477986653645833 +[g_only s1] step 7900/12000 loss=4.5726 gn=0.89 lr=2.06e-04 tps=17441 mem=6.3G lam=None Gm=0.3448435465494792 +[g_only s1] step 7920/12000 loss=4.4234 gn=0.89 lr=2.04e-04 tps=17442 mem=6.3G lam=None Gm=0.342498779296875 +[g_only s1] step 7940/12000 loss=4.4795 gn=0.88 lr=2.03e-04 tps=17442 mem=6.3G lam=None Gm=0.3476460774739583 +[g_only s1] step 7960/12000 loss=4.5352 gn=0.89 lr=2.02e-04 tps=17443 mem=6.3G lam=None Gm=0.3520151774088542 +[g_only s1] step 7980/12000 loss=4.2547 gn=0.91 lr=2.00e-04 tps=17443 mem=6.3G lam=None Gm=0.3500162760416667 +[g_only s1] step 8000/12000 loss=4.3374 gn=0.85 lr=1.99e-04 tps=17443 mem=6.3G lam=None Gm=0.3435160319010417 +[g_only s1] EVAL step 8000: val_loss=4.5351 val_ppl=93.24 +[g_only s1] step 8020/12000 loss=4.6167 gn=0.87 lr=1.98e-04 tps=17428 mem=6.3G lam=None Gm=0.3466695149739583 +[g_only s1] step 8040/12000 loss=4.4102 gn=0.84 lr=1.97e-04 tps=17429 mem=6.3G lam=None Gm=0.3445485432942708 +[g_only s1] step 8060/12000 loss=4.2926 gn=0.84 lr=1.95e-04 tps=17429 mem=6.3G lam=None Gm=0.345306396484375 +[g_only s1] step 8080/12000 loss=4.4008 gn=0.89 lr=1.94e-04 tps=17430 mem=6.3G lam=None Gm=0.3501739501953125 +[g_only s1] step 8100/12000 loss=4.4035 gn=0.86 lr=1.93e-04 tps=17430 mem=6.3G lam=None Gm=0.3456268310546875 +[g_only s1] step 8120/12000 loss=4.4312 gn=0.81 lr=1.92e-04 tps=17431 mem=6.3G lam=None Gm=0.3487701416015625 +[g_only s1] step 8140/12000 loss=4.3963 gn=0.86 lr=1.90e-04 tps=17431 mem=6.3G lam=None Gm=0.3452504475911458 +[g_only s1] step 8160/12000 loss=4.5097 gn=0.91 lr=1.89e-04 tps=17431 mem=6.3G lam=None Gm=0.3496297200520833 +[g_only s1] step 8180/12000 loss=4.4499 gn=0.83 lr=1.88e-04 tps=17432 mem=6.3G lam=None Gm=0.34393310546875 +[g_only s1] step 8200/12000 loss=4.5048 gn=0.94 lr=1.87e-04 tps=17432 mem=6.3G lam=None Gm=0.3425801595052083 +[g_only s1] step 8220/12000 loss=4.3682 gn=0.85 lr=1.86e-04 tps=17432 mem=6.3G lam=None Gm=0.3408304850260417 +[g_only s1] step 8240/12000 loss=4.4301 gn=0.89 lr=1.84e-04 tps=17433 mem=6.3G lam=None Gm=0.3484039306640625 +[g_only s1] step 8260/12000 loss=4.3398 gn=0.95 lr=1.83e-04 tps=17433 mem=6.3G lam=None Gm=0.3397216796875 +[g_only s1] step 8280/12000 loss=4.4119 gn=0.87 lr=1.82e-04 tps=17434 mem=6.3G lam=None Gm=0.343505859375 +[g_only s1] step 8300/12000 loss=4.5753 gn=0.89 lr=1.81e-04 tps=17434 mem=6.3G lam=None Gm=0.347320556640625 +[g_only s1] step 8320/12000 loss=4.2291 gn=0.83 lr=1.80e-04 tps=17434 mem=6.3G lam=None Gm=0.346160888671875 +[g_only s1] step 8340/12000 loss=4.4097 gn=0.86 lr=1.78e-04 tps=17435 mem=6.3G lam=None Gm=0.3459218343098958 +[g_only s1] step 8360/12000 loss=4.3999 gn=0.86 lr=1.77e-04 tps=17435 mem=6.3G lam=None Gm=0.3469441731770833 +[g_only s1] step 8380/12000 loss=4.3663 gn=0.83 lr=1.76e-04 tps=17435 mem=6.3G lam=None Gm=0.3463541666666667 +[g_only s1] step 8400/12000 loss=4.5603 gn=0.84 lr=1.75e-04 tps=17436 mem=6.3G lam=None Gm=0.3504435221354167 +[g_only s1] step 8420/12000 loss=4.3685 gn=0.87 lr=1.74e-04 tps=17436 mem=6.3G lam=None Gm=0.3453623453776042 +[g_only s1] step 8440/12000 loss=4.3701 gn=0.86 lr=1.72e-04 tps=17437 mem=6.3G lam=None Gm=0.3419189453125 +[g_only s1] step 8460/12000 loss=4.4153 gn=0.85 lr=1.71e-04 tps=17437 mem=6.3G lam=None Gm=0.3439381917317708 +[g_only s1] step 8480/12000 loss=4.5329 gn=0.84 lr=1.70e-04 tps=17437 mem=6.3G lam=None Gm=0.3443094889322917 +[g_only s1] step 8500/12000 loss=4.4951 gn=0.93 lr=1.69e-04 tps=17438 mem=6.3G lam=None Gm=0.3422648111979167 +[g_only s1] EVAL step 8500: val_loss=4.4879 val_ppl=88.94 +[g_only s1] step 8520/12000 loss=4.4292 gn=0.88 lr=1.68e-04 tps=17423 mem=6.3G lam=None Gm=0.3441619873046875 +[g_only s1] step 8540/12000 loss=4.3848 gn=0.88 lr=1.67e-04 tps=17424 mem=6.3G lam=None Gm=0.3461507161458333 +[g_only s1] step 8560/12000 loss=4.3542 gn=0.88 lr=1.66e-04 tps=17424 mem=6.3G lam=None Gm=0.345916748046875 +[g_only s1] step 8580/12000 loss=4.4924 gn=0.86 lr=1.64e-04 tps=17425 mem=6.3G lam=None Gm=0.3447774251302083 +[g_only s1] step 8600/12000 loss=4.5276 gn=0.91 lr=1.63e-04 tps=17425 mem=6.3G lam=None Gm=0.3455607096354167 +[g_only s1] step 8620/12000 loss=4.5100 gn=0.85 lr=1.62e-04 tps=17425 mem=6.3G lam=None Gm=0.3460489908854167 +[g_only s1] step 8640/12000 loss=4.3994 gn=0.89 lr=1.61e-04 tps=17426 mem=6.3G lam=None Gm=0.3389689127604167 +[g_only s1] step 8660/12000 loss=4.4581 gn=0.92 lr=1.60e-04 tps=17426 mem=6.3G lam=None Gm=0.3431650797526042 +[g_only s1] step 8680/12000 loss=4.3686 gn=0.88 lr=1.59e-04 tps=17427 mem=6.3G lam=None Gm=0.3427225748697917 +[g_only s1] step 8700/12000 loss=4.3486 gn=0.91 lr=1.58e-04 tps=17427 mem=6.3G lam=None Gm=0.343109130859375 +[g_only s1] step 8720/12000 loss=4.4552 gn=0.86 lr=1.57e-04 tps=17427 mem=6.3G lam=None Gm=0.3496602376302083 +[g_only s1] step 8740/12000 loss=4.5637 gn=0.87 lr=1.55e-04 tps=17428 mem=6.3G lam=None Gm=0.344146728515625 +[g_only s1] step 8760/12000 loss=4.3910 gn=0.86 lr=1.54e-04 tps=17428 mem=6.3G lam=None Gm=0.34295654296875 +[g_only s1] step 8780/12000 loss=4.1548 gn=0.89 lr=1.53e-04 tps=17429 mem=6.3G lam=None Gm=0.3359273274739583 +[g_only s1] step 8800/12000 loss=4.3992 gn=0.91 lr=1.52e-04 tps=17429 mem=6.3G lam=None Gm=0.3452962239583333 +[g_only s1] step 8820/12000 loss=4.3801 gn=0.90 lr=1.51e-04 tps=17429 mem=6.3G lam=None Gm=0.342376708984375 +[g_only s1] step 8840/12000 loss=4.2372 gn=0.90 lr=1.50e-04 tps=17430 mem=6.3G lam=None Gm=0.346954345703125 +[g_only s1] step 8860/12000 loss=4.3782 gn=0.89 lr=1.49e-04 tps=17430 mem=6.3G lam=None Gm=0.340118408203125 +[g_only s1] step 8880/12000 loss=4.5026 gn=0.93 lr=1.48e-04 tps=17431 mem=6.3G lam=None Gm=0.3446451822916667 +[g_only s1] step 8900/12000 loss=4.5459 gn=0.94 lr=1.47e-04 tps=17431 mem=6.3G lam=None Gm=0.3467203776041667 +[g_only s1] step 8920/12000 loss=4.3749 gn=0.87 lr=1.46e-04 tps=17431 mem=6.3G lam=None Gm=0.3452351888020833 +[g_only s1] step 8940/12000 loss=4.3753 gn=0.91 lr=1.45e-04 tps=17432 mem=6.3G lam=None Gm=0.348388671875 +[g_only s1] step 8960/12000 loss=4.4106 gn=0.86 lr=1.44e-04 tps=17432 mem=6.3G lam=None Gm=0.34912109375 +[g_only s1] step 8980/12000 loss=4.4529 gn=0.86 lr=1.43e-04 tps=17433 mem=6.3G lam=None Gm=0.3380228678385417 +[g_only s1] step 9000/12000 loss=4.4250 gn=0.86 lr=1.42e-04 tps=17433 mem=6.3G lam=None Gm=0.3426462809244792 +[g_only s1] EVAL step 9000: val_loss=4.4613 val_ppl=86.60 +[g_only s1] step 9020/12000 loss=4.4518 gn=0.89 lr=1.41e-04 tps=17420 mem=6.3G lam=None Gm=0.3425140380859375 +[g_only s1] step 9040/12000 loss=4.4072 gn=0.91 lr=1.40e-04 tps=17420 mem=6.3G lam=None Gm=0.3379770914713542 +[g_only s1] step 9060/12000 loss=4.4366 gn=0.86 lr=1.39e-04 tps=17421 mem=6.3G lam=None Gm=0.3402506510416667 +[g_only s1] step 9080/12000 loss=4.4452 gn=0.92 lr=1.38e-04 tps=17421 mem=6.3G lam=None Gm=0.3455607096354167 +[g_only s1] step 9100/12000 loss=4.3587 gn=0.95 lr=1.37e-04 tps=17421 mem=6.3G lam=None Gm=0.3425394694010417 +[g_only s1] step 9120/12000 loss=4.2216 gn=0.88 lr=1.36e-04 tps=17422 mem=6.3G lam=None Gm=0.3385874430338542 +[g_only s1] step 9140/12000 loss=4.4817 gn=0.90 lr=1.35e-04 tps=17422 mem=6.3G lam=None Gm=0.3408559163411458 +[g_only s1] step 9160/12000 loss=4.5499 gn=0.87 lr=1.34e-04 tps=17423 mem=6.3G lam=None Gm=0.3458709716796875 +[g_only s1] step 9180/12000 loss=4.3372 gn=0.90 lr=1.33e-04 tps=17423 mem=6.3G lam=None Gm=0.3390146891276042 +[g_only s1] step 9200/12000 loss=4.4585 gn=0.89 lr=1.32e-04 tps=17423 mem=6.3G lam=None Gm=0.3442586263020833 +[g_only s1] step 9220/12000 loss=4.3447 gn=0.90 lr=1.31e-04 tps=17424 mem=6.3G lam=None Gm=0.3439534505208333 +[g_only s1] step 9240/12000 loss=4.3832 gn=0.87 lr=1.30e-04 tps=17424 mem=6.3G lam=None Gm=0.340789794921875 +[g_only s1] step 9260/12000 loss=4.4588 gn=0.87 lr=1.29e-04 tps=17425 mem=6.3G lam=None Gm=0.3490702311197917 +[g_only s1] step 9280/12000 loss=4.4507 gn=0.89 lr=1.28e-04 tps=17425 mem=6.3G lam=None Gm=0.3438008626302083 +[g_only s1] step 9300/12000 loss=4.5250 gn=0.96 lr=1.27e-04 tps=17425 mem=6.3G lam=None Gm=0.346954345703125 +[g_only s1] step 9320/12000 loss=4.2828 gn=0.89 lr=1.26e-04 tps=17426 mem=6.3G lam=None Gm=0.3429412841796875 +[g_only s1] step 9340/12000 loss=4.2865 gn=0.91 lr=1.25e-04 tps=17426 mem=6.3G lam=None Gm=0.343109130859375 +[g_only s1] step 9360/12000 loss=4.4664 gn=0.95 lr=1.24e-04 tps=17426 mem=6.3G lam=None Gm=0.3453114827473958 +[g_only s1] step 9380/12000 loss=4.3892 gn=0.93 lr=1.23e-04 tps=17426 mem=6.3G lam=None Gm=0.3511098225911458 +[g_only s1] step 9400/12000 loss=4.3418 gn=0.95 lr=1.22e-04 tps=17427 mem=6.3G lam=None Gm=0.3474833170572917 +[g_only s1] step 9420/12000 loss=4.2386 gn=0.92 lr=1.21e-04 tps=17427 mem=6.3G lam=None Gm=0.3439687093098958 +[g_only s1] step 9440/12000 loss=4.3383 gn=0.91 lr=1.20e-04 tps=17427 mem=6.3G lam=None Gm=0.343475341796875 +[g_only s1] step 9460/12000 loss=4.3077 gn=0.92 lr=1.19e-04 tps=17428 mem=6.3G lam=None Gm=0.3419545491536458 +[g_only s1] step 9480/12000 loss=4.3888 gn=0.88 lr=1.19e-04 tps=17428 mem=6.3G lam=None Gm=0.3418680826822917 +[g_only s1] step 9500/12000 loss=4.3588 gn=0.92 lr=1.18e-04 tps=17429 mem=6.3G lam=None Gm=0.3478240966796875 +[g_only s1] EVAL step 9500: val_loss=4.4539 val_ppl=85.96 +[g_only s1] step 9520/12000 loss=4.4112 gn=0.95 lr=1.17e-04 tps=17416 mem=6.3G lam=None Gm=0.347320556640625 +[g_only s1] step 9540/12000 loss=4.2881 gn=0.91 lr=1.16e-04 tps=17416 mem=6.3G lam=None Gm=0.344085693359375 +[g_only s1] step 9560/12000 loss=4.4107 gn=0.98 lr=1.15e-04 tps=17416 mem=6.3G lam=None Gm=0.3399200439453125 +[g_only s1] step 9580/12000 loss=4.4347 gn=0.95 lr=1.14e-04 tps=17417 mem=6.3G lam=None Gm=0.3471527099609375 +[g_only s1] step 9600/12000 loss=4.3429 gn=0.90 lr=1.13e-04 tps=17417 mem=6.3G lam=None Gm=0.343017578125 +[g_only s1] step 9620/12000 loss=4.3567 gn=0.93 lr=1.12e-04 tps=17417 mem=6.3G lam=None Gm=0.3414306640625 +[g_only s1] step 9640/12000 loss=4.2836 gn=0.90 lr=1.12e-04 tps=17418 mem=6.3G lam=None Gm=0.3373311360677083 +[g_only s1] step 9660/12000 loss=4.5443 gn=0.94 lr=1.11e-04 tps=17418 mem=6.3G lam=None Gm=0.3383433024088542 +[g_only s1] step 9680/12000 loss=4.3200 gn=0.88 lr=1.10e-04 tps=17418 mem=6.3G lam=None Gm=0.3417612711588542 +[g_only s1] step 9700/12000 loss=4.4131 gn=0.92 lr=1.09e-04 tps=17419 mem=6.3G lam=None Gm=0.3448994954427083 +[g_only s1] step 9720/12000 loss=4.4827 gn=0.98 lr=1.08e-04 tps=17419 mem=6.3G lam=None Gm=0.3452606201171875 +[g_only s1] step 9740/12000 loss=4.2266 gn=0.90 lr=1.07e-04 tps=17419 mem=6.3G lam=None Gm=0.344390869140625 +[g_only s1] step 9760/12000 loss=4.4611 gn=0.92 lr=1.07e-04 tps=17420 mem=6.3G lam=None Gm=0.3394775390625 +[g_only s1] step 9780/12000 loss=4.3450 gn=0.89 lr=1.06e-04 tps=17420 mem=6.3G lam=None Gm=0.339080810546875 +[g_only s1] step 9800/12000 loss=4.4436 gn=0.90 lr=1.05e-04 tps=17420 mem=6.3G lam=None Gm=0.3434193929036458 +[g_only s1] step 9820/12000 loss=4.2889 gn=0.91 lr=1.04e-04 tps=17421 mem=6.3G lam=None Gm=0.3376057942708333 +[g_only s1] step 9840/12000 loss=4.4762 gn=0.97 lr=1.03e-04 tps=17421 mem=6.3G lam=None Gm=0.3425852457682292 +[g_only s1] step 9860/12000 loss=4.5204 gn=1.01 lr=1.03e-04 tps=17422 mem=6.3G lam=None Gm=0.3434906005859375 +[g_only s1] step 9880/12000 loss=4.3619 gn=0.93 lr=1.02e-04 tps=17422 mem=6.3G lam=None Gm=0.3449045817057292 +[g_only s1] step 9900/12000 loss=4.3387 gn=0.88 lr=1.01e-04 tps=17422 mem=6.3G lam=None Gm=0.3357696533203125 +[g_only s1] step 9920/12000 loss=4.3049 gn=0.90 lr=1.00e-04 tps=17423 mem=6.3G lam=None Gm=0.3383839925130208 +[g_only s1] step 9940/12000 loss=4.2693 gn=0.90 lr=9.96e-05 tps=17423 mem=6.3G lam=None Gm=0.3371683756510417 +[g_only s1] step 9960/12000 loss=4.3392 gn=0.99 lr=9.89e-05 tps=17423 mem=6.3G lam=None Gm=0.3377939860026042 +[g_only s1] step 9980/12000 loss=4.4052 gn=0.88 lr=9.81e-05 tps=17424 mem=6.3G lam=None Gm=0.3376973470052083 +[g_only s1] step 10000/12000 loss=4.3294 gn=0.95 lr=9.74e-05 tps=17424 mem=6.3G lam=None Gm=0.3370513916015625 +[g_only s1] EVAL step 10000: val_loss=4.4170 val_ppl=82.84 +[g_only s1] step 10020/12000 loss=4.6541 gn=1.07 lr=9.67e-05 tps=17412 mem=6.3G lam=None Gm=0.3438873291015625 +[g_only s1] step 10040/12000 loss=4.2397 gn=0.93 lr=9.59e-05 tps=17412 mem=6.3G lam=None Gm=0.3339945475260417 +[g_only s1] step 10060/12000 loss=4.4760 gn=0.92 lr=9.52e-05 tps=17413 mem=6.3G lam=None Gm=0.3466949462890625 +[g_only s1] step 10080/12000 loss=4.2452 gn=0.92 lr=9.45e-05 tps=17413 mem=6.3G lam=None Gm=0.3431752522786458 +[g_only s1] step 10100/12000 loss=4.3089 gn=0.95 lr=9.38e-05 tps=17413 mem=6.3G lam=None Gm=0.34588623046875 +[g_only s1] step 10120/12000 loss=4.3334 gn=0.90 lr=9.31e-05 tps=17414 mem=6.3G lam=None Gm=0.34326171875 +[g_only s1] step 10140/12000 loss=4.2560 gn=0.92 lr=9.24e-05 tps=17414 mem=6.3G lam=None Gm=0.3359425862630208 +[g_only s1] step 10160/12000 loss=4.3676 gn=0.92 lr=9.18e-05 tps=17414 mem=6.3G lam=None Gm=0.3437093098958333 +[g_only s1] step 10180/12000 loss=4.2738 gn=0.88 lr=9.11e-05 tps=17415 mem=6.3G lam=None Gm=0.3369140625 +[g_only s1] step 10200/12000 loss=4.4665 gn=0.98 lr=9.04e-05 tps=17415 mem=6.3G lam=None Gm=0.3410288492838542 +[g_only s1] step 10220/12000 loss=4.3919 gn=0.98 lr=8.98e-05 tps=17415 mem=6.3G lam=None Gm=0.3459879557291667 +[g_only s1] step 10240/12000 loss=4.3236 gn=0.93 lr=8.91e-05 tps=17416 mem=6.3G lam=None Gm=0.3351491292317708 +[g_only s1] step 10260/12000 loss=4.4446 gn=0.90 lr=8.85e-05 tps=17416 mem=6.3G lam=None Gm=0.3447977701822917 +[g_only s1] step 10280/12000 loss=4.3505 gn=0.90 lr=8.78e-05 tps=17416 mem=6.3G lam=None Gm=0.3380889892578125 +[g_only s1] step 10300/12000 loss=4.2589 gn=0.95 lr=8.72e-05 tps=17416 mem=6.3G lam=None Gm=0.3329315185546875 +[g_only s1] step 10320/12000 loss=4.2955 gn=0.92 lr=8.66e-05 tps=17417 mem=6.3G lam=None Gm=0.3466084798177083 +[g_only s1] step 10340/12000 loss=4.4688 gn=0.96 lr=8.59e-05 tps=17417 mem=6.3G lam=None Gm=0.342254638671875 +[g_only s1] step 10360/12000 loss=4.3390 gn=0.90 lr=8.53e-05 tps=17417 mem=6.3G lam=None Gm=0.3377176920572917 +[g_only s1] step 10380/12000 loss=4.3567 gn=0.94 lr=8.47e-05 tps=17418 mem=6.3G lam=None Gm=0.341552734375 +[g_only s1] step 10400/12000 loss=4.4487 gn=0.96 lr=8.41e-05 tps=17418 mem=6.3G lam=None Gm=0.344696044921875 +[g_only s1] step 10420/12000 loss=4.4076 gn=1.02 lr=8.35e-05 tps=17418 mem=6.3G lam=None Gm=0.3439280192057292 +[g_only s1] step 10440/12000 loss=4.4203 gn=0.95 lr=8.30e-05 tps=17419 mem=6.3G lam=None Gm=0.3411966959635417 +[g_only s1] step 10460/12000 loss=4.2431 gn=0.92 lr=8.24e-05 tps=17419 mem=6.3G lam=None Gm=0.3404286702473958 +[g_only s1] step 10480/12000 loss=4.3366 gn=0.91 lr=8.18e-05 tps=17420 mem=6.3G lam=None Gm=0.3381144205729167 +[g_only s1] step 10500/12000 loss=4.5189 gn=1.13 lr=8.12e-05 tps=17420 mem=6.3G lam=None Gm=0.3447113037109375 +[g_only s1] EVAL step 10500: val_loss=4.4128 val_ppl=82.50 +[g_only s1] step 10520/12000 loss=4.3543 gn=0.95 lr=8.07e-05 tps=17408 mem=6.3G lam=None Gm=0.3406168619791667 +[g_only s1] step 10540/12000 loss=4.4604 gn=0.97 lr=8.01e-05 tps=17409 mem=6.3G lam=None Gm=0.3383127848307292 +[g_only s1] step 10560/12000 loss=4.4361 gn=0.98 lr=7.96e-05 tps=17409 mem=6.3G lam=None Gm=0.3411916097005208 +[g_only s1] step 10580/12000 loss=4.3035 gn=0.93 lr=7.91e-05 tps=17409 mem=6.3G lam=None Gm=0.3337809244791667 +[g_only s1] step 10600/12000 loss=4.2435 gn=0.96 lr=7.85e-05 tps=17410 mem=6.3G lam=None Gm=0.3373921712239583 +[g_only s1] step 10620/12000 loss=4.3396 gn=0.98 lr=7.80e-05 tps=17410 mem=6.3G lam=None Gm=0.3430328369140625 +[g_only s1] step 10640/12000 loss=4.4442 gn=0.99 lr=7.75e-05 tps=17411 mem=6.3G lam=None Gm=0.3413543701171875 +[g_only s1] step 10660/12000 loss=4.2533 gn=0.91 lr=7.70e-05 tps=17411 mem=6.3G lam=None Gm=0.341522216796875 +[g_only s1] step 10680/12000 loss=4.2707 gn=0.93 lr=7.65e-05 tps=17411 mem=6.3G lam=None Gm=0.337493896484375 +[g_only s1] step 10700/12000 loss=4.3471 gn=0.89 lr=7.60e-05 tps=17412 mem=6.3G lam=None Gm=0.3366495768229167 +[g_only s1] step 10720/12000 loss=4.3579 gn=0.98 lr=7.55e-05 tps=17412 mem=6.3G lam=None Gm=0.3453114827473958 +[g_only s1] step 10740/12000 loss=4.4034 gn=0.95 lr=7.50e-05 tps=17412 mem=6.3G lam=None Gm=0.3436838785807292 +[g_only s1] step 10760/12000 loss=4.3118 gn=0.90 lr=7.46e-05 tps=17413 mem=6.3G lam=None Gm=0.3383026123046875 +[g_only s1] step 10780/12000 loss=4.3338 gn=0.93 lr=7.41e-05 tps=17413 mem=6.3G lam=None Gm=0.3419901529947917 +[g_only s1] step 10800/12000 loss=4.4123 gn=1.01 lr=7.37e-05 tps=17413 mem=6.3G lam=None Gm=0.3407694498697917 +[g_only s1] step 10820/12000 loss=4.5067 gn=0.94 lr=7.32e-05 tps=17414 mem=6.3G lam=None Gm=0.3417714436848958 +[g_only s1] step 10840/12000 loss=4.1900 gn=0.92 lr=7.28e-05 tps=17414 mem=6.3G lam=None Gm=0.3349253336588542 +[g_only s1] step 10860/12000 loss=4.3787 gn=0.89 lr=7.23e-05 tps=17414 mem=6.3G lam=None Gm=0.3438618977864583 +[g_only s1] step 10880/12000 loss=4.2095 gn=0.98 lr=7.19e-05 tps=17415 mem=6.3G lam=None Gm=0.3380126953125 +[g_only s1] step 10900/12000 loss=4.3433 gn=0.94 lr=7.15e-05 tps=17415 mem=6.3G lam=None Gm=0.340362548828125 +[g_only s1] step 10920/12000 loss=4.3524 gn=0.98 lr=7.11e-05 tps=17415 mem=6.3G lam=None Gm=0.3405253092447917 +[g_only s1] step 10940/12000 loss=4.2931 gn=0.94 lr=7.07e-05 tps=17415 mem=6.3G lam=None Gm=0.3349507649739583 +[g_only s1] step 10960/12000 loss=4.3011 gn=0.96 lr=7.03e-05 tps=17416 mem=6.3G lam=None Gm=0.33526611328125 +[g_only s1] step 10980/12000 loss=4.5073 gn=1.03 lr=6.99e-05 tps=17416 mem=6.3G lam=None Gm=0.34539794921875 +[g_only s1] step 11000/12000 loss=4.3077 gn=0.94 lr=6.95e-05 tps=17417 mem=6.3G lam=None Gm=0.3430938720703125 +[g_only s1] EVAL step 11000: val_loss=4.3975 val_ppl=81.25 +[g_only s1] step 11020/12000 loss=4.1165 gn=0.96 lr=6.91e-05 tps=17405 mem=6.3G lam=None Gm=0.3385976155598958 +[g_only s1] step 11040/12000 loss=4.3224 gn=1.00 lr=6.88e-05 tps=17406 mem=6.3G lam=None Gm=0.341827392578125 +[g_only s1] step 11060/12000 loss=4.3877 gn=0.98 lr=6.84e-05 tps=17406 mem=6.3G lam=None Gm=0.338714599609375 +[g_only s1] step 11080/12000 loss=4.3953 gn=1.00 lr=6.81e-05 tps=17407 mem=6.3G lam=None Gm=0.3358357747395833 +[g_only s1] step 11100/12000 loss=4.4088 gn=0.95 lr=6.77e-05 tps=17407 mem=6.3G lam=None Gm=0.3376820882161458 +[g_only s1] step 11120/12000 loss=4.3852 gn=0.94 lr=6.74e-05 tps=17407 mem=6.3G lam=None Gm=0.3446807861328125 +[g_only s1] step 11140/12000 loss=4.2661 gn=0.95 lr=6.70e-05 tps=17407 mem=6.3G lam=None Gm=0.3387400309244792 +[g_only s1] step 11160/12000 loss=4.2860 gn=0.98 lr=6.67e-05 tps=17408 mem=6.3G lam=None Gm=0.3320973714192708 +[g_only s1] step 11180/12000 loss=4.3063 gn=0.95 lr=6.64e-05 tps=17408 mem=6.3G lam=None Gm=0.3386789957682292 +[g_only s1] step 11200/12000 loss=4.4966 gn=1.15 lr=6.61e-05 tps=17408 mem=6.3G lam=None Gm=0.3391825358072917 +[g_only s1] step 11220/12000 loss=4.3698 gn=1.05 lr=6.58e-05 tps=17409 mem=6.3G lam=None Gm=0.33642578125 +[g_only s1] step 11240/12000 loss=4.2725 gn=0.98 lr=6.55e-05 tps=17409 mem=6.3G lam=None Gm=0.3370310465494792 +[g_only s1] step 11260/12000 loss=4.2949 gn=0.93 lr=6.52e-05 tps=17409 mem=6.3G lam=None Gm=0.3425750732421875 +[g_only s1] step 11280/12000 loss=4.1959 gn=1.00 lr=6.49e-05 tps=17410 mem=6.3G lam=None Gm=0.3360493977864583 +[g_only s1] step 11300/12000 loss=4.4029 gn=0.96 lr=6.47e-05 tps=17410 mem=6.3G lam=None Gm=0.3442179361979167 +[g_only s1] step 11320/12000 loss=4.2226 gn=0.98 lr=6.44e-05 tps=17410 mem=6.3G lam=None Gm=0.3378550211588542 +[g_only s1] step 11340/12000 loss=4.3866 gn=1.05 lr=6.42e-05 tps=17411 mem=6.3G lam=None Gm=0.3390299479166667 +[g_only s1] step 11360/12000 loss=4.2741 gn=0.96 lr=6.39e-05 tps=17411 mem=6.3G lam=None Gm=0.3345591227213542 +[g_only s1] step 11380/12000 loss=4.3405 gn=1.05 lr=6.37e-05 tps=17411 mem=6.3G lam=None Gm=0.3398640950520833 +[g_only s1] step 11400/12000 loss=4.3957 gn=0.95 lr=6.34e-05 tps=17412 mem=6.3G lam=None Gm=0.3379465738932292 +[g_only s1] step 11420/12000 loss=4.0872 gn=0.96 lr=6.32e-05 tps=17412 mem=6.3G lam=None Gm=0.3337860107421875 +[g_only s1] step 11440/12000 loss=4.1850 gn=0.95 lr=6.30e-05 tps=17412 mem=6.3G lam=None Gm=0.334716796875 +[g_only s1] step 11460/12000 loss=4.3156 gn=0.99 lr=6.28e-05 tps=17412 mem=6.3G lam=None Gm=0.3395589192708333 +[g_only s1] step 11480/12000 loss=4.3726 gn=0.93 lr=6.26e-05 tps=17413 mem=6.3G lam=None Gm=0.34063720703125 +[g_only s1] step 11500/12000 loss=4.5183 gn=1.02 lr=6.24e-05 tps=17413 mem=6.3G lam=None Gm=0.3369191487630208 +[g_only s1] EVAL step 11500: val_loss=4.3883 val_ppl=80.50 +[g_only s1] step 11520/12000 loss=4.2934 gn=0.97 lr=6.22e-05 tps=17403 mem=6.3G lam=None Gm=0.3368581136067708 +[g_only s1] step 11540/12000 loss=4.1957 gn=0.96 lr=6.20e-05 tps=17403 mem=6.3G lam=None Gm=0.3339894612630208 +[g_only s1] step 11560/12000 loss=4.3291 gn=1.04 lr=6.19e-05 tps=17403 mem=6.3G lam=None Gm=0.3370259602864583 +[g_only s1] step 11580/12000 loss=4.3693 gn=0.95 lr=6.17e-05 tps=17404 mem=6.3G lam=None Gm=0.3382822672526042 +[g_only s1] step 11600/12000 loss=4.1896 gn=0.98 lr=6.15e-05 tps=17404 mem=6.3G lam=None Gm=0.3362325032552083 +[g_only s1] step 11620/12000 loss=4.4307 gn=1.02 lr=6.14e-05 tps=17404 mem=6.3G lam=None Gm=0.34210205078125 +[g_only s1] step 11640/12000 loss=4.3685 gn=0.99 lr=6.12e-05 tps=17405 mem=6.3G lam=None Gm=0.3374481201171875 +[g_only s1] step 11660/12000 loss=4.3698 gn=0.96 lr=6.11e-05 tps=17405 mem=6.3G lam=None Gm=0.3376820882161458 +[g_only s1] step 11680/12000 loss=4.2802 gn=0.94 lr=6.10e-05 tps=17405 mem=6.3G lam=None Gm=0.3402557373046875 +[g_only s1] step 11700/12000 loss=4.2997 gn=0.94 lr=6.09e-05 tps=17406 mem=6.3G lam=None Gm=0.3365681966145833 +[g_only s1] step 11720/12000 loss=4.1742 gn=0.95 lr=6.07e-05 tps=17406 mem=6.3G lam=None Gm=0.3364308675130208 +[g_only s1] step 11740/12000 loss=4.3930 gn=1.01 lr=6.06e-05 tps=17406 mem=6.3G lam=None Gm=0.3430684407552083 +[g_only s1] step 11760/12000 loss=4.2830 gn=0.97 lr=6.06e-05 tps=17407 mem=6.3G lam=None Gm=0.3402913411458333 +[g_only s1] step 11780/12000 loss=4.2707 gn=0.98 lr=6.05e-05 tps=17407 mem=6.3G lam=None Gm=0.3309783935546875 +[g_only s1] step 11800/12000 loss=4.2349 gn=1.02 lr=6.04e-05 tps=17407 mem=6.3G lam=None Gm=0.3384145100911458 +[g_only s1] step 11820/12000 loss=4.3239 gn=0.93 lr=6.03e-05 tps=17407 mem=6.3G lam=None Gm=0.3376719156901042 +[g_only s1] step 11840/12000 loss=4.2944 gn=0.97 lr=6.02e-05 tps=17408 mem=6.3G lam=None Gm=0.34051513671875 +[g_only s1] step 11860/12000 loss=4.3041 gn=0.97 lr=6.02e-05 tps=17408 mem=6.3G lam=None Gm=0.3346099853515625 +[g_only s1] step 11880/12000 loss=4.2649 gn=0.97 lr=6.01e-05 tps=17408 mem=6.3G lam=None Gm=0.3377126057942708 +[g_only s1] step 11900/12000 loss=4.4074 gn=1.16 lr=6.01e-05 tps=17409 mem=6.3G lam=None Gm=0.3348744710286458 +[g_only s1] step 11920/12000 loss=4.2446 gn=1.13 lr=6.01e-05 tps=17409 mem=6.3G lam=None Gm=0.332000732421875 +[g_only s1] step 11940/12000 loss=4.2996 gn=0.97 lr=6.00e-05 tps=17409 mem=6.3G lam=None Gm=0.3362681070963542 +[g_only s1] step 11960/12000 loss=4.2211 gn=0.98 lr=6.00e-05 tps=17410 mem=6.3G lam=None Gm=0.3381398518880208 +[g_only s1] step 11980/12000 loss=4.3103 gn=0.98 lr=6.00e-05 tps=17410 mem=6.3G lam=None Gm=0.3359120686848958 +[g_only s1] EVAL step 11999: val_loss=4.3892 val_ppl=80.58 +[init] variant=full seed=1 params=63.98M +[init] use_S=True use_G=True +[full s1] step 0/12000 loss=10.9020 gn=10.46 lr=3.00e-06 tps=7237 mem=5.9G lam=0.12692801654338837 Gm=0.4995858383675416 +[full s1] step 20/12000 loss=9.7599 gn=2.11 lr=6.30e-05 tps=14555 mem=6.4G lam=0.1269188163181146 Gm=0.4347797520458698 +[full s1] step 40/12000 loss=9.1248 gn=1.63 lr=1.23e-04 tps=14995 mem=6.4G lam=0.12684423414369425 Gm=0.41958451891938847 +[full s1] step 60/12000 loss=8.2397 gn=1.45 lr=1.83e-04 tps=15143 mem=6.4G lam=0.12674915107587972 Gm=0.4125487109025319 +[full s1] step 80/12000 loss=7.3904 gn=0.81 lr=2.43e-04 tps=15216 mem=6.4G lam=0.12662707827985287 Gm=0.4209214635193348 +[full s1] step 100/12000 loss=7.2791 gn=0.63 lr=3.03e-04 tps=15239 mem=6.4G lam=0.12651526803771654 Gm=0.42261641720930737 +[full s1] step 120/12000 loss=7.0610 gn=0.65 lr=3.63e-04 tps=15256 mem=6.4G lam=0.1264491273711125 Gm=0.4161597763498624 +[full s1] step 140/12000 loss=6.8002 gn=0.60 lr=4.23e-04 tps=15274 mem=6.4G lam=0.12636985008915266 Gm=0.4149595449368159 +[full s1] step 160/12000 loss=6.7819 gn=0.53 lr=4.83e-04 tps=15284 mem=6.4G lam=0.1263305749744177 Gm=0.4113170268634955 +[full s1] step 180/12000 loss=6.7422 gn=0.54 lr=5.43e-04 tps=15289 mem=6.4G lam=0.1262933313846588 Gm=0.39049290244777995 +[full s1] step 200/12000 loss=6.5951 gn=0.56 lr=6.00e-04 tps=15295 mem=6.4G lam=0.12629335994521776 Gm=0.39893319581945735 +[full s1] step 220/12000 loss=6.6112 gn=0.54 lr=6.00e-04 tps=15303 mem=6.4G lam=0.1262515423198541 Gm=0.39553646246592206 +[full s1] step 240/12000 loss=6.6158 gn=0.50 lr=6.00e-04 tps=15307 mem=6.4G lam=0.126265499740839 Gm=0.38961820552746457 +[full s1] step 260/12000 loss=6.6230 gn=0.53 lr=6.00e-04 tps=15312 mem=6.4G lam=0.1262509444107612 Gm=0.39975054810444516 +[full s1] step 280/12000 loss=6.6984 gn=0.52 lr=6.00e-04 tps=15315 mem=6.4G lam=0.12626273992160955 Gm=0.4032938467959563 +[full s1] step 300/12000 loss=6.5449 gn=0.50 lr=6.00e-04 tps=15318 mem=6.4G lam=0.12629778248568377 Gm=0.39266059547662735 +[full s1] step 320/12000 loss=6.4674 gn=0.47 lr=6.00e-04 tps=15321 mem=6.4G lam=0.1263255880524715 Gm=0.4033467322587967 +[full s1] step 340/12000 loss=6.4858 gn=0.47 lr=6.00e-04 tps=15323 mem=6.4G lam=0.12631176691502333 Gm=0.41551605239510536 +[full s1] step 360/12000 loss=6.2349 gn=0.48 lr=6.00e-04 tps=15323 mem=6.4G lam=0.12636415101587772 Gm=0.3895490989089012 +[full s1] step 380/12000 loss=6.3028 gn=0.46 lr=6.00e-04 tps=15323 mem=6.4G lam=0.12646240182220936 Gm=0.4022495821118355 +[full s1] step 400/12000 loss=6.2624 gn=0.47 lr=6.00e-04 tps=15324 mem=6.4G lam=0.12657801620662212 Gm=0.4217920737961928 +[full s1] step 420/12000 loss=6.3604 gn=0.46 lr=6.00e-04 tps=15327 mem=6.4G lam=0.1266706862176458 Gm=0.408292718231678 +[full s1] step 440/12000 loss=6.2654 gn=0.50 lr=5.99e-04 tps=15328 mem=6.4G lam=0.126765969209373 Gm=0.4287540478010972 +[full s1] step 460/12000 loss=6.3276 gn=0.50 lr=5.99e-04 tps=15321 mem=6.4G lam=0.12691457755863667 Gm=0.4263387819131215 +[full s1] step 480/12000 loss=6.1970 gn=0.47 lr=5.99e-04 tps=15326 mem=6.4G lam=0.127042882454892 Gm=0.4007243352631728 +[full s1] step 500/12000 loss=6.3228 gn=0.48 lr=5.99e-04 tps=15327 mem=6.4G lam=0.1271931075801452 Gm=0.45453549176454544 +[full s1] EVAL step 500: val_loss=6.2490 val_ppl=517.51 +[full s1] step 520/12000 loss=6.1670 gn=0.52 lr=5.99e-04 tps=15104 mem=6.4G lam=0.1273673657948772 Gm=0.46657221764326096 +[full s1] step 540/12000 loss=6.2539 gn=0.48 lr=5.99e-04 tps=15111 mem=6.4G lam=0.12753031154473624 Gm=0.4478190665443738 +[full s1] step 560/12000 loss=6.1391 gn=0.46 lr=5.99e-04 tps=15116 mem=6.4G lam=0.12767995055764914 Gm=0.46714531257748604 +[full s1] step 580/12000 loss=6.0933 gn=0.52 lr=5.99e-04 tps=15124 mem=6.4G lam=0.12791364857306084 Gm=0.44959703211983043 +[full s1] step 600/12000 loss=6.1374 gn=0.50 lr=5.98e-04 tps=15131 mem=6.4G lam=0.12814220475653806 Gm=0.49712130924065906 +[full s1] step 620/12000 loss=6.2420 gn=0.46 lr=5.98e-04 tps=15132 mem=6.4G lam=0.12846932963778576 Gm=0.5221152193844318 +[full s1] step 640/12000 loss=6.2345 gn=0.53 lr=5.98e-04 tps=15130 mem=6.4G lam=0.12868615301946798 Gm=0.48895440623164177 +[full s1] step 660/12000 loss=5.9665 gn=0.49 lr=5.98e-04 tps=15117 mem=6.4G lam=0.1289492187400659 Gm=0.5151641977330049 +[full s1] step 680/12000 loss=6.0001 gn=0.52 lr=5.98e-04 tps=15110 mem=6.4G lam=0.12923613532135883 Gm=0.4942400325089693 +[full s1] step 700/12000 loss=5.9869 gn=0.48 lr=5.98e-04 tps=15101 mem=6.4G lam=0.12954013825704655 Gm=0.5145861580967903 +[full s1] step 720/12000 loss=6.0693 gn=0.52 lr=5.97e-04 tps=15091 mem=6.4G lam=0.12988313163320223 Gm=0.5187281928956509 +[full s1] step 740/12000 loss=5.9842 gn=0.48 lr=5.97e-04 tps=15079 mem=6.4G lam=0.130269353898863 Gm=0.5354160778224468 +[full s1] step 760/12000 loss=6.0367 gn=0.47 lr=5.97e-04 tps=15069 mem=6.4G lam=0.13060892187058926 Gm=0.5487556271255016 +[full s1] step 780/12000 loss=6.0188 gn=0.49 lr=5.97e-04 tps=15066 mem=6.4G lam=0.13101794136067232 Gm=0.5220799595117569 +[full s1] step 800/12000 loss=5.9387 gn=0.54 lr=5.97e-04 tps=15064 mem=6.4G lam=0.1314238477498293 Gm=0.5579956161479155 +[full s1] step 820/12000 loss=5.8457 gn=0.55 lr=5.96e-04 tps=15060 mem=6.4G lam=0.1318300971761346 Gm=0.5208019266525904 +[full s1] step 840/12000 loss=5.7736 gn=0.53 lr=5.96e-04 tps=15065 mem=6.4G lam=0.13224542948106924 Gm=0.5339502487331629 +[full s1] step 860/12000 loss=5.6955 gn=0.50 lr=5.96e-04 tps=15070 mem=6.4G lam=0.13265944675852856 Gm=0.550466320787867 +[full s1] step 880/12000 loss=5.7634 gn=0.53 lr=5.96e-04 tps=15076 mem=6.4G lam=0.13301804071913162 Gm=0.5376573378841082 +[full s1] step 900/12000 loss=5.6893 gn=0.50 lr=5.95e-04 tps=15081 mem=6.4G lam=0.13344700230906406 Gm=0.5317534307638804 +[full s1] step 920/12000 loss=5.8748 gn=0.51 lr=5.95e-04 tps=15086 mem=6.4G lam=0.13387259965141615 Gm=0.5500236116349697 +[full s1] step 940/12000 loss=5.7740 gn=0.50 lr=5.95e-04 tps=15092 mem=6.4G lam=0.13431155091772476 Gm=0.539446696639061 +[full s1] step 960/12000 loss=5.9132 gn=0.49 lr=5.94e-04 tps=15096 mem=6.4G lam=0.134713439270854 Gm=0.5332141350954771 +[full s1] step 980/12000 loss=5.6280 gn=0.52 lr=5.94e-04 tps=15100 mem=6.4G lam=0.1351950994382302 Gm=0.5231226769586405 +[full s1] step 1000/12000 loss=5.7491 gn=0.48 lr=5.94e-04 tps=15105 mem=6.4G lam=0.13570013331870237 Gm=0.5336024121691784 +[full s1] EVAL step 1000: val_loss=5.7838 val_ppl=324.98 +[full s1] step 1020/12000 loss=5.6672 gn=0.54 lr=5.94e-04 tps=14996 mem=6.4G lam=0.1361740588520964 Gm=0.5432699968417486 +[full s1] step 1040/12000 loss=5.7247 gn=0.55 lr=5.93e-04 tps=15002 mem=6.4G lam=0.13660040590912104 Gm=0.5442267836381992 +[full s1] step 1060/12000 loss=5.7762 gn=0.52 lr=5.93e-04 tps=15009 mem=6.4G lam=0.13701843253026405 Gm=0.5248871830602487 +[full s1] step 1080/12000 loss=5.5813 gn=0.53 lr=5.93e-04 tps=15015 mem=6.4G lam=0.13744254782795906 Gm=0.5443419044216474 +[full s1] step 1100/12000 loss=5.7918 gn=0.53 lr=5.92e-04 tps=15020 mem=6.4G lam=0.13791457129021487 Gm=0.5344842790315548 +[full s1] step 1120/12000 loss=5.6508 gn=0.60 lr=5.92e-04 tps=15027 mem=6.4G lam=0.13837703472624222 Gm=0.5129796254138151 +[full s1] step 1140/12000 loss=5.6843 gn=0.49 lr=5.92e-04 tps=15031 mem=6.4G lam=0.13889473676681519 Gm=0.5187325533479452 +[full s1] step 1160/12000 loss=5.6596 gn=0.54 lr=5.91e-04 tps=15037 mem=6.4G lam=0.13932060108830532 Gm=0.5307530878732601 +[full s1] step 1180/12000 loss=5.7027 gn=0.53 lr=5.91e-04 tps=15042 mem=6.4G lam=0.1397992434600989 Gm=0.5250659603625536 +[full s1] step 1200/12000 loss=5.6508 gn=0.54 lr=5.90e-04 tps=15046 mem=6.4G lam=0.14024506974965334 Gm=0.5201472174376249 +[full s1] step 1220/12000 loss=5.8270 gn=0.52 lr=5.90e-04 tps=15050 mem=6.4G lam=0.14067930790285268 Gm=0.5142818142970403 +[full s1] step 1240/12000 loss=5.6051 gn=0.55 lr=5.90e-04 tps=15054 mem=6.4G lam=0.1411820643891891 Gm=0.5086755454540253 +[full s1] step 1260/12000 loss=5.5883 gn=0.57 lr=5.89e-04 tps=15058 mem=6.4G lam=0.14161355855564275 Gm=0.5008563883602619 +[full s1] step 1280/12000 loss=5.4934 gn=0.63 lr=5.89e-04 tps=15062 mem=6.4G lam=0.14210093518098196 Gm=0.5111561908076206 +[full s1] step 1300/12000 loss=5.5969 gn=0.52 lr=5.89e-04 tps=15067 mem=6.4G lam=0.14261136731753746 Gm=0.49679382890462875 +[full s1] step 1320/12000 loss=5.4730 gn=0.53 lr=5.88e-04 tps=15070 mem=6.4G lam=0.14309041605641445 Gm=0.49579310851792496 +[full s1] step 1340/12000 loss=5.7192 gn=0.62 lr=5.88e-04 tps=15074 mem=6.4G lam=0.14360974449664354 Gm=0.48786865050594014 +[full s1] step 1360/12000 loss=5.5509 gn=0.56 lr=5.87e-04 tps=15078 mem=6.4G lam=0.144102250225842 Gm=0.49437601305544376 +[full s1] step 1380/12000 loss=5.4662 gn=0.53 lr=5.87e-04 tps=15082 mem=6.4G lam=0.1445565310617288 Gm=0.4992907239745061 +[full s1] step 1400/12000 loss=5.6736 gn=0.54 lr=5.86e-04 tps=15086 mem=6.4G lam=0.14502672757953405 Gm=0.48423727105061215 +[full s1] step 1420/12000 loss=5.4428 gn=0.54 lr=5.86e-04 tps=15089 mem=6.4G lam=0.14550234439472357 Gm=0.47503861722846824 +[full s1] step 1440/12000 loss=5.5681 gn=0.53 lr=5.85e-04 tps=15092 mem=6.4G lam=0.14598066949596009 Gm=0.49535263205568 +[full s1] step 1460/12000 loss=5.5947 gn=0.60 lr=5.85e-04 tps=15095 mem=6.4G lam=0.1464797913407286 Gm=0.48866956618924934 +[full s1] step 1480/12000 loss=5.4859 gn=0.54 lr=5.84e-04 tps=15098 mem=6.4G lam=0.1469409835214416 Gm=0.4797961926087737 +[full s1] step 1500/12000 loss=5.2924 gn=0.64 lr=5.84e-04 tps=15101 mem=6.4G lam=0.14740437424431244 Gm=0.49094694790740806 +[full s1] EVAL step 1500: val_loss=5.4998 val_ppl=244.63 +[full s1] step 1520/12000 loss=5.4853 gn=0.57 lr=5.83e-04 tps=15029 mem=6.4G lam=0.14792320939401785 Gm=0.49411463861664134 +[full s1] step 1540/12000 loss=5.4689 gn=0.58 lr=5.83e-04 tps=15034 mem=6.4G lam=0.14836191199719906 Gm=0.4858884538213412 +[full s1] step 1560/12000 loss=5.4707 gn=0.56 lr=5.82e-04 tps=15037 mem=6.4G lam=0.14881866735716662 Gm=0.46717609899739426 +[full s1] step 1580/12000 loss=5.1545 gn=0.53 lr=5.82e-04 tps=15036 mem=6.4G lam=0.14933893146614233 Gm=0.4865201183905204 +[full s1] step 1600/12000 loss=5.2384 gn=0.57 lr=5.81e-04 tps=15035 mem=6.4G lam=0.14979441122462353 Gm=0.45702402604122955 +[full s1] step 1620/12000 loss=5.2430 gn=0.58 lr=5.81e-04 tps=15035 mem=6.4G lam=0.15030948352068663 Gm=0.4778065225109458 +[full s1] step 1640/12000 loss=5.2453 gn=0.55 lr=5.80e-04 tps=15036 mem=6.4G lam=0.15086202509701252 Gm=0.46451487361143035 +[full s1] step 1660/12000 loss=5.3797 gn=0.61 lr=5.80e-04 tps=15037 mem=6.4G lam=0.1513464174543818 Gm=0.46604346701254445 +[full s1] step 1680/12000 loss=5.1902 gn=0.55 lr=5.79e-04 tps=15038 mem=6.4G lam=0.15187862142920494 Gm=0.46669851150363684 +[full s1] step 1700/12000 loss=5.2776 gn=0.53 lr=5.79e-04 tps=15038 mem=6.4G lam=0.1523896176367998 Gm=0.46310600141684216 +[full s1] step 1720/12000 loss=5.3419 gn=0.54 lr=5.78e-04 tps=15039 mem=6.4G lam=0.15285915229469538 Gm=0.4667445082838337 +[full s1] step 1740/12000 loss=5.2747 gn=0.56 lr=5.78e-04 tps=15041 mem=6.4G lam=0.1533452638735374 Gm=0.4592539019261797 +[full s1] step 1760/12000 loss=5.3220 gn=0.57 lr=5.77e-04 tps=15041 mem=6.4G lam=0.15379787981510162 Gm=0.4685449091096719 +[full s1] step 1780/12000 loss=5.3250 gn=0.55 lr=5.76e-04 tps=15041 mem=6.4G lam=0.15428432046125332 Gm=0.4611059765641888 +[full s1] step 1800/12000 loss=5.4217 gn=0.62 lr=5.76e-04 tps=15041 mem=6.4G lam=0.1548013056938847 Gm=0.4686073859532674 +[full s1] step 1820/12000 loss=5.4223 gn=0.56 lr=5.75e-04 tps=15042 mem=6.4G lam=0.15531393947700659 Gm=0.45914786743621033 +[full s1] step 1840/12000 loss=5.1765 gn=0.60 lr=5.75e-04 tps=15043 mem=6.4G lam=0.1557793046037356 Gm=0.4511067798982064 +[full s1] step 1860/12000 loss=5.2717 gn=0.62 lr=5.74e-04 tps=15043 mem=6.4G lam=0.15625694238891205 Gm=0.45839044141272706 +[full s1] step 1880/12000 loss=5.1241 gn=0.58 lr=5.73e-04 tps=15044 mem=6.4G lam=0.15679655193040767 Gm=0.45472002153595287 +[full s1] step 1900/12000 loss=5.3495 gn=0.59 lr=5.73e-04 tps=15044 mem=6.4G lam=0.15727647269765535 Gm=0.4490290687729915 +[full s1] step 1920/12000 loss=5.3751 gn=0.62 lr=5.72e-04 tps=15045 mem=6.4G lam=0.15773574045548835 Gm=0.4572247117757797 +[full s1] step 1940/12000 loss=5.4825 gn=0.56 lr=5.72e-04 tps=15045 mem=6.4G lam=0.15828591274718443 Gm=0.4475600505247712 +[full s1] step 1960/12000 loss=5.1259 gn=0.58 lr=5.71e-04 tps=15047 mem=6.4G lam=0.1587673577790459 Gm=0.43393760329733294 +[full s1] step 1980/12000 loss=5.2515 gn=0.60 lr=5.70e-04 tps=15048 mem=6.4G lam=0.15932540564487377 Gm=0.4404917157565554 +[full s1] step 2000/12000 loss=5.3138 gn=0.62 lr=5.70e-04 tps=15048 mem=6.4G lam=0.15990498196333647 Gm=0.44254679962371785 +[full s1] EVAL step 2000: val_loss=5.3397 val_ppl=208.44 +[full s1] step 2020/12000 loss=5.2256 gn=0.62 lr=5.69e-04 tps=14993 mem=6.4G lam=0.16048197324077287 Gm=0.4400143741319577 +[full s1] step 2040/12000 loss=5.2774 gn=0.59 lr=5.68e-04 tps=14994 mem=6.4G lam=0.1609654643883308 Gm=0.44182249748458463 +[full s1] step 2060/12000 loss=5.2358 gn=0.56 lr=5.68e-04 tps=14994 mem=6.4G lam=0.16149225706855455 Gm=0.44253334620346624 +[full s1] step 2080/12000 loss=5.2488 gn=0.62 lr=5.67e-04 tps=14995 mem=6.4G lam=0.1619664803147316 Gm=0.4571499613424142 +[full s1] step 2100/12000 loss=5.2864 gn=0.56 lr=5.66e-04 tps=14996 mem=6.4G lam=0.16239299376805624 Gm=0.42950112217416364 +[full s1] step 2120/12000 loss=5.1488 gn=0.56 lr=5.65e-04 tps=14997 mem=6.4G lam=0.16285897139459848 Gm=0.4387456787129243 +[full s1] step 2140/12000 loss=5.1108 gn=0.67 lr=5.65e-04 tps=14998 mem=6.4G lam=0.16338208566109338 Gm=0.4414132108601431 +[full s1] step 2160/12000 loss=5.1806 gn=0.58 lr=5.64e-04 tps=14999 mem=6.4G lam=0.16390493356933197 Gm=0.43529749289155006 +[full s1] step 2180/12000 loss=5.1872 gn=0.60 lr=5.63e-04 tps=15001 mem=6.4G lam=0.16443953973551592 Gm=0.4204845493659377 +[full s1] step 2200/12000 loss=5.1405 gn=0.58 lr=5.63e-04 tps=15003 mem=6.4G lam=0.16494531774272522 Gm=0.42809480025122565 +[full s1] step 2220/12000 loss=5.2787 gn=0.60 lr=5.62e-04 tps=15004 mem=6.4G lam=0.1654657982289791 Gm=0.42853371193632483 +[full s1] step 2240/12000 loss=5.0631 gn=0.58 lr=5.61e-04 tps=15005 mem=6.4G lam=0.1660358545680841 Gm=0.42935371507580083 +[full s1] step 2260/12000 loss=5.0517 gn=0.57 lr=5.60e-04 tps=15006 mem=6.4G lam=0.1665091086179018 Gm=0.42736871757855016 +[full s1] step 2280/12000 loss=5.1714 gn=0.63 lr=5.60e-04 tps=15008 mem=6.4G lam=0.16707039314011732 Gm=0.4278221872324745 +[full s1] step 2300/12000 loss=5.0942 gn=0.58 lr=5.59e-04 tps=15009 mem=6.4G lam=0.16755416740973791 Gm=0.42178585814932984 +[full s1] step 2320/12000 loss=4.9725 gn=0.63 lr=5.58e-04 tps=15010 mem=6.4G lam=0.16808387668182453 Gm=0.4178652417225142 +[full s1] step 2340/12000 loss=5.1535 gn=0.58 lr=5.57e-04 tps=15011 mem=6.4G lam=0.1685764662300547 Gm=0.4267597170546651 +[full s1] step 2360/12000 loss=5.2220 gn=0.62 lr=5.57e-04 tps=15012 mem=6.4G lam=0.16904318425804377 Gm=0.42788540354619425 +[full s1] step 2380/12000 loss=5.1750 gn=0.64 lr=5.56e-04 tps=15013 mem=6.4G lam=0.169654021350046 Gm=0.42358839480827254 +[full s1] step 2400/12000 loss=5.0820 gn=0.65 lr=5.55e-04 tps=15015 mem=6.4G lam=0.17022556470086178 Gm=0.4310847794016202 +[full s1] step 2420/12000 loss=5.1206 gn=0.60 lr=5.54e-04 tps=15015 mem=6.4G lam=0.17069197011490664 Gm=0.41329746910681325 +[full s1] step 2440/12000 loss=5.1212 gn=0.64 lr=5.53e-04 tps=15016 mem=6.4G lam=0.1712501710280776 Gm=0.42022023210301995 +[full s1] step 2460/12000 loss=5.1352 gn=0.62 lr=5.53e-04 tps=15017 mem=6.4G lam=0.17179489663491646 Gm=0.42324206481377286 +[full s1] step 2480/12000 loss=5.1137 gn=0.61 lr=5.52e-04 tps=15019 mem=6.4G lam=0.17226049428184828 Gm=0.41971947128574055 +[full s1] step 2500/12000 loss=5.1318 gn=0.67 lr=5.51e-04 tps=15020 mem=6.4G lam=0.17278560002644858 Gm=0.41679671173915267 +[full s1] EVAL step 2500: val_loss=5.1686 val_ppl=175.66 +[full s1] step 2520/12000 loss=5.2157 gn=0.68 lr=5.50e-04 tps=14976 mem=6.4G lam=0.1733467181523641 Gm=0.4191951248794794 +[full s1] step 2540/12000 loss=5.1745 gn=0.65 lr=5.49e-04 tps=14977 mem=6.4G lam=0.17391117879499993 Gm=0.4185059356192748 +[full s1] step 2560/12000 loss=5.1510 gn=0.62 lr=5.48e-04 tps=14979 mem=6.4G lam=0.17443909402936697 Gm=0.4222052649905284 +[full s1] step 2580/12000 loss=4.9942 gn=0.67 lr=5.48e-04 tps=14980 mem=6.4G lam=0.17497245067109665 Gm=0.4096009056083858 +[full s1] step 2600/12000 loss=5.0684 gn=0.63 lr=5.47e-04 tps=14981 mem=6.4G lam=0.17552269777903953 Gm=0.4104561192604403 +[full s1] step 2620/12000 loss=4.9777 gn=0.63 lr=5.46e-04 tps=14982 mem=6.4G lam=0.17607388831675053 Gm=0.4036020350952943 +[full s1] step 2640/12000 loss=5.1787 gn=0.66 lr=5.45e-04 tps=14984 mem=6.4G lam=0.17660792544484138 Gm=0.4197468024988969 +[full s1] step 2660/12000 loss=5.0063 gn=0.64 lr=5.44e-04 tps=14985 mem=6.4G lam=0.1771103631084164 Gm=0.40465116982037824 +[full s1] step 2680/12000 loss=5.0146 gn=0.67 lr=5.43e-04 tps=14986 mem=6.4G lam=0.17758611869066954 Gm=0.4134108276727299 +[full s1] step 2700/12000 loss=5.1650 gn=0.69 lr=5.42e-04 tps=14987 mem=6.4G lam=0.17816964350640774 Gm=0.40519791515544057 +[full s1] step 2720/12000 loss=5.0139 gn=0.64 lr=5.41e-04 tps=14989 mem=6.4G lam=0.17870614056785902 Gm=0.4123427518643439 +[full s1] step 2740/12000 loss=5.1602 gn=0.66 lr=5.41e-04 tps=14990 mem=6.4G lam=0.17928303064157566 Gm=0.41071807344754535 +[full s1] step 2760/12000 loss=4.9844 gn=0.75 lr=5.40e-04 tps=14991 mem=6.4G lam=0.17973885405808687 Gm=0.4159242582196991 +[full s1] step 2780/12000 loss=5.1117 gn=0.67 lr=5.39e-04 tps=14992 mem=6.4G lam=0.18017095141112804 Gm=0.40580977173522115 +[full s1] step 2800/12000 loss=4.8802 gn=0.65 lr=5.38e-04 tps=14993 mem=6.4G lam=0.1806250453616182 Gm=0.40025877781833213 +[full s1] step 2820/12000 loss=5.1923 gn=0.66 lr=5.37e-04 tps=14994 mem=6.4G lam=0.1811371094857653 Gm=0.40688813136269647 +[full s1] step 2840/12000 loss=5.0334 gn=0.67 lr=5.36e-04 tps=14994 mem=6.4G lam=0.18173559724042812 Gm=0.4058922144273917 +[full s1] step 2860/12000 loss=4.9748 gn=0.64 lr=5.35e-04 tps=14996 mem=6.4G lam=0.18220047559589148 Gm=0.4108061242538194 +[full s1] step 2880/12000 loss=5.0345 gn=0.71 lr=5.34e-04 tps=14997 mem=6.4G lam=0.18270055763423443 Gm=0.3939345497637987 +[full s1] step 2900/12000 loss=5.0399 gn=0.66 lr=5.33e-04 tps=14998 mem=6.4G lam=0.18319883042325577 Gm=0.3984602933439116 +[full s1] step 2920/12000 loss=4.9872 gn=0.81 lr=5.32e-04 tps=14999 mem=6.4G lam=0.18365457964440188 Gm=0.3983884967553119 +[full s1] step 2940/12000 loss=5.0609 gn=0.65 lr=5.31e-04 tps=15000 mem=6.4G lam=0.18414263365169367 Gm=0.3946270962866644 +[full s1] step 2960/12000 loss=4.8556 gn=0.68 lr=5.30e-04 tps=15002 mem=6.4G lam=0.18457439665993056 Gm=0.3923480113347371 +[full s1] step 2980/12000 loss=4.9490 gn=0.61 lr=5.29e-04 tps=15002 mem=6.4G lam=0.18497683635602394 Gm=0.39650011404107016 +[full s1] step 3000/12000 loss=4.9598 gn=0.70 lr=5.28e-04 tps=15003 mem=6.4G lam=0.1854154352719585 Gm=0.3968866447297235 +[full s1] EVAL step 3000: val_loss=5.0315 val_ppl=153.16 +[full s1] step 3020/12000 loss=4.9881 gn=0.66 lr=5.27e-04 tps=14967 mem=6.4G lam=0.18591970826188722 Gm=0.39868138792614144 +[full s1] step 3040/12000 loss=4.8237 gn=0.62 lr=5.26e-04 tps=14967 mem=6.4G lam=0.18643991245577732 Gm=0.39237305149435997 +[full s1] step 3060/12000 loss=4.9882 gn=0.69 lr=5.25e-04 tps=14968 mem=6.4G lam=0.1870363550260663 Gm=0.3957376967494686 +[full s1] step 3080/12000 loss=4.8105 gn=0.69 lr=5.24e-04 tps=14969 mem=6.4G lam=0.18751942863066992 Gm=0.3923614569939673 +[full s1] step 3100/12000 loss=4.9827 gn=0.68 lr=5.23e-04 tps=14971 mem=6.4G lam=0.1879943838963906 Gm=0.3923491245756547 +[full s1] step 3120/12000 loss=4.9718 gn=0.71 lr=5.22e-04 tps=14972 mem=6.4G lam=0.18843579292297363 Gm=0.4027436982529859 +[full s1] step 3140/12000 loss=4.9303 gn=0.68 lr=5.21e-04 tps=14973 mem=6.4G lam=0.18894681055098772 Gm=0.39573678203547996 +[full s1] step 3160/12000 loss=4.9548 gn=0.66 lr=5.20e-04 tps=14974 mem=6.4G lam=0.1894544381648302 Gm=0.39118253346532583 +[full s1] step 3180/12000 loss=4.9884 gn=0.80 lr=5.19e-04 tps=14975 mem=6.4G lam=0.1899393325050672 Gm=0.3957154434174299 +[full s1] step 3200/12000 loss=4.7724 gn=0.65 lr=5.18e-04 tps=14976 mem=6.4G lam=0.1903568860143423 Gm=0.38460491349299747 +[full s1] step 3220/12000 loss=4.9610 gn=0.77 lr=5.17e-04 tps=14976 mem=6.4G lam=0.1908245130131642 Gm=0.39597682701423764 +[full s1] step 3240/12000 loss=5.0215 gn=0.70 lr=5.16e-04 tps=14977 mem=6.4G lam=0.19130467313031355 Gm=0.3976783783485492 +[full s1] step 3260/12000 loss=4.8778 gn=0.67 lr=5.15e-04 tps=14978 mem=6.4G lam=0.1918546153853337 Gm=0.389945635261635 +[full s1] step 3280/12000 loss=4.8228 gn=0.75 lr=5.14e-04 tps=14979 mem=6.4G lam=0.1923148650676012 Gm=0.3916345052421093 +[full s1] step 3300/12000 loss=4.9939 gn=0.68 lr=5.13e-04 tps=14980 mem=6.4G lam=0.19285838368038336 Gm=0.39586646544436616 +[full s1] step 3320/12000 loss=5.0226 gn=0.69 lr=5.12e-04 tps=14981 mem=6.4G lam=0.19338323455303907 Gm=0.39684691388780874 +[full s1] step 3340/12000 loss=4.8594 gn=0.65 lr=5.11e-04 tps=14982 mem=6.4G lam=0.19379915048678717 Gm=0.3942444010948141 +[full s1] step 3360/12000 loss=4.7907 gn=0.66 lr=5.10e-04 tps=14983 mem=6.4G lam=0.1942577799782157 Gm=0.3938445254849891 +[full s1] step 3380/12000 loss=5.0478 gn=0.67 lr=5.09e-04 tps=14984 mem=6.4G lam=0.1947017451748252 Gm=0.3959193304181099 +[full s1] step 3400/12000 loss=5.1334 gn=0.70 lr=5.08e-04 tps=14985 mem=6.4G lam=0.19520254681507745 Gm=0.3968056154747804 +[full s1] step 3420/12000 loss=5.0416 gn=0.77 lr=5.07e-04 tps=14986 mem=6.4G lam=0.19567940601458153 Gm=0.3865506239235401 +[full s1] step 3440/12000 loss=4.8358 gn=0.69 lr=5.06e-04 tps=14987 mem=6.4G lam=0.19615983155866465 Gm=0.3851083545014262 +[full s1] step 3460/12000 loss=4.9627 gn=0.70 lr=5.05e-04 tps=14987 mem=6.4G lam=0.19670069621255 Gm=0.38711169579376775 +[full s1] step 3480/12000 loss=4.8570 gn=0.71 lr=5.03e-04 tps=14988 mem=6.4G lam=0.1972318027789394 Gm=0.390395356186976 +[full s1] step 3500/12000 loss=4.8909 gn=0.72 lr=5.02e-04 tps=14989 mem=6.4G lam=0.19771316864838204 Gm=0.38419649897453684 +[full s1] EVAL step 3500: val_loss=4.9867 val_ppl=146.46 +[full s1] step 3520/12000 loss=4.8320 gn=0.69 lr=5.01e-04 tps=14958 mem=6.4G lam=0.19817129212121168 Gm=0.3906831302059193 +[full s1] step 3540/12000 loss=4.9267 gn=0.72 lr=5.00e-04 tps=14959 mem=6.4G lam=0.19855653091023365 Gm=0.3963502161204815 +[full s1] step 3560/12000 loss=4.9063 gn=0.69 lr=4.99e-04 tps=14959 mem=6.4G lam=0.19909179055442414 Gm=0.38569319900125265 +[full s1] step 3580/12000 loss=4.9846 gn=0.76 lr=4.98e-04 tps=14960 mem=6.4G lam=0.19963449022422233 Gm=0.3832656890153885 +[full s1] step 3600/12000 loss=5.0507 gn=0.73 lr=4.97e-04 tps=14960 mem=6.4G lam=0.20007767590383688 Gm=0.389210515966018 +[full s1] step 3620/12000 loss=4.9047 gn=0.72 lr=4.96e-04 tps=14961 mem=6.4G lam=0.2005883470798532 Gm=0.3864594671564798 +[full s1] step 3640/12000 loss=4.7740 gn=0.77 lr=4.94e-04 tps=14962 mem=6.4G lam=0.2011195303251346 Gm=0.38664374127984047 +[full s1] step 3660/12000 loss=4.7301 gn=0.71 lr=4.93e-04 tps=14963 mem=6.4G lam=0.20154940647383532 Gm=0.37231302199264366 +[full s1] step 3680/12000 loss=4.7127 gn=0.66 lr=4.92e-04 tps=14964 mem=6.4G lam=0.20195918809622526 Gm=0.38360304951978225 +[full s1] step 3700/12000 loss=4.9417 gn=0.73 lr=4.91e-04 tps=14965 mem=6.4G lam=0.20242032315582037 Gm=0.38928530846411985 +[full s1] step 3720/12000 loss=4.8841 gn=0.70 lr=4.90e-04 tps=14966 mem=6.4G lam=0.20285006643583378 Gm=0.3776211114600301 +[full s1] step 3740/12000 loss=4.8984 gn=0.70 lr=4.89e-04 tps=14966 mem=6.4G lam=0.20331771858036518 Gm=0.3829444528867801 +[full s1] step 3760/12000 loss=4.8615 gn=0.73 lr=4.88e-04 tps=14967 mem=6.4G lam=0.20384814900656542 Gm=0.3852472440339625 +[full s1] step 3780/12000 loss=4.7740 gn=0.73 lr=4.86e-04 tps=14968 mem=6.4G lam=0.20426123340924582 Gm=0.3814217286805312 +[full s1] step 3800/12000 loss=4.7904 gn=0.71 lr=4.85e-04 tps=14969 mem=6.4G lam=0.20474778302013874 Gm=0.38044378510676324 +[full s1] step 3820/12000 loss=4.8357 gn=0.69 lr=4.84e-04 tps=14969 mem=6.4G lam=0.20519877411425114 Gm=0.3849569895149519 +[full s1] step 3840/12000 loss=4.9221 gn=0.69 lr=4.83e-04 tps=14970 mem=6.4G lam=0.20564522873610258 Gm=0.3805615307452778 +[full s1] step 3860/12000 loss=4.7970 gn=0.74 lr=4.82e-04 tps=14971 mem=6.4G lam=0.2061215927824378 Gm=0.37187275802716613 +[full s1] step 3880/12000 loss=4.6664 gn=0.67 lr=4.80e-04 tps=14972 mem=6.4G lam=0.2066107140854001 Gm=0.3797315318758289 +[full s1] step 3900/12000 loss=4.7961 gn=0.75 lr=4.79e-04 tps=14973 mem=6.4G lam=0.20712714393933615 Gm=0.37564280272151035 +[full s1] step 3920/12000 loss=4.9016 gn=0.73 lr=4.78e-04 tps=14973 mem=6.4G lam=0.20762438358118138 Gm=0.3796211796967934 +[full s1] step 3940/12000 loss=4.7344 gn=0.70 lr=4.77e-04 tps=14974 mem=6.4G lam=0.20810106055190167 Gm=0.37997589260339737 +[full s1] step 3960/12000 loss=4.9707 gn=0.72 lr=4.76e-04 tps=14974 mem=6.4G lam=0.2086110496893525 Gm=0.37405933486297727 +[full s1] step 3980/12000 loss=5.0192 gn=0.80 lr=4.74e-04 tps=14975 mem=6.4G lam=0.20913751485447088 Gm=0.3787727613622944 +[full s1] step 4000/12000 loss=4.9264 gn=0.73 lr=4.73e-04 tps=14976 mem=6.4G lam=0.20954076293855906 Gm=0.37509435741230845 +[full s1] EVAL step 4000: val_loss=4.8459 val_ppl=127.21 +[full s1] step 4020/12000 loss=4.5730 gn=0.66 lr=4.72e-04 tps=14949 mem=6.4G lam=0.20995215947429338 Gm=0.3806492208192746 +[full s1] step 4040/12000 loss=4.8364 gn=0.72 lr=4.71e-04 tps=14950 mem=6.4G lam=0.21045360248535872 Gm=0.3771408104027311 +[full s1] step 4060/12000 loss=4.9125 gn=0.78 lr=4.70e-04 tps=14951 mem=6.4G lam=0.2110036676749587 Gm=0.3686733143404126 +[full s1] step 4080/12000 loss=4.7035 gn=0.72 lr=4.68e-04 tps=14951 mem=6.4G lam=0.21147102086494365 Gm=0.3686879393644631 +[full s1] step 4100/12000 loss=4.8377 gn=0.75 lr=4.67e-04 tps=14952 mem=6.4G lam=0.21188004625340304 Gm=0.37463575570533675 +[full s1] step 4120/12000 loss=4.8264 gn=0.78 lr=4.66e-04 tps=14952 mem=6.4G lam=0.21229513517270485 Gm=0.37910475620689493 +[full s1] step 4140/12000 loss=4.8938 gn=0.75 lr=4.65e-04 tps=14953 mem=6.4G lam=0.21282581984996796 Gm=0.3812202579962711 +[full s1] step 4160/12000 loss=4.7213 gn=0.76 lr=4.63e-04 tps=14954 mem=6.4G lam=0.21322089744110903 Gm=0.3751378464512527 +[full s1] step 4180/12000 loss=4.9601 gn=0.76 lr=4.62e-04 tps=14955 mem=6.4G lam=0.21367353418221077 Gm=0.37702339196888107 +[full s1] step 4200/12000 loss=4.6669 gn=0.74 lr=4.61e-04 tps=14956 mem=6.4G lam=0.2141635020573934 Gm=0.3800294892862439 +[full s1] step 4220/12000 loss=4.7361 gn=0.73 lr=4.60e-04 tps=14957 mem=6.4G lam=0.2146925569201509 Gm=0.37625679032256204 +[full s1] step 4240/12000 loss=4.6893 gn=0.68 lr=4.58e-04 tps=14958 mem=6.4G lam=0.21521606078992286 Gm=0.37928203664099175 +[full s1] step 4260/12000 loss=4.7335 gn=0.72 lr=4.57e-04 tps=14959 mem=6.4G lam=0.2157188830897212 Gm=0.3730606504250318 +[full s1] step 4280/12000 loss=4.8633 gn=0.82 lr=4.56e-04 tps=14960 mem=6.4G lam=0.21619992734243473 Gm=0.38272624167924124 +[full s1] step 4300/12000 loss=4.9130 gn=0.74 lr=4.54e-04 tps=14960 mem=6.4G lam=0.21671670023351908 Gm=0.37422184894482297 +[full s1] step 4320/12000 loss=4.7026 gn=0.72 lr=4.53e-04 tps=14961 mem=6.4G lam=0.2172152598698934 Gm=0.37501811461212736 +[full s1] step 4340/12000 loss=4.7478 gn=0.70 lr=4.52e-04 tps=14961 mem=6.4G lam=0.21766561611245075 Gm=0.37222437079374987 +[full s1] step 4360/12000 loss=4.8249 gn=0.75 lr=4.51e-04 tps=14962 mem=6.4G lam=0.2182499667008718 Gm=0.37534508605798084 +[full s1] step 4380/12000 loss=4.7023 gn=0.74 lr=4.49e-04 tps=14963 mem=6.4G lam=0.21872822598864636 Gm=0.37400468112900853 +[full s1] step 4400/12000 loss=4.7954 gn=0.76 lr=4.48e-04 tps=14963 mem=6.4G lam=0.2191599498813351 Gm=0.37007787901287276 +[full s1] step 4420/12000 loss=4.7934 gn=0.77 lr=4.47e-04 tps=14964 mem=6.4G lam=0.2196408354987701 Gm=0.3758167006696264 +[full s1] step 4440/12000 loss=4.7060 gn=0.76 lr=4.45e-04 tps=14965 mem=6.4G lam=0.2201397263755401 Gm=0.37172647185313207 +[full s1] step 4460/12000 loss=4.5242 gn=0.74 lr=4.44e-04 tps=14965 mem=6.4G lam=0.22058281457672516 Gm=0.3712911718369772 +[full s1] step 4480/12000 loss=4.6588 gn=0.70 lr=4.43e-04 tps=14965 mem=6.4G lam=0.22119973910351595 Gm=0.37429977968956035 +[full s1] step 4500/12000 loss=4.9386 gn=0.72 lr=4.42e-04 tps=14966 mem=6.4G lam=0.221638402591149 Gm=0.36432913481257856 +[full s1] EVAL step 4500: val_loss=4.7628 val_ppl=117.07 +[full s1] step 4520/12000 loss=4.8204 gn=0.78 lr=4.40e-04 tps=14942 mem=6.4G lam=0.22209187783300877 Gm=0.37729213216031593 +[full s1] step 4540/12000 loss=4.7875 gn=0.72 lr=4.39e-04 tps=14943 mem=6.4G lam=0.22255874735613665 Gm=0.37067429806726676 +[full s1] step 4560/12000 loss=4.6597 gn=0.73 lr=4.38e-04 tps=14944 mem=6.4G lam=0.22309325262904167 Gm=0.36570760514587164 +[full s1] step 4580/12000 loss=4.8381 gn=0.78 lr=4.36e-04 tps=14945 mem=6.4G lam=0.22356062040974697 Gm=0.38268600264564157 +[full s1] step 4600/12000 loss=4.6668 gn=0.82 lr=4.35e-04 tps=14945 mem=6.4G lam=0.2240753968556722 Gm=0.3727439334616065 +[full s1] step 4620/12000 loss=4.6802 gn=0.80 lr=4.34e-04 tps=14946 mem=6.4G lam=0.2244894653558731 Gm=0.3703835306999584 +[full s1] step 4640/12000 loss=4.8249 gn=0.78 lr=4.32e-04 tps=14947 mem=6.4G lam=0.224920516833663 Gm=0.36716553568840027 +[full s1] step 4660/12000 loss=4.6951 gn=0.76 lr=4.31e-04 tps=14948 mem=6.4G lam=0.2254283813138803 Gm=0.3668460439269741 +[full s1] step 4680/12000 loss=4.8195 gn=0.74 lr=4.30e-04 tps=14949 mem=6.4G lam=0.22585028813531002 Gm=0.3677620122519632 +[full s1] step 4700/12000 loss=4.7601 gn=0.79 lr=4.28e-04 tps=14949 mem=6.4G lam=0.2261975103368362 Gm=0.3657983403342466 +[full s1] step 4720/12000 loss=4.7832 gn=0.77 lr=4.27e-04 tps=14950 mem=6.4G lam=0.22659939713776112 Gm=0.3694853999962409 +[full s1] step 4740/12000 loss=4.7498 gn=0.76 lr=4.26e-04 tps=14951 mem=6.4G lam=0.226980771869421 Gm=0.3716899997089058 +[full s1] step 4760/12000 loss=4.8067 gn=0.75 lr=4.24e-04 tps=14952 mem=6.4G lam=0.22744736510018507 Gm=0.3633918932173401 +[full s1] step 4780/12000 loss=4.6168 gn=0.82 lr=4.23e-04 tps=14953 mem=6.4G lam=0.22802451035628715 Gm=0.36724060455647606 +[full s1] step 4800/12000 loss=4.5786 gn=0.75 lr=4.22e-04 tps=14953 mem=6.4G lam=0.22842577969034514 Gm=0.36432265241940814 +[full s1] step 4820/12000 loss=4.7484 gn=0.70 lr=4.20e-04 tps=14954 mem=6.4G lam=0.2288939127077659 Gm=0.3663145396082352 +[full s1] step 4840/12000 loss=4.7424 gn=0.76 lr=4.19e-04 tps=14955 mem=6.4G lam=0.22933503209302822 Gm=0.3626585485568891 +[full s1] step 4860/12000 loss=4.8278 gn=0.70 lr=4.18e-04 tps=14956 mem=6.4G lam=0.22985996150722107 Gm=0.36458040758346516 +[full s1] step 4880/12000 loss=4.6147 gn=0.74 lr=4.16e-04 tps=14957 mem=6.4G lam=0.2302693168943127 Gm=0.37304140836931765 +[full s1] step 4900/12000 loss=4.7266 gn=0.73 lr=4.15e-04 tps=14957 mem=6.4G lam=0.23078510196258625 Gm=0.36698070998924476 +[full s1] step 4920/12000 loss=4.7565 gn=0.73 lr=4.13e-04 tps=14958 mem=6.4G lam=0.23123980034142733 Gm=0.36269292514771223 +[full s1] step 4940/12000 loss=4.5155 gn=0.77 lr=4.12e-04 tps=14958 mem=6.4G lam=0.2316896210735043 Gm=0.3555888255747656 +[full s1] step 4960/12000 loss=4.7286 gn=0.83 lr=4.11e-04 tps=14959 mem=6.4G lam=0.23206498908499876 Gm=0.3665480197717746 +[full s1] step 4980/12000 loss=4.8296 gn=0.75 lr=4.09e-04 tps=14960 mem=6.4G lam=0.23243206708381572 Gm=0.36154867607789737 +[full s1] step 5000/12000 loss=4.7071 gn=0.83 lr=4.08e-04 tps=14960 mem=6.4G lam=0.23282447333137193 Gm=0.36828932468779385 +[full s1] EVAL step 5000: val_loss=4.7319 val_ppl=113.52 +[full s1] step 5020/12000 loss=4.6582 gn=0.76 lr=4.07e-04 tps=14938 mem=6.4G lam=0.2333515811090668 Gm=0.3608268881992747 +[full s1] step 5040/12000 loss=4.6099 gn=0.77 lr=4.05e-04 tps=14939 mem=6.4G lam=0.23382905467102924 Gm=0.36591486919981736 +[full s1] step 5060/12000 loss=4.7399 gn=0.75 lr=4.04e-04 tps=14940 mem=6.4G lam=0.23427053758253655 Gm=0.36017881869338453 +[full s1] step 5080/12000 loss=4.7424 gn=0.79 lr=4.02e-04 tps=14940 mem=6.4G lam=0.23478025694688162 Gm=0.3644076492637396 +[full s1] step 5100/12000 loss=4.6320 gn=0.79 lr=4.01e-04 tps=14941 mem=6.4G lam=0.2351724530259768 Gm=0.3611690717904518 +[full s1] step 5120/12000 loss=4.7514 gn=0.73 lr=4.00e-04 tps=14942 mem=6.4G lam=0.23561857702831426 Gm=0.3644266583335896 +[full s1] step 5140/12000 loss=4.7018 gn=0.72 lr=3.98e-04 tps=14942 mem=6.4G lam=0.23607436319192251 Gm=0.3563565543542306 +[full s1] step 5160/12000 loss=4.7354 gn=0.77 lr=3.97e-04 tps=14943 mem=6.4G lam=0.23655954655259848 Gm=0.36777909006923437 +[full s1] step 5180/12000 loss=4.7646 gn=0.83 lr=3.95e-04 tps=14944 mem=6.4G lam=0.23710218320290247 Gm=0.3640374782650421 +[full s1] step 5200/12000 loss=4.7405 gn=0.82 lr=3.94e-04 tps=14944 mem=6.4G lam=0.23754268481085697 Gm=0.3649633709186067 +[full s1] step 5220/12000 loss=4.5415 gn=0.76 lr=3.93e-04 tps=14945 mem=6.4G lam=0.23798670681814352 Gm=0.3614440616996338 +[full s1] step 5240/12000 loss=4.8775 gn=0.82 lr=3.91e-04 tps=14945 mem=6.4G lam=0.23840092246731123 Gm=0.3659230852499604 +[full s1] step 5260/12000 loss=4.6211 gn=0.77 lr=3.90e-04 tps=14946 mem=6.4G lam=0.2388043226674199 Gm=0.36463975238924223 +[full s1] step 5280/12000 loss=4.5235 gn=0.75 lr=3.88e-04 tps=14947 mem=6.4G lam=0.23911674538006386 Gm=0.36188951080354553 +[full s1] step 5300/12000 loss=4.6641 gn=0.78 lr=3.87e-04 tps=14947 mem=6.4G lam=0.23948627368857464 Gm=0.3604513762208323 +[full s1] step 5320/12000 loss=4.6375 gn=0.77 lr=3.86e-04 tps=14948 mem=6.4G lam=0.23991193156689405 Gm=0.3604962717120846 +[full s1] step 5340/12000 loss=4.7242 gn=0.77 lr=3.84e-04 tps=14949 mem=6.4G lam=0.2403493756428361 Gm=0.3652848042547703 +[full s1] step 5360/12000 loss=4.4989 gn=0.76 lr=3.83e-04 tps=14949 mem=6.4G lam=0.2407885016873479 Gm=0.3511878954401861 +[full s1] step 5380/12000 loss=4.5223 gn=0.82 lr=3.81e-04 tps=14950 mem=6.4G lam=0.24116604837278524 Gm=0.3664879105829944 +[full s1] step 5400/12000 loss=4.5505 gn=0.76 lr=3.80e-04 tps=14951 mem=6.4G lam=0.24158635331938663 Gm=0.36109936485687893 +[full s1] step 5420/12000 loss=4.6328 gn=0.74 lr=3.79e-04 tps=14951 mem=6.4G lam=0.24204647106428942 Gm=0.35352831791775924 +[full s1] step 5440/12000 loss=4.7690 gn=0.79 lr=3.77e-04 tps=14952 mem=6.4G lam=0.24247047708680233 Gm=0.36126538272947073 +[full s1] step 5460/12000 loss=4.6578 gn=0.79 lr=3.76e-04 tps=14953 mem=6.4G lam=0.24286914492646852 Gm=0.3565722156781703 +[full s1] step 5480/12000 loss=4.6526 gn=0.77 lr=3.74e-04 tps=14954 mem=6.4G lam=0.24332997233917317 Gm=0.35821183398365974 +[full s1] step 5500/12000 loss=4.6424 gn=0.81 lr=3.73e-04 tps=14954 mem=6.4G lam=0.24383877466122308 Gm=0.3584972876124084 +[full s1] EVAL step 5500: val_loss=4.6700 val_ppl=106.70 +[full s1] step 5520/12000 loss=4.4968 gn=0.76 lr=3.72e-04 tps=14935 mem=6.4G lam=0.24429708377768597 Gm=0.3590108932306369 +[full s1] step 5540/12000 loss=4.6257 gn=0.83 lr=3.70e-04 tps=14936 mem=6.4G lam=0.24473298185815415 Gm=0.3618115121498704 +[full s1] step 5560/12000 loss=4.6501 gn=0.80 lr=3.69e-04 tps=14936 mem=6.4G lam=0.24521021420756975 Gm=0.3657219629579534 +[full s1] step 5580/12000 loss=4.4843 gn=0.77 lr=3.67e-04 tps=14937 mem=6.4G lam=0.24560947405795255 Gm=0.3584932784239451 +[full s1] step 5600/12000 loss=4.6933 gn=0.84 lr=3.66e-04 tps=14938 mem=6.4G lam=0.24600102876623473 Gm=0.35319007793441415 +[full s1] step 5620/12000 loss=4.6973 gn=0.79 lr=3.64e-04 tps=14939 mem=6.4G lam=0.24634672546138367 Gm=0.35618873957234126 +[full s1] step 5640/12000 loss=4.6290 gn=0.76 lr=3.63e-04 tps=14939 mem=6.4G lam=0.2467571816717585 Gm=0.35739188532655436 +[full s1] step 5660/12000 loss=4.7465 gn=0.82 lr=3.62e-04 tps=14940 mem=6.4G lam=0.24718666914850473 Gm=0.3566814415777723 +[full s1] step 5680/12000 loss=4.7806 gn=0.83 lr=3.60e-04 tps=14941 mem=6.4G lam=0.24764590648313364 Gm=0.3571089179410289 +[full s1] step 5700/12000 loss=4.5906 gn=0.79 lr=3.59e-04 tps=14942 mem=6.4G lam=0.2480700397863984 Gm=0.352999451259772 +[full s1] step 5720/12000 loss=4.4877 gn=0.82 lr=3.57e-04 tps=14943 mem=6.4G lam=0.248445271824797 Gm=0.35537890042178333 +[full s1] step 5740/12000 loss=4.6142 gn=0.78 lr=3.56e-04 tps=14943 mem=6.4G lam=0.24892320235570273 Gm=0.35398017001959187 +[full s1] step 5760/12000 loss=4.6783 gn=0.76 lr=3.54e-04 tps=14944 mem=6.4G lam=0.2492934986948967 Gm=0.3527222261764109 +[full s1] step 5780/12000 loss=4.6088 gn=0.80 lr=3.53e-04 tps=14945 mem=6.4G lam=0.24963949931164583 Gm=0.35647912230342627 +[full s1] step 5800/12000 loss=4.5868 gn=0.81 lr=3.52e-04 tps=14945 mem=6.4G lam=0.2500800058866541 Gm=0.3555862130597234 +[full s1] step 5820/12000 loss=4.5615 gn=0.76 lr=3.50e-04 tps=14946 mem=6.4G lam=0.25037563281754655 Gm=0.35599654835338396 +[full s1] step 5840/12000 loss=4.5559 gn=0.81 lr=3.49e-04 tps=14946 mem=6.4G lam=0.250719763028125 Gm=0.354915084472547 +[full s1] step 5860/12000 loss=4.5479 gn=0.76 lr=3.47e-04 tps=14947 mem=6.4G lam=0.2510459451004863 Gm=0.35425161422851187 +[full s1] step 5880/12000 loss=4.5322 gn=0.77 lr=3.46e-04 tps=14947 mem=6.4G lam=0.25139578897506 Gm=0.35190622399871546 +[full s1] step 5900/12000 loss=4.4984 gn=0.75 lr=3.44e-04 tps=14948 mem=6.4G lam=0.25180616540213424 Gm=0.35850760193231207 +[full s1] step 5920/12000 loss=4.6180 gn=0.79 lr=3.43e-04 tps=14948 mem=6.4G lam=0.2521887455756466 Gm=0.3557229045157631 +[full s1] step 5940/12000 loss=4.5917 gn=0.83 lr=3.41e-04 tps=14949 mem=6.4G lam=0.25253822685529786 Gm=0.35688715195283294 +[full s1] step 5960/12000 loss=4.7372 gn=0.83 lr=3.40e-04 tps=14950 mem=6.4G lam=0.2529512144004305 Gm=0.3608042790244023 +[full s1] step 5980/12000 loss=4.5293 gn=0.77 lr=3.39e-04 tps=14950 mem=6.4G lam=0.25331244245171547 Gm=0.34914714004844427 +[full s1] step 6000/12000 loss=4.5567 gn=0.84 lr=3.37e-04 tps=14951 mem=6.4G lam=0.2536272695288062 Gm=0.349960605148226 +[full s1] EVAL step 6000: val_loss=4.6830 val_ppl=108.10 +[full s1] step 6020/12000 loss=4.6428 gn=0.83 lr=3.36e-04 tps=14933 mem=6.4G lam=0.2539903636400898 Gm=0.35764261754229665 +[full s1] step 6040/12000 loss=4.5083 gn=0.81 lr=3.34e-04 tps=14934 mem=6.4G lam=0.2544513779381911 Gm=0.3578381221741438 +[full s1] step 6060/12000 loss=4.4339 gn=0.75 lr=3.33e-04 tps=14935 mem=6.4G lam=0.2547763880963127 Gm=0.34918899554759264 +[full s1] step 6080/12000 loss=4.4353 gn=0.81 lr=3.31e-04 tps=14935 mem=6.4G lam=0.25517619370172423 Gm=0.35176146410716075 +[full s1] step 6100/12000 loss=4.4557 gn=0.77 lr=3.30e-04 tps=14936 mem=6.4G lam=0.2555143569285671 Gm=0.352242373318101 +[full s1] step 6120/12000 loss=4.5537 gn=0.78 lr=3.29e-04 tps=14936 mem=6.4G lam=0.25584579569598037 Gm=0.3511818471985559 +[full s1] step 6140/12000 loss=4.4793 gn=0.78 lr=3.27e-04 tps=14936 mem=6.4G lam=0.256162135861814 Gm=0.3465487485130628 +[full s1] step 6160/12000 loss=4.4885 gn=0.83 lr=3.26e-04 tps=14937 mem=6.4G lam=0.2565131727606058 Gm=0.35290113432953757 +[full s1] step 6180/12000 loss=4.6240 gn=0.89 lr=3.24e-04 tps=14938 mem=6.4G lam=0.2568654340381424 Gm=0.3530101820360869 +[full s1] step 6200/12000 loss=4.5488 gn=0.80 lr=3.23e-04 tps=14938 mem=6.4G lam=0.2571197769915064 Gm=0.3543983727383117 +[full s1] step 6220/12000 loss=4.5817 gn=0.82 lr=3.21e-04 tps=14939 mem=6.4G lam=0.25745477589468163 Gm=0.3473441960910956 +[full s1] step 6240/12000 loss=4.5175 gn=0.83 lr=3.20e-04 tps=14939 mem=6.4G lam=0.2578473376731078 Gm=0.35217455339928466 +[full s1] step 6260/12000 loss=4.5420 gn=0.81 lr=3.19e-04 tps=14939 mem=6.4G lam=0.25821321550756693 Gm=0.35635251831263304 +[full s1] step 6280/12000 loss=4.6135 gn=0.82 lr=3.17e-04 tps=14940 mem=6.4G lam=0.25856153449664515 Gm=0.3508106314887603 +[full s1] step 6300/12000 loss=4.4767 gn=0.82 lr=3.16e-04 tps=14940 mem=6.4G lam=0.25889758486300707 Gm=0.3486305832241972 +[full s1] step 6320/12000 loss=4.6251 gn=0.83 lr=3.14e-04 tps=14941 mem=6.4G lam=0.25918776510904235 Gm=0.34927801275625825 +[full s1] step 6340/12000 loss=4.4562 gn=0.82 lr=3.13e-04 tps=14942 mem=6.4G lam=0.2595130726695061 Gm=0.3484239195628713 +[full s1] step 6360/12000 loss=4.5474 gn=0.85 lr=3.11e-04 tps=14942 mem=6.4G lam=0.25988756337513524 Gm=0.35815933633906144 +[full s1] step 6380/12000 loss=4.5841 gn=0.86 lr=3.10e-04 tps=14943 mem=6.4G lam=0.26029288582503796 Gm=0.35128619133805233 +[full s1] step 6400/12000 loss=4.5216 gn=0.84 lr=3.08e-04 tps=14943 mem=6.4G lam=0.260696267709136 Gm=0.35674855252727866 +[full s1] step 6420/12000 loss=4.5732 gn=0.84 lr=3.07e-04 tps=14944 mem=6.4G lam=0.26106796599924564 Gm=0.3527419176728775 +[full s1] step 6440/12000 loss=4.5421 gn=0.80 lr=3.06e-04 tps=14945 mem=6.4G lam=0.2614184453462561 Gm=0.35877512434187037 +[full s1] step 6460/12000 loss=4.6197 gn=0.83 lr=3.04e-04 tps=14945 mem=6.4G lam=0.26175570612152416 Gm=0.35162812906006974 +[full s1] step 6480/12000 loss=4.4143 gn=0.81 lr=3.03e-04 tps=14946 mem=6.4G lam=0.2621135550240676 Gm=0.347916381744047 +[full s1] step 6500/12000 loss=4.5035 gn=0.81 lr=3.01e-04 tps=14946 mem=6.4G lam=0.26240216195583344 Gm=0.3524323896660159 +[full s1] EVAL step 6500: val_loss=4.6314 val_ppl=102.66 +[full s1] step 6520/12000 loss=4.4807 gn=0.76 lr=3.00e-04 tps=14929 mem=6.4G lam=0.2627305034548044 Gm=0.3442771936922024 +[full s1] step 6540/12000 loss=4.6244 gn=0.82 lr=2.98e-04 tps=14930 mem=6.4G lam=0.2630973511065046 Gm=0.3450061803838859 +[full s1] step 6560/12000 loss=4.5799 gn=0.83 lr=2.97e-04 tps=14930 mem=6.4G lam=0.26348271903892356 Gm=0.35383700113743544 +[full s1] step 6580/12000 loss=4.5218 gn=0.78 lr=2.96e-04 tps=14931 mem=6.4G lam=0.26382218146075803 Gm=0.34668865248871344 +[full s1] step 6600/12000 loss=4.3273 gn=0.77 lr=2.94e-04 tps=14932 mem=6.4G lam=0.264100993052125 Gm=0.3442901980597526 +[full s1] step 6620/12000 loss=4.7469 gn=0.89 lr=2.93e-04 tps=14932 mem=6.4G lam=0.26442968907455605 Gm=0.345340166396151 +[full s1] step 6640/12000 loss=4.5376 gn=0.84 lr=2.91e-04 tps=14933 mem=6.4G lam=0.26477554192145664 Gm=0.3517673848352085 +[full s1] step 6660/12000 loss=4.4978 gn=0.83 lr=2.90e-04 tps=14933 mem=6.4G lam=0.26517605905731517 Gm=0.35189354776715237 +[full s1] step 6680/12000 loss=4.4925 gn=0.77 lr=2.88e-04 tps=14933 mem=6.4G lam=0.2654828081528346 Gm=0.3473820990572373 +[full s1] step 6700/12000 loss=4.4991 gn=0.87 lr=2.87e-04 tps=14934 mem=6.4G lam=0.2658550764123599 Gm=0.345112725161016 +[full s1] step 6720/12000 loss=4.5188 gn=0.85 lr=2.86e-04 tps=14934 mem=6.4G lam=0.2661411914353569 Gm=0.3507055505178869 +[full s1] step 6740/12000 loss=4.4881 gn=0.83 lr=2.84e-04 tps=14935 mem=6.4G lam=0.2664645789191127 Gm=0.3466409300453961 +[full s1] step 6760/12000 loss=4.5663 gn=0.80 lr=2.83e-04 tps=14936 mem=6.4G lam=0.2667553663874666 Gm=0.35291034712766606 +[full s1] step 6780/12000 loss=4.5480 gn=0.80 lr=2.81e-04 tps=14936 mem=6.4G lam=0.2670265457903345 Gm=0.34344490244984627 +[full s1] step 6800/12000 loss=4.4953 gn=0.90 lr=2.80e-04 tps=14937 mem=6.4G lam=0.26728153973817825 Gm=0.3486279157611231 +[full s1] step 6820/12000 loss=4.6706 gn=0.84 lr=2.79e-04 tps=14938 mem=6.4G lam=0.2676221566895644 Gm=0.34811893198639154 +[full s1] step 6840/12000 loss=4.5650 gn=0.85 lr=2.77e-04 tps=14939 mem=6.4G lam=0.2679725084453821 Gm=0.3540763700536142 +[full s1] step 6860/12000 loss=4.4599 gn=0.89 lr=2.76e-04 tps=14939 mem=6.4G lam=0.2682387555638949 Gm=0.34175121209894616 +[full s1] step 6880/12000 loss=4.4988 gn=0.85 lr=2.74e-04 tps=14940 mem=6.4G lam=0.26855796047796804 Gm=0.34704178432002664 +[full s1] step 6900/12000 loss=4.4817 gn=0.89 lr=2.73e-04 tps=14941 mem=6.4G lam=0.268855062002937 Gm=0.3510745828971267 +[full s1] step 6920/12000 loss=4.4361 gn=0.85 lr=2.72e-04 tps=14941 mem=6.4G lam=0.2692021771023671 Gm=0.3504328644679238 +[full s1] step 6940/12000 loss=4.4315 gn=0.84 lr=2.70e-04 tps=14942 mem=6.4G lam=0.2695480001469453 Gm=0.3522706561877082 +[full s1] step 6960/12000 loss=4.4889 gn=0.82 lr=2.69e-04 tps=14942 mem=6.4G lam=0.26979978444675606 Gm=0.34353846684098244 +[full s1] step 6980/12000 loss=4.3626 gn=0.87 lr=2.67e-04 tps=14943 mem=6.4G lam=0.27012288104742765 Gm=0.3484779993693034 +[full s1] step 7000/12000 loss=4.3533 gn=0.84 lr=2.66e-04 tps=14944 mem=6.4G lam=0.2704320962851246 Gm=0.3413316130948563 +[full s1] EVAL step 7000: val_loss=4.5675 val_ppl=96.30 +[full s1] step 7020/12000 loss=4.5250 gn=0.81 lr=2.65e-04 tps=14929 mem=6.4G lam=0.27072847727686167 Gm=0.35441024011621874 +[full s1] step 7040/12000 loss=4.5956 gn=0.83 lr=2.63e-04 tps=14929 mem=6.4G lam=0.27101048609862727 Gm=0.35044005342448753 +[full s1] step 7060/12000 loss=4.5223 gn=0.85 lr=2.62e-04 tps=14930 mem=6.4G lam=0.27134962659329176 Gm=0.35434069111943245 +[full s1] step 7080/12000 loss=4.4169 gn=0.80 lr=2.60e-04 tps=14930 mem=6.4G lam=0.2716083771859606 Gm=0.3480169920561214 +[full s1] step 7100/12000 loss=4.5593 gn=0.87 lr=2.59e-04 tps=14931 mem=6.4G lam=0.2718857803071539 Gm=0.3462352875309686 +[full s1] step 7120/12000 loss=4.4615 gn=0.82 lr=2.58e-04 tps=14931 mem=6.4G lam=0.2721764150386055 Gm=0.34436867603411275 +[full s1] step 7140/12000 loss=4.3958 gn=0.85 lr=2.56e-04 tps=14932 mem=6.4G lam=0.2724112216383219 Gm=0.3394071236252785 +[full s1] step 7160/12000 loss=4.3688 gn=0.82 lr=2.55e-04 tps=14932 mem=6.4G lam=0.2726918300613761 Gm=0.3426256822422147 +[full s1] step 7180/12000 loss=4.5763 gn=0.83 lr=2.53e-04 tps=14932 mem=6.4G lam=0.2729436152925094 Gm=0.3485044447394709 +[full s1] step 7200/12000 loss=4.4564 gn=0.83 lr=2.52e-04 tps=14932 mem=6.4G lam=0.2732290243729949 Gm=0.3466080612658213 +[full s1] step 7220/12000 loss=4.5503 gn=0.94 lr=2.51e-04 tps=14930 mem=6.4G lam=0.27355622965842485 Gm=0.3470041834904502 +[full s1] step 7240/12000 loss=4.4654 gn=0.85 lr=2.49e-04 tps=14929 mem=6.4G lam=0.27387895838667947 Gm=0.3466790091091146 +[full s1] step 7260/12000 loss=4.5085 gn=0.85 lr=2.48e-04 tps=14928 mem=6.4G lam=0.27417855244129896 Gm=0.3487608864282568 +[full s1] step 7280/12000 loss=4.5986 gn=0.82 lr=2.47e-04 tps=14928 mem=6.4G lam=0.27446882830311853 Gm=0.34791149649148184 +[full s1] step 7300/12000 loss=4.4058 gn=0.83 lr=2.45e-04 tps=14926 mem=6.4G lam=0.2747661418591936 Gm=0.34430325240828097 +[full s1] step 7320/12000 loss=4.4706 gn=0.78 lr=2.44e-04 tps=14924 mem=6.4G lam=0.2750614869097869 Gm=0.3505839213418464 +[full s1] step 7340/12000 loss=4.4927 gn=0.83 lr=2.42e-04 tps=14923 mem=6.4G lam=0.27532961933563155 Gm=0.34687839603672427 +[full s1] step 7360/12000 loss=4.5015 gn=0.82 lr=2.41e-04 tps=14923 mem=6.4G lam=0.27558561973273754 Gm=0.34004239753509563 +[full s1] step 7380/12000 loss=4.3470 gn=0.87 lr=2.40e-04 tps=14923 mem=6.4G lam=0.2758992662032445 Gm=0.34796322501885396 +[full s1] step 7400/12000 loss=4.3679 gn=0.85 lr=2.38e-04 tps=14923 mem=6.4G lam=0.2761421278119087 Gm=0.338308632063369 +[full s1] step 7420/12000 loss=4.5362 gn=0.87 lr=2.37e-04 tps=14924 mem=6.4G lam=0.2764232136930029 Gm=0.34449541723976534 +[full s1] step 7440/12000 loss=4.5642 gn=0.90 lr=2.36e-04 tps=14924 mem=6.4G lam=0.27672313526272774 Gm=0.35585038394977647 +[full s1] step 7460/12000 loss=4.4377 gn=0.87 lr=2.34e-04 tps=14925 mem=6.4G lam=0.2770028092587988 Gm=0.34772513279070455 +[full s1] step 7480/12000 loss=4.6787 gn=0.90 lr=2.33e-04 tps=14926 mem=6.4G lam=0.2772649625937144 Gm=0.3457111962294827 +[full s1] step 7500/12000 loss=4.3044 gn=0.81 lr=2.32e-04 tps=14926 mem=6.4G lam=0.2775505604222417 Gm=0.33973450027406216 +[full s1] EVAL step 7500: val_loss=4.5447 val_ppl=94.13 +[full s1] step 7520/12000 loss=4.5353 gn=0.88 lr=2.30e-04 tps=14912 mem=6.4G lam=0.27782221293697756 Gm=0.3388730324804783 +[full s1] step 7540/12000 loss=4.3972 gn=0.84 lr=2.29e-04 tps=14913 mem=6.4G lam=0.27811516014238197 Gm=0.33819003216922283 +[full s1] step 7560/12000 loss=4.4230 gn=0.91 lr=2.28e-04 tps=14913 mem=6.4G lam=0.2783182244747877 Gm=0.35280541811759275 +[full s1] step 7580/12000 loss=4.3663 gn=0.82 lr=2.26e-04 tps=14914 mem=6.4G lam=0.27858708581576747 Gm=0.34702396268645924 +[full s1] step 7600/12000 loss=4.4039 gn=0.83 lr=2.25e-04 tps=14915 mem=6.4G lam=0.2788164659092824 Gm=0.34671472494180006 +[full s1] step 7620/12000 loss=4.3879 gn=0.86 lr=2.24e-04 tps=14915 mem=6.4G lam=0.2790191853418946 Gm=0.3446747342435022 +[full s1] step 7640/12000 loss=4.4111 gn=0.83 lr=2.22e-04 tps=14916 mem=6.4G lam=0.27925100705275935 Gm=0.3431756311717133 +[full s1] step 7660/12000 loss=4.4908 gn=0.88 lr=2.21e-04 tps=14917 mem=6.4G lam=0.27953738532960415 Gm=0.3548006162357827 +[full s1] step 7680/12000 loss=4.3667 gn=0.91 lr=2.20e-04 tps=14918 mem=6.4G lam=0.27974383036295575 Gm=0.35047381355737645 +[full s1] step 7700/12000 loss=4.4932 gn=0.91 lr=2.18e-04 tps=14918 mem=6.4G lam=0.2799709451695283 Gm=0.3415645866965254 +[full s1] step 7720/12000 loss=4.5219 gn=0.91 lr=2.17e-04 tps=14919 mem=6.4G lam=0.2801798991858959 Gm=0.35005578383182484 +[full s1] step 7740/12000 loss=4.3572 gn=0.85 lr=2.16e-04 tps=14920 mem=6.4G lam=0.28042268225302297 Gm=0.33441605341310304 +[full s1] step 7760/12000 loss=4.5671 gn=0.93 lr=2.15e-04 tps=14920 mem=6.4G lam=0.28063109548141557 Gm=0.34774880995973945 +[full s1] step 7780/12000 loss=4.4349 gn=0.87 lr=2.13e-04 tps=14921 mem=6.4G lam=0.2809249771138032 Gm=0.34316958324052393 +[full s1] step 7800/12000 loss=4.4773 gn=0.91 lr=2.12e-04 tps=14922 mem=6.4G lam=0.28111171691368025 Gm=0.35026451852172613 +[full s1] step 7820/12000 loss=4.3583 gn=0.87 lr=2.11e-04 tps=14922 mem=6.4G lam=0.28134031873196363 Gm=0.34219653873393935 +[full s1] step 7840/12000 loss=4.5019 gn=0.89 lr=2.09e-04 tps=14923 mem=6.4G lam=0.2815718005100886 Gm=0.34490649371097487 +[full s1] step 7860/12000 loss=4.4308 gn=0.95 lr=2.08e-04 tps=14924 mem=6.4G lam=0.28179345310976106 Gm=0.3450868207340439 +[full s1] step 7880/12000 loss=4.3853 gn=0.91 lr=2.07e-04 tps=14924 mem=6.4G lam=0.28200966740647954 Gm=0.34551286697387695 +[full s1] step 7900/12000 loss=4.5589 gn=0.90 lr=2.06e-04 tps=14925 mem=6.4G lam=0.2822326800475518 Gm=0.34515832302471 +[full s1] step 7920/12000 loss=4.4201 gn=0.94 lr=2.04e-04 tps=14926 mem=6.4G lam=0.2824596722299854 Gm=0.33799145463854074 +[full s1] step 7940/12000 loss=4.4698 gn=0.89 lr=2.03e-04 tps=14927 mem=6.4G lam=0.28266944953550893 Gm=0.3475161101669073 +[full s1] step 7960/12000 loss=4.5404 gn=0.88 lr=2.02e-04 tps=14927 mem=6.4G lam=0.2829163148999214 Gm=0.3514249010477215 +[full s1] step 7980/12000 loss=4.2521 gn=0.97 lr=2.00e-04 tps=14928 mem=6.4G lam=0.28313401776055497 Gm=0.34654874727129936 +[full s1] step 8000/12000 loss=4.3487 gn=0.89 lr=1.99e-04 tps=14929 mem=6.4G lam=0.28336625173687935 Gm=0.34060760742674273 +[full s1] EVAL step 8000: val_loss=4.5359 val_ppl=93.30 +[full s1] step 8020/12000 loss=4.6194 gn=0.88 lr=1.98e-04 tps=14916 mem=6.4G lam=0.28356551968803007 Gm=0.33989962299043935 +[full s1] step 8040/12000 loss=4.4237 gn=0.88 lr=1.97e-04 tps=14916 mem=6.4G lam=0.28381693239013356 Gm=0.33963298161203664 +[full s1] step 8060/12000 loss=4.3031 gn=0.95 lr=1.95e-04 tps=14917 mem=6.4G lam=0.2840121618161599 Gm=0.34052017827828723 +[full s1] step 8080/12000 loss=4.3901 gn=0.86 lr=1.94e-04 tps=14917 mem=6.4G lam=0.2841923947756489 Gm=0.34592763311229646 +[full s1] step 8100/12000 loss=4.3874 gn=0.90 lr=1.93e-04 tps=14918 mem=6.4G lam=0.2843751922870676 Gm=0.3466095264690618 +[full s1] step 8120/12000 loss=4.4470 gn=0.82 lr=1.92e-04 tps=14919 mem=6.4G lam=0.2845909384389718 Gm=0.3473951699367414 +[full s1] step 8140/12000 loss=4.4008 gn=0.90 lr=1.90e-04 tps=14920 mem=6.4G lam=0.28481190496434766 Gm=0.34228154163186747 +[full s1] step 8160/12000 loss=4.4916 gn=0.93 lr=1.89e-04 tps=14921 mem=6.4G lam=0.28499374849100906 Gm=0.34821502996298176 +[full s1] step 8180/12000 loss=4.4430 gn=0.86 lr=1.88e-04 tps=14922 mem=6.4G lam=0.2852474596972267 Gm=0.3383662384779503 +[full s1] step 8200/12000 loss=4.4935 gn=0.95 lr=1.87e-04 tps=14922 mem=6.4G lam=0.28545837849378586 Gm=0.33968524479617673 +[full s1] step 8220/12000 loss=4.3733 gn=0.85 lr=1.86e-04 tps=14923 mem=6.4G lam=0.2856840845197439 Gm=0.33713511082654196 +[full s1] step 8240/12000 loss=4.4306 gn=0.91 lr=1.84e-04 tps=14924 mem=6.4G lam=0.2859370264535149 Gm=0.34765029783981544 +[full s1] step 8260/12000 loss=4.3420 gn=0.89 lr=1.83e-04 tps=14924 mem=6.4G lam=0.2861492410302162 Gm=0.33553352137096226 +[full s1] step 8280/12000 loss=4.4152 gn=0.88 lr=1.82e-04 tps=14925 mem=6.4G lam=0.2863436782111724 Gm=0.34241331182420254 +[full s1] step 8300/12000 loss=4.5829 gn=0.95 lr=1.81e-04 tps=14925 mem=6.4G lam=0.28654965820411843 Gm=0.34562258422374725 +[full s1] step 8320/12000 loss=4.2256 gn=0.87 lr=1.80e-04 tps=14926 mem=6.4G lam=0.2867387927447756 Gm=0.34407483918281895 +[full s1] step 8340/12000 loss=4.4181 gn=0.90 lr=1.78e-04 tps=14927 mem=6.4G lam=0.28688709406803053 Gm=0.3451058574331303 +[full s1] step 8360/12000 loss=4.4040 gn=0.94 lr=1.77e-04 tps=14927 mem=6.4G lam=0.2870911819239457 Gm=0.3455642448582997 +[full s1] step 8380/12000 loss=4.3678 gn=0.83 lr=1.76e-04 tps=14928 mem=6.4G lam=0.2873180058474342 Gm=0.3413767210052659 +[full s1] step 8400/12000 loss=4.5617 gn=0.87 lr=1.75e-04 tps=14928 mem=6.4G lam=0.28746493129680556 Gm=0.34726010480274755 +[full s1] step 8420/12000 loss=4.3705 gn=0.86 lr=1.74e-04 tps=14929 mem=6.4G lam=0.2876405840118726 Gm=0.33901106705889106 +[full s1] step 8440/12000 loss=4.3640 gn=0.86 lr=1.72e-04 tps=14930 mem=6.4G lam=0.28781865754475194 Gm=0.33993038449746865 +[full s1] step 8460/12000 loss=4.4044 gn=0.85 lr=1.71e-04 tps=14930 mem=6.4G lam=0.2880376462514202 Gm=0.33927446107069653 +[full s1] step 8480/12000 loss=4.5315 gn=0.94 lr=1.70e-04 tps=14931 mem=6.4G lam=0.2882019927104314 Gm=0.3433768747684856 +[full s1] step 8500/12000 loss=4.4813 gn=0.92 lr=1.69e-04 tps=14932 mem=6.4G lam=0.2884171347444256 Gm=0.3420118213010331 +[full s1] EVAL step 8500: val_loss=4.4849 val_ppl=88.67 +[full s1] step 8520/12000 loss=4.4112 gn=0.91 lr=1.68e-04 tps=14919 mem=6.4G lam=0.28863336611539125 Gm=0.34064470945547026 +[full s1] step 8540/12000 loss=4.3787 gn=0.87 lr=1.67e-04 tps=14920 mem=6.4G lam=0.2887981891011198 Gm=0.33992780186235905 +[full s1] step 8560/12000 loss=4.3594 gn=0.94 lr=1.66e-04 tps=14920 mem=6.4G lam=0.28894921640555066 Gm=0.34065695805475116 +[full s1] step 8580/12000 loss=4.4981 gn=0.91 lr=1.64e-04 tps=14921 mem=6.4G lam=0.2891603596508503 Gm=0.34208472034273046 +[full s1] step 8600/12000 loss=4.5444 gn=0.94 lr=1.63e-04 tps=14922 mem=6.4G lam=0.2894141674041748 Gm=0.34263875102624297 +[full s1] step 8620/12000 loss=4.5039 gn=0.90 lr=1.62e-04 tps=14922 mem=6.4G lam=0.2896295487880707 Gm=0.3410064401881148 +[full s1] step 8640/12000 loss=4.3977 gn=0.90 lr=1.61e-04 tps=14923 mem=6.4G lam=0.28979032083104056 Gm=0.3374485025803248 +[full s1] step 8660/12000 loss=4.4353 gn=0.89 lr=1.60e-04 tps=14923 mem=6.4G lam=0.2899776327734192 Gm=0.3356814038318892 +[full s1] step 8680/12000 loss=4.3705 gn=0.93 lr=1.59e-04 tps=14924 mem=6.4G lam=0.2901350660249591 Gm=0.33909701618055504 +[full s1] step 8700/12000 loss=4.3455 gn=0.95 lr=1.58e-04 tps=14924 mem=6.4G lam=0.2903583204994599 Gm=0.3397424171368281 +[full s1] step 8720/12000 loss=4.4348 gn=0.89 lr=1.57e-04 tps=14925 mem=6.4G lam=0.2905528728539745 Gm=0.34344047160508734 +[full s1] step 8740/12000 loss=4.5670 gn=0.90 lr=1.55e-04 tps=14926 mem=6.4G lam=0.29072649621715146 Gm=0.34288869503264624 +[full s1] step 8760/12000 loss=4.3854 gn=0.92 lr=1.54e-04 tps=14926 mem=6.4G lam=0.2908831536769867 Gm=0.34687677848463255 +[full s1] step 8780/12000 loss=4.1509 gn=0.90 lr=1.53e-04 tps=14927 mem=6.4G lam=0.2910599196329713 Gm=0.3337502205589165 +[full s1] step 8800/12000 loss=4.4159 gn=0.92 lr=1.52e-04 tps=14928 mem=6.4G lam=0.2912237076088786 Gm=0.3469098535521577 +[full s1] step 8820/12000 loss=4.3894 gn=0.92 lr=1.51e-04 tps=14928 mem=6.4G lam=0.29135065836211044 Gm=0.3413136218053599 +[full s1] step 8840/12000 loss=4.2369 gn=0.92 lr=1.50e-04 tps=14929 mem=6.4G lam=0.2915183479587237 Gm=0.34311963253033656 +[full s1] step 8860/12000 loss=4.3793 gn=0.92 lr=1.49e-04 tps=14929 mem=6.4G lam=0.291716108409067 Gm=0.33896452421322465 +[full s1] step 8880/12000 loss=4.5032 gn=0.93 lr=1.48e-04 tps=14930 mem=6.4G lam=0.29186240521570045 Gm=0.34011702743979794 +[full s1] step 8900/12000 loss=4.5487 gn=0.94 lr=1.47e-04 tps=14930 mem=6.4G lam=0.29197167977690697 Gm=0.3456936933410664 +[full s1] step 8920/12000 loss=4.3577 gn=0.87 lr=1.46e-04 tps=14931 mem=6.4G lam=0.29215598354736966 Gm=0.3427434970314304 +[full s1] step 8940/12000 loss=4.3950 gn=0.91 lr=1.45e-04 tps=14931 mem=6.4G lam=0.29236572204778594 Gm=0.343122077640146 +[full s1] step 8960/12000 loss=4.4019 gn=0.91 lr=1.44e-04 tps=14932 mem=6.4G lam=0.29251775424927473 Gm=0.34454215434379876 +[full s1] step 8980/12000 loss=4.4295 gn=0.89 lr=1.43e-04 tps=14933 mem=6.4G lam=0.29265144436309737 Gm=0.33634375664405525 +[full s1] step 9000/12000 loss=4.4158 gn=0.90 lr=1.42e-04 tps=14933 mem=6.4G lam=0.29282454525430995 Gm=0.33981903999422985 +[full s1] EVAL step 9000: val_loss=4.4577 val_ppl=86.29 +[full s1] step 9020/12000 loss=4.4485 gn=0.90 lr=1.41e-04 tps=14922 mem=6.4G lam=0.29298380482941866 Gm=0.3389382710835586 +[full s1] step 9040/12000 loss=4.4121 gn=0.95 lr=1.40e-04 tps=14922 mem=6.4G lam=0.29313348854581517 Gm=0.3328123997586469 +[full s1] step 9060/12000 loss=4.4505 gn=0.90 lr=1.39e-04 tps=14923 mem=6.4G lam=0.2933033686131239 Gm=0.3366988787117104 +[full s1] step 9080/12000 loss=4.4300 gn=0.92 lr=1.38e-04 tps=14923 mem=6.4G lam=0.29349551318834227 Gm=0.34200999955646694 +[full s1] step 9100/12000 loss=4.3491 gn=0.96 lr=1.37e-04 tps=14924 mem=6.4G lam=0.293619377228121 Gm=0.34261049500977 +[full s1] step 9120/12000 loss=4.2266 gn=0.89 lr=1.36e-04 tps=14925 mem=6.4G lam=0.29376209154725075 Gm=0.3361261432679991 +[full s1] step 9140/12000 loss=4.4795 gn=0.91 lr=1.35e-04 tps=14925 mem=6.4G lam=0.29394931967059773 Gm=0.34007107249150675 +[full s1] step 9160/12000 loss=4.5415 gn=0.86 lr=1.34e-04 tps=14926 mem=6.4G lam=0.2941182792807619 Gm=0.3459863947549214 +[full s1] step 9180/12000 loss=4.3405 gn=0.95 lr=1.33e-04 tps=14926 mem=6.4G lam=0.29430665355175734 Gm=0.3391829081811011 +[full s1] step 9200/12000 loss=4.4593 gn=0.94 lr=1.32e-04 tps=14927 mem=6.4G lam=0.29446953255683184 Gm=0.3425008775666356 +[full s1] step 9220/12000 loss=4.3569 gn=0.98 lr=1.31e-04 tps=14928 mem=6.4G lam=0.2946350971857707 Gm=0.340011556710427 +[full s1] step 9240/12000 loss=4.3734 gn=0.91 lr=1.30e-04 tps=14928 mem=6.4G lam=0.2947903222714861 Gm=0.3398952594337364 +[full s1] step 9260/12000 loss=4.4604 gn=0.94 lr=1.29e-04 tps=14929 mem=6.4G lam=0.29491288866847754 Gm=0.3459074574833115 +[full s1] step 9280/12000 loss=4.4536 gn=0.90 lr=1.28e-04 tps=14929 mem=6.4G lam=0.29505225891868275 Gm=0.33944013466437656 +[full s1] step 9300/12000 loss=4.5365 gn=1.05 lr=1.27e-04 tps=14930 mem=6.4G lam=0.2951832925900817 Gm=0.34641445335000753 +[full s1] step 9320/12000 loss=4.2866 gn=0.90 lr=1.26e-04 tps=14930 mem=6.4G lam=0.29534027073532343 Gm=0.33782087328533333 +[full s1] step 9340/12000 loss=4.2800 gn=0.97 lr=1.25e-04 tps=14931 mem=6.4G lam=0.2955094895636042 Gm=0.33738527726382017 +[full s1] step 9360/12000 loss=4.4683 gn=0.93 lr=1.24e-04 tps=14931 mem=6.4G lam=0.2956567853689194 Gm=0.34047717197487753 +[full s1] step 9380/12000 loss=4.3898 gn=0.98 lr=1.23e-04 tps=14932 mem=6.4G lam=0.2957918103784323 Gm=0.34681121236644685 +[full s1] step 9400/12000 loss=4.3547 gn=0.96 lr=1.22e-04 tps=14932 mem=6.4G lam=0.2959379243354003 Gm=0.34507056636114913 +[full s1] step 9420/12000 loss=4.2422 gn=0.93 lr=1.21e-04 tps=14933 mem=6.4G lam=0.29608920713265735 Gm=0.34414994926191866 +[full s1] step 9440/12000 loss=4.3252 gn=0.90 lr=1.20e-04 tps=14934 mem=6.4G lam=0.2962161048005025 Gm=0.34234863286837935 +[full s1] step 9460/12000 loss=4.3062 gn=0.92 lr=1.19e-04 tps=14934 mem=6.4G lam=0.29629969876259565 Gm=0.34004356983738643 +[full s1] step 9480/12000 loss=4.3740 gn=0.90 lr=1.19e-04 tps=14935 mem=6.4G lam=0.29643203256030876 Gm=0.34055806308363873 +[full s1] step 9500/12000 loss=4.3638 gn=0.91 lr=1.18e-04 tps=14936 mem=6.4G lam=0.29654729800919694 Gm=0.3484238611999899 +[full s1] EVAL step 9500: val_loss=4.4497 val_ppl=85.60 +[full s1] step 9520/12000 loss=4.4091 gn=0.95 lr=1.17e-04 tps=14924 mem=6.4G lam=0.2967016731078426 Gm=0.34734264404202503 +[full s1] step 9540/12000 loss=4.2859 gn=0.95 lr=1.16e-04 tps=14925 mem=6.4G lam=0.2968842424452305 Gm=0.34081506977478665 +[full s1] step 9560/12000 loss=4.4024 gn=1.00 lr=1.15e-04 tps=14926 mem=6.4G lam=0.2969930050894618 Gm=0.33961647904167575 +[full s1] step 9580/12000 loss=4.4401 gn=0.97 lr=1.14e-04 tps=14926 mem=6.4G lam=0.29711972549557686 Gm=0.3427540115856876 +[full s1] step 9600/12000 loss=4.3353 gn=0.90 lr=1.13e-04 tps=14927 mem=6.4G lam=0.29729169482986134 Gm=0.33877854763219756 +[full s1] step 9620/12000 loss=4.3646 gn=0.97 lr=1.12e-04 tps=14927 mem=6.4G lam=0.29742314169804257 Gm=0.33801450735578936 +[full s1] step 9640/12000 loss=4.2695 gn=0.91 lr=1.12e-04 tps=14928 mem=6.4G lam=0.29755173654605943 Gm=0.3370431348060568 +[full s1] step 9660/12000 loss=4.5212 gn=0.96 lr=1.11e-04 tps=14928 mem=6.4G lam=0.29764372824380797 Gm=0.3354852390475571 +[full s1] step 9680/12000 loss=4.3249 gn=0.89 lr=1.10e-04 tps=14929 mem=6.4G lam=0.2977282904709379 Gm=0.339896086215352 +[full s1] step 9700/12000 loss=4.4092 gn=0.94 lr=1.09e-04 tps=14930 mem=6.4G lam=0.2978308942789833 Gm=0.3441615041034917 +[full s1] step 9720/12000 loss=4.4923 gn=0.96 lr=1.08e-04 tps=14930 mem=6.4G lam=0.2979542603716254 Gm=0.3414705368534972 +[full s1] step 9740/12000 loss=4.2265 gn=0.91 lr=1.07e-04 tps=14931 mem=6.4G lam=0.29808915468553704 Gm=0.3430751849276324 +[full s1] step 9760/12000 loss=4.4587 gn=0.96 lr=1.07e-04 tps=14931 mem=6.4G lam=0.29819946611920994 Gm=0.34178857536365587 +[full s1] step 9780/12000 loss=4.3601 gn=0.95 lr=1.06e-04 tps=14932 mem=6.4G lam=0.2983569260686636 Gm=0.33922408032231033 +[full s1] step 9800/12000 loss=4.4493 gn=0.94 lr=1.05e-04 tps=14932 mem=6.4G lam=0.29845513838032883 Gm=0.3417251609886686 +[full s1] step 9820/12000 loss=4.2771 gn=0.93 lr=1.04e-04 tps=14933 mem=6.4G lam=0.29855115339159966 Gm=0.3345434626874824 +[full s1] step 9840/12000 loss=4.4832 gn=1.07 lr=1.03e-04 tps=14933 mem=6.4G lam=0.2986542194460829 Gm=0.3405516595424463 +[full s1] step 9860/12000 loss=4.5135 gn=1.03 lr=1.03e-04 tps=14934 mem=6.4G lam=0.29873531746367615 Gm=0.34181329421699047 +[full s1] step 9880/12000 loss=4.3429 gn=0.94 lr=1.02e-04 tps=14934 mem=6.4G lam=0.2988433415691058 Gm=0.3430671164145072 +[full s1] step 9900/12000 loss=4.3271 gn=0.90 lr=1.01e-04 tps=14935 mem=6.4G lam=0.29895817550520104 Gm=0.3322411383657406 +[full s1] step 9920/12000 loss=4.3049 gn=0.97 lr=1.00e-04 tps=14935 mem=6.4G lam=0.2990503925830126 Gm=0.3373091338823239 +[full s1] step 9940/12000 loss=4.2659 gn=0.90 lr=9.96e-05 tps=14936 mem=6.4G lam=0.2991308659935991 Gm=0.33665582824808854 +[full s1] step 9960/12000 loss=4.3386 gn=1.05 lr=9.89e-05 tps=14936 mem=6.4G lam=0.29923958486566943 Gm=0.33687299117445946 +[full s1] step 9980/12000 loss=4.3934 gn=0.92 lr=9.81e-05 tps=14937 mem=6.4G lam=0.29932749706010026 Gm=0.3354289364845802 +[full s1] step 10000/12000 loss=4.3248 gn=0.93 lr=9.74e-05 tps=14937 mem=6.4G lam=0.29943383981784183 Gm=0.3334744846603523 +[full s1] EVAL step 10000: val_loss=4.4155 val_ppl=82.72 +[full s1] step 10020/12000 loss=4.6457 gn=1.00 lr=9.67e-05 tps=14927 mem=6.4G lam=0.2995408872763316 Gm=0.34047093490759534 +[full s1] step 10040/12000 loss=4.2386 gn=0.94 lr=9.59e-05 tps=14927 mem=6.4G lam=0.29959512564043206 Gm=0.33106089492018026 +[full s1] step 10060/12000 loss=4.4709 gn=0.95 lr=9.52e-05 tps=14928 mem=6.4G lam=0.29971005860716105 Gm=0.34144469009091455 +[full s1] step 10080/12000 loss=4.2390 gn=0.95 lr=9.45e-05 tps=14928 mem=6.4G lam=0.2998379360263546 Gm=0.3406158823830386 +[full s1] step 10100/12000 loss=4.2826 gn=0.97 lr=9.38e-05 tps=14929 mem=6.4G lam=0.29997461506476003 Gm=0.34258005674928427 +[full s1] step 10120/12000 loss=4.3194 gn=0.93 lr=9.31e-05 tps=14929 mem=6.4G lam=0.30009393704434234 Gm=0.33917338983155787 +[full s1] step 10140/12000 loss=4.2669 gn=0.97 lr=9.24e-05 tps=14930 mem=6.4G lam=0.3001837246119976 Gm=0.3344895443879068 +[full s1] step 10160/12000 loss=4.3698 gn=0.92 lr=9.18e-05 tps=14930 mem=6.4G lam=0.30028996833910543 Gm=0.3406371967867017 +[full s1] step 10180/12000 loss=4.2715 gn=0.89 lr=9.11e-05 tps=14931 mem=6.4G lam=0.30039458597699803 Gm=0.3327654702588916 +[full s1] step 10200/12000 loss=4.4606 gn=1.06 lr=9.04e-05 tps=14931 mem=6.4G lam=0.30047296173870564 Gm=0.3368753941419224 +[full s1] step 10220/12000 loss=4.3747 gn=0.98 lr=8.98e-05 tps=14932 mem=6.4G lam=0.3005556231364608 Gm=0.341253148081402 +[full s1] step 10240/12000 loss=4.3191 gn=0.97 lr=8.91e-05 tps=14932 mem=6.4G lam=0.3006566911935806 Gm=0.33171902170094353 +[full s1] step 10260/12000 loss=4.4345 gn=0.95 lr=8.85e-05 tps=14933 mem=6.4G lam=0.30073328129947186 Gm=0.3398340597438316 +[full s1] step 10280/12000 loss=4.3463 gn=0.94 lr=8.78e-05 tps=14933 mem=6.4G lam=0.30081363767385483 Gm=0.33307005232200027 +[full s1] step 10300/12000 loss=4.2536 gn=0.97 lr=8.72e-05 tps=14934 mem=6.4G lam=0.30089299753308296 Gm=0.33023961385091144 +[full s1] step 10320/12000 loss=4.2933 gn=0.95 lr=8.66e-05 tps=14934 mem=6.4G lam=0.3009818975503246 Gm=0.34072927323480445 +[full s1] step 10340/12000 loss=4.4629 gn=1.06 lr=8.59e-05 tps=14935 mem=6.4G lam=0.30108450135837 Gm=0.33758410019800067 +[full s1] step 10360/12000 loss=4.3415 gn=0.93 lr=8.53e-05 tps=14935 mem=6.4G lam=0.3011891804635525 Gm=0.33347883688596386 +[full s1] step 10380/12000 loss=4.3555 gn=0.98 lr=8.47e-05 tps=14936 mem=6.4G lam=0.30129477630058926 Gm=0.34035456118484336 +[full s1] step 10400/12000 loss=4.4628 gn=1.07 lr=8.41e-05 tps=14936 mem=6.4G lam=0.30138971470296383 Gm=0.339464097439001 +[full s1] step 10420/12000 loss=4.3992 gn=1.03 lr=8.35e-05 tps=14937 mem=6.4G lam=0.3014794485643506 Gm=0.34288667057019967 +[full s1] step 10440/12000 loss=4.4234 gn=1.00 lr=8.30e-05 tps=14937 mem=6.4G lam=0.3015691541756193 Gm=0.33816341077908874 +[full s1] step 10460/12000 loss=4.2463 gn=0.94 lr=8.24e-05 tps=14938 mem=6.4G lam=0.3016777755692601 Gm=0.3383414135314524 +[full s1] step 10480/12000 loss=4.3354 gn=0.96 lr=8.18e-05 tps=14938 mem=6.4G lam=0.3017521755148967 Gm=0.3337688127843042 +[full s1] step 10500/12000 loss=4.5112 gn=1.23 lr=8.12e-05 tps=14939 mem=6.4G lam=0.30183420578638714 Gm=0.3411858434168001 +[full s1] EVAL step 10500: val_loss=4.4097 val_ppl=82.24 +[full s1] step 10520/12000 loss=4.3546 gn=1.01 lr=8.07e-05 tps=14929 mem=6.4G lam=0.3019057046622038 Gm=0.3370377699223657 +[full s1] step 10540/12000 loss=4.4548 gn=0.97 lr=8.01e-05 tps=14929 mem=6.4G lam=0.30200791110595065 Gm=0.3331927874435981 +[full s1] step 10560/12000 loss=4.4226 gn=0.99 lr=7.96e-05 tps=14929 mem=6.4G lam=0.30211212548116845 Gm=0.34042731773418683 +[full s1] step 10580/12000 loss=4.3043 gn=0.96 lr=7.91e-05 tps=14930 mem=6.4G lam=0.3021982526406646 Gm=0.33403980180931586 +[full s1] step 10600/12000 loss=4.2360 gn=0.97 lr=7.85e-05 tps=14930 mem=6.4G lam=0.30227868507305783 Gm=0.33552397081318003 +[full s1] step 10620/12000 loss=4.3307 gn=1.08 lr=7.80e-05 tps=14931 mem=6.4G lam=0.30237143735090893 Gm=0.3409980114083737 +[full s1] step 10640/12000 loss=4.4558 gn=1.05 lr=7.75e-05 tps=14931 mem=6.4G lam=0.3024684761961301 Gm=0.34044065947333974 +[full s1] step 10660/12000 loss=4.2592 gn=0.91 lr=7.70e-05 tps=14932 mem=6.4G lam=0.3025721811378996 Gm=0.34107425234590966 +[full s1] step 10680/12000 loss=4.2663 gn=0.97 lr=7.65e-05 tps=14932 mem=6.4G lam=0.30267492060859996 Gm=0.33487795627055067 +[full s1] step 10700/12000 loss=4.3257 gn=0.92 lr=7.60e-05 tps=14933 mem=6.4G lam=0.30275496405859786 Gm=0.33367965277284384 +[full s1] step 10720/12000 loss=4.3500 gn=1.02 lr=7.55e-05 tps=14933 mem=6.4G lam=0.3028086358681321 Gm=0.3409408323932439 +[full s1] step 10740/12000 loss=4.4052 gn=1.00 lr=7.50e-05 tps=14934 mem=6.4G lam=0.30290188267827034 Gm=0.3410857390457143 +[full s1] step 10760/12000 loss=4.3106 gn=0.94 lr=7.46e-05 tps=14934 mem=6.4G lam=0.30298712352911633 Gm=0.3348843969094257 +[full s1] step 10780/12000 loss=4.3370 gn=1.03 lr=7.41e-05 tps=14935 mem=6.4G lam=0.30306046300878126 Gm=0.33898684607508284 +[full s1] step 10800/12000 loss=4.4019 gn=1.04 lr=7.37e-05 tps=14935 mem=6.4G lam=0.3031277833506465 Gm=0.3363374946638942 +[full s1] step 10820/12000 loss=4.5053 gn=0.99 lr=7.32e-05 tps=14936 mem=6.4G lam=0.3032203586772084 Gm=0.33997396922980744 +[full s1] step 10840/12000 loss=4.1911 gn=0.93 lr=7.28e-05 tps=14936 mem=6.4G lam=0.30329576258858043 Gm=0.33203533858371276 +[full s1] step 10860/12000 loss=4.3689 gn=0.94 lr=7.23e-05 tps=14937 mem=6.4G lam=0.3033824913824598 Gm=0.33734686113893986 +[full s1] step 10880/12000 loss=4.2149 gn=0.99 lr=7.19e-05 tps=14937 mem=6.4G lam=0.30348380530873936 Gm=0.3345504488485555 +[full s1] step 10900/12000 loss=4.3469 gn=0.96 lr=7.15e-05 tps=14938 mem=6.4G lam=0.3035652969653408 Gm=0.33736404636874795 +[full s1] step 10920/12000 loss=4.3480 gn=1.00 lr=7.11e-05 tps=14938 mem=6.4G lam=0.30365455833574134 Gm=0.3356605110069116 +[full s1] step 10940/12000 loss=4.2970 gn=0.96 lr=7.07e-05 tps=14939 mem=6.4G lam=0.3037222384785612 Gm=0.33223293457801145 +[full s1] step 10960/12000 loss=4.3061 gn=0.98 lr=7.03e-05 tps=14939 mem=6.4G lam=0.3038029440989097 Gm=0.33268481593889493 +[full s1] step 10980/12000 loss=4.4968 gn=1.02 lr=6.99e-05 tps=14940 mem=6.4G lam=0.303904262992243 Gm=0.34465672137836617 +[full s1] step 11000/12000 loss=4.3102 gn=0.97 lr=6.95e-05 tps=14940 mem=6.4G lam=0.30398521448175114 Gm=0.3398795675020665 +[full s1] EVAL step 11000: val_loss=4.3951 val_ppl=81.05 +[full s1] step 11020/12000 loss=4.1053 gn=0.96 lr=6.91e-05 tps=14930 mem=6.4G lam=0.3040475870172183 Gm=0.33876407992405194 +[full s1] step 11040/12000 loss=4.3203 gn=1.06 lr=6.88e-05 tps=14931 mem=6.4G lam=0.3041220245261987 Gm=0.33934111753478646 +[full s1] step 11060/12000 loss=4.3862 gn=1.04 lr=6.84e-05 tps=14931 mem=6.4G lam=0.30422475654631853 Gm=0.3382815996495386 +[full s1] step 11080/12000 loss=4.3889 gn=0.98 lr=6.81e-05 tps=14932 mem=6.4G lam=0.3043117892617981 Gm=0.33478747998985153 +[full s1] step 11100/12000 loss=4.4040 gn=0.97 lr=6.77e-05 tps=14932 mem=6.4G lam=0.304379681746165 Gm=0.33671689926025766 +[full s1] step 11120/12000 loss=4.3857 gn=0.99 lr=6.74e-05 tps=14932 mem=6.4G lam=0.3044529479617874 Gm=0.3414290250899891 +[full s1] step 11140/12000 loss=4.2459 gn=0.98 lr=6.70e-05 tps=14933 mem=6.4G lam=0.3045361203451951 Gm=0.33513817222168046 +[full s1] step 11160/12000 loss=4.2834 gn=1.00 lr=6.67e-05 tps=14934 mem=6.4G lam=0.30461655867596465 Gm=0.33398131855453056 +[full s1] step 11180/12000 loss=4.3002 gn=0.94 lr=6.64e-05 tps=14934 mem=6.4G lam=0.30468419070045155 Gm=0.3358771277125925 +[full s1] step 11200/12000 loss=4.4866 gn=1.10 lr=6.61e-05 tps=14935 mem=6.4G lam=0.3047658372670412 Gm=0.3363290848210454 +[full s1] step 11220/12000 loss=4.3439 gn=1.03 lr=6.58e-05 tps=14935 mem=6.4G lam=0.30484327549735707 Gm=0.33613793497594696 +[full s1] step 11240/12000 loss=4.2670 gn=0.96 lr=6.55e-05 tps=14936 mem=6.4G lam=0.3048963031421105 Gm=0.3335481498700877 +[full s1] step 11260/12000 loss=4.2845 gn=1.00 lr=6.52e-05 tps=14936 mem=6.4G lam=0.3049839961652954 Gm=0.3378124154017617 +[full s1] step 11280/12000 loss=4.1842 gn=1.06 lr=6.49e-05 tps=14936 mem=6.4G lam=0.30507458187639713 Gm=0.33554028424744803 +[full s1] step 11300/12000 loss=4.3883 gn=1.05 lr=6.47e-05 tps=14937 mem=6.4G lam=0.3051457942153017 Gm=0.3417727934041371 +[full s1] step 11320/12000 loss=4.2014 gn=0.95 lr=6.44e-05 tps=14937 mem=6.4G lam=0.3052071801697214 Gm=0.33881257148459554 +[full s1] step 11340/12000 loss=4.3843 gn=0.98 lr=6.42e-05 tps=14937 mem=6.4G lam=0.3052649113039176 Gm=0.3361024232581258 +[full s1] step 11360/12000 loss=4.2728 gn=0.97 lr=6.39e-05 tps=14938 mem=6.4G lam=0.3053384091084202 Gm=0.3341764595049123 +[full s1] step 11380/12000 loss=4.3156 gn=1.03 lr=6.37e-05 tps=14938 mem=6.4G lam=0.30539992979417246 Gm=0.33833564491942525 +[full s1] step 11400/12000 loss=4.3896 gn=0.97 lr=6.34e-05 tps=14939 mem=6.4G lam=0.305472475476563 Gm=0.33463978317255777 +[full s1] step 11420/12000 loss=4.0742 gn=0.99 lr=6.32e-05 tps=14939 mem=6.4G lam=0.3055427831908067 Gm=0.3312970826712747 +[full s1] step 11440/12000 loss=4.1876 gn=0.99 lr=6.30e-05 tps=14939 mem=6.4G lam=0.3056138039877017 Gm=0.3308415880116324 +[full s1] step 11460/12000 loss=4.3175 gn=1.02 lr=6.28e-05 tps=14940 mem=6.4G lam=0.3056925230969985 Gm=0.3361895938093464 +[full s1] step 11480/12000 loss=4.3738 gn=0.96 lr=6.26e-05 tps=14940 mem=6.4G lam=0.3057495007912318 Gm=0.3343528201803565 +[full s1] step 11500/12000 loss=4.5042 gn=1.05 lr=6.24e-05 tps=14941 mem=6.4G lam=0.3058134081463019 Gm=0.33373715456885594 +[full s1] EVAL step 11500: val_loss=4.3875 val_ppl=80.44 +[full s1] step 11520/12000 loss=4.2996 gn=0.98 lr=6.22e-05 tps=14931 mem=6.4G lam=0.30590026701490086 Gm=0.3364600057248026 +[full s1] step 11540/12000 loss=4.1952 gn=1.04 lr=6.20e-05 tps=14931 mem=6.4G lam=0.3059872568895419 Gm=0.33031873777508736 +[full s1] step 11560/12000 loss=4.3289 gn=1.05 lr=6.19e-05 tps=14932 mem=6.4G lam=0.30604644026607275 Gm=0.3335089859707902 +[full s1] step 11580/12000 loss=4.3603 gn=0.98 lr=6.17e-05 tps=14932 mem=6.4G lam=0.3060865756124258 Gm=0.33480301103554666 +[full s1] step 11600/12000 loss=4.1658 gn=0.98 lr=6.15e-05 tps=14933 mem=6.4G lam=0.306142745539546 Gm=0.33385506831109524 +[full s1] step 11620/12000 loss=4.4249 gn=1.03 lr=6.14e-05 tps=14933 mem=6.4G lam=0.3062308228885134 Gm=0.33734663265446824 +[full s1] step 11640/12000 loss=4.3571 gn=1.02 lr=6.12e-05 tps=14934 mem=6.4G lam=0.30630997226883966 Gm=0.33328206464648247 +[full s1] step 11660/12000 loss=4.3633 gn=1.01 lr=6.11e-05 tps=14934 mem=6.4G lam=0.3063824021567901 Gm=0.3354764223719637 +[full s1] step 11680/12000 loss=4.2658 gn=0.98 lr=6.10e-05 tps=14934 mem=6.4G lam=0.3064426699032386 Gm=0.33739031789203483 +[full s1] step 11700/12000 loss=4.3020 gn=0.96 lr=6.09e-05 tps=14935 mem=6.4G lam=0.30652635730803013 Gm=0.33484403354426223 +[full s1] step 11720/12000 loss=4.1780 gn=0.99 lr=6.07e-05 tps=14935 mem=6.4G lam=0.306582841090858 Gm=0.33021775384744007 +[full s1] step 11740/12000 loss=4.3830 gn=1.00 lr=6.06e-05 tps=14936 mem=6.4G lam=0.3066388402755062 Gm=0.34156534215435386 +[full s1] step 11760/12000 loss=4.2660 gn=1.00 lr=6.06e-05 tps=14936 mem=6.4G lam=0.3066916608562072 Gm=0.3384451263894637 +[full s1] step 11780/12000 loss=4.2634 gn=1.00 lr=6.05e-05 tps=14936 mem=6.4G lam=0.3067607646808028 Gm=0.3284249833474557 +[full s1] step 11800/12000 loss=4.2302 gn=0.96 lr=6.04e-05 tps=14937 mem=6.4G lam=0.3068497485170762 Gm=0.33474206971004605 +[full s1] step 11820/12000 loss=4.3132 gn=0.97 lr=6.03e-05 tps=14937 mem=6.4G lam=0.3069329826782147 Gm=0.33670608171572286 +[full s1] step 11840/12000 loss=4.2899 gn=1.01 lr=6.02e-05 tps=14937 mem=6.4G lam=0.30698336971302825 Gm=0.33943870834385353 +[full s1] step 11860/12000 loss=4.2990 gn=1.05 lr=6.02e-05 tps=14938 mem=6.4G lam=0.30705410055816174 Gm=0.3348457107786089 +[full s1] step 11880/12000 loss=4.2576 gn=1.00 lr=6.01e-05 tps=14938 mem=6.4G lam=0.3071059910580516 Gm=0.33525876033430296 +[full s1] step 11900/12000 loss=4.3941 gn=1.09 lr=6.01e-05 tps=14939 mem=6.4G lam=0.3071805763368805 Gm=0.33361885766498744 +[full s1] step 11920/12000 loss=4.2474 gn=1.02 lr=6.01e-05 tps=14939 mem=6.4G lam=0.30724938493222 Gm=0.3293047013382117 +[full s1] step 11940/12000 loss=4.3131 gn=1.05 lr=6.00e-05 tps=14939 mem=6.4G lam=0.30732027099778253 Gm=0.3317298356754084 +[full s1] step 11960/12000 loss=4.2182 gn=1.03 lr=6.00e-05 tps=14940 mem=6.4G lam=0.30739027882615727 Gm=0.3333937887412806 +[full s1] step 11980/12000 loss=4.3019 gn=0.98 lr=6.00e-05 tps=14940 mem=6.4G lam=0.3074745563790202 Gm=0.3332850156972806 +[full s1] EVAL step 11999: val_loss=4.3824 val_ppl=80.03 +[init] variant=baseline seed=2 params=63.38M +[init] use_S=False use_G=False +[baseline s2] step 0/12000 loss=10.8749 gn=14.16 lr=3.00e-06 tps=9207 mem=5.5G lam=None Gm=None +[baseline s2] step 20/12000 loss=9.8900 gn=2.46 lr=6.30e-05 tps=17904 mem=6.0G lam=None Gm=None +[baseline s2] step 40/12000 loss=9.0933 gn=1.81 lr=1.23e-04 tps=18313 mem=6.0G lam=None Gm=None +[baseline s2] step 60/12000 loss=8.1959 gn=1.53 lr=1.83e-04 tps=18468 mem=6.0G lam=None Gm=None +[baseline s2] step 80/12000 loss=7.5603 gn=0.81 lr=2.43e-04 tps=18535 mem=6.0G lam=None Gm=None +[baseline s2] step 100/12000 loss=7.2565 gn=0.93 lr=3.03e-04 tps=18572 mem=6.0G lam=None Gm=None +[baseline s2] step 120/12000 loss=7.1677 gn=0.81 lr=3.63e-04 tps=18584 mem=6.0G lam=None Gm=None +[baseline s2] step 140/12000 loss=7.0294 gn=0.65 lr=4.23e-04 tps=18603 mem=6.0G lam=None Gm=None +[baseline s2] step 160/12000 loss=6.7335 gn=0.59 lr=4.83e-04 tps=18606 mem=6.0G lam=None Gm=None +[baseline s2] step 180/12000 loss=6.8797 gn=0.81 lr=5.43e-04 tps=18534 mem=6.0G lam=None Gm=None +[baseline s2] step 200/12000 loss=6.6347 gn=0.58 lr=6.00e-04 tps=18544 mem=6.0G lam=None Gm=None +[baseline s2] step 220/12000 loss=6.6389 gn=0.68 lr=6.00e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s2] step 240/12000 loss=6.5923 gn=0.54 lr=6.00e-04 tps=18555 mem=6.0G lam=None Gm=None +[baseline s2] step 260/12000 loss=6.4208 gn=0.61 lr=6.00e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s2] step 280/12000 loss=6.5221 gn=0.54 lr=6.00e-04 tps=18558 mem=6.0G lam=None Gm=None +[baseline s2] step 300/12000 loss=6.6633 gn=0.55 lr=6.00e-04 tps=18560 mem=6.0G lam=None Gm=None +[baseline s2] step 320/12000 loss=6.6060 gn=0.52 lr=6.00e-04 tps=18560 mem=6.0G lam=None Gm=None +[baseline s2] step 340/12000 loss=6.4719 gn=0.64 lr=6.00e-04 tps=18559 mem=6.0G lam=None Gm=None +[baseline s2] step 360/12000 loss=6.4077 gn=0.58 lr=6.00e-04 tps=18557 mem=6.0G lam=None Gm=None +[baseline s2] step 380/12000 loss=6.2800 gn=0.53 lr=6.00e-04 tps=18553 mem=6.0G lam=None Gm=None +[baseline s2] step 400/12000 loss=6.4819 gn=0.55 lr=6.00e-04 tps=18551 mem=6.0G lam=None Gm=None +[baseline s2] step 420/12000 loss=6.3039 gn=0.49 lr=6.00e-04 tps=18549 mem=6.0G lam=None Gm=None +[baseline s2] step 440/12000 loss=6.3192 gn=0.57 lr=5.99e-04 tps=18541 mem=6.0G lam=None Gm=None +[baseline s2] step 460/12000 loss=6.2589 gn=0.50 lr=5.99e-04 tps=18537 mem=6.0G lam=None Gm=None +[baseline s2] step 480/12000 loss=6.4996 gn=0.51 lr=5.99e-04 tps=18533 mem=6.0G lam=None Gm=None +[baseline s2] step 500/12000 loss=6.3373 gn=0.54 lr=5.99e-04 tps=18530 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 500: val_loss=6.2497 val_ppl=517.87 +[baseline s2] step 520/12000 loss=6.3706 gn=0.60 lr=5.99e-04 tps=18272 mem=6.0G lam=None Gm=None +[baseline s2] step 540/12000 loss=6.1220 gn=0.52 lr=5.99e-04 tps=18275 mem=6.0G lam=None Gm=None +[baseline s2] step 560/12000 loss=6.0757 gn=0.58 lr=5.99e-04 tps=18277 mem=6.0G lam=None Gm=None +[baseline s2] step 580/12000 loss=5.9226 gn=0.53 lr=5.99e-04 tps=18281 mem=6.0G lam=None Gm=None +[baseline s2] step 600/12000 loss=6.0739 gn=0.54 lr=5.98e-04 tps=18285 mem=6.0G lam=None Gm=None +[baseline s2] step 620/12000 loss=6.2340 gn=0.58 lr=5.98e-04 tps=18286 mem=6.0G lam=None Gm=None +[baseline s2] step 640/12000 loss=6.1460 gn=0.52 lr=5.98e-04 tps=18287 mem=6.0G lam=None Gm=None +[baseline s2] step 660/12000 loss=6.1388 gn=0.51 lr=5.98e-04 tps=18289 mem=6.0G lam=None Gm=None +[baseline s2] step 680/12000 loss=6.0913 gn=0.49 lr=5.98e-04 tps=18289 mem=6.0G lam=None Gm=None +[baseline s2] step 700/12000 loss=5.9831 gn=0.51 lr=5.98e-04 tps=18288 mem=6.0G lam=None Gm=None +[baseline s2] step 720/12000 loss=6.0651 gn=0.67 lr=5.97e-04 tps=18288 mem=6.0G lam=None Gm=None +[baseline s2] step 740/12000 loss=6.0266 gn=0.59 lr=5.97e-04 tps=18287 mem=6.0G lam=None Gm=None +[baseline s2] step 760/12000 loss=6.0556 gn=0.54 lr=5.97e-04 tps=18287 mem=6.0G lam=None Gm=None +[baseline s2] step 780/12000 loss=5.8008 gn=0.53 lr=5.97e-04 tps=18260 mem=6.0G lam=None Gm=None +[baseline s2] step 800/12000 loss=5.9242 gn=0.58 lr=5.97e-04 tps=18220 mem=6.0G lam=None Gm=None +[baseline s2] step 820/12000 loss=5.8746 gn=0.56 lr=5.96e-04 tps=18203 mem=6.0G lam=None Gm=None +[baseline s2] step 840/12000 loss=5.9288 gn=0.52 lr=5.96e-04 tps=18160 mem=6.0G lam=None Gm=None +[baseline s2] step 860/12000 loss=5.9023 gn=0.52 lr=5.96e-04 tps=18148 mem=6.0G lam=None Gm=None +[baseline s2] step 880/12000 loss=5.9954 gn=0.56 lr=5.96e-04 tps=18146 mem=6.0G lam=None Gm=None +[baseline s2] step 900/12000 loss=5.8027 gn=0.51 lr=5.95e-04 tps=18137 mem=6.0G lam=None Gm=None +[baseline s2] step 920/12000 loss=5.9452 gn=0.52 lr=5.95e-04 tps=18134 mem=6.0G lam=None Gm=None +[baseline s2] step 940/12000 loss=5.8537 gn=0.52 lr=5.95e-04 tps=18124 mem=6.0G lam=None Gm=None +[baseline s2] step 960/12000 loss=5.7457 gn=0.52 lr=5.94e-04 tps=18119 mem=6.0G lam=None Gm=None +[baseline s2] step 980/12000 loss=5.8428 gn=0.54 lr=5.94e-04 tps=18118 mem=6.0G lam=None Gm=None +[baseline s2] step 1000/12000 loss=5.7543 gn=0.55 lr=5.94e-04 tps=18112 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 1000: val_loss=5.8213 val_ppl=337.40 +[baseline s2] step 1020/12000 loss=5.7059 gn=0.57 lr=5.94e-04 tps=17979 mem=6.0G lam=None Gm=None +[baseline s2] step 1040/12000 loss=5.8308 gn=0.53 lr=5.93e-04 tps=17978 mem=6.0G lam=None Gm=None +[baseline s2] step 1060/12000 loss=5.7086 gn=0.77 lr=5.93e-04 tps=17978 mem=6.0G lam=None Gm=None +[baseline s2] step 1080/12000 loss=5.7880 gn=0.54 lr=5.93e-04 tps=17967 mem=6.0G lam=None Gm=None +[baseline s2] step 1100/12000 loss=5.6179 gn=0.54 lr=5.92e-04 tps=17963 mem=6.0G lam=None Gm=None +[baseline s2] step 1120/12000 loss=5.7119 gn=0.57 lr=5.92e-04 tps=17958 mem=6.0G lam=None Gm=None +[baseline s2] step 1140/12000 loss=5.7682 gn=0.65 lr=5.92e-04 tps=17949 mem=6.0G lam=None Gm=None +[baseline s2] step 1160/12000 loss=5.6893 gn=0.55 lr=5.91e-04 tps=17944 mem=6.0G lam=None Gm=None +[baseline s2] step 1180/12000 loss=5.5356 gn=0.52 lr=5.91e-04 tps=17942 mem=6.0G lam=None Gm=None +[baseline s2] step 1200/12000 loss=5.5755 gn=0.56 lr=5.90e-04 tps=17943 mem=6.0G lam=None Gm=None +[baseline s2] step 1220/12000 loss=5.6348 gn=0.53 lr=5.90e-04 tps=17945 mem=6.0G lam=None Gm=None +[baseline s2] step 1240/12000 loss=5.5541 gn=0.56 lr=5.90e-04 tps=17947 mem=6.0G lam=None Gm=None +[baseline s2] step 1260/12000 loss=5.5058 gn=0.57 lr=5.89e-04 tps=17926 mem=6.0G lam=None Gm=None +[baseline s2] step 1280/12000 loss=5.4844 gn=0.65 lr=5.89e-04 tps=17902 mem=6.0G lam=None Gm=None +[baseline s2] step 1300/12000 loss=5.6122 gn=0.56 lr=5.89e-04 tps=17892 mem=6.0G lam=None Gm=None +[baseline s2] step 1320/12000 loss=5.4803 gn=0.67 lr=5.88e-04 tps=17871 mem=6.0G lam=None Gm=None +[baseline s2] step 1340/12000 loss=5.6722 gn=0.59 lr=5.88e-04 tps=17858 mem=6.0G lam=None Gm=None +[baseline s2] step 1360/12000 loss=5.5027 gn=0.60 lr=5.87e-04 tps=17860 mem=6.0G lam=None Gm=None +[baseline s2] step 1380/12000 loss=5.4970 gn=0.57 lr=5.87e-04 tps=17861 mem=6.0G lam=None Gm=None +[baseline s2] step 1400/12000 loss=5.4201 gn=0.58 lr=5.86e-04 tps=17862 mem=6.0G lam=None Gm=None +[baseline s2] step 1420/12000 loss=5.4740 gn=0.58 lr=5.86e-04 tps=17864 mem=6.0G lam=None Gm=None +[baseline s2] step 1440/12000 loss=5.5222 gn=0.53 lr=5.85e-04 tps=17866 mem=6.0G lam=None Gm=None +[baseline s2] step 1460/12000 loss=5.4468 gn=0.59 lr=5.85e-04 tps=17870 mem=6.0G lam=None Gm=None +[baseline s2] step 1480/12000 loss=5.3762 gn=0.61 lr=5.84e-04 tps=17872 mem=6.0G lam=None Gm=None +[baseline s2] step 1500/12000 loss=5.6424 gn=0.61 lr=5.84e-04 tps=17876 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 1500: val_loss=5.5226 val_ppl=250.28 +[baseline s2] step 1520/12000 loss=5.4338 gn=0.60 lr=5.83e-04 tps=17798 mem=6.0G lam=None Gm=None +[baseline s2] step 1540/12000 loss=5.4297 gn=0.53 lr=5.83e-04 tps=17804 mem=6.0G lam=None Gm=None +[baseline s2] step 1560/12000 loss=5.6238 gn=0.57 lr=5.82e-04 tps=17810 mem=6.0G lam=None Gm=None +[baseline s2] step 1580/12000 loss=5.5063 gn=0.56 lr=5.82e-04 tps=17816 mem=6.0G lam=None Gm=None +[baseline s2] step 1600/12000 loss=5.4285 gn=0.58 lr=5.81e-04 tps=17821 mem=6.0G lam=None Gm=None +[baseline s2] step 1620/12000 loss=5.4064 gn=0.58 lr=5.81e-04 tps=17827 mem=6.0G lam=None Gm=None +[baseline s2] step 1640/12000 loss=5.4608 gn=0.58 lr=5.80e-04 tps=17833 mem=6.0G lam=None Gm=None +[baseline s2] step 1660/12000 loss=5.1927 gn=0.58 lr=5.80e-04 tps=17840 mem=6.0G lam=None Gm=None +[baseline s2] step 1680/12000 loss=5.4620 gn=0.61 lr=5.79e-04 tps=17845 mem=6.0G lam=None Gm=None +[baseline s2] step 1700/12000 loss=5.5456 gn=0.60 lr=5.79e-04 tps=17851 mem=6.0G lam=None Gm=None +[baseline s2] step 1720/12000 loss=5.3360 gn=0.59 lr=5.78e-04 tps=17857 mem=6.0G lam=None Gm=None +[baseline s2] step 1740/12000 loss=5.3891 gn=0.57 lr=5.78e-04 tps=17862 mem=6.0G lam=None Gm=None +[baseline s2] step 1760/12000 loss=5.2660 gn=0.60 lr=5.77e-04 tps=17868 mem=6.0G lam=None Gm=None +[baseline s2] step 1780/12000 loss=5.2895 gn=0.59 lr=5.76e-04 tps=17874 mem=6.0G lam=None Gm=None +[baseline s2] step 1800/12000 loss=5.4679 gn=0.60 lr=5.76e-04 tps=17879 mem=6.0G lam=None Gm=None +[baseline s2] step 1820/12000 loss=5.5026 gn=0.60 lr=5.75e-04 tps=17885 mem=6.0G lam=None Gm=None +[baseline s2] step 1840/12000 loss=5.4545 gn=0.61 lr=5.75e-04 tps=17890 mem=6.0G lam=None Gm=None +[baseline s2] step 1860/12000 loss=5.4563 gn=0.69 lr=5.74e-04 tps=17895 mem=6.0G lam=None Gm=None +[baseline s2] step 1880/12000 loss=5.2029 gn=0.59 lr=5.73e-04 tps=17900 mem=6.0G lam=None Gm=None +[baseline s2] step 1900/12000 loss=5.2576 gn=0.66 lr=5.73e-04 tps=17906 mem=6.0G lam=None Gm=None +[baseline s2] step 1920/12000 loss=5.4604 gn=0.60 lr=5.72e-04 tps=17912 mem=6.0G lam=None Gm=None +[baseline s2] step 1940/12000 loss=5.5938 gn=0.73 lr=5.72e-04 tps=17917 mem=6.0G lam=None Gm=None +[baseline s2] step 1960/12000 loss=5.1999 gn=0.57 lr=5.71e-04 tps=17922 mem=6.0G lam=None Gm=None +[baseline s2] step 1980/12000 loss=5.1771 gn=0.61 lr=5.70e-04 tps=17927 mem=6.0G lam=None Gm=None +[baseline s2] step 2000/12000 loss=5.1362 gn=0.63 lr=5.70e-04 tps=17932 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 2000: val_loss=5.3292 val_ppl=206.27 +[baseline s2] step 2020/12000 loss=5.2381 gn=0.65 lr=5.69e-04 tps=17875 mem=6.0G lam=None Gm=None +[baseline s2] step 2040/12000 loss=5.3236 gn=0.63 lr=5.68e-04 tps=17880 mem=6.0G lam=None Gm=None +[baseline s2] step 2060/12000 loss=5.3933 gn=0.67 lr=5.68e-04 tps=17885 mem=6.0G lam=None Gm=None +[baseline s2] step 2080/12000 loss=5.3978 gn=0.63 lr=5.67e-04 tps=17890 mem=6.0G lam=None Gm=None +[baseline s2] step 2100/12000 loss=5.2233 gn=0.64 lr=5.66e-04 tps=17896 mem=6.0G lam=None Gm=None +[baseline s2] step 2120/12000 loss=5.3804 gn=0.69 lr=5.65e-04 tps=17901 mem=6.0G lam=None Gm=None +[baseline s2] step 2140/12000 loss=5.1374 gn=0.59 lr=5.65e-04 tps=17906 mem=6.0G lam=None Gm=None +[baseline s2] step 2160/12000 loss=5.1991 gn=0.61 lr=5.64e-04 tps=17911 mem=6.0G lam=None Gm=None +[baseline s2] step 2180/12000 loss=5.3169 gn=0.61 lr=5.63e-04 tps=17916 mem=6.0G lam=None Gm=None +[baseline s2] step 2200/12000 loss=5.0953 gn=0.66 lr=5.63e-04 tps=17921 mem=6.0G lam=None Gm=None +[baseline s2] step 2220/12000 loss=5.1888 gn=0.63 lr=5.62e-04 tps=17925 mem=6.0G lam=None Gm=None +[baseline s2] step 2240/12000 loss=5.3063 gn=0.61 lr=5.61e-04 tps=17930 mem=6.0G lam=None Gm=None +[baseline s2] step 2260/12000 loss=5.1129 gn=0.65 lr=5.60e-04 tps=17936 mem=6.0G lam=None Gm=None +[baseline s2] step 2280/12000 loss=5.1532 gn=0.68 lr=5.60e-04 tps=17940 mem=6.0G lam=None Gm=None +[baseline s2] step 2300/12000 loss=5.3501 gn=0.65 lr=5.59e-04 tps=17944 mem=6.0G lam=None Gm=None +[baseline s2] step 2320/12000 loss=5.3051 gn=0.69 lr=5.58e-04 tps=17949 mem=6.0G lam=None Gm=None +[baseline s2] step 2340/12000 loss=5.1381 gn=0.63 lr=5.57e-04 tps=17954 mem=6.0G lam=None Gm=None +[baseline s2] step 2360/12000 loss=5.1453 gn=0.66 lr=5.57e-04 tps=17958 mem=6.0G lam=None Gm=None +[baseline s2] step 2380/12000 loss=5.1235 gn=0.69 lr=5.56e-04 tps=17963 mem=6.0G lam=None Gm=None +[baseline s2] step 2400/12000 loss=5.1089 gn=0.67 lr=5.55e-04 tps=17967 mem=6.0G lam=None Gm=None +[baseline s2] step 2420/12000 loss=5.2969 gn=0.71 lr=5.54e-04 tps=17971 mem=6.0G lam=None Gm=None +[baseline s2] step 2440/12000 loss=5.1374 gn=0.61 lr=5.53e-04 tps=17975 mem=6.0G lam=None Gm=None +[baseline s2] step 2460/12000 loss=5.0720 gn=0.67 lr=5.53e-04 tps=17980 mem=6.0G lam=None Gm=None +[baseline s2] step 2480/12000 loss=5.0739 gn=0.63 lr=5.52e-04 tps=17983 mem=6.0G lam=None Gm=None +[baseline s2] step 2500/12000 loss=5.1792 gn=0.65 lr=5.51e-04 tps=17986 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 2500: val_loss=5.1943 val_ppl=180.24 +[baseline s2] step 2520/12000 loss=4.9885 gn=0.64 lr=5.50e-04 tps=17940 mem=6.0G lam=None Gm=None +[baseline s2] step 2540/12000 loss=5.0786 gn=0.62 lr=5.49e-04 tps=17945 mem=6.0G lam=None Gm=None +[baseline s2] step 2560/12000 loss=5.1922 gn=0.73 lr=5.48e-04 tps=17949 mem=6.0G lam=None Gm=None +[baseline s2] step 2580/12000 loss=5.1703 gn=0.72 lr=5.48e-04 tps=17953 mem=6.0G lam=None Gm=None +[baseline s2] step 2600/12000 loss=5.0678 gn=0.65 lr=5.47e-04 tps=17957 mem=6.0G lam=None Gm=None +[baseline s2] step 2620/12000 loss=4.9971 gn=0.67 lr=5.46e-04 tps=17961 mem=6.0G lam=None Gm=None +[baseline s2] step 2640/12000 loss=5.1102 gn=0.72 lr=5.45e-04 tps=17966 mem=6.0G lam=None Gm=None +[baseline s2] step 2660/12000 loss=5.1644 gn=0.66 lr=5.44e-04 tps=17970 mem=6.0G lam=None Gm=None +[baseline s2] step 2680/12000 loss=5.1350 gn=0.70 lr=5.43e-04 tps=17973 mem=6.0G lam=None Gm=None +[baseline s2] step 2700/12000 loss=5.1410 gn=0.65 lr=5.42e-04 tps=17977 mem=6.0G lam=None Gm=None +[baseline s2] step 2720/12000 loss=5.0211 gn=0.68 lr=5.41e-04 tps=17981 mem=6.0G lam=None Gm=None +[baseline s2] step 2740/12000 loss=5.0992 gn=0.67 lr=5.41e-04 tps=17985 mem=6.0G lam=None Gm=None +[baseline s2] step 2760/12000 loss=5.0405 gn=0.72 lr=5.40e-04 tps=17989 mem=6.0G lam=None Gm=None +[baseline s2] step 2780/12000 loss=4.9688 gn=0.66 lr=5.39e-04 tps=17992 mem=6.0G lam=None Gm=None +[baseline s2] step 2800/12000 loss=5.0098 gn=0.69 lr=5.38e-04 tps=17996 mem=6.0G lam=None Gm=None +[baseline s2] step 2820/12000 loss=4.9943 gn=0.70 lr=5.37e-04 tps=18000 mem=6.0G lam=None Gm=None +[baseline s2] step 2840/12000 loss=5.1396 gn=0.71 lr=5.36e-04 tps=18003 mem=6.0G lam=None Gm=None +[baseline s2] step 2860/12000 loss=4.8740 gn=0.69 lr=5.35e-04 tps=18007 mem=6.0G lam=None Gm=None +[baseline s2] step 2880/12000 loss=5.0437 gn=0.67 lr=5.34e-04 tps=18010 mem=6.0G lam=None Gm=None +[baseline s2] step 2900/12000 loss=5.0202 gn=0.70 lr=5.33e-04 tps=18013 mem=6.0G lam=None Gm=None +[baseline s2] step 2920/12000 loss=5.0618 gn=0.69 lr=5.32e-04 tps=18017 mem=6.0G lam=None Gm=None +[baseline s2] step 2940/12000 loss=5.1798 gn=0.77 lr=5.31e-04 tps=18020 mem=6.0G lam=None Gm=None +[baseline s2] step 2960/12000 loss=4.8949 gn=0.69 lr=5.30e-04 tps=18023 mem=6.0G lam=None Gm=None +[baseline s2] step 2980/12000 loss=5.0007 gn=0.62 lr=5.29e-04 tps=18027 mem=6.0G lam=None Gm=None +[baseline s2] step 3000/12000 loss=4.9178 gn=0.73 lr=5.28e-04 tps=18030 mem=6.0G lam=None Gm=None +[baseline s2] EVAL step 3000: val_loss=5.0695 val_ppl=159.09 +[baseline s2] step 3020/12000 loss=5.0901 gn=0.68 lr=5.27e-04 tps=17991 mem=6.0G lam=None Gm=None +[baseline s2] step 3040/12000 loss=5.0436 gn=0.74 lr=5.26e-04 tps=17994 mem=6.0G lam=None Gm=None +[baseline s2] step 3060/12000 loss=4.9765 gn=0.69 lr=5.25e-04 tps=17998 mem=6.0G lam=None Gm=None +[baseline s2] step 3080/12000 loss=4.8776 gn=0.70 lr=5.24e-04 tps=18001 mem=6.0G lam=None Gm=None +[baseline s2] step 3100/12000 loss=5.0084 gn=0.68 lr=5.23e-04 tps=18004 mem=6.0G lam=None Gm=None +[baseline s2] step 3120/12000 loss=4.7524 gn=0.67 lr=5.22e-04 tps=18007 mem=6.0G lam=None Gm=None +[baseline s2] step 3140/12000 loss=4.8998 gn=0.65 lr=5.21e-04 tps=18010 mem=6.0G lam=None Gm=None +[baseline s2] step 3160/12000 loss=4.8517 gn=0.70 lr=5.20e-04 tps=18014 mem=6.0G lam=None Gm=None +[baseline s2] step 3180/12000 loss=4.9756 gn=0.67 lr=5.19e-04 tps=18017 mem=6.0G lam=None Gm=None +[baseline s2] step 3200/12000 loss=4.8999 gn=0.65 lr=5.18e-04 tps=18019 mem=6.0G lam=None Gm=None +[baseline s2] step 3220/12000 loss=4.9688 gn=0.73 lr=5.17e-04 tps=18023 mem=6.0G lam=None Gm=None \ No newline at end of file diff --git a/runs_optimized.log b/runs_optimized.log new file mode 100644 index 0000000000000000000000000000000000000000..3ed8b4560b43ee115fc1b163dc66f91fe55b086f --- /dev/null +++ b/runs_optimized.log @@ -0,0 +1,7404 @@ +[init] variant=baseline seed=2 params=63.38M +[init] use_S=False use_G=False +[baseline s2] step 0/12000 loss=10.8750 gn=14.16 lr=3.00e-06 tps=10577 mem=4.8G lam=None Gm=None +[baseline s2] step 20/12000 loss=9.8900 gn=2.46 lr=6.30e-05 tps=23284 mem=5.4G lam=None Gm=None +[baseline s2] step 40/12000 loss=9.0932 gn=1.81 lr=1.23e-04 tps=24010 mem=5.4G lam=None Gm=None +[baseline s2] step 60/12000 loss=8.1959 gn=1.53 lr=1.83e-04 tps=24276 mem=5.4G lam=None Gm=None +[baseline s2] step 80/12000 loss=7.5603 gn=0.81 lr=2.43e-04 tps=24383 mem=5.4G lam=None Gm=None +[baseline s2] step 100/12000 loss=7.2510 gn=0.88 lr=3.03e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 120/12000 loss=7.1561 gn=0.66 lr=3.63e-04 tps=24534 mem=5.4G lam=None Gm=None +[baseline s2] step 140/12000 loss=7.0276 gn=0.60 lr=4.23e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s2] step 160/12000 loss=6.7371 gn=0.61 lr=4.83e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s2] step 180/12000 loss=6.8895 gn=0.95 lr=5.43e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s2] step 200/12000 loss=6.6327 gn=0.61 lr=6.00e-04 tps=24636 mem=5.4G lam=None Gm=None +[baseline s2] step 220/12000 loss=6.6379 gn=0.64 lr=6.00e-04 tps=24648 mem=5.4G lam=None Gm=None +[baseline s2] step 240/12000 loss=6.5912 gn=0.54 lr=6.00e-04 tps=24658 mem=5.4G lam=None Gm=None +[baseline s2] step 260/12000 loss=6.4165 gn=0.55 lr=6.00e-04 tps=24665 mem=5.4G lam=None Gm=None +[baseline s2] step 280/12000 loss=6.5223 gn=0.51 lr=6.00e-04 tps=24668 mem=5.4G lam=None Gm=None +[baseline s2] step 300/12000 loss=6.6553 gn=0.50 lr=6.00e-04 tps=24674 mem=5.4G lam=None Gm=None +[baseline s2] step 320/12000 loss=6.6083 gn=0.55 lr=6.00e-04 tps=24678 mem=5.4G lam=None Gm=None +[baseline s2] step 340/12000 loss=6.4743 gn=0.66 lr=6.00e-04 tps=24681 mem=5.4G lam=None Gm=None +[baseline s2] step 360/12000 loss=6.4046 gn=0.52 lr=6.00e-04 tps=24680 mem=5.4G lam=None Gm=None +[baseline s2] step 380/12000 loss=6.2807 gn=0.52 lr=6.00e-04 tps=24685 mem=5.4G lam=None Gm=None +[baseline s2] step 400/12000 loss=6.4778 gn=0.55 lr=6.00e-04 tps=24686 mem=5.4G lam=None Gm=None +[baseline s2] step 420/12000 loss=6.3125 gn=0.52 lr=6.00e-04 tps=24688 mem=5.4G lam=None Gm=None +[baseline s2] step 440/12000 loss=6.3159 gn=0.52 lr=5.99e-04 tps=24691 mem=5.4G lam=None Gm=None +[baseline s2] step 460/12000 loss=6.2589 gn=0.51 lr=5.99e-04 tps=24694 mem=5.4G lam=None Gm=None +[baseline s2] step 480/12000 loss=6.5029 gn=0.50 lr=5.99e-04 tps=24695 mem=5.4G lam=None Gm=None +[baseline s2] step 500/12000 loss=6.3362 gn=0.51 lr=5.99e-04 tps=24698 mem=5.4G lam=None Gm=None +[baseline s2] step 520/12000 loss=6.3607 gn=0.61 lr=5.99e-04 tps=24699 mem=5.4G lam=None Gm=None +[baseline s2] step 540/12000 loss=6.1166 gn=0.49 lr=5.99e-04 tps=24700 mem=5.4G lam=None Gm=None +[baseline s2] step 560/12000 loss=6.0782 gn=0.57 lr=5.99e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 580/12000 loss=5.9290 gn=0.57 lr=5.99e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 600/12000 loss=6.0703 gn=0.54 lr=5.98e-04 tps=24703 mem=5.4G lam=None Gm=None +[baseline s2] step 620/12000 loss=6.2358 gn=0.57 lr=5.98e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 640/12000 loss=6.1436 gn=0.51 lr=5.98e-04 tps=24701 mem=5.4G lam=None Gm=None +[baseline s2] step 660/12000 loss=6.1439 gn=0.52 lr=5.98e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 680/12000 loss=6.0958 gn=0.48 lr=5.98e-04 tps=24703 mem=5.4G lam=None Gm=None +[baseline s2] step 700/12000 loss=5.9854 gn=0.50 lr=5.98e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 720/12000 loss=6.0724 gn=0.68 lr=5.97e-04 tps=24703 mem=5.4G lam=None Gm=None +[baseline s2] step 740/12000 loss=6.0242 gn=0.55 lr=5.97e-04 tps=24702 mem=5.4G lam=None Gm=None +[baseline s2] step 760/12000 loss=6.0489 gn=0.52 lr=5.97e-04 tps=24701 mem=5.4G lam=None Gm=None +[baseline s2] step 780/12000 loss=5.8150 gn=0.55 lr=5.97e-04 tps=24701 mem=5.4G lam=None Gm=None +[baseline s2] step 800/12000 loss=5.9205 gn=0.56 lr=5.97e-04 tps=24699 mem=5.4G lam=None Gm=None +[baseline s2] step 820/12000 loss=5.8744 gn=0.52 lr=5.96e-04 tps=24699 mem=5.4G lam=None Gm=None +[baseline s2] step 840/12000 loss=5.9294 gn=0.52 lr=5.96e-04 tps=24700 mem=5.4G lam=None Gm=None +[baseline s2] step 860/12000 loss=5.9070 gn=0.53 lr=5.96e-04 tps=24698 mem=5.4G lam=None Gm=None +[baseline s2] step 880/12000 loss=5.9999 gn=0.58 lr=5.96e-04 tps=24699 mem=5.4G lam=None Gm=None +[baseline s2] step 900/12000 loss=5.7975 gn=0.52 lr=5.95e-04 tps=24698 mem=5.4G lam=None Gm=None +[baseline s2] step 920/12000 loss=5.9521 gn=0.51 lr=5.95e-04 tps=24697 mem=5.4G lam=None Gm=None +[baseline s2] step 940/12000 loss=5.8597 gn=0.53 lr=5.95e-04 tps=24697 mem=5.4G lam=None Gm=None +[baseline s2] step 960/12000 loss=5.7564 gn=0.53 lr=5.94e-04 tps=24696 mem=5.4G lam=None Gm=None +[baseline s2] step 980/12000 loss=5.8445 gn=0.53 lr=5.94e-04 tps=24695 mem=5.4G lam=None Gm=None +[baseline s2] step 1000/12000 loss=5.7497 gn=0.52 lr=5.94e-04 tps=24695 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 1000: val_loss=5.7538 val_ppl=315.37 +[baseline s2] step 1020/12000 loss=5.7033 gn=0.62 lr=5.94e-04 tps=24611 mem=5.4G lam=None Gm=None +[baseline s2] step 1040/12000 loss=5.8408 gn=0.53 lr=5.93e-04 tps=24611 mem=5.4G lam=None Gm=None +[baseline s2] step 1060/12000 loss=5.7424 gn=1.17 lr=5.93e-04 tps=24612 mem=5.4G lam=None Gm=None +[baseline s2] step 1080/12000 loss=5.7935 gn=0.55 lr=5.93e-04 tps=24612 mem=5.4G lam=None Gm=None +[baseline s2] step 1100/12000 loss=5.6278 gn=0.53 lr=5.92e-04 tps=24612 mem=5.4G lam=None Gm=None +[baseline s2] step 1120/12000 loss=5.7116 gn=0.57 lr=5.92e-04 tps=24613 mem=5.4G lam=None Gm=None +[baseline s2] step 1140/12000 loss=5.7687 gn=0.63 lr=5.92e-04 tps=24613 mem=5.4G lam=None Gm=None +[baseline s2] step 1160/12000 loss=5.6925 gn=0.56 lr=5.91e-04 tps=24614 mem=5.4G lam=None Gm=None +[baseline s2] step 1180/12000 loss=5.5274 gn=0.51 lr=5.91e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s2] step 1200/12000 loss=5.5728 gn=0.52 lr=5.90e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s2] step 1220/12000 loss=5.6383 gn=0.55 lr=5.90e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s2] step 1240/12000 loss=5.5621 gn=0.57 lr=5.90e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s2] step 1260/12000 loss=5.5086 gn=0.62 lr=5.89e-04 tps=24616 mem=5.4G lam=None Gm=None +[baseline s2] step 1280/12000 loss=5.4788 gn=0.59 lr=5.89e-04 tps=24617 mem=5.4G lam=None Gm=None +[baseline s2] step 1300/12000 loss=5.6166 gn=0.58 lr=5.89e-04 tps=24617 mem=5.4G lam=None Gm=None +[baseline s2] step 1320/12000 loss=5.4920 gn=0.86 lr=5.88e-04 tps=24617 mem=5.4G lam=None Gm=None +[baseline s2] step 1340/12000 loss=5.6783 gn=0.60 lr=5.88e-04 tps=24619 mem=5.4G lam=None Gm=None +[baseline s2] step 1360/12000 loss=5.5024 gn=0.60 lr=5.87e-04 tps=24619 mem=5.4G lam=None Gm=None +[baseline s2] step 1380/12000 loss=5.5061 gn=0.58 lr=5.87e-04 tps=24618 mem=5.4G lam=None Gm=None +[baseline s2] step 1400/12000 loss=5.4236 gn=0.59 lr=5.86e-04 tps=24618 mem=5.4G lam=None Gm=None +[baseline s2] step 1420/12000 loss=5.4754 gn=0.60 lr=5.86e-04 tps=24619 mem=5.4G lam=None Gm=None +[baseline s2] step 1440/12000 loss=5.5151 gn=0.55 lr=5.85e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s2] step 1460/12000 loss=5.4482 gn=0.59 lr=5.85e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1480/12000 loss=5.3770 gn=0.59 lr=5.84e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1500/12000 loss=5.6392 gn=0.60 lr=5.84e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1520/12000 loss=5.4308 gn=0.58 lr=5.83e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1540/12000 loss=5.4362 gn=0.58 lr=5.83e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1560/12000 loss=5.6259 gn=0.56 lr=5.82e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s2] step 1580/12000 loss=5.5142 gn=0.58 lr=5.82e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1600/12000 loss=5.4262 gn=0.59 lr=5.81e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1620/12000 loss=5.3924 gn=0.56 lr=5.81e-04 tps=24623 mem=5.4G lam=None Gm=None +[baseline s2] step 1640/12000 loss=5.4651 gn=0.62 lr=5.80e-04 tps=24623 mem=5.4G lam=None Gm=None +[baseline s2] step 1660/12000 loss=5.1832 gn=0.56 lr=5.80e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1680/12000 loss=5.4649 gn=0.60 lr=5.79e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s2] step 1700/12000 loss=5.5367 gn=0.63 lr=5.79e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1720/12000 loss=5.3410 gn=0.61 lr=5.78e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1740/12000 loss=5.3916 gn=0.60 lr=5.78e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1760/12000 loss=5.2565 gn=0.58 lr=5.77e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1780/12000 loss=5.2891 gn=0.60 lr=5.76e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s2] step 1800/12000 loss=5.4674 gn=0.64 lr=5.76e-04 tps=24624 mem=5.4G lam=None Gm=None +[baseline s2] step 1820/12000 loss=5.5029 gn=0.62 lr=5.75e-04 tps=24623 mem=5.4G lam=None Gm=None +[baseline s2] step 1840/12000 loss=5.4460 gn=0.60 lr=5.75e-04 tps=24624 mem=5.4G lam=None Gm=None +[baseline s2] step 1860/12000 loss=5.4602 gn=0.65 lr=5.74e-04 tps=24625 mem=5.4G lam=None Gm=None +[baseline s2] step 1880/12000 loss=5.1999 gn=0.60 lr=5.73e-04 tps=24625 mem=5.4G lam=None Gm=None +[baseline s2] step 1900/12000 loss=5.2626 gn=0.66 lr=5.73e-04 tps=24625 mem=5.4G lam=None Gm=None +[baseline s2] step 1920/12000 loss=5.4516 gn=0.60 lr=5.72e-04 tps=24626 mem=5.4G lam=None Gm=None +[baseline s2] step 1940/12000 loss=5.5736 gn=0.66 lr=5.72e-04 tps=24626 mem=5.4G lam=None Gm=None +[baseline s2] step 1960/12000 loss=5.2042 gn=0.62 lr=5.71e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s2] step 1980/12000 loss=5.1854 gn=0.64 lr=5.70e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s2] step 2000/12000 loss=5.1393 gn=0.64 lr=5.70e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 2000: val_loss=5.3100 val_ppl=202.36 +[baseline s2] step 2020/12000 loss=5.2413 gn=0.61 lr=5.69e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s2] step 2040/12000 loss=5.3210 gn=0.64 lr=5.68e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s2] step 2060/12000 loss=5.3837 gn=0.62 lr=5.68e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s2] step 2080/12000 loss=5.3966 gn=0.62 lr=5.67e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s2] step 2100/12000 loss=5.2224 gn=0.62 lr=5.66e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s2] step 2120/12000 loss=5.3651 gn=0.73 lr=5.65e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s2] step 2140/12000 loss=5.1481 gn=0.58 lr=5.65e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s2] step 2160/12000 loss=5.1919 gn=0.58 lr=5.64e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s2] step 2180/12000 loss=5.3271 gn=0.64 lr=5.63e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s2] step 2200/12000 loss=5.0951 gn=0.66 lr=5.63e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2220/12000 loss=5.1905 gn=0.64 lr=5.62e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2240/12000 loss=5.2993 gn=0.63 lr=5.61e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2260/12000 loss=5.1102 gn=0.67 lr=5.60e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2280/12000 loss=5.1446 gn=0.64 lr=5.60e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2300/12000 loss=5.3535 gn=0.63 lr=5.59e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2320/12000 loss=5.3077 gn=0.71 lr=5.58e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s2] step 2340/12000 loss=5.1387 gn=0.67 lr=5.57e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s2] step 2360/12000 loss=5.1461 gn=0.62 lr=5.57e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s2] step 2380/12000 loss=5.1401 gn=0.73 lr=5.56e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s2] step 2400/12000 loss=5.1165 gn=0.64 lr=5.55e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s2] step 2420/12000 loss=5.2999 gn=0.71 lr=5.54e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s2] step 2440/12000 loss=5.1309 gn=0.63 lr=5.53e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s2] step 2460/12000 loss=5.0747 gn=0.63 lr=5.53e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s2] step 2480/12000 loss=5.0640 gn=0.61 lr=5.52e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s2] step 2500/12000 loss=5.1823 gn=0.62 lr=5.51e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s2] step 2520/12000 loss=5.0019 gn=0.64 lr=5.50e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s2] step 2540/12000 loss=5.0956 gn=0.67 lr=5.49e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s2] step 2560/12000 loss=5.1891 gn=0.69 lr=5.48e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s2] step 2580/12000 loss=5.1725 gn=0.69 lr=5.48e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s2] step 2600/12000 loss=5.0546 gn=0.61 lr=5.47e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2620/12000 loss=5.0008 gn=0.67 lr=5.46e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2640/12000 loss=5.1112 gn=0.69 lr=5.45e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2660/12000 loss=5.1628 gn=0.63 lr=5.44e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2680/12000 loss=5.1423 gn=0.65 lr=5.43e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2700/12000 loss=5.1572 gn=0.67 lr=5.42e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s2] step 2720/12000 loss=5.0150 gn=0.67 lr=5.41e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s2] step 2740/12000 loss=5.1006 gn=0.68 lr=5.41e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s2] step 2760/12000 loss=5.0551 gn=0.69 lr=5.40e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s2] step 2780/12000 loss=4.9708 gn=0.71 lr=5.39e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s2] step 2800/12000 loss=5.0075 gn=0.66 lr=5.38e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s2] step 2820/12000 loss=4.9968 gn=0.69 lr=5.37e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s2] step 2840/12000 loss=5.1391 gn=0.65 lr=5.36e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s2] step 2860/12000 loss=4.8885 gn=0.70 lr=5.35e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s2] step 2880/12000 loss=5.0521 gn=0.69 lr=5.34e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 2900/12000 loss=5.0190 gn=0.72 lr=5.33e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 2920/12000 loss=5.0757 gn=0.70 lr=5.32e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 2940/12000 loss=5.1936 gn=0.74 lr=5.31e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 2960/12000 loss=4.9030 gn=0.69 lr=5.30e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 2980/12000 loss=5.0096 gn=0.68 lr=5.29e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] step 3000/12000 loss=4.9161 gn=0.75 lr=5.28e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 3000: val_loss=5.0673 val_ppl=158.74 +[baseline s2] step 3020/12000 loss=5.1060 gn=0.71 lr=5.27e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3040/12000 loss=5.0390 gn=0.72 lr=5.26e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3060/12000 loss=4.9804 gn=0.71 lr=5.25e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3080/12000 loss=4.8768 gn=0.71 lr=5.24e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3100/12000 loss=5.0038 gn=0.68 lr=5.23e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3120/12000 loss=4.7592 gn=0.68 lr=5.22e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3140/12000 loss=4.8967 gn=0.65 lr=5.21e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3160/12000 loss=4.8543 gn=0.75 lr=5.20e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s2] step 3180/12000 loss=4.9785 gn=0.67 lr=5.19e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s2] step 3200/12000 loss=4.9032 gn=0.68 lr=5.18e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s2] step 3220/12000 loss=4.9669 gn=0.71 lr=5.17e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s2] step 3240/12000 loss=5.0290 gn=0.78 lr=5.16e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s2] step 3260/12000 loss=5.1008 gn=0.74 lr=5.15e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s2] step 3280/12000 loss=4.8796 gn=0.74 lr=5.14e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s2] step 3300/12000 loss=4.7714 gn=0.70 lr=5.13e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s2] step 3320/12000 loss=4.9077 gn=0.94 lr=5.12e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s2] step 3340/12000 loss=4.9204 gn=0.69 lr=5.11e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s2] step 3360/12000 loss=4.8516 gn=0.73 lr=5.10e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s2] step 3380/12000 loss=4.9774 gn=0.72 lr=5.09e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s2] step 3400/12000 loss=4.8835 gn=0.71 lr=5.08e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s2] step 3420/12000 loss=5.0099 gn=0.69 lr=5.07e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s2] step 3440/12000 loss=4.9351 gn=0.73 lr=5.06e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s2] step 3460/12000 loss=4.9346 gn=0.72 lr=5.05e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s2] step 3480/12000 loss=4.9543 gn=0.74 lr=5.03e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s2] step 3500/12000 loss=4.8586 gn=0.72 lr=5.02e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s2] step 3520/12000 loss=4.9026 gn=0.71 lr=5.01e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s2] step 3540/12000 loss=4.8745 gn=0.69 lr=5.00e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s2] step 3560/12000 loss=4.8160 gn=0.73 lr=4.99e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s2] step 3580/12000 loss=4.8909 gn=0.68 lr=4.98e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s2] step 3600/12000 loss=4.7622 gn=0.71 lr=4.97e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s2] step 3620/12000 loss=4.8796 gn=0.83 lr=4.96e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s2] step 3640/12000 loss=4.8112 gn=0.74 lr=4.94e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s2] step 3660/12000 loss=4.8813 gn=0.76 lr=4.93e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s2] step 3680/12000 loss=4.9093 gn=0.73 lr=4.92e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s2] step 3700/12000 loss=4.8343 gn=0.83 lr=4.91e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s2] step 3720/12000 loss=4.9926 gn=0.73 lr=4.90e-04 tps=24553 mem=5.4G lam=None Gm=None +[baseline s2] step 3740/12000 loss=4.8789 gn=0.75 lr=4.89e-04 tps=24552 mem=5.4G lam=None Gm=None +[baseline s2] step 3760/12000 loss=4.8164 gn=0.73 lr=4.88e-04 tps=24550 mem=5.4G lam=None Gm=None +[baseline s2] step 3780/12000 loss=4.8229 gn=0.78 lr=4.86e-04 tps=24547 mem=5.4G lam=None Gm=None +[baseline s2] step 3800/12000 loss=4.9285 gn=0.77 lr=4.85e-04 tps=24546 mem=5.4G lam=None Gm=None +[baseline s2] step 3820/12000 loss=4.8501 gn=0.68 lr=4.84e-04 tps=24544 mem=5.4G lam=None Gm=None +[baseline s2] step 3840/12000 loss=5.0163 gn=0.75 lr=4.83e-04 tps=24543 mem=5.4G lam=None Gm=None +[baseline s2] step 3860/12000 loss=4.6614 gn=0.73 lr=4.82e-04 tps=24543 mem=5.4G lam=None Gm=None +[baseline s2] step 3880/12000 loss=4.7731 gn=0.76 lr=4.80e-04 tps=24541 mem=5.4G lam=None Gm=None +[baseline s2] step 3900/12000 loss=4.7752 gn=0.70 lr=4.79e-04 tps=24540 mem=5.4G lam=None Gm=None +[baseline s2] step 3920/12000 loss=4.8330 gn=0.77 lr=4.78e-04 tps=24539 mem=5.4G lam=None Gm=None +[baseline s2] step 3940/12000 loss=4.9468 gn=0.73 lr=4.77e-04 tps=24538 mem=5.4G lam=None Gm=None +[baseline s2] step 3960/12000 loss=4.7281 gn=0.74 lr=4.76e-04 tps=24538 mem=5.4G lam=None Gm=None +[baseline s2] step 3980/12000 loss=4.6554 gn=0.72 lr=4.74e-04 tps=24538 mem=5.4G lam=None Gm=None +[baseline s2] step 4000/12000 loss=4.8807 gn=0.76 lr=4.73e-04 tps=24538 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 4000: val_loss=4.8799 val_ppl=131.62 +[baseline s2] step 4020/12000 loss=4.9156 gn=0.76 lr=4.72e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4040/12000 loss=4.6776 gn=0.73 lr=4.71e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4060/12000 loss=4.7289 gn=0.76 lr=4.70e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4080/12000 loss=4.8534 gn=0.78 lr=4.68e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4100/12000 loss=4.7338 gn=0.78 lr=4.67e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4120/12000 loss=4.9276 gn=0.78 lr=4.66e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4140/12000 loss=4.8401 gn=0.80 lr=4.65e-04 tps=24517 mem=5.4G lam=None Gm=None +[baseline s2] step 4160/12000 loss=4.7581 gn=0.67 lr=4.63e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4180/12000 loss=4.8409 gn=0.71 lr=4.62e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4200/12000 loss=4.6837 gn=0.75 lr=4.61e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4220/12000 loss=4.7246 gn=0.72 lr=4.60e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4240/12000 loss=4.7632 gn=0.70 lr=4.58e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4260/12000 loss=4.6483 gn=0.75 lr=4.57e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4280/12000 loss=4.7232 gn=0.75 lr=4.56e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4300/12000 loss=4.7886 gn=0.83 lr=4.54e-04 tps=24518 mem=5.4G lam=None Gm=None +[baseline s2] step 4320/12000 loss=4.8516 gn=0.75 lr=4.53e-04 tps=24519 mem=5.4G lam=None Gm=None +[baseline s2] step 4340/12000 loss=4.8114 gn=0.76 lr=4.52e-04 tps=24519 mem=5.4G lam=None Gm=None +[baseline s2] step 4360/12000 loss=4.6966 gn=0.75 lr=4.51e-04 tps=24519 mem=5.4G lam=None Gm=None +[baseline s2] step 4380/12000 loss=4.6752 gn=0.78 lr=4.49e-04 tps=24519 mem=5.4G lam=None Gm=None +[baseline s2] step 4400/12000 loss=4.7335 gn=0.87 lr=4.48e-04 tps=24519 mem=5.4G lam=None Gm=None +[baseline s2] step 4420/12000 loss=4.7860 gn=0.78 lr=4.47e-04 tps=24520 mem=5.4G lam=None Gm=None +[baseline s2] step 4440/12000 loss=4.7694 gn=0.73 lr=4.45e-04 tps=24520 mem=5.4G lam=None Gm=None +[baseline s2] step 4460/12000 loss=4.9003 gn=0.79 lr=4.44e-04 tps=24520 mem=5.4G lam=None Gm=None +[baseline s2] step 4480/12000 loss=4.8528 gn=0.78 lr=4.43e-04 tps=24520 mem=5.4G lam=None Gm=None +[baseline s2] step 4500/12000 loss=4.7151 gn=0.77 lr=4.42e-04 tps=24520 mem=5.4G lam=None Gm=None +[baseline s2] step 4520/12000 loss=4.8365 gn=0.83 lr=4.40e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4540/12000 loss=4.7833 gn=0.78 lr=4.39e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4560/12000 loss=4.7831 gn=0.79 lr=4.38e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4580/12000 loss=4.6168 gn=0.80 lr=4.36e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4600/12000 loss=4.6349 gn=0.77 lr=4.35e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4620/12000 loss=4.6786 gn=0.75 lr=4.34e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4640/12000 loss=4.7191 gn=0.83 lr=4.32e-04 tps=24521 mem=5.4G lam=None Gm=None +[baseline s2] step 4660/12000 loss=4.8729 gn=0.82 lr=4.31e-04 tps=24522 mem=5.4G lam=None Gm=None +[baseline s2] step 4680/12000 loss=4.8911 gn=0.85 lr=4.30e-04 tps=24522 mem=5.4G lam=None Gm=None +[baseline s2] step 4700/12000 loss=4.7192 gn=0.80 lr=4.28e-04 tps=24522 mem=5.4G lam=None Gm=None +[baseline s2] step 4720/12000 loss=4.5936 gn=0.97 lr=4.27e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4740/12000 loss=4.9064 gn=0.79 lr=4.26e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4760/12000 loss=4.5381 gn=0.94 lr=4.24e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4780/12000 loss=4.5101 gn=0.74 lr=4.23e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4800/12000 loss=4.7904 gn=0.79 lr=4.22e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4820/12000 loss=4.6938 gn=0.84 lr=4.20e-04 tps=24523 mem=5.4G lam=None Gm=None +[baseline s2] step 4840/12000 loss=4.7655 gn=0.78 lr=4.19e-04 tps=24524 mem=5.4G lam=None Gm=None +[baseline s2] step 4860/12000 loss=4.7910 gn=0.85 lr=4.18e-04 tps=24524 mem=5.4G lam=None Gm=None +[baseline s2] step 4880/12000 loss=4.6310 gn=0.80 lr=4.16e-04 tps=24524 mem=5.4G lam=None Gm=None +[baseline s2] step 4900/12000 loss=4.7144 gn=0.78 lr=4.15e-04 tps=24524 mem=5.4G lam=None Gm=None +[baseline s2] step 4920/12000 loss=4.6795 gn=0.80 lr=4.13e-04 tps=24525 mem=5.4G lam=None Gm=None +[baseline s2] step 4940/12000 loss=4.6651 gn=0.78 lr=4.12e-04 tps=24524 mem=5.4G lam=None Gm=None +[baseline s2] step 4960/12000 loss=4.6630 gn=0.76 lr=4.11e-04 tps=24525 mem=5.4G lam=None Gm=None +[baseline s2] step 4980/12000 loss=4.6364 gn=0.80 lr=4.09e-04 tps=24525 mem=5.4G lam=None Gm=None +[baseline s2] step 5000/12000 loss=4.5577 gn=0.78 lr=4.08e-04 tps=24525 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 5000: val_loss=4.7564 val_ppl=116.32 +[baseline s2] step 5020/12000 loss=4.8210 gn=0.82 lr=4.07e-04 tps=24508 mem=5.4G lam=None Gm=None +[baseline s2] step 5040/12000 loss=4.6326 gn=0.83 lr=4.05e-04 tps=24508 mem=5.4G lam=None Gm=None +[baseline s2] step 5060/12000 loss=4.6473 gn=0.76 lr=4.04e-04 tps=24508 mem=5.4G lam=None Gm=None +[baseline s2] step 5080/12000 loss=4.5698 gn=0.81 lr=4.02e-04 tps=24509 mem=5.4G lam=None Gm=None +[baseline s2] step 5100/12000 loss=4.6713 gn=0.79 lr=4.01e-04 tps=24509 mem=5.4G lam=None Gm=None +[baseline s2] step 5120/12000 loss=4.5064 gn=0.82 lr=4.00e-04 tps=24509 mem=5.4G lam=None Gm=None +[baseline s2] step 5140/12000 loss=4.8275 gn=0.96 lr=3.98e-04 tps=24509 mem=5.4G lam=None Gm=None +[baseline s2] step 5160/12000 loss=4.8611 gn=0.83 lr=3.97e-04 tps=24509 mem=5.4G lam=None Gm=None +[baseline s2] step 5180/12000 loss=4.6400 gn=0.77 lr=3.95e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5200/12000 loss=4.6540 gn=0.81 lr=3.94e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5220/12000 loss=4.7390 gn=0.83 lr=3.93e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5240/12000 loss=4.6094 gn=0.79 lr=3.91e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5260/12000 loss=4.4636 gn=0.88 lr=3.90e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5280/12000 loss=4.8032 gn=0.82 lr=3.88e-04 tps=24510 mem=5.4G lam=None Gm=None +[baseline s2] step 5300/12000 loss=4.7143 gn=0.87 lr=3.87e-04 tps=24511 mem=5.4G lam=None Gm=None +[baseline s2] step 5320/12000 loss=4.7416 gn=0.82 lr=3.86e-04 tps=24511 mem=5.4G lam=None Gm=None +[baseline s2] step 5340/12000 loss=4.4873 gn=0.78 lr=3.84e-04 tps=24511 mem=5.4G lam=None Gm=None +[baseline s2] step 5360/12000 loss=4.7313 gn=0.86 lr=3.83e-04 tps=24511 mem=5.4G lam=None Gm=None +[baseline s2] step 5380/12000 loss=4.5003 gn=0.83 lr=3.81e-04 tps=24512 mem=5.4G lam=None Gm=None +[baseline s2] step 5400/12000 loss=4.5531 gn=0.83 lr=3.80e-04 tps=24512 mem=5.4G lam=None Gm=None +[baseline s2] step 5420/12000 loss=4.7137 gn=0.84 lr=3.79e-04 tps=24512 mem=5.4G lam=None Gm=None +[baseline s2] step 5440/12000 loss=4.5608 gn=0.83 lr=3.77e-04 tps=24512 mem=5.4G lam=None Gm=None +[baseline s2] step 5460/12000 loss=4.7692 gn=0.83 lr=3.76e-04 tps=24513 mem=5.4G lam=None Gm=None +[baseline s2] step 5480/12000 loss=4.5587 gn=0.81 lr=3.74e-04 tps=24513 mem=5.4G lam=None Gm=None +[baseline s2] step 5500/12000 loss=4.8842 gn=0.90 lr=3.73e-04 tps=24513 mem=5.4G lam=None Gm=None +[baseline s2] step 5520/12000 loss=4.6244 gn=0.83 lr=3.72e-04 tps=24513 mem=5.4G lam=None Gm=None +[baseline s2] step 5540/12000 loss=4.5847 gn=0.81 lr=3.70e-04 tps=24514 mem=5.4G lam=None Gm=None +[baseline s2] step 5560/12000 loss=4.6672 gn=0.77 lr=3.69e-04 tps=24514 mem=5.4G lam=None Gm=None +[baseline s2] step 5580/12000 loss=4.5957 gn=0.78 lr=3.67e-04 tps=24514 mem=5.4G lam=None Gm=None +[baseline s2] step 5600/12000 loss=4.7661 gn=0.79 lr=3.66e-04 tps=24514 mem=5.4G lam=None Gm=None +[baseline s2] step 5620/12000 loss=4.6897 gn=0.83 lr=3.64e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5640/12000 loss=4.5408 gn=0.81 lr=3.63e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5660/12000 loss=4.5730 gn=0.83 lr=3.62e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5680/12000 loss=4.6738 gn=0.90 lr=3.60e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5700/12000 loss=4.5760 gn=0.84 lr=3.59e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5720/12000 loss=4.5357 gn=0.83 lr=3.57e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5740/12000 loss=4.5703 gn=0.77 lr=3.56e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5760/12000 loss=4.3541 gn=0.82 lr=3.54e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5780/12000 loss=4.5547 gn=0.85 lr=3.53e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5800/12000 loss=4.6449 gn=0.89 lr=3.52e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5820/12000 loss=4.5985 gn=0.84 lr=3.50e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5840/12000 loss=4.6148 gn=0.81 lr=3.49e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5860/12000 loss=4.6591 gn=0.84 lr=3.47e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5880/12000 loss=4.6547 gn=0.86 lr=3.46e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5900/12000 loss=4.8037 gn=0.84 lr=3.44e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5920/12000 loss=4.6525 gn=0.89 lr=3.43e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5940/12000 loss=4.6162 gn=0.83 lr=3.41e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5960/12000 loss=4.6543 gn=0.79 lr=3.40e-04 tps=24515 mem=5.4G lam=None Gm=None +[baseline s2] step 5980/12000 loss=4.5205 gn=0.89 lr=3.39e-04 tps=24516 mem=5.4G lam=None Gm=None +[baseline s2] step 6000/12000 loss=4.5464 gn=0.80 lr=3.37e-04 tps=24516 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 6000: val_loss=4.6399 val_ppl=103.54 +[baseline s2] step 6020/12000 loss=4.4760 gn=0.80 lr=3.36e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6040/12000 loss=4.6230 gn=0.81 lr=3.34e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6060/12000 loss=4.5241 gn=0.83 lr=3.33e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6080/12000 loss=4.5929 gn=0.82 lr=3.31e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6100/12000 loss=4.5589 gn=0.91 lr=3.30e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6120/12000 loss=4.4627 gn=0.80 lr=3.29e-04 tps=24502 mem=5.4G lam=None Gm=None +[baseline s2] step 6140/12000 loss=4.5531 gn=0.90 lr=3.27e-04 tps=24503 mem=5.4G lam=None Gm=None +[baseline s2] step 6160/12000 loss=4.6352 gn=0.77 lr=3.26e-04 tps=24503 mem=5.4G lam=None Gm=None +[baseline s2] step 6180/12000 loss=4.5381 gn=0.86 lr=3.24e-04 tps=24503 mem=5.4G lam=None Gm=None +[baseline s2] step 6200/12000 loss=4.5956 gn=0.80 lr=3.23e-04 tps=24503 mem=5.4G lam=None Gm=None +[baseline s2] step 6220/12000 loss=4.5126 gn=0.82 lr=3.21e-04 tps=24503 mem=5.4G lam=None Gm=None +[baseline s2] step 6240/12000 loss=4.4595 gn=0.86 lr=3.20e-04 tps=24504 mem=5.4G lam=None Gm=None +[baseline s2] step 6260/12000 loss=4.4504 gn=0.84 lr=3.19e-04 tps=24504 mem=5.4G lam=None Gm=None +[baseline s2] step 6280/12000 loss=4.7457 gn=0.89 lr=3.17e-04 tps=24504 mem=5.4G lam=None Gm=None +[baseline s2] step 6300/12000 loss=4.7300 gn=0.86 lr=3.16e-04 tps=24504 mem=5.4G lam=None Gm=None +[baseline s2] step 6320/12000 loss=4.4945 gn=0.88 lr=3.14e-04 tps=24504 mem=5.4G lam=None Gm=None +[baseline s2] step 6340/12000 loss=4.6298 gn=0.88 lr=3.13e-04 tps=24498 mem=5.4G lam=None Gm=None +[baseline s2] step 6360/12000 loss=4.5112 gn=0.82 lr=3.11e-04 tps=24491 mem=5.4G lam=None Gm=None +[baseline s2] step 6380/12000 loss=4.5595 gn=0.82 lr=3.10e-04 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] step 6400/12000 loss=4.3849 gn=0.90 lr=3.08e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 6420/12000 loss=4.5745 gn=0.90 lr=3.07e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 6440/12000 loss=4.7533 gn=1.00 lr=3.06e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6460/12000 loss=4.5414 gn=0.88 lr=3.04e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6480/12000 loss=4.5867 gn=1.02 lr=3.03e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6500/12000 loss=4.4080 gn=0.88 lr=3.01e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6520/12000 loss=4.6720 gn=0.90 lr=3.00e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6540/12000 loss=4.5919 gn=1.03 lr=2.98e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6560/12000 loss=4.5856 gn=0.89 lr=2.97e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6580/12000 loss=4.4449 gn=0.88 lr=2.96e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 6600/12000 loss=4.6338 gn=0.87 lr=2.94e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6620/12000 loss=4.4757 gn=0.90 lr=2.93e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6640/12000 loss=4.5839 gn=0.93 lr=2.91e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6660/12000 loss=4.6808 gn=0.91 lr=2.90e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6680/12000 loss=4.5170 gn=0.90 lr=2.88e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6700/12000 loss=4.6042 gn=0.85 lr=2.87e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6720/12000 loss=4.6150 gn=0.86 lr=2.86e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6740/12000 loss=4.5812 gn=0.88 lr=2.84e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 6760/12000 loss=4.6048 gn=0.87 lr=2.83e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6780/12000 loss=4.5564 gn=0.83 lr=2.81e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6800/12000 loss=4.5766 gn=0.91 lr=2.80e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6820/12000 loss=4.5402 gn=0.88 lr=2.79e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6840/12000 loss=4.6239 gn=0.82 lr=2.77e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6860/12000 loss=4.5919 gn=0.90 lr=2.76e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6880/12000 loss=4.4787 gn=0.88 lr=2.74e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6900/12000 loss=4.5814 gn=0.96 lr=2.73e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6920/12000 loss=4.4094 gn=0.87 lr=2.72e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6940/12000 loss=4.7680 gn=0.98 lr=2.70e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 6960/12000 loss=4.5283 gn=0.86 lr=2.69e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 6980/12000 loss=4.4810 gn=0.91 lr=2.67e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 7000/12000 loss=4.5807 gn=0.91 lr=2.66e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 7000: val_loss=4.5845 val_ppl=97.95 +[baseline s2] step 7020/12000 loss=4.4681 gn=0.91 lr=2.65e-04 tps=24470 mem=5.4G lam=None Gm=None +[baseline s2] step 7040/12000 loss=4.5863 gn=0.90 lr=2.63e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7060/12000 loss=4.6179 gn=0.90 lr=2.62e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7080/12000 loss=4.4789 gn=0.93 lr=2.60e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7100/12000 loss=4.4992 gn=0.96 lr=2.59e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7120/12000 loss=4.4417 gn=0.91 lr=2.58e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7140/12000 loss=4.5867 gn=0.86 lr=2.56e-04 tps=24471 mem=5.4G lam=None Gm=None +[baseline s2] step 7160/12000 loss=4.4411 gn=0.87 lr=2.55e-04 tps=24472 mem=5.4G lam=None Gm=None +[baseline s2] step 7180/12000 loss=4.4940 gn=0.92 lr=2.53e-04 tps=24472 mem=5.4G lam=None Gm=None +[baseline s2] step 7200/12000 loss=4.5565 gn=0.94 lr=2.52e-04 tps=24472 mem=5.4G lam=None Gm=None +[baseline s2] step 7220/12000 loss=4.5816 gn=0.85 lr=2.51e-04 tps=24472 mem=5.4G lam=None Gm=None +[baseline s2] step 7240/12000 loss=4.3989 gn=0.95 lr=2.49e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 7260/12000 loss=4.5919 gn=0.94 lr=2.48e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 7280/12000 loss=4.4653 gn=0.95 lr=2.47e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 7300/12000 loss=4.4634 gn=0.89 lr=2.45e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 7320/12000 loss=4.6225 gn=0.93 lr=2.44e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 7340/12000 loss=4.3958 gn=0.88 lr=2.42e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 7360/12000 loss=4.3835 gn=0.94 lr=2.41e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 7380/12000 loss=4.5974 gn=0.92 lr=2.40e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 7400/12000 loss=4.4533 gn=0.89 lr=2.38e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 7420/12000 loss=4.6735 gn=0.86 lr=2.37e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 7440/12000 loss=4.4286 gn=0.88 lr=2.36e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 7460/12000 loss=4.5438 gn=0.83 lr=2.34e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 7480/12000 loss=4.5134 gn=0.85 lr=2.33e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 7500/12000 loss=4.4794 gn=0.96 lr=2.32e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 7520/12000 loss=4.3648 gn=0.87 lr=2.30e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 7540/12000 loss=4.4806 gn=0.91 lr=2.29e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 7560/12000 loss=4.5912 gn=0.85 lr=2.28e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 7580/12000 loss=4.5037 gn=0.99 lr=2.26e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 7600/12000 loss=4.4753 gn=0.95 lr=2.25e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 7620/12000 loss=4.7114 gn=0.97 lr=2.24e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 7640/12000 loss=4.4468 gn=0.96 lr=2.22e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 7660/12000 loss=4.3725 gn=0.88 lr=2.21e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 7680/12000 loss=4.3304 gn=0.87 lr=2.20e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 7700/12000 loss=4.4731 gn=0.85 lr=2.18e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 7720/12000 loss=4.5302 gn=0.91 lr=2.17e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 7740/12000 loss=4.4269 gn=0.94 lr=2.16e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 7760/12000 loss=4.5671 gn=0.96 lr=2.15e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 7780/12000 loss=4.6095 gn=1.00 lr=2.13e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 7800/12000 loss=4.5023 gn=0.91 lr=2.12e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 7820/12000 loss=4.4582 gn=0.88 lr=2.11e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 7840/12000 loss=4.3607 gn=0.85 lr=2.09e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 7860/12000 loss=4.4191 gn=0.96 lr=2.08e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 7880/12000 loss=4.4817 gn=1.00 lr=2.07e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 7900/12000 loss=4.4671 gn=0.95 lr=2.06e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 7920/12000 loss=4.5612 gn=0.94 lr=2.04e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 7940/12000 loss=4.4426 gn=0.94 lr=2.03e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 7960/12000 loss=4.5158 gn=0.89 lr=2.02e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 7980/12000 loss=4.4775 gn=0.96 lr=2.00e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8000/12000 loss=4.4443 gn=0.93 lr=1.99e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 8000: val_loss=4.4795 val_ppl=88.19 +[baseline s2] step 8020/12000 loss=4.4939 gn=0.89 lr=1.98e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 8040/12000 loss=4.5203 gn=0.87 lr=1.97e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 8060/12000 loss=4.3950 gn=0.93 lr=1.95e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 8080/12000 loss=4.4551 gn=0.89 lr=1.94e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 8100/12000 loss=4.5019 gn=0.95 lr=1.93e-04 tps=24473 mem=5.4G lam=None Gm=None +[baseline s2] step 8120/12000 loss=4.5365 gn=0.95 lr=1.92e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 8140/12000 loss=4.4458 gn=0.93 lr=1.90e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 8160/12000 loss=4.5658 gn=0.93 lr=1.89e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 8180/12000 loss=4.5512 gn=1.04 lr=1.88e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 8200/12000 loss=4.4998 gn=0.92 lr=1.87e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 8220/12000 loss=4.5214 gn=0.98 lr=1.86e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 8240/12000 loss=4.4199 gn=0.99 lr=1.84e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 8260/12000 loss=4.4742 gn=0.93 lr=1.83e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 8280/12000 loss=4.4538 gn=0.95 lr=1.82e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 8300/12000 loss=4.5002 gn=0.98 lr=1.81e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 8320/12000 loss=4.4406 gn=0.93 lr=1.80e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 8340/12000 loss=4.4753 gn=0.94 lr=1.78e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 8360/12000 loss=4.4817 gn=0.94 lr=1.77e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 8380/12000 loss=4.4455 gn=0.97 lr=1.76e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 8400/12000 loss=4.3891 gn=0.98 lr=1.75e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 8420/12000 loss=4.5914 gn=0.98 lr=1.74e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 8440/12000 loss=4.4744 gn=1.01 lr=1.72e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 8460/12000 loss=4.4161 gn=0.98 lr=1.71e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 8480/12000 loss=4.3666 gn=0.92 lr=1.70e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 8500/12000 loss=4.4056 gn=1.07 lr=1.69e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 8520/12000 loss=4.5156 gn=0.95 lr=1.68e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 8540/12000 loss=4.5933 gn=1.04 lr=1.67e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 8560/12000 loss=4.3166 gn=1.00 lr=1.66e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 8580/12000 loss=4.4901 gn=0.95 lr=1.64e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 8600/12000 loss=4.3776 gn=1.00 lr=1.63e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 8620/12000 loss=4.4049 gn=0.95 lr=1.62e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 8640/12000 loss=4.4045 gn=1.02 lr=1.61e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 8660/12000 loss=4.4600 gn=0.94 lr=1.60e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 8680/12000 loss=4.3228 gn=1.01 lr=1.59e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 8700/12000 loss=4.2025 gn=0.95 lr=1.58e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 8720/12000 loss=4.3865 gn=0.99 lr=1.57e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 8740/12000 loss=4.4301 gn=0.96 lr=1.55e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 8760/12000 loss=4.5345 gn=0.99 lr=1.54e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 8780/12000 loss=4.5318 gn=0.98 lr=1.53e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 8800/12000 loss=4.3483 gn=0.93 lr=1.52e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 8820/12000 loss=4.3682 gn=0.98 lr=1.51e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 8840/12000 loss=4.5127 gn=0.98 lr=1.50e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 8860/12000 loss=4.4020 gn=0.92 lr=1.49e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 8880/12000 loss=4.4031 gn=0.98 lr=1.48e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8900/12000 loss=4.4809 gn=0.94 lr=1.47e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8920/12000 loss=4.3576 gn=0.95 lr=1.46e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8940/12000 loss=4.4390 gn=0.95 lr=1.45e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8960/12000 loss=4.4192 gn=0.95 lr=1.44e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 8980/12000 loss=4.3382 gn=0.93 lr=1.43e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9000/12000 loss=4.3714 gn=0.95 lr=1.42e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 9000: val_loss=4.5084 val_ppl=90.77 +[baseline s2] step 9020/12000 loss=4.3756 gn=1.00 lr=1.41e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 9040/12000 loss=4.5438 gn=0.95 lr=1.40e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 9060/12000 loss=4.4095 gn=0.99 lr=1.39e-04 tps=24474 mem=5.4G lam=None Gm=None +[baseline s2] step 9080/12000 loss=4.3128 gn=0.93 lr=1.38e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 9100/12000 loss=4.4544 gn=1.00 lr=1.37e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 9120/12000 loss=4.3765 gn=1.00 lr=1.36e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 9140/12000 loss=4.3105 gn=0.94 lr=1.35e-04 tps=24475 mem=5.4G lam=None Gm=None +[baseline s2] step 9160/12000 loss=4.4492 gn=0.95 lr=1.34e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9180/12000 loss=4.3707 gn=1.18 lr=1.33e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9200/12000 loss=4.4855 gn=1.08 lr=1.32e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9220/12000 loss=4.3372 gn=1.10 lr=1.31e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9240/12000 loss=4.3630 gn=0.97 lr=1.30e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9260/12000 loss=4.3701 gn=0.97 lr=1.29e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9280/12000 loss=4.2564 gn=0.94 lr=1.28e-04 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 9300/12000 loss=4.4958 gn=1.13 lr=1.27e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 9320/12000 loss=4.3463 gn=1.06 lr=1.26e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 9340/12000 loss=4.3646 gn=1.03 lr=1.25e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 9360/12000 loss=4.1255 gn=0.94 lr=1.24e-04 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 9380/12000 loss=4.5111 gn=1.00 lr=1.23e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 9400/12000 loss=4.4225 gn=0.95 lr=1.22e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 9420/12000 loss=4.3750 gn=1.06 lr=1.21e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 9440/12000 loss=4.2833 gn=0.94 lr=1.20e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 9460/12000 loss=4.4514 gn=1.00 lr=1.19e-04 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 9480/12000 loss=4.2629 gn=1.07 lr=1.19e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 9500/12000 loss=4.4260 gn=1.04 lr=1.18e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 9520/12000 loss=4.2903 gn=1.03 lr=1.17e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 9540/12000 loss=4.2660 gn=1.02 lr=1.16e-04 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 9560/12000 loss=4.3123 gn=0.93 lr=1.15e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 9580/12000 loss=4.2310 gn=0.97 lr=1.14e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 9600/12000 loss=4.3840 gn=1.05 lr=1.13e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 9620/12000 loss=4.4763 gn=0.97 lr=1.12e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 9640/12000 loss=4.4091 gn=0.98 lr=1.12e-04 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 9660/12000 loss=4.2529 gn=0.94 lr=1.11e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 9680/12000 loss=4.2271 gn=1.06 lr=1.10e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 9700/12000 loss=4.2462 gn=1.01 lr=1.09e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 9720/12000 loss=4.2580 gn=1.03 lr=1.08e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 9740/12000 loss=4.3593 gn=1.06 lr=1.07e-04 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 9760/12000 loss=4.2055 gn=0.98 lr=1.07e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 9780/12000 loss=4.3588 gn=1.01 lr=1.06e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 9800/12000 loss=4.4148 gn=1.05 lr=1.05e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 9820/12000 loss=4.4309 gn=1.07 lr=1.04e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 9840/12000 loss=4.1737 gn=1.00 lr=1.03e-04 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 9860/12000 loss=4.4313 gn=1.12 lr=1.03e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9880/12000 loss=4.5246 gn=1.08 lr=1.02e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9900/12000 loss=4.3348 gn=1.03 lr=1.01e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9920/12000 loss=4.3965 gn=0.99 lr=1.00e-04 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9940/12000 loss=4.3446 gn=1.15 lr=9.96e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 9960/12000 loss=4.2145 gn=0.98 lr=9.89e-05 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] step 9980/12000 loss=4.3279 gn=1.00 lr=9.81e-05 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] step 10000/12000 loss=4.2260 gn=1.00 lr=9.74e-05 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 10000: val_loss=4.4160 val_ppl=82.77 +[baseline s2] step 10020/12000 loss=4.5025 gn=1.06 lr=9.67e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 10040/12000 loss=4.5274 gn=1.11 lr=9.59e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 10060/12000 loss=4.5175 gn=1.06 lr=9.52e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 10080/12000 loss=4.3512 gn=1.09 lr=9.45e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 10100/12000 loss=4.4953 gn=1.01 lr=9.38e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10120/12000 loss=4.2978 gn=1.10 lr=9.31e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10140/12000 loss=4.2888 gn=1.06 lr=9.24e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10160/12000 loss=4.3918 gn=1.04 lr=9.18e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10180/12000 loss=4.3930 gn=1.04 lr=9.11e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10200/12000 loss=4.2989 gn=0.98 lr=9.04e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10220/12000 loss=4.3962 gn=1.11 lr=8.98e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10240/12000 loss=4.3197 gn=1.05 lr=8.91e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 10260/12000 loss=4.3405 gn=1.03 lr=8.85e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 10280/12000 loss=4.4689 gn=1.05 lr=8.78e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 10300/12000 loss=4.3150 gn=1.09 lr=8.72e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 10320/12000 loss=4.5014 gn=1.14 lr=8.66e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 10340/12000 loss=4.3076 gn=1.04 lr=8.59e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 10360/12000 loss=4.3357 gn=0.99 lr=8.53e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10380/12000 loss=4.4475 gn=1.14 lr=8.47e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10400/12000 loss=4.4480 gn=1.02 lr=8.41e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10420/12000 loss=4.2670 gn=1.06 lr=8.35e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10440/12000 loss=4.2085 gn=1.01 lr=8.30e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10460/12000 loss=4.3598 gn=1.10 lr=8.24e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10480/12000 loss=4.3828 gn=1.14 lr=8.18e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10500/12000 loss=4.2675 gn=0.95 lr=8.12e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 10520/12000 loss=4.2372 gn=0.99 lr=8.07e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 10540/12000 loss=4.1061 gn=1.06 lr=8.01e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 10560/12000 loss=4.3606 gn=1.01 lr=7.96e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 10580/12000 loss=4.4164 gn=1.12 lr=7.91e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 10600/12000 loss=4.4618 gn=1.12 lr=7.85e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 10620/12000 loss=4.3941 gn=1.06 lr=7.80e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 10640/12000 loss=4.3801 gn=1.01 lr=7.75e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 10660/12000 loss=4.3425 gn=1.04 lr=7.70e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 10680/12000 loss=4.4470 gn=1.05 lr=7.65e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 10700/12000 loss=4.4588 gn=1.12 lr=7.60e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 10720/12000 loss=4.3411 gn=0.98 lr=7.55e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10740/12000 loss=4.2116 gn=0.96 lr=7.50e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10760/12000 loss=4.3222 gn=1.09 lr=7.46e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10780/12000 loss=4.4475 gn=1.05 lr=7.41e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10800/12000 loss=4.3984 gn=1.12 lr=7.37e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10820/12000 loss=4.3278 gn=1.01 lr=7.32e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10840/12000 loss=4.3210 gn=1.12 lr=7.28e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10860/12000 loss=4.3138 gn=1.02 lr=7.23e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10880/12000 loss=4.4470 gn=1.11 lr=7.19e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 10900/12000 loss=4.4933 gn=1.02 lr=7.15e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 10920/12000 loss=4.5340 gn=1.40 lr=7.11e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 10940/12000 loss=4.5428 gn=1.06 lr=7.07e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 10960/12000 loss=4.2972 gn=1.04 lr=7.03e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 10980/12000 loss=4.1743 gn=1.01 lr=6.99e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11000/12000 loss=4.4010 gn=1.13 lr=6.95e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 11000: val_loss=4.4136 val_ppl=82.56 +[baseline s2] step 11020/12000 loss=4.3446 gn=1.18 lr=6.91e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11040/12000 loss=4.2812 gn=1.09 lr=6.88e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11060/12000 loss=4.4026 gn=1.18 lr=6.84e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11080/12000 loss=4.3082 gn=1.10 lr=6.81e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11100/12000 loss=4.4435 gn=1.07 lr=6.77e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11120/12000 loss=4.1808 gn=1.03 lr=6.74e-05 tps=24476 mem=5.4G lam=None Gm=None +[baseline s2] step 11140/12000 loss=4.3441 gn=1.06 lr=6.70e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 11160/12000 loss=4.3614 gn=1.03 lr=6.67e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 11180/12000 loss=4.3473 gn=1.04 lr=6.64e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 11200/12000 loss=4.1937 gn=1.05 lr=6.61e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 11220/12000 loss=4.1452 gn=1.03 lr=6.58e-05 tps=24477 mem=5.4G lam=None Gm=None +[baseline s2] step 11240/12000 loss=4.3399 gn=1.09 lr=6.55e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11260/12000 loss=4.3627 gn=1.15 lr=6.52e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11280/12000 loss=4.3266 gn=1.15 lr=6.49e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11300/12000 loss=4.3646 gn=1.12 lr=6.47e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11320/12000 loss=4.2386 gn=1.08 lr=6.44e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11340/12000 loss=4.4211 gn=1.17 lr=6.42e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11360/12000 loss=4.3221 gn=1.11 lr=6.39e-05 tps=24478 mem=5.4G lam=None Gm=None +[baseline s2] step 11380/12000 loss=4.3463 gn=1.13 lr=6.37e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 11400/12000 loss=4.3828 gn=1.25 lr=6.34e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 11420/12000 loss=4.3420 gn=1.07 lr=6.32e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 11440/12000 loss=4.2111 gn=1.10 lr=6.30e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 11460/12000 loss=4.2808 gn=1.13 lr=6.28e-05 tps=24479 mem=5.4G lam=None Gm=None +[baseline s2] step 11480/12000 loss=4.2532 gn=1.08 lr=6.26e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11500/12000 loss=4.3658 gn=1.06 lr=6.24e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11520/12000 loss=4.4604 gn=1.08 lr=6.22e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11540/12000 loss=4.3404 gn=1.16 lr=6.20e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11560/12000 loss=4.2987 gn=1.13 lr=6.19e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11580/12000 loss=4.5038 gn=1.25 lr=6.17e-05 tps=24480 mem=5.4G lam=None Gm=None +[baseline s2] step 11600/12000 loss=4.2134 gn=1.10 lr=6.15e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11620/12000 loss=4.3272 gn=1.06 lr=6.14e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11640/12000 loss=4.2579 gn=0.99 lr=6.12e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11660/12000 loss=4.2954 gn=1.07 lr=6.11e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11680/12000 loss=4.3198 gn=1.04 lr=6.10e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11700/12000 loss=4.3523 gn=1.42 lr=6.09e-05 tps=24481 mem=5.4G lam=None Gm=None +[baseline s2] step 11720/12000 loss=4.3050 gn=1.04 lr=6.07e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11740/12000 loss=4.3416 gn=1.11 lr=6.06e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11760/12000 loss=4.2325 gn=1.20 lr=6.06e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11780/12000 loss=4.4253 gn=1.13 lr=6.05e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11800/12000 loss=4.3335 gn=1.12 lr=6.04e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11820/12000 loss=4.4911 gn=1.08 lr=6.03e-05 tps=24482 mem=5.4G lam=None Gm=None +[baseline s2] step 11840/12000 loss=4.2704 gn=1.03 lr=6.02e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11860/12000 loss=4.2268 gn=1.12 lr=6.02e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11880/12000 loss=4.3498 gn=1.15 lr=6.01e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11900/12000 loss=4.3060 gn=1.07 lr=6.01e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11920/12000 loss=4.2161 gn=1.09 lr=6.01e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11940/12000 loss=4.2089 gn=1.24 lr=6.00e-05 tps=24483 mem=5.4G lam=None Gm=None +[baseline s2] step 11960/12000 loss=4.3081 gn=1.10 lr=6.00e-05 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] step 11980/12000 loss=4.2962 gn=1.22 lr=6.00e-05 tps=24484 mem=5.4G lam=None Gm=None +[baseline s2] EVAL step 11999: val_loss=4.4108 val_ppl=82.34 +[init] variant=s_only seed=2 params=63.97M +[init] use_S=True use_G=False +[s_only s2] step 0/12000 loss=10.9409 gn=14.18 lr=3.00e-06 tps=8127 mem=5.0G lam=0.12692801654338837 Gm=None +[s_only s2] step 20/12000 loss=9.8878 gn=1.90 lr=6.30e-05 tps=19566 mem=5.5G lam=0.12692837851742902 Gm=None +[s_only s2] step 40/12000 loss=9.0777 gn=1.76 lr=1.23e-04 tps=20318 mem=5.5G lam=0.12686570671697459 Gm=None +[s_only s2] step 60/12000 loss=8.1781 gn=1.52 lr=1.83e-04 tps=20585 mem=5.5G lam=0.1267942531655232 Gm=None +[s_only s2] step 80/12000 loss=7.5601 gn=0.87 lr=2.43e-04 tps=20698 mem=5.5G lam=0.1266943079729875 Gm=None +[s_only s2] step 100/12000 loss=7.2493 gn=0.69 lr=3.03e-04 tps=20789 mem=5.5G lam=0.12660098386307558 Gm=None +[s_only s2] step 120/12000 loss=7.1497 gn=0.62 lr=3.63e-04 tps=20835 mem=5.5G lam=0.1265279327829679 Gm=None +[s_only s2] step 140/12000 loss=7.0324 gn=0.67 lr=4.23e-04 tps=20864 mem=5.5G lam=0.12647979520261288 Gm=None +[s_only s2] step 160/12000 loss=6.7340 gn=0.57 lr=4.83e-04 tps=20898 mem=5.5G lam=0.1264337282627821 Gm=None +[s_only s2] step 180/12000 loss=6.8651 gn=0.70 lr=5.43e-04 tps=20926 mem=5.5G lam=0.12645062804222107 Gm=None +[s_only s2] step 200/12000 loss=6.6360 gn=0.58 lr=6.00e-04 tps=20942 mem=5.5G lam=0.12645902298390865 Gm=None +[s_only s2] step 220/12000 loss=6.6467 gn=0.58 lr=6.00e-04 tps=20955 mem=5.5G lam=0.12642931938171387 Gm=None +[s_only s2] step 240/12000 loss=6.5953 gn=0.52 lr=6.00e-04 tps=20980 mem=5.5G lam=0.12646159219245115 Gm=None +[s_only s2] step 260/12000 loss=6.4275 gn=0.52 lr=6.00e-04 tps=20987 mem=5.5G lam=0.1264669212202231 Gm=None +[s_only s2] step 280/12000 loss=6.5163 gn=0.50 lr=6.00e-04 tps=21002 mem=5.5G lam=0.1264281782011191 Gm=None +[s_only s2] step 300/12000 loss=6.6659 gn=0.49 lr=6.00e-04 tps=21011 mem=5.5G lam=0.12640829725811878 Gm=None +[s_only s2] step 320/12000 loss=6.6135 gn=0.52 lr=6.00e-04 tps=21017 mem=5.5G lam=0.12643759263058504 Gm=None +[s_only s2] step 340/12000 loss=6.4844 gn=0.53 lr=6.00e-04 tps=21026 mem=5.5G lam=0.12648962500194708 Gm=None +[s_only s2] step 360/12000 loss=6.4062 gn=0.54 lr=6.00e-04 tps=21029 mem=5.5G lam=0.12656068118909994 Gm=None +[s_only s2] step 380/12000 loss=6.2789 gn=0.52 lr=6.00e-04 tps=21039 mem=5.5G lam=0.1266042236238718 Gm=None +[s_only s2] step 400/12000 loss=6.4868 gn=0.54 lr=6.00e-04 tps=21040 mem=5.5G lam=0.12663555362572274 Gm=None +[s_only s2] step 420/12000 loss=6.3157 gn=0.55 lr=6.00e-04 tps=21045 mem=5.5G lam=0.12671906097481647 Gm=None +[s_only s2] step 440/12000 loss=6.3087 gn=0.51 lr=5.99e-04 tps=21048 mem=5.5G lam=0.1267982693389058 Gm=None +[s_only s2] step 460/12000 loss=6.2591 gn=0.50 lr=5.99e-04 tps=21054 mem=5.5G lam=0.12690138320128122 Gm=None +[s_only s2] step 480/12000 loss=6.4973 gn=0.49 lr=5.99e-04 tps=21053 mem=5.5G lam=0.12700510739038387 Gm=None +[s_only s2] step 500/12000 loss=6.3397 gn=0.56 lr=5.99e-04 tps=21040 mem=5.5G lam=0.12715404046078524 Gm=None +[s_only s2] step 520/12000 loss=6.3580 gn=0.52 lr=5.99e-04 tps=21047 mem=5.5G lam=0.12734212602178255 Gm=None +[s_only s2] step 540/12000 loss=6.0999 gn=0.49 lr=5.99e-04 tps=21051 mem=5.5G lam=0.12745685689151287 Gm=None +[s_only s2] step 560/12000 loss=6.0617 gn=0.56 lr=5.99e-04 tps=21055 mem=5.5G lam=0.12762061941126981 Gm=None +[s_only s2] step 580/12000 loss=5.9280 gn=0.53 lr=5.99e-04 tps=21057 mem=5.5G lam=0.12780083467562994 Gm=None +[s_only s2] step 600/12000 loss=6.0698 gn=0.53 lr=5.98e-04 tps=21061 mem=5.5G lam=0.12803988282879195 Gm=None +[s_only s2] step 620/12000 loss=6.2320 gn=0.55 lr=5.98e-04 tps=21064 mem=5.5G lam=0.12824469866851965 Gm=None +[s_only s2] step 640/12000 loss=6.1538 gn=0.53 lr=5.98e-04 tps=21067 mem=5.5G lam=0.12845907794932523 Gm=None +[s_only s2] step 660/12000 loss=6.1433 gn=0.53 lr=5.98e-04 tps=21070 mem=5.5G lam=0.1287163874755303 Gm=None +[s_only s2] step 680/12000 loss=6.0798 gn=0.51 lr=5.98e-04 tps=21069 mem=5.5G lam=0.1289759629095594 Gm=None +[s_only s2] step 700/12000 loss=5.9808 gn=0.49 lr=5.98e-04 tps=21072 mem=5.5G lam=0.1292309695854783 Gm=None +[s_only s2] step 720/12000 loss=6.0470 gn=0.67 lr=5.97e-04 tps=21073 mem=5.5G lam=0.12961956827590862 Gm=None +[s_only s2] step 740/12000 loss=6.0230 gn=0.59 lr=5.97e-04 tps=21070 mem=5.5G lam=0.12988734431564808 Gm=None +[s_only s2] step 760/12000 loss=6.0327 gn=0.55 lr=5.97e-04 tps=21071 mem=5.5G lam=0.13020737345019975 Gm=None +[s_only s2] step 780/12000 loss=5.7937 gn=0.50 lr=5.97e-04 tps=21072 mem=5.5G lam=0.13058073415110508 Gm=None +[s_only s2] step 800/12000 loss=5.9079 gn=0.55 lr=5.97e-04 tps=21074 mem=5.5G lam=0.13093745801597834 Gm=None +[s_only s2] step 820/12000 loss=5.8624 gn=0.54 lr=5.96e-04 tps=21073 mem=5.5G lam=0.1313253886376818 Gm=None +[s_only s2] step 840/12000 loss=5.9264 gn=0.50 lr=5.96e-04 tps=21075 mem=5.5G lam=0.1317227923621734 Gm=None +[s_only s2] step 860/12000 loss=5.8980 gn=0.52 lr=5.96e-04 tps=21077 mem=5.5G lam=0.132078694490095 Gm=None +[s_only s2] step 880/12000 loss=5.9966 gn=0.57 lr=5.96e-04 tps=21080 mem=5.5G lam=0.1324861655011773 Gm=None +[s_only s2] step 900/12000 loss=5.7909 gn=0.54 lr=5.95e-04 tps=21080 mem=5.5G lam=0.1329219943533341 Gm=None +[s_only s2] step 920/12000 loss=5.9468 gn=0.54 lr=5.95e-04 tps=21083 mem=5.5G lam=0.13340729009360075 Gm=None +[s_only s2] step 940/12000 loss=5.8571 gn=0.53 lr=5.95e-04 tps=21083 mem=5.5G lam=0.1338638523593545 Gm=None +[s_only s2] step 960/12000 loss=5.7505 gn=0.53 lr=5.94e-04 tps=21080 mem=5.5G lam=0.1343357814475894 Gm=None +[s_only s2] step 980/12000 loss=5.8418 gn=0.55 lr=5.94e-04 tps=21074 mem=5.5G lam=0.13474212369571129 Gm=None +[s_only s2] step 1000/12000 loss=5.7472 gn=0.55 lr=5.94e-04 tps=21065 mem=5.5G lam=0.13519146665930748 Gm=None +[s_only s2] EVAL step 1000: val_loss=5.7436 val_ppl=312.20 +[s_only s2] step 1020/12000 loss=5.6972 gn=0.57 lr=5.94e-04 tps=20991 mem=5.5G lam=0.13562108296900988 Gm=None +[s_only s2] step 1040/12000 loss=5.8324 gn=0.54 lr=5.93e-04 tps=20990 mem=5.5G lam=0.1361229456961155 Gm=None +[s_only s2] step 1060/12000 loss=5.6892 gn=0.71 lr=5.93e-04 tps=20990 mem=5.5G lam=0.1366096499065558 Gm=None +[s_only s2] step 1080/12000 loss=5.7767 gn=0.55 lr=5.93e-04 tps=20985 mem=5.5G lam=0.13713131193071604 Gm=None +[s_only s2] step 1100/12000 loss=5.6259 gn=0.54 lr=5.92e-04 tps=20983 mem=5.5G lam=0.1376005799199144 Gm=None +[s_only s2] step 1120/12000 loss=5.6984 gn=0.55 lr=5.92e-04 tps=20981 mem=5.5G lam=0.13808964751660824 Gm=None +[s_only s2] step 1140/12000 loss=5.7655 gn=0.62 lr=5.92e-04 tps=20976 mem=5.5G lam=0.1385601075987021 Gm=None +[s_only s2] step 1160/12000 loss=5.6847 gn=0.53 lr=5.91e-04 tps=20979 mem=5.5G lam=0.13905498230208954 Gm=None +[s_only s2] step 1180/12000 loss=5.5205 gn=0.52 lr=5.91e-04 tps=20971 mem=5.5G lam=0.1395608065649867 Gm=None +[s_only s2] step 1200/12000 loss=5.5616 gn=0.53 lr=5.90e-04 tps=20973 mem=5.5G lam=0.14009722167005143 Gm=None +[s_only s2] step 1220/12000 loss=5.6232 gn=0.51 lr=5.90e-04 tps=20975 mem=5.5G lam=0.14061182737350464 Gm=None +[s_only s2] step 1240/12000 loss=5.5668 gn=0.53 lr=5.90e-04 tps=20977 mem=5.5G lam=0.14108367574711642 Gm=None +[s_only s2] step 1260/12000 loss=5.4882 gn=0.54 lr=5.89e-04 tps=20977 mem=5.5G lam=0.14163944280395904 Gm=None +[s_only s2] step 1280/12000 loss=5.4828 gn=0.69 lr=5.89e-04 tps=20979 mem=5.5G lam=0.14213442026327053 Gm=None +[s_only s2] step 1300/12000 loss=5.6276 gn=0.55 lr=5.89e-04 tps=20980 mem=5.5G lam=0.14264978220065436 Gm=None +[s_only s2] step 1320/12000 loss=5.4835 gn=0.64 lr=5.88e-04 tps=20981 mem=5.5G lam=0.14318145190676054 Gm=None +[s_only s2] step 1340/12000 loss=5.6784 gn=0.57 lr=5.88e-04 tps=20982 mem=5.5G lam=0.14363846431175867 Gm=None +[s_only s2] step 1360/12000 loss=5.5024 gn=0.59 lr=5.87e-04 tps=20984 mem=5.5G lam=0.14412274025380611 Gm=None +[s_only s2] step 1380/12000 loss=5.5042 gn=0.57 lr=5.87e-04 tps=20986 mem=5.5G lam=0.14457225458075604 Gm=None +[s_only s2] step 1400/12000 loss=5.4176 gn=0.59 lr=5.86e-04 tps=20988 mem=5.5G lam=0.1451071264843146 Gm=None +[s_only s2] step 1420/12000 loss=5.4803 gn=0.64 lr=5.86e-04 tps=20990 mem=5.5G lam=0.14556187018752098 Gm=None +[s_only s2] step 1440/12000 loss=5.5241 gn=0.57 lr=5.85e-04 tps=20992 mem=5.5G lam=0.1460501728579402 Gm=None +[s_only s2] step 1460/12000 loss=5.4356 gn=0.59 lr=5.85e-04 tps=20994 mem=5.5G lam=0.14660542582472166 Gm=None +[s_only s2] step 1480/12000 loss=5.3876 gn=0.58 lr=5.84e-04 tps=20997 mem=5.5G lam=0.14709274005144835 Gm=None +[s_only s2] step 1500/12000 loss=5.6475 gn=0.58 lr=5.84e-04 tps=20998 mem=5.5G lam=0.147624634206295 Gm=None +[s_only s2] step 1520/12000 loss=5.4229 gn=0.61 lr=5.83e-04 tps=21000 mem=5.5G lam=0.14811959645400444 Gm=None +[s_only s2] step 1540/12000 loss=5.4234 gn=0.55 lr=5.83e-04 tps=21001 mem=5.5G lam=0.14862913514177004 Gm=None +[s_only s2] step 1560/12000 loss=5.6252 gn=0.58 lr=5.82e-04 tps=21003 mem=5.5G lam=0.14908674856026968 Gm=None +[s_only s2] step 1580/12000 loss=5.5253 gn=0.56 lr=5.82e-04 tps=21006 mem=5.5G lam=0.1496563327188293 Gm=None +[s_only s2] step 1600/12000 loss=5.4266 gn=0.60 lr=5.81e-04 tps=21007 mem=5.5G lam=0.15021418469647566 Gm=None +[s_only s2] step 1620/12000 loss=5.3972 gn=0.56 lr=5.81e-04 tps=21009 mem=5.5G lam=0.1507384612535437 Gm=None +[s_only s2] step 1640/12000 loss=5.4671 gn=0.57 lr=5.80e-04 tps=21011 mem=5.5G lam=0.15124856370190778 Gm=None +[s_only s2] step 1660/12000 loss=5.1833 gn=0.63 lr=5.80e-04 tps=21012 mem=5.5G lam=0.15179214936991534 Gm=None +[s_only s2] step 1680/12000 loss=5.4532 gn=0.55 lr=5.79e-04 tps=21014 mem=5.5G lam=0.15230250606934229 Gm=None +[s_only s2] step 1700/12000 loss=5.5286 gn=0.60 lr=5.79e-04 tps=21017 mem=5.5G lam=0.152803473174572 Gm=None +[s_only s2] step 1720/12000 loss=5.3298 gn=0.60 lr=5.78e-04 tps=21019 mem=5.5G lam=0.1533876716469725 Gm=None +[s_only s2] step 1740/12000 loss=5.3847 gn=0.58 lr=5.78e-04 tps=21018 mem=5.5G lam=0.15392273850739002 Gm=None +[s_only s2] step 1760/12000 loss=5.2585 gn=0.60 lr=5.77e-04 tps=21020 mem=5.5G lam=0.15444788864503303 Gm=None +[s_only s2] step 1780/12000 loss=5.2923 gn=0.62 lr=5.76e-04 tps=21021 mem=5.5G lam=0.15498756027470031 Gm=None +[s_only s2] step 1800/12000 loss=5.4629 gn=0.64 lr=5.76e-04 tps=21023 mem=5.5G lam=0.1555917477235198 Gm=None +[s_only s2] step 1820/12000 loss=5.4925 gn=0.59 lr=5.75e-04 tps=21025 mem=5.5G lam=0.1561817629262805 Gm=None +[s_only s2] step 1840/12000 loss=5.4388 gn=0.60 lr=5.75e-04 tps=21026 mem=5.5G lam=0.15670432150363922 Gm=None +[s_only s2] step 1860/12000 loss=5.4345 gn=0.62 lr=5.74e-04 tps=21027 mem=5.5G lam=0.15726616761336723 Gm=None +[s_only s2] step 1880/12000 loss=5.2145 gn=0.60 lr=5.73e-04 tps=21028 mem=5.5G lam=0.15779087785631418 Gm=None +[s_only s2] step 1900/12000 loss=5.2557 gn=0.60 lr=5.73e-04 tps=21029 mem=5.5G lam=0.15831719214717546 Gm=None +[s_only s2] step 1920/12000 loss=5.4326 gn=0.58 lr=5.72e-04 tps=21030 mem=5.5G lam=0.15885187809666 Gm=None +[s_only s2] step 1940/12000 loss=5.5683 gn=0.72 lr=5.72e-04 tps=21030 mem=5.5G lam=0.15938908948252598 Gm=None +[s_only s2] step 1960/12000 loss=5.1853 gn=0.55 lr=5.71e-04 tps=21031 mem=5.5G lam=0.15991612430661917 Gm=None +[s_only s2] step 1980/12000 loss=5.1713 gn=0.59 lr=5.70e-04 tps=21033 mem=5.5G lam=0.16047911842664084 Gm=None +[s_only s2] step 2000/12000 loss=5.1310 gn=0.63 lr=5.70e-04 tps=21034 mem=5.5G lam=0.16100293211638927 Gm=None +[s_only s2] EVAL step 2000: val_loss=5.3019 val_ppl=200.73 +[s_only s2] step 2020/12000 loss=5.2197 gn=0.65 lr=5.69e-04 tps=20997 mem=5.5G lam=0.1615820483615001 Gm=None +[s_only s2] step 2040/12000 loss=5.3244 gn=0.63 lr=5.68e-04 tps=20999 mem=5.5G lam=0.16213112759093443 Gm=None +[s_only s2] step 2060/12000 loss=5.3826 gn=0.64 lr=5.68e-04 tps=21000 mem=5.5G lam=0.16268985780576864 Gm=None +[s_only s2] step 2080/12000 loss=5.4054 gn=0.64 lr=5.67e-04 tps=21000 mem=5.5G lam=0.1632603518664837 Gm=None +[s_only s2] step 2100/12000 loss=5.2131 gn=0.66 lr=5.66e-04 tps=21000 mem=5.5G lam=0.16379532249023518 Gm=None +[s_only s2] step 2120/12000 loss=5.3849 gn=0.74 lr=5.65e-04 tps=21001 mem=5.5G lam=0.16432241164147854 Gm=None +[s_only s2] step 2140/12000 loss=5.1461 gn=0.60 lr=5.65e-04 tps=21003 mem=5.5G lam=0.16498092034210762 Gm=None +[s_only s2] step 2160/12000 loss=5.1845 gn=0.56 lr=5.64e-04 tps=21004 mem=5.5G lam=0.16553983620057502 Gm=None +[s_only s2] step 2180/12000 loss=5.3137 gn=0.64 lr=5.63e-04 tps=21006 mem=5.5G lam=0.16605860833078623 Gm=None +[s_only s2] step 2200/12000 loss=5.0871 gn=0.67 lr=5.63e-04 tps=21006 mem=5.5G lam=0.16659335481623808 Gm=None +[s_only s2] step 2220/12000 loss=5.2001 gn=0.63 lr=5.62e-04 tps=21008 mem=5.5G lam=0.1671676750605305 Gm=None +[s_only s2] step 2240/12000 loss=5.3065 gn=0.65 lr=5.61e-04 tps=21009 mem=5.5G lam=0.1677684352422754 Gm=None +[s_only s2] step 2260/12000 loss=5.0935 gn=0.68 lr=5.60e-04 tps=21011 mem=5.5G lam=0.16831384226679802 Gm=None +[s_only s2] step 2280/12000 loss=5.1523 gn=0.65 lr=5.60e-04 tps=21012 mem=5.5G lam=0.16890539011607567 Gm=None +[s_only s2] step 2300/12000 loss=5.3405 gn=0.62 lr=5.59e-04 tps=21013 mem=5.5G lam=0.16950044532616934 Gm=None +[s_only s2] step 2320/12000 loss=5.2990 gn=0.67 lr=5.58e-04 tps=21014 mem=5.5G lam=0.17006774774442115 Gm=None +[s_only s2] step 2340/12000 loss=5.1318 gn=0.67 lr=5.57e-04 tps=21015 mem=5.5G lam=0.17066771630197763 Gm=None +[s_only s2] step 2360/12000 loss=5.1349 gn=0.61 lr=5.57e-04 tps=21016 mem=5.5G lam=0.17124903885026774 Gm=None +[s_only s2] step 2380/12000 loss=5.1133 gn=0.75 lr=5.56e-04 tps=21016 mem=5.5G lam=0.17179550509899855 Gm=None +[s_only s2] step 2400/12000 loss=5.1177 gn=0.65 lr=5.55e-04 tps=21018 mem=5.5G lam=0.17235980027665695 Gm=None +[s_only s2] step 2420/12000 loss=5.2629 gn=0.71 lr=5.54e-04 tps=21019 mem=5.5G lam=0.17294246703386307 Gm=None +[s_only s2] step 2440/12000 loss=5.1383 gn=0.61 lr=5.53e-04 tps=21020 mem=5.5G lam=0.17352001337955394 Gm=None +[s_only s2] step 2460/12000 loss=5.0694 gn=0.63 lr=5.53e-04 tps=21020 mem=5.5G lam=0.17409420013427734 Gm=None +[s_only s2] step 2480/12000 loss=5.0616 gn=0.62 lr=5.52e-04 tps=21021 mem=5.5G lam=0.1747010381271442 Gm=None +[s_only s2] step 2500/12000 loss=5.1712 gn=0.68 lr=5.51e-04 tps=21020 mem=5.5G lam=0.17530828590194383 Gm=None +[s_only s2] step 2520/12000 loss=5.0002 gn=0.64 lr=5.50e-04 tps=21021 mem=5.5G lam=0.17590311573197445 Gm=None +[s_only s2] step 2540/12000 loss=5.0820 gn=0.65 lr=5.49e-04 tps=21022 mem=5.5G lam=0.17645126394927502 Gm=None +[s_only s2] step 2560/12000 loss=5.1879 gn=0.72 lr=5.48e-04 tps=21023 mem=5.5G lam=0.177005589629213 Gm=None +[s_only s2] step 2580/12000 loss=5.1718 gn=0.70 lr=5.48e-04 tps=21024 mem=5.5G lam=0.17756234792371592 Gm=None +[s_only s2] step 2600/12000 loss=5.0597 gn=0.63 lr=5.47e-04 tps=21025 mem=5.5G lam=0.17819232710947594 Gm=None +[s_only s2] step 2620/12000 loss=4.9844 gn=0.65 lr=5.46e-04 tps=21025 mem=5.5G lam=0.17872155426690975 Gm=None +[s_only s2] step 2640/12000 loss=5.1047 gn=0.66 lr=5.45e-04 tps=21027 mem=5.5G lam=0.17935810051858425 Gm=None +[s_only s2] step 2660/12000 loss=5.1570 gn=0.67 lr=5.44e-04 tps=21028 mem=5.5G lam=0.17990141020466885 Gm=None +[s_only s2] step 2680/12000 loss=5.1196 gn=0.79 lr=5.43e-04 tps=21030 mem=5.5G lam=0.18049528046200672 Gm=None +[s_only s2] step 2700/12000 loss=5.1473 gn=0.71 lr=5.42e-04 tps=21031 mem=5.5G lam=0.1810732992986838 Gm=None +[s_only s2] step 2720/12000 loss=5.0116 gn=0.69 lr=5.41e-04 tps=21032 mem=5.5G lam=0.18164366949349642 Gm=None +[s_only s2] step 2740/12000 loss=5.0951 gn=0.69 lr=5.41e-04 tps=21033 mem=5.5G lam=0.1821500655884544 Gm=None +[s_only s2] step 2760/12000 loss=5.0446 gn=0.72 lr=5.40e-04 tps=21034 mem=5.5G lam=0.1827440056949854 Gm=None +[s_only s2] step 2780/12000 loss=4.9542 gn=0.63 lr=5.39e-04 tps=21034 mem=5.5G lam=0.18333740656574568 Gm=None +[s_only s2] step 2800/12000 loss=5.0076 gn=0.68 lr=5.38e-04 tps=21035 mem=5.5G lam=0.18387206022938093 Gm=None +[s_only s2] step 2820/12000 loss=4.9958 gn=0.70 lr=5.37e-04 tps=21032 mem=5.5G lam=0.1844573626294732 Gm=None +[s_only s2] step 2840/12000 loss=5.1228 gn=0.67 lr=5.36e-04 tps=21031 mem=5.5G lam=0.18507957799981037 Gm=None +[s_only s2] step 2860/12000 loss=4.8876 gn=0.65 lr=5.35e-04 tps=21031 mem=5.5G lam=0.18568130830923715 Gm=None +[s_only s2] step 2880/12000 loss=5.0341 gn=0.68 lr=5.34e-04 tps=21033 mem=5.5G lam=0.18624555133283138 Gm=None +[s_only s2] step 2900/12000 loss=5.0157 gn=0.73 lr=5.33e-04 tps=21034 mem=5.5G lam=0.18677870370447636 Gm=None +[s_only s2] step 2920/12000 loss=5.0657 gn=0.67 lr=5.32e-04 tps=21034 mem=5.5G lam=0.18737626199920973 Gm=None +[s_only s2] step 2940/12000 loss=5.1668 gn=0.69 lr=5.31e-04 tps=21035 mem=5.5G lam=0.18794211776306233 Gm=None +[s_only s2] step 2960/12000 loss=4.9012 gn=0.71 lr=5.30e-04 tps=21036 mem=5.5G lam=0.1885195386906465 Gm=None +[s_only s2] step 2980/12000 loss=4.9894 gn=0.66 lr=5.29e-04 tps=21037 mem=5.5G lam=0.18904351287831864 Gm=None +[s_only s2] step 3000/12000 loss=4.8917 gn=0.69 lr=5.28e-04 tps=21038 mem=5.5G lam=0.18962034489959478 Gm=None +[s_only s2] EVAL step 3000: val_loss=5.0579 val_ppl=157.26 +[s_only s2] step 3020/12000 loss=5.0862 gn=0.72 lr=5.27e-04 tps=21014 mem=5.5G lam=0.1902163171519836 Gm=None +[s_only s2] step 3040/12000 loss=5.0385 gn=0.72 lr=5.26e-04 tps=21014 mem=5.5G lam=0.19070918609698614 Gm=None +[s_only s2] step 3060/12000 loss=4.9726 gn=0.71 lr=5.25e-04 tps=21016 mem=5.5G lam=0.19126395508646965 Gm=None +[s_only s2] step 3080/12000 loss=4.8679 gn=0.70 lr=5.24e-04 tps=21016 mem=5.5G lam=0.1918899022663633 Gm=None +[s_only s2] step 3100/12000 loss=4.9908 gn=0.67 lr=5.23e-04 tps=21017 mem=5.5G lam=0.19249269304176173 Gm=None +[s_only s2] step 3120/12000 loss=4.7421 gn=0.75 lr=5.22e-04 tps=21017 mem=5.5G lam=0.1930630775168538 Gm=None +[s_only s2] step 3140/12000 loss=4.8803 gn=0.65 lr=5.21e-04 tps=21018 mem=5.5G lam=0.19371144690861306 Gm=None +[s_only s2] step 3160/12000 loss=4.8600 gn=0.70 lr=5.20e-04 tps=21018 mem=5.5G lam=0.19428702723234892 Gm=None +[s_only s2] step 3180/12000 loss=4.9482 gn=0.63 lr=5.19e-04 tps=21019 mem=5.5G lam=0.1949081982796391 Gm=None +[s_only s2] step 3200/12000 loss=4.8904 gn=0.66 lr=5.18e-04 tps=21019 mem=5.5G lam=0.19549890824904045 Gm=None +[s_only s2] step 3220/12000 loss=4.9621 gn=0.72 lr=5.17e-04 tps=21020 mem=5.5G lam=0.19612916931509972 Gm=None +[s_only s2] step 3240/12000 loss=5.0179 gn=0.74 lr=5.16e-04 tps=21020 mem=5.5G lam=0.19662492039302984 Gm=None +[s_only s2] step 3260/12000 loss=5.0713 gn=0.70 lr=5.15e-04 tps=21021 mem=5.5G lam=0.19715873307238022 Gm=None +[s_only s2] step 3280/12000 loss=4.8717 gn=0.84 lr=5.14e-04 tps=21019 mem=5.5G lam=0.19774328420559564 Gm=None +[s_only s2] step 3300/12000 loss=4.7735 gn=0.75 lr=5.13e-04 tps=21020 mem=5.5G lam=0.19831778233249983 Gm=None +[s_only s2] step 3320/12000 loss=4.8820 gn=0.92 lr=5.12e-04 tps=21021 mem=5.5G lam=0.19882962697496018 Gm=None +[s_only s2] step 3340/12000 loss=4.9176 gn=0.73 lr=5.11e-04 tps=21022 mem=5.5G lam=0.19934438013782105 Gm=None +[s_only s2] step 3360/12000 loss=4.8560 gn=0.74 lr=5.10e-04 tps=21022 mem=5.5G lam=0.19995553512126207 Gm=None +[s_only s2] step 3380/12000 loss=4.9874 gn=0.72 lr=5.09e-04 tps=21022 mem=5.5G lam=0.20045332113901773 Gm=None +[s_only s2] step 3400/12000 loss=4.8939 gn=0.77 lr=5.08e-04 tps=21023 mem=5.5G lam=0.20099328427265087 Gm=None +[s_only s2] step 3420/12000 loss=5.0178 gn=0.69 lr=5.07e-04 tps=21024 mem=5.5G lam=0.2015475214769443 Gm=None +[s_only s2] step 3440/12000 loss=4.9290 gn=0.77 lr=5.06e-04 tps=21025 mem=5.5G lam=0.20206240937113762 Gm=None +[s_only s2] step 3460/12000 loss=4.9359 gn=0.70 lr=5.05e-04 tps=21025 mem=5.5G lam=0.20268456917256117 Gm=None +[s_only s2] step 3480/12000 loss=4.9550 gn=0.72 lr=5.03e-04 tps=21026 mem=5.5G lam=0.20332708023488522 Gm=None +[s_only s2] step 3500/12000 loss=4.8538 gn=0.71 lr=5.02e-04 tps=21027 mem=5.5G lam=0.20391590365519127 Gm=None +[s_only s2] step 3520/12000 loss=4.9069 gn=0.72 lr=5.01e-04 tps=21027 mem=5.5G lam=0.20442970904211202 Gm=None +[s_only s2] step 3540/12000 loss=4.8764 gn=0.72 lr=5.00e-04 tps=21028 mem=5.5G lam=0.2050004784638683 Gm=None +[s_only s2] step 3560/12000 loss=4.8175 gn=0.76 lr=4.99e-04 tps=21029 mem=5.5G lam=0.20555343634138504 Gm=None +[s_only s2] step 3580/12000 loss=4.8992 gn=0.70 lr=4.98e-04 tps=21030 mem=5.5G lam=0.20615917971978584 Gm=None +[s_only s2] step 3600/12000 loss=4.7644 gn=0.69 lr=4.97e-04 tps=21031 mem=5.5G lam=0.20671730612715086 Gm=None +[s_only s2] step 3620/12000 loss=4.8802 gn=0.85 lr=4.96e-04 tps=21032 mem=5.5G lam=0.2072356352582574 Gm=None +[s_only s2] step 3640/12000 loss=4.8161 gn=0.80 lr=4.94e-04 tps=21033 mem=5.5G lam=0.20776245091110468 Gm=None +[s_only s2] step 3660/12000 loss=4.8753 gn=0.76 lr=4.93e-04 tps=21034 mem=5.5G lam=0.2083150368804733 Gm=None +[s_only s2] step 3680/12000 loss=4.9047 gn=0.80 lr=4.92e-04 tps=21034 mem=5.5G lam=0.20880986719081798 Gm=None +[s_only s2] step 3700/12000 loss=4.8082 gn=0.80 lr=4.91e-04 tps=21035 mem=5.5G lam=0.2093805857002735 Gm=None +[s_only s2] step 3720/12000 loss=4.9947 gn=0.74 lr=4.90e-04 tps=21036 mem=5.5G lam=0.20986641198396683 Gm=None +[s_only s2] step 3740/12000 loss=4.8793 gn=0.78 lr=4.89e-04 tps=21036 mem=5.5G lam=0.210326025262475 Gm=None +[s_only s2] step 3760/12000 loss=4.8135 gn=0.80 lr=4.88e-04 tps=21037 mem=5.5G lam=0.21089446265250444 Gm=None +[s_only s2] step 3780/12000 loss=4.7994 gn=0.76 lr=4.86e-04 tps=21037 mem=5.5G lam=0.21149151865392923 Gm=None +[s_only s2] step 3800/12000 loss=4.9176 gn=0.72 lr=4.85e-04 tps=21037 mem=5.5G lam=0.2120856357117494 Gm=None +[s_only s2] step 3820/12000 loss=4.8636 gn=0.72 lr=4.84e-04 tps=21037 mem=5.5G lam=0.21253252495080233 Gm=None +[s_only s2] step 3840/12000 loss=5.0157 gn=0.77 lr=4.83e-04 tps=21038 mem=5.5G lam=0.2130688981463512 Gm=None +[s_only s2] step 3860/12000 loss=4.6536 gn=0.76 lr=4.82e-04 tps=21039 mem=5.5G lam=0.21374133291343847 Gm=None +[s_only s2] step 3880/12000 loss=4.7757 gn=0.76 lr=4.80e-04 tps=21039 mem=5.5G lam=0.21432387673606476 Gm=None +[s_only s2] step 3900/12000 loss=4.7757 gn=0.69 lr=4.79e-04 tps=21040 mem=5.5G lam=0.21485952070603767 Gm=None +[s_only s2] step 3920/12000 loss=4.8344 gn=0.75 lr=4.78e-04 tps=21040 mem=5.5G lam=0.21542129199951887 Gm=None +[s_only s2] step 3940/12000 loss=4.9616 gn=0.73 lr=4.77e-04 tps=21041 mem=5.5G lam=0.21584463367859522 Gm=None +[s_only s2] step 3960/12000 loss=4.7114 gn=0.73 lr=4.76e-04 tps=21041 mem=5.5G lam=0.21637693171699843 Gm=None +[s_only s2] step 3980/12000 loss=4.6547 gn=0.79 lr=4.74e-04 tps=21042 mem=5.5G lam=0.21690085747589669 Gm=None +[s_only s2] step 4000/12000 loss=4.8692 gn=0.76 lr=4.73e-04 tps=21042 mem=5.5G lam=0.21754630841314793 Gm=None +[s_only s2] EVAL step 4000: val_loss=4.8798 val_ppl=131.60 +[s_only s2] step 4020/12000 loss=4.9070 gn=0.75 lr=4.72e-04 tps=21024 mem=5.5G lam=0.21807953622192144 Gm=None +[s_only s2] step 4040/12000 loss=4.6684 gn=0.76 lr=4.71e-04 tps=21022 mem=5.5G lam=0.21866723832984766 Gm=None +[s_only s2] step 4060/12000 loss=4.7220 gn=0.72 lr=4.70e-04 tps=21023 mem=5.5G lam=0.2192204132055243 Gm=None +[s_only s2] step 4080/12000 loss=4.8480 gn=0.83 lr=4.68e-04 tps=21023 mem=5.5G lam=0.21977955009788275 Gm=None +[s_only s2] step 4100/12000 loss=4.7337 gn=0.75 lr=4.67e-04 tps=21023 mem=5.5G lam=0.22024597072352967 Gm=None +[s_only s2] step 4120/12000 loss=4.9463 gn=0.80 lr=4.66e-04 tps=21024 mem=5.5G lam=0.22079447687913975 Gm=None +[s_only s2] step 4140/12000 loss=4.8453 gn=0.83 lr=4.65e-04 tps=21024 mem=5.5G lam=0.22136733680963516 Gm=None +[s_only s2] step 4160/12000 loss=4.7554 gn=0.70 lr=4.63e-04 tps=21025 mem=5.5G lam=0.22182494215667248 Gm=None +[s_only s2] step 4180/12000 loss=4.8287 gn=0.73 lr=4.62e-04 tps=21026 mem=5.5G lam=0.22229902384181818 Gm=None +[s_only s2] step 4200/12000 loss=4.6911 gn=0.76 lr=4.61e-04 tps=21026 mem=5.5G lam=0.22284341820826134 Gm=None +[s_only s2] step 4220/12000 loss=4.7184 gn=0.74 lr=4.60e-04 tps=21027 mem=5.5G lam=0.22335255021850267 Gm=None +[s_only s2] step 4240/12000 loss=4.7465 gn=0.73 lr=4.58e-04 tps=21027 mem=5.5G lam=0.2238574189444383 Gm=None +[s_only s2] step 4260/12000 loss=4.6283 gn=0.76 lr=4.57e-04 tps=21028 mem=5.5G lam=0.22441158692042032 Gm=None +[s_only s2] step 4280/12000 loss=4.7371 gn=0.73 lr=4.56e-04 tps=21028 mem=5.5G lam=0.22498197760432959 Gm=None +[s_only s2] step 4300/12000 loss=4.7816 gn=0.85 lr=4.54e-04 tps=21028 mem=5.5G lam=0.22546239228298268 Gm=None +[s_only s2] step 4320/12000 loss=4.8514 gn=0.82 lr=4.53e-04 tps=21028 mem=5.5G lam=0.2260184483602643 Gm=None +[s_only s2] step 4340/12000 loss=4.8134 gn=0.77 lr=4.52e-04 tps=21029 mem=5.5G lam=0.22659847171356282 Gm=None +[s_only s2] step 4360/12000 loss=4.6826 gn=0.78 lr=4.51e-04 tps=21029 mem=5.5G lam=0.2270673637588819 Gm=None +[s_only s2] step 4380/12000 loss=4.6772 gn=0.77 lr=4.49e-04 tps=21029 mem=5.5G lam=0.22754385291288295 Gm=None +[s_only s2] step 4400/12000 loss=4.7273 gn=0.86 lr=4.48e-04 tps=21030 mem=5.5G lam=0.2280224555482467 Gm=None +[s_only s2] step 4420/12000 loss=4.7704 gn=0.75 lr=4.47e-04 tps=21031 mem=5.5G lam=0.22852246773739657 Gm=None +[s_only s2] step 4440/12000 loss=4.7524 gn=0.78 lr=4.45e-04 tps=21031 mem=5.5G lam=0.22891566498825947 Gm=None +[s_only s2] step 4460/12000 loss=4.8988 gn=0.79 lr=4.44e-04 tps=21031 mem=5.5G lam=0.22933202485243478 Gm=None +[s_only s2] step 4480/12000 loss=4.8488 gn=0.78 lr=4.43e-04 tps=21032 mem=5.5G lam=0.22984327748417854 Gm=None +[s_only s2] step 4500/12000 loss=4.6877 gn=0.72 lr=4.42e-04 tps=21032 mem=5.5G lam=0.23033169532815614 Gm=None +[s_only s2] step 4520/12000 loss=4.8235 gn=0.82 lr=4.40e-04 tps=21033 mem=5.5G lam=0.23096610171099505 Gm=None +[s_only s2] step 4540/12000 loss=4.7711 gn=0.83 lr=4.39e-04 tps=21033 mem=5.5G lam=0.23148034668217102 Gm=None +[s_only s2] step 4560/12000 loss=4.7958 gn=0.80 lr=4.38e-04 tps=21033 mem=5.5G lam=0.23198267072439194 Gm=None +[s_only s2] step 4580/12000 loss=4.6269 gn=0.77 lr=4.36e-04 tps=21033 mem=5.5G lam=0.23249882366508245 Gm=None +[s_only s2] step 4600/12000 loss=4.6341 gn=0.79 lr=4.35e-04 tps=21033 mem=5.5G lam=0.23296099218229452 Gm=None +[s_only s2] step 4620/12000 loss=4.6835 gn=0.76 lr=4.34e-04 tps=21033 mem=5.5G lam=0.23343835522731146 Gm=None +[s_only s2] step 4640/12000 loss=4.7083 gn=0.82 lr=4.32e-04 tps=21033 mem=5.5G lam=0.23392422745625177 Gm=None +[s_only s2] step 4660/12000 loss=4.8662 gn=0.78 lr=4.31e-04 tps=21033 mem=5.5G lam=0.23433506612976393 Gm=None +[s_only s2] step 4680/12000 loss=4.8813 gn=0.87 lr=4.30e-04 tps=21033 mem=5.5G lam=0.23492382590969405 Gm=None +[s_only s2] step 4700/12000 loss=4.7064 gn=0.80 lr=4.28e-04 tps=21033 mem=5.5G lam=0.23534104165931544 Gm=None +[s_only s2] step 4720/12000 loss=4.6062 gn=1.36 lr=4.27e-04 tps=21033 mem=5.5G lam=0.2357162625218431 Gm=None +[s_only s2] step 4740/12000 loss=4.9094 gn=0.77 lr=4.26e-04 tps=21033 mem=5.5G lam=0.23618849646300077 Gm=None +[s_only s2] step 4760/12000 loss=4.5429 gn=0.89 lr=4.24e-04 tps=21034 mem=5.5G lam=0.23679729861517748 Gm=None +[s_only s2] step 4780/12000 loss=4.5058 gn=0.83 lr=4.23e-04 tps=21034 mem=5.5G lam=0.2373175024986267 Gm=None +[s_only s2] step 4800/12000 loss=4.7830 gn=0.78 lr=4.22e-04 tps=21035 mem=5.5G lam=0.23780154964576164 Gm=None +[s_only s2] step 4820/12000 loss=4.6880 gn=0.79 lr=4.20e-04 tps=21034 mem=5.5G lam=0.23826238668213287 Gm=None +[s_only s2] step 4840/12000 loss=4.7686 gn=0.79 lr=4.19e-04 tps=21035 mem=5.5G lam=0.23875076106439033 Gm=None +[s_only s2] step 4860/12000 loss=4.7693 gn=0.88 lr=4.18e-04 tps=21036 mem=5.5G lam=0.2391461500277122 Gm=None +[s_only s2] step 4880/12000 loss=4.6182 gn=0.83 lr=4.16e-04 tps=21036 mem=5.5G lam=0.23952643014490604 Gm=None +[s_only s2] step 4900/12000 loss=4.7098 gn=0.77 lr=4.15e-04 tps=21037 mem=5.5G lam=0.2400260434175531 Gm=None +[s_only s2] step 4920/12000 loss=4.6796 gn=0.84 lr=4.13e-04 tps=21038 mem=5.5G lam=0.24049460298071304 Gm=None +[s_only s2] step 4940/12000 loss=4.6495 gn=0.77 lr=4.12e-04 tps=21038 mem=5.5G lam=0.2409783893575271 Gm=None +[s_only s2] step 4960/12000 loss=4.6383 gn=0.74 lr=4.11e-04 tps=21038 mem=5.5G lam=0.24148680735379457 Gm=None +[s_only s2] step 4980/12000 loss=4.6651 gn=0.81 lr=4.09e-04 tps=21039 mem=5.5G lam=0.24200281966477633 Gm=None +[s_only s2] step 5000/12000 loss=4.5629 gn=0.78 lr=4.08e-04 tps=21039 mem=5.5G lam=0.24247589986771345 Gm=None +[s_only s2] EVAL step 5000: val_loss=4.7466 val_ppl=115.20 +[s_only s2] step 5020/12000 loss=4.8215 gn=0.81 lr=4.07e-04 tps=21024 mem=5.5G lam=0.24291378383835158 Gm=None +[s_only s2] step 5040/12000 loss=4.6320 gn=0.85 lr=4.05e-04 tps=21024 mem=5.5G lam=0.2433711833630999 Gm=None +[s_only s2] step 5060/12000 loss=4.6536 gn=0.84 lr=4.04e-04 tps=21024 mem=5.5G lam=0.24388939204315344 Gm=None +[s_only s2] step 5080/12000 loss=4.5595 gn=0.82 lr=4.02e-04 tps=21024 mem=5.5G lam=0.244293423059086 Gm=None +[s_only s2] step 5100/12000 loss=4.6617 gn=0.81 lr=4.01e-04 tps=21024 mem=5.5G lam=0.2447816121081511 Gm=None +[s_only s2] step 5120/12000 loss=4.5055 gn=0.82 lr=4.00e-04 tps=21024 mem=5.5G lam=0.24521219109495482 Gm=None +[s_only s2] step 5140/12000 loss=4.8169 gn=0.93 lr=3.98e-04 tps=21024 mem=5.5G lam=0.24565046653151512 Gm=None +[s_only s2] step 5160/12000 loss=4.8525 gn=0.84 lr=3.97e-04 tps=21025 mem=5.5G lam=0.24613430630415678 Gm=None +[s_only s2] step 5180/12000 loss=4.6406 gn=0.78 lr=3.95e-04 tps=21025 mem=5.5G lam=0.24664599324266115 Gm=None +[s_only s2] step 5200/12000 loss=4.6817 gn=0.86 lr=3.94e-04 tps=21025 mem=5.5G lam=0.24718484623978534 Gm=None +[s_only s2] step 5220/12000 loss=4.7443 gn=0.90 lr=3.93e-04 tps=21026 mem=5.5G lam=0.24770667621244988 Gm=None +[s_only s2] step 5240/12000 loss=4.6084 gn=0.80 lr=3.91e-04 tps=21026 mem=5.5G lam=0.24814249544094005 Gm=None +[s_only s2] step 5260/12000 loss=4.4819 gn=0.90 lr=3.90e-04 tps=21026 mem=5.5G lam=0.24867582973092794 Gm=None +[s_only s2] step 5280/12000 loss=4.8128 gn=0.87 lr=3.88e-04 tps=21026 mem=5.5G lam=0.24914405308663845 Gm=None +[s_only s2] step 5300/12000 loss=4.7053 gn=0.87 lr=3.87e-04 tps=21027 mem=5.5G lam=0.24957690046479306 Gm=None +[s_only s2] step 5320/12000 loss=4.7270 gn=0.82 lr=3.86e-04 tps=21027 mem=5.5G lam=0.2500547183056672 Gm=None +[s_only s2] step 5340/12000 loss=4.4953 gn=0.81 lr=3.84e-04 tps=21027 mem=5.5G lam=0.2504766738663117 Gm=None +[s_only s2] step 5360/12000 loss=4.7218 gn=0.88 lr=3.83e-04 tps=21028 mem=5.5G lam=0.25097494665533304 Gm=None +[s_only s2] step 5380/12000 loss=4.4935 gn=0.91 lr=3.81e-04 tps=21028 mem=5.5G lam=0.2514585942650835 Gm=None +[s_only s2] step 5400/12000 loss=4.5447 gn=0.84 lr=3.80e-04 tps=21028 mem=5.5G lam=0.2519325663646062 Gm=None +[s_only s2] step 5420/12000 loss=4.7171 gn=0.82 lr=3.79e-04 tps=21028 mem=5.5G lam=0.25239215915401775 Gm=None +[s_only s2] step 5440/12000 loss=4.5434 gn=0.79 lr=3.77e-04 tps=21028 mem=5.5G lam=0.25276057111720246 Gm=None +[s_only s2] step 5460/12000 loss=4.7746 gn=0.84 lr=3.76e-04 tps=21028 mem=5.5G lam=0.2531544156372547 Gm=None +[s_only s2] step 5480/12000 loss=4.5499 gn=0.84 lr=3.74e-04 tps=21028 mem=5.5G lam=0.2536056600511074 Gm=None +[s_only s2] step 5500/12000 loss=4.8874 gn=0.87 lr=3.73e-04 tps=21029 mem=5.5G lam=0.25403946234534186 Gm=None +[s_only s2] step 5520/12000 loss=4.6116 gn=0.90 lr=3.72e-04 tps=21029 mem=5.5G lam=0.2545148752008875 Gm=None +[s_only s2] step 5540/12000 loss=4.5915 gn=0.86 lr=3.70e-04 tps=21029 mem=5.5G lam=0.2550393861408035 Gm=None +[s_only s2] step 5560/12000 loss=4.6540 gn=0.82 lr=3.69e-04 tps=21029 mem=5.5G lam=0.2554669848953684 Gm=None +[s_only s2] step 5580/12000 loss=4.5967 gn=0.81 lr=3.67e-04 tps=21027 mem=5.5G lam=0.2559390064949791 Gm=None +[s_only s2] step 5600/12000 loss=4.7827 gn=0.81 lr=3.66e-04 tps=21025 mem=5.5G lam=0.256461293126146 Gm=None +[s_only s2] step 5620/12000 loss=4.6911 gn=0.88 lr=3.64e-04 tps=21023 mem=5.5G lam=0.2569750587766369 Gm=None +[s_only s2] step 5640/12000 loss=4.5293 gn=0.85 lr=3.63e-04 tps=21013 mem=5.5G lam=0.25740332963565987 Gm=None +[s_only s2] step 5660/12000 loss=4.5895 gn=0.82 lr=3.62e-04 tps=21008 mem=5.5G lam=0.2579368744045496 Gm=None +[s_only s2] step 5680/12000 loss=4.6626 gn=0.82 lr=3.60e-04 tps=21005 mem=5.5G lam=0.2583879418671131 Gm=None +[s_only s2] step 5700/12000 loss=4.5807 gn=0.88 lr=3.59e-04 tps=21006 mem=5.5G lam=0.25863895534227294 Gm=None +[s_only s2] step 5720/12000 loss=4.5422 gn=0.84 lr=3.57e-04 tps=21006 mem=5.5G lam=0.25900476550062496 Gm=None +[s_only s2] step 5740/12000 loss=4.5706 gn=0.90 lr=3.56e-04 tps=21007 mem=5.5G lam=0.2594257574528456 Gm=None +[s_only s2] step 5760/12000 loss=4.3503 gn=0.83 lr=3.54e-04 tps=21006 mem=5.5G lam=0.2598218002046148 Gm=None +[s_only s2] step 5780/12000 loss=4.5548 gn=0.81 lr=3.53e-04 tps=21004 mem=5.5G lam=0.2602734516064326 Gm=None +[s_only s2] step 5800/12000 loss=4.6328 gn=0.87 lr=3.52e-04 tps=21004 mem=5.5G lam=0.2606243460128705 Gm=None +[s_only s2] step 5820/12000 loss=4.5983 gn=0.86 lr=3.50e-04 tps=21004 mem=5.5G lam=0.2610365664586425 Gm=None +[s_only s2] step 5840/12000 loss=4.6195 gn=0.80 lr=3.49e-04 tps=21005 mem=5.5G lam=0.2614124271397789 Gm=None +[s_only s2] step 5860/12000 loss=4.6459 gn=0.87 lr=3.47e-04 tps=21005 mem=5.5G lam=0.2618840370948116 Gm=None +[s_only s2] step 5880/12000 loss=4.6381 gn=0.91 lr=3.46e-04 tps=21005 mem=5.5G lam=0.26229391650607187 Gm=None +[s_only s2] step 5900/12000 loss=4.8002 gn=0.91 lr=3.44e-04 tps=21005 mem=5.5G lam=0.2626849866161744 Gm=None +[s_only s2] step 5920/12000 loss=4.6385 gn=0.90 lr=3.43e-04 tps=21004 mem=5.5G lam=0.26311251396934193 Gm=None +[s_only s2] step 5940/12000 loss=4.6251 gn=0.95 lr=3.41e-04 tps=21004 mem=5.5G lam=0.2635484794154763 Gm=None +[s_only s2] step 5960/12000 loss=4.6726 gn=0.84 lr=3.40e-04 tps=21004 mem=5.5G lam=0.2639696712916096 Gm=None +[s_only s2] step 5980/12000 loss=4.5163 gn=0.87 lr=3.39e-04 tps=21004 mem=5.5G lam=0.2643654244020581 Gm=None +[s_only s2] step 6000/12000 loss=4.5479 gn=0.82 lr=3.37e-04 tps=21004 mem=5.5G lam=0.2646653664608796 Gm=None +[s_only s2] EVAL step 6000: val_loss=4.6343 val_ppl=102.96 +[s_only s2] step 6020/12000 loss=4.4766 gn=0.83 lr=3.36e-04 tps=20992 mem=5.5G lam=0.26500156583885354 Gm=None +[s_only s2] step 6040/12000 loss=4.6161 gn=0.84 lr=3.34e-04 tps=20992 mem=5.5G lam=0.2653660637636979 Gm=None +[s_only s2] step 6060/12000 loss=4.5263 gn=0.84 lr=3.33e-04 tps=20993 mem=5.5G lam=0.26578407268971205 Gm=None +[s_only s2] step 6080/12000 loss=4.6075 gn=0.88 lr=3.31e-04 tps=20994 mem=5.5G lam=0.2662042152757446 Gm=None +[s_only s2] step 6100/12000 loss=4.5586 gn=0.85 lr=3.30e-04 tps=20994 mem=5.5G lam=0.2666562845309575 Gm=None +[s_only s2] step 6120/12000 loss=4.4676 gn=0.82 lr=3.29e-04 tps=20995 mem=5.5G lam=0.26710641880830127 Gm=None +[s_only s2] step 6140/12000 loss=4.5325 gn=0.90 lr=3.27e-04 tps=20995 mem=5.5G lam=0.2673940894504388 Gm=None +[s_only s2] step 6160/12000 loss=4.6396 gn=0.83 lr=3.26e-04 tps=20996 mem=5.5G lam=0.26776444508383673 Gm=None +[s_only s2] step 6180/12000 loss=4.5402 gn=0.88 lr=3.24e-04 tps=20996 mem=5.5G lam=0.26814319379627705 Gm=None +[s_only s2] step 6200/12000 loss=4.5944 gn=0.82 lr=3.23e-04 tps=20996 mem=5.5G lam=0.2685485426336527 Gm=None +[s_only s2] step 6220/12000 loss=4.4887 gn=0.83 lr=3.21e-04 tps=20997 mem=5.5G lam=0.2689185906201601 Gm=None +[s_only s2] step 6240/12000 loss=4.4506 gn=0.89 lr=3.20e-04 tps=20997 mem=5.5G lam=0.2693471970657508 Gm=None +[s_only s2] step 6260/12000 loss=4.4527 gn=0.88 lr=3.19e-04 tps=20997 mem=5.5G lam=0.2697340085481604 Gm=None +[s_only s2] step 6280/12000 loss=4.7481 gn=0.85 lr=3.17e-04 tps=20997 mem=5.5G lam=0.27000636979937553 Gm=None +[s_only s2] step 6300/12000 loss=4.7200 gn=0.83 lr=3.16e-04 tps=20998 mem=5.5G lam=0.2703995270033677 Gm=None +[s_only s2] step 6320/12000 loss=4.4808 gn=0.89 lr=3.14e-04 tps=20998 mem=5.5G lam=0.2707295262565215 Gm=None +[s_only s2] step 6340/12000 loss=4.6306 gn=0.88 lr=3.13e-04 tps=20998 mem=5.5G lam=0.271109270863235 Gm=None +[s_only s2] step 6360/12000 loss=4.5046 gn=0.83 lr=3.11e-04 tps=20998 mem=5.5G lam=0.27148185949772596 Gm=None +[s_only s2] step 6380/12000 loss=4.5578 gn=0.91 lr=3.10e-04 tps=20998 mem=5.5G lam=0.2719736456250151 Gm=None +[s_only s2] step 6400/12000 loss=4.3656 gn=0.90 lr=3.08e-04 tps=20999 mem=5.5G lam=0.2724094359825055 Gm=None +[s_only s2] step 6420/12000 loss=4.6035 gn=0.88 lr=3.07e-04 tps=20999 mem=5.5G lam=0.27273204891632 Gm=None +[s_only s2] step 6440/12000 loss=4.7357 gn=0.94 lr=3.06e-04 tps=20999 mem=5.5G lam=0.27313968787590664 Gm=None +[s_only s2] step 6460/12000 loss=4.5279 gn=0.94 lr=3.04e-04 tps=20999 mem=5.5G lam=0.2734393747523427 Gm=None +[s_only s2] step 6480/12000 loss=4.5903 gn=0.99 lr=3.03e-04 tps=21000 mem=5.5G lam=0.2737533909579118 Gm=None +[s_only s2] step 6500/12000 loss=4.4109 gn=0.96 lr=3.01e-04 tps=21000 mem=5.5G lam=0.27401096125443775 Gm=None +[s_only s2] step 6520/12000 loss=4.6709 gn=0.91 lr=3.00e-04 tps=21001 mem=5.5G lam=0.27432002623875934 Gm=None +[s_only s2] step 6540/12000 loss=4.5738 gn=0.97 lr=2.98e-04 tps=21001 mem=5.5G lam=0.2746452623978257 Gm=None +[s_only s2] step 6560/12000 loss=4.5833 gn=0.92 lr=2.97e-04 tps=21002 mem=5.5G lam=0.2749651127184431 Gm=None +[s_only s2] step 6580/12000 loss=4.4566 gn=0.95 lr=2.96e-04 tps=21002 mem=5.5G lam=0.27529759611934423 Gm=None +[s_only s2] step 6600/12000 loss=4.6283 gn=0.95 lr=2.94e-04 tps=21002 mem=5.5G lam=0.27562828455120325 Gm=None +[s_only s2] step 6620/12000 loss=4.4858 gn=0.92 lr=2.93e-04 tps=21002 mem=5.5G lam=0.2759853570411603 Gm=None +[s_only s2] step 6640/12000 loss=4.5837 gn=0.87 lr=2.91e-04 tps=21003 mem=5.5G lam=0.2763743670657277 Gm=None +[s_only s2] step 6660/12000 loss=4.6822 gn=0.92 lr=2.90e-04 tps=21003 mem=5.5G lam=0.2767472357178728 Gm=None +[s_only s2] step 6680/12000 loss=4.5132 gn=0.87 lr=2.88e-04 tps=21004 mem=5.5G lam=0.27713999586800736 Gm=None +[s_only s2] step 6700/12000 loss=4.6031 gn=0.88 lr=2.87e-04 tps=21004 mem=5.5G lam=0.27746275905519724 Gm=None +[s_only s2] step 6720/12000 loss=4.6037 gn=0.83 lr=2.86e-04 tps=21004 mem=5.5G lam=0.27782616081337136 Gm=None +[s_only s2] step 6740/12000 loss=4.5656 gn=0.92 lr=2.84e-04 tps=21005 mem=5.5G lam=0.2781501868739724 Gm=None +[s_only s2] step 6760/12000 loss=4.5960 gn=0.85 lr=2.83e-04 tps=21005 mem=5.5G lam=0.2785565185671051 Gm=None +[s_only s2] step 6780/12000 loss=4.5497 gn=0.86 lr=2.81e-04 tps=21006 mem=5.5G lam=0.2789055447404583 Gm=None +[s_only s2] step 6800/12000 loss=4.6013 gn=0.94 lr=2.80e-04 tps=21006 mem=5.5G lam=0.27922236422697705 Gm=None +[s_only s2] step 6820/12000 loss=4.5503 gn=0.92 lr=2.79e-04 tps=21006 mem=5.5G lam=0.27953907412787277 Gm=None +[s_only s2] step 6840/12000 loss=4.6190 gn=0.88 lr=2.77e-04 tps=21006 mem=5.5G lam=0.27984337353458005 Gm=None +[s_only s2] step 6860/12000 loss=4.5892 gn=0.90 lr=2.76e-04 tps=21006 mem=5.5G lam=0.2801496246829629 Gm=None +[s_only s2] step 6880/12000 loss=4.4947 gn=0.95 lr=2.74e-04 tps=21006 mem=5.5G lam=0.28054239694029093 Gm=None +[s_only s2] step 6900/12000 loss=4.5574 gn=0.98 lr=2.73e-04 tps=21006 mem=5.5G lam=0.2808959602067868 Gm=None +[s_only s2] step 6920/12000 loss=4.4189 gn=0.90 lr=2.72e-04 tps=21007 mem=5.5G lam=0.2812091599528988 Gm=None +[s_only s2] step 6940/12000 loss=4.7499 gn=0.99 lr=2.70e-04 tps=21007 mem=5.5G lam=0.2814779194692771 Gm=None +[s_only s2] step 6960/12000 loss=4.5215 gn=0.86 lr=2.69e-04 tps=21007 mem=5.5G lam=0.2817836885030071 Gm=None +[s_only s2] step 6980/12000 loss=4.4607 gn=0.91 lr=2.67e-04 tps=21007 mem=5.5G lam=0.2820315894981225 Gm=None +[s_only s2] step 7000/12000 loss=4.5782 gn=0.95 lr=2.66e-04 tps=21007 mem=5.5G lam=0.2823164627576868 Gm=None +[s_only s2] EVAL step 7000: val_loss=4.5828 val_ppl=97.78 +[s_only s2] step 7020/12000 loss=4.4524 gn=0.89 lr=2.65e-04 tps=20996 mem=5.5G lam=0.2826612290615837 Gm=None +[s_only s2] step 7040/12000 loss=4.5758 gn=0.90 lr=2.63e-04 tps=20994 mem=5.5G lam=0.28294239720950526 Gm=None +[s_only s2] step 7060/12000 loss=4.6185 gn=0.93 lr=2.62e-04 tps=20993 mem=5.5G lam=0.2831963129962484 Gm=None +[s_only s2] step 7080/12000 loss=4.4799 gn=0.97 lr=2.60e-04 tps=20992 mem=5.5G lam=0.2834468698129058 Gm=None +[s_only s2] step 7100/12000 loss=4.4896 gn=0.98 lr=2.59e-04 tps=20990 mem=5.5G lam=0.2837455316136281 Gm=None +[s_only s2] step 7120/12000 loss=4.4254 gn=0.99 lr=2.58e-04 tps=20989 mem=5.5G lam=0.2840733025223017 Gm=None +[s_only s2] step 7140/12000 loss=4.5674 gn=0.87 lr=2.56e-04 tps=20989 mem=5.5G lam=0.2844091371322672 Gm=None +[s_only s2] step 7160/12000 loss=4.4387 gn=0.88 lr=2.55e-04 tps=20989 mem=5.5G lam=0.2847208607320984 Gm=None +[s_only s2] step 7180/12000 loss=4.5078 gn=0.96 lr=2.53e-04 tps=20990 mem=5.5G lam=0.2850275399784247 Gm=None +[s_only s2] step 7200/12000 loss=4.5544 gn=0.98 lr=2.52e-04 tps=20991 mem=5.5G lam=0.2853563331688444 Gm=None +[s_only s2] step 7220/12000 loss=4.5675 gn=0.92 lr=2.51e-04 tps=20991 mem=5.5G lam=0.2857344712441166 Gm=None +[s_only s2] step 7240/12000 loss=4.3896 gn=0.97 lr=2.49e-04 tps=20990 mem=5.5G lam=0.28605805803090334 Gm=None +[s_only s2] step 7260/12000 loss=4.5818 gn=1.04 lr=2.48e-04 tps=20991 mem=5.5G lam=0.28628035355359316 Gm=None +[s_only s2] step 7280/12000 loss=4.4656 gn=1.00 lr=2.47e-04 tps=20991 mem=5.5G lam=0.28663209453225136 Gm=None +[s_only s2] step 7300/12000 loss=4.4579 gn=0.90 lr=2.45e-04 tps=20991 mem=5.5G lam=0.28688729864855606 Gm=None +[s_only s2] step 7320/12000 loss=4.6268 gn=0.95 lr=2.44e-04 tps=20991 mem=5.5G lam=0.2871602090696494 Gm=None +[s_only s2] step 7340/12000 loss=4.4007 gn=1.01 lr=2.42e-04 tps=20991 mem=5.5G lam=0.2874564950664838 Gm=None +[s_only s2] step 7360/12000 loss=4.3810 gn=0.93 lr=2.41e-04 tps=20990 mem=5.5G lam=0.2877280519654353 Gm=None +[s_only s2] step 7380/12000 loss=4.5855 gn=0.92 lr=2.40e-04 tps=20990 mem=5.5G lam=0.2880757578338186 Gm=None +[s_only s2] step 7400/12000 loss=4.4468 gn=0.88 lr=2.38e-04 tps=20990 mem=5.5G lam=0.28835653358449537 Gm=None +[s_only s2] step 7420/12000 loss=4.6655 gn=0.99 lr=2.37e-04 tps=20990 mem=5.5G lam=0.28862268570810556 Gm=None +[s_only s2] step 7440/12000 loss=4.4377 gn=0.90 lr=2.36e-04 tps=20990 mem=5.5G lam=0.2889397374043862 Gm=None +[s_only s2] step 7460/12000 loss=4.5335 gn=0.95 lr=2.34e-04 tps=20990 mem=5.5G lam=0.2891930139934023 Gm=None +[s_only s2] step 7480/12000 loss=4.5097 gn=0.92 lr=2.33e-04 tps=20990 mem=5.5G lam=0.2894794186577201 Gm=None +[s_only s2] step 7500/12000 loss=4.4596 gn=0.96 lr=2.32e-04 tps=20990 mem=5.5G lam=0.28979209375878173 Gm=None +[s_only s2] step 7520/12000 loss=4.3656 gn=0.91 lr=2.30e-04 tps=20990 mem=5.5G lam=0.2900073140238722 Gm=None +[s_only s2] step 7540/12000 loss=4.4714 gn=0.95 lr=2.29e-04 tps=20990 mem=5.5G lam=0.2902587990586956 Gm=None +[s_only s2] step 7560/12000 loss=4.5902 gn=0.92 lr=2.28e-04 tps=20989 mem=5.5G lam=0.29052977822721004 Gm=None +[s_only s2] step 7580/12000 loss=4.4811 gn=1.02 lr=2.26e-04 tps=20989 mem=5.5G lam=0.29086567151049775 Gm=None +[s_only s2] step 7600/12000 loss=4.4666 gn=0.92 lr=2.25e-04 tps=20989 mem=5.5G lam=0.2911681147913138 Gm=None +[s_only s2] step 7620/12000 loss=4.7128 gn=1.04 lr=2.24e-04 tps=20989 mem=5.5G lam=0.2914774042243759 Gm=None +[s_only s2] step 7640/12000 loss=4.4441 gn=0.98 lr=2.22e-04 tps=20989 mem=5.5G lam=0.29175439663231373 Gm=None +[s_only s2] step 7660/12000 loss=4.3823 gn=0.92 lr=2.21e-04 tps=20989 mem=5.5G lam=0.2920082062482834 Gm=None +[s_only s2] step 7680/12000 loss=4.3410 gn=0.94 lr=2.20e-04 tps=20989 mem=5.5G lam=0.29227580533673364 Gm=None +[s_only s2] step 7700/12000 loss=4.4888 gn=0.90 lr=2.18e-04 tps=20989 mem=5.5G lam=0.29256780569752056 Gm=None +[s_only s2] step 7720/12000 loss=4.5366 gn=0.95 lr=2.17e-04 tps=20988 mem=5.5G lam=0.29287401648859185 Gm=None +[s_only s2] step 7740/12000 loss=4.4267 gn=1.02 lr=2.16e-04 tps=20988 mem=5.5G lam=0.2931306092068553 Gm=None +[s_only s2] step 7760/12000 loss=4.5676 gn=1.08 lr=2.15e-04 tps=20988 mem=5.5G lam=0.29338400283207494 Gm=None +[s_only s2] step 7780/12000 loss=4.6100 gn=0.92 lr=2.13e-04 tps=20989 mem=5.5G lam=0.2936224543179075 Gm=None +[s_only s2] step 7800/12000 loss=4.5010 gn=1.00 lr=2.12e-04 tps=20989 mem=5.5G lam=0.29387457203119993 Gm=None +[s_only s2] step 7820/12000 loss=4.4602 gn=0.91 lr=2.11e-04 tps=20988 mem=5.5G lam=0.2942239890495936 Gm=None +[s_only s2] step 7840/12000 loss=4.3721 gn=0.89 lr=2.09e-04 tps=20987 mem=5.5G lam=0.2944861330712835 Gm=None +[s_only s2] step 7860/12000 loss=4.4138 gn=0.87 lr=2.08e-04 tps=20987 mem=5.5G lam=0.29470052880545455 Gm=None +[s_only s2] step 7880/12000 loss=4.4845 gn=0.98 lr=2.07e-04 tps=20985 mem=5.5G lam=0.2949250877524416 Gm=None +[s_only s2] step 7900/12000 loss=4.4786 gn=0.92 lr=2.06e-04 tps=20985 mem=5.5G lam=0.2951602293178439 Gm=None +[s_only s2] step 7920/12000 loss=4.5555 gn=0.97 lr=2.04e-04 tps=20985 mem=5.5G lam=0.29533701390028 Gm=None +[s_only s2] step 7940/12000 loss=4.4515 gn=0.95 lr=2.03e-04 tps=20985 mem=5.5G lam=0.2955960410957535 Gm=None +[s_only s2] step 7960/12000 loss=4.5059 gn=0.98 lr=2.02e-04 tps=20985 mem=5.5G lam=0.2958082702631752 Gm=None +[s_only s2] step 7980/12000 loss=4.4774 gn=1.01 lr=2.00e-04 tps=20985 mem=5.5G lam=0.29603456892073154 Gm=None +[s_only s2] step 8000/12000 loss=4.4320 gn=1.00 lr=1.99e-04 tps=20985 mem=5.5G lam=0.2962665806214015 Gm=None +[s_only s2] EVAL step 8000: val_loss=4.4831 val_ppl=88.51 +[s_only s2] step 8020/12000 loss=4.4902 gn=0.98 lr=1.98e-04 tps=20975 mem=5.5G lam=0.2965219095349312 Gm=None +[s_only s2] step 8040/12000 loss=4.5090 gn=0.91 lr=1.97e-04 tps=20975 mem=5.5G lam=0.29672919772565365 Gm=None +[s_only s2] step 8060/12000 loss=4.3810 gn=0.95 lr=1.95e-04 tps=20974 mem=5.5G lam=0.29699624174584943 Gm=None +[s_only s2] step 8080/12000 loss=4.4645 gn=0.94 lr=1.94e-04 tps=20973 mem=5.5G lam=0.2972711365049084 Gm=None +[s_only s2] step 8100/12000 loss=4.4989 gn=0.98 lr=1.93e-04 tps=20973 mem=5.5G lam=0.2974848508213957 Gm=None +[s_only s2] step 8120/12000 loss=4.5247 gn=0.94 lr=1.92e-04 tps=20972 mem=5.5G lam=0.29773142592360574 Gm=None +[s_only s2] step 8140/12000 loss=4.4470 gn=0.94 lr=1.90e-04 tps=20972 mem=5.5G lam=0.29799348457405966 Gm=None +[s_only s2] step 8160/12000 loss=4.5624 gn=0.93 lr=1.89e-04 tps=20971 mem=5.5G lam=0.29824015218764544 Gm=None +[s_only s2] step 8180/12000 loss=4.5429 gn=1.12 lr=1.88e-04 tps=20970 mem=5.5G lam=0.29847285089393455 Gm=None +[s_only s2] step 8200/12000 loss=4.5022 gn=0.92 lr=1.87e-04 tps=20969 mem=5.5G lam=0.2987157544121146 Gm=None +[s_only s2] step 8220/12000 loss=4.5136 gn=0.97 lr=1.86e-04 tps=20969 mem=5.5G lam=0.29893200534085435 Gm=None +[s_only s2] step 8240/12000 loss=4.4181 gn=1.01 lr=1.84e-04 tps=20969 mem=5.5G lam=0.29914099754144746 Gm=None +[s_only s2] step 8260/12000 loss=4.4736 gn=0.93 lr=1.83e-04 tps=20969 mem=5.5G lam=0.29931600391864777 Gm=None +[s_only s2] step 8280/12000 loss=4.4647 gn=1.02 lr=1.82e-04 tps=20969 mem=5.5G lam=0.2995398637528221 Gm=None +[s_only s2] step 8300/12000 loss=4.4875 gn=1.00 lr=1.81e-04 tps=20969 mem=5.5G lam=0.2997386644904812 Gm=None +[s_only s2] step 8320/12000 loss=4.4281 gn=0.98 lr=1.80e-04 tps=20969 mem=5.5G lam=0.29993723581234616 Gm=None +[s_only s2] step 8340/12000 loss=4.4713 gn=1.01 lr=1.78e-04 tps=20969 mem=5.5G lam=0.3001044470195969 Gm=None +[s_only s2] step 8360/12000 loss=4.4814 gn=1.02 lr=1.77e-04 tps=20970 mem=5.5G lam=0.30030416045337915 Gm=None +[s_only s2] step 8380/12000 loss=4.4389 gn=0.92 lr=1.76e-04 tps=20969 mem=5.5G lam=0.30053397330145043 Gm=None +[s_only s2] step 8400/12000 loss=4.3835 gn=1.05 lr=1.75e-04 tps=20969 mem=5.5G lam=0.3007811267549793 Gm=None +[s_only s2] step 8420/12000 loss=4.5865 gn=1.09 lr=1.74e-04 tps=20969 mem=5.5G lam=0.30098992958664894 Gm=None +[s_only s2] step 8440/12000 loss=4.4792 gn=1.11 lr=1.72e-04 tps=20969 mem=5.5G lam=0.3011693594356378 Gm=None +[s_only s2] step 8460/12000 loss=4.4277 gn=1.06 lr=1.71e-04 tps=20970 mem=5.5G lam=0.3013659439360102 Gm=None +[s_only s2] step 8480/12000 loss=4.3680 gn=0.93 lr=1.70e-04 tps=20969 mem=5.5G lam=0.30158438937117654 Gm=None +[s_only s2] step 8500/12000 loss=4.3968 gn=1.03 lr=1.69e-04 tps=20970 mem=5.5G lam=0.3017741885657112 Gm=None +[s_only s2] step 8520/12000 loss=4.5309 gn=0.99 lr=1.68e-04 tps=20970 mem=5.5G lam=0.30195903964340687 Gm=None +[s_only s2] step 8540/12000 loss=4.5775 gn=1.07 lr=1.67e-04 tps=20970 mem=5.5G lam=0.30213839809099835 Gm=None +[s_only s2] step 8560/12000 loss=4.3158 gn=1.04 lr=1.66e-04 tps=20970 mem=5.5G lam=0.3023693251113097 Gm=None +[s_only s2] step 8580/12000 loss=4.4819 gn=0.94 lr=1.64e-04 tps=20970 mem=5.5G lam=0.30250710683564347 Gm=None +[s_only s2] step 8600/12000 loss=4.3794 gn=0.99 lr=1.63e-04 tps=20971 mem=5.5G lam=0.3027069090555112 Gm=None +[s_only s2] step 8620/12000 loss=4.4076 gn=1.09 lr=1.62e-04 tps=20970 mem=5.5G lam=0.3029084202523033 Gm=None +[s_only s2] step 8640/12000 loss=4.4043 gn=0.98 lr=1.61e-04 tps=20970 mem=5.5G lam=0.30310207574317855 Gm=None +[s_only s2] step 8660/12000 loss=4.4555 gn=0.96 lr=1.60e-04 tps=20970 mem=5.5G lam=0.303305072709918 Gm=None +[s_only s2] step 8680/12000 loss=4.3299 gn=1.06 lr=1.59e-04 tps=20970 mem=5.5G lam=0.303483239064614 Gm=None +[s_only s2] step 8700/12000 loss=4.1906 gn=0.93 lr=1.58e-04 tps=20970 mem=5.5G lam=0.3036634648839633 Gm=None +[s_only s2] step 8720/12000 loss=4.3846 gn=1.02 lr=1.57e-04 tps=20970 mem=5.5G lam=0.30385657120496035 Gm=None +[s_only s2] step 8740/12000 loss=4.4477 gn=1.01 lr=1.55e-04 tps=20970 mem=5.5G lam=0.30404316044102114 Gm=None +[s_only s2] step 8760/12000 loss=4.5362 gn=1.01 lr=1.54e-04 tps=20970 mem=5.5G lam=0.3042232890923818 Gm=None +[s_only s2] step 8780/12000 loss=4.5379 gn=0.97 lr=1.53e-04 tps=20970 mem=5.5G lam=0.3043933504571517 Gm=None +[s_only s2] step 8800/12000 loss=4.3611 gn=0.98 lr=1.52e-04 tps=20970 mem=5.5G lam=0.30460307964434225 Gm=None +[s_only s2] step 8820/12000 loss=4.3624 gn=0.96 lr=1.51e-04 tps=20971 mem=5.5G lam=0.30480420341094333 Gm=None +[s_only s2] step 8840/12000 loss=4.5180 gn=0.99 lr=1.50e-04 tps=20971 mem=5.5G lam=0.3049757042899728 Gm=None +[s_only s2] step 8860/12000 loss=4.4020 gn=1.01 lr=1.49e-04 tps=20972 mem=5.5G lam=0.30509729745487374 Gm=None +[s_only s2] step 8880/12000 loss=4.3958 gn=0.98 lr=1.48e-04 tps=20971 mem=5.5G lam=0.3052949759488304 Gm=None +[s_only s2] step 8900/12000 loss=4.4798 gn=0.99 lr=1.47e-04 tps=20971 mem=5.5G lam=0.30547469730178517 Gm=None +[s_only s2] step 8920/12000 loss=4.3575 gn=1.04 lr=1.46e-04 tps=20971 mem=5.5G lam=0.30562735131631297 Gm=None +[s_only s2] step 8940/12000 loss=4.4563 gn=1.00 lr=1.45e-04 tps=20971 mem=5.5G lam=0.30581385393937427 Gm=None +[s_only s2] step 8960/12000 loss=4.4123 gn=1.00 lr=1.44e-04 tps=20971 mem=5.5G lam=0.30603296495974064 Gm=None +[s_only s2] step 8980/12000 loss=4.3359 gn=0.97 lr=1.43e-04 tps=20971 mem=5.5G lam=0.3061758146310846 Gm=None +[s_only s2] step 9000/12000 loss=4.3643 gn=0.94 lr=1.42e-04 tps=20971 mem=5.5G lam=0.3063194140171011 Gm=None +[s_only s2] EVAL step 9000: val_loss=4.5041 val_ppl=90.38 +[s_only s2] step 9020/12000 loss=4.3840 gn=0.99 lr=1.41e-04 tps=20963 mem=5.5G lam=0.30647911100337905 Gm=None +[s_only s2] step 9040/12000 loss=4.5390 gn=1.03 lr=1.40e-04 tps=20963 mem=5.5G lam=0.3066239729523659 Gm=None +[s_only s2] step 9060/12000 loss=4.4078 gn=1.09 lr=1.39e-04 tps=20964 mem=5.5G lam=0.30680373031646013 Gm=None +[s_only s2] step 9080/12000 loss=4.3251 gn=0.97 lr=1.38e-04 tps=20963 mem=5.5G lam=0.30694302792350453 Gm=None +[s_only s2] step 9100/12000 loss=4.4442 gn=1.07 lr=1.37e-04 tps=20964 mem=5.5G lam=0.3071571169421077 Gm=None +[s_only s2] step 9120/12000 loss=4.3842 gn=1.01 lr=1.36e-04 tps=20964 mem=5.5G lam=0.3073020074516535 Gm=None +[s_only s2] step 9140/12000 loss=4.3158 gn=0.95 lr=1.35e-04 tps=20964 mem=5.5G lam=0.3075289170568188 Gm=None +[s_only s2] step 9160/12000 loss=4.4253 gn=1.08 lr=1.34e-04 tps=20964 mem=5.5G lam=0.3077426416178544 Gm=None +[s_only s2] step 9180/12000 loss=4.3729 gn=1.08 lr=1.33e-04 tps=20964 mem=5.5G lam=0.30788091383874416 Gm=None +[s_only s2] step 9200/12000 loss=4.4746 gn=1.11 lr=1.32e-04 tps=20964 mem=5.5G lam=0.3080643971140186 Gm=None +[s_only s2] step 9220/12000 loss=4.3319 gn=1.15 lr=1.31e-04 tps=20964 mem=5.5G lam=0.3081953537960847 Gm=None +[s_only s2] step 9240/12000 loss=4.3566 gn=0.99 lr=1.30e-04 tps=20964 mem=5.5G lam=0.30835285782814026 Gm=None +[s_only s2] step 9260/12000 loss=4.3542 gn=1.04 lr=1.29e-04 tps=20964 mem=5.5G lam=0.3085153366749485 Gm=None +[s_only s2] step 9280/12000 loss=4.2465 gn=1.03 lr=1.28e-04 tps=20964 mem=5.5G lam=0.3086527433867256 Gm=None +[s_only s2] step 9300/12000 loss=4.4913 gn=1.17 lr=1.27e-04 tps=20964 mem=5.5G lam=0.3088077005619804 Gm=None +[s_only s2] step 9320/12000 loss=4.3543 gn=1.06 lr=1.26e-04 tps=20964 mem=5.5G lam=0.3089525445053975 Gm=None +[s_only s2] step 9340/12000 loss=4.3647 gn=1.05 lr=1.25e-04 tps=20964 mem=5.5G lam=0.30910664579520625 Gm=None +[s_only s2] step 9360/12000 loss=4.1153 gn=0.97 lr=1.24e-04 tps=20961 mem=5.5G lam=0.3092357652882735 Gm=None +[s_only s2] step 9380/12000 loss=4.5149 gn=1.08 lr=1.23e-04 tps=20956 mem=5.5G lam=0.3093980907772978 Gm=None +[s_only s2] step 9400/12000 loss=4.4281 gn=1.01 lr=1.22e-04 tps=20953 mem=5.5G lam=0.30953816665957373 Gm=None +[s_only s2] step 9420/12000 loss=4.3762 gn=1.06 lr=1.21e-04 tps=20951 mem=5.5G lam=0.3097144455338518 Gm=None +[s_only s2] step 9440/12000 loss=4.2774 gn=0.98 lr=1.20e-04 tps=20951 mem=5.5G lam=0.30990738545854885 Gm=None +[s_only s2] step 9460/12000 loss=4.4457 gn=0.98 lr=1.19e-04 tps=20950 mem=5.5G lam=0.31005147751420736 Gm=None +[s_only s2] step 9480/12000 loss=4.2702 gn=1.13 lr=1.19e-04 tps=20950 mem=5.5G lam=0.3101800136889021 Gm=None +[s_only s2] step 9500/12000 loss=4.4242 gn=1.08 lr=1.18e-04 tps=20950 mem=5.5G lam=0.31034424373259145 Gm=None +[s_only s2] step 9520/12000 loss=4.2842 gn=0.99 lr=1.17e-04 tps=20950 mem=5.5G lam=0.3105243795240919 Gm=None +[s_only s2] step 9540/12000 loss=4.2599 gn=1.05 lr=1.16e-04 tps=20950 mem=5.5G lam=0.3106787158176303 Gm=None +[s_only s2] step 9560/12000 loss=4.3144 gn=0.97 lr=1.15e-04 tps=20949 mem=5.5G lam=0.3108053067699075 Gm=None +[s_only s2] step 9580/12000 loss=4.2402 gn=0.99 lr=1.14e-04 tps=20950 mem=5.5G lam=0.3109419218574961 Gm=None +[s_only s2] step 9600/12000 loss=4.3911 gn=1.12 lr=1.13e-04 tps=20950 mem=5.5G lam=0.3110789271692435 Gm=None +[s_only s2] step 9620/12000 loss=4.4814 gn=0.96 lr=1.12e-04 tps=20950 mem=5.5G lam=0.31120830836395424 Gm=None +[s_only s2] step 9640/12000 loss=4.4022 gn=0.99 lr=1.12e-04 tps=20950 mem=5.5G lam=0.3113517450789611 Gm=None +[s_only s2] step 9660/12000 loss=4.2334 gn=0.97 lr=1.11e-04 tps=20950 mem=5.5G lam=0.3114693419386943 Gm=None +[s_only s2] step 9680/12000 loss=4.2363 gn=1.00 lr=1.10e-04 tps=20951 mem=5.5G lam=0.3116099542627732 Gm=None +[s_only s2] step 9700/12000 loss=4.2547 gn=1.07 lr=1.09e-04 tps=20950 mem=5.5G lam=0.311699609272182 Gm=None +[s_only s2] step 9720/12000 loss=4.2704 gn=1.03 lr=1.08e-04 tps=20951 mem=5.5G lam=0.31183134764432907 Gm=None +[s_only s2] step 9740/12000 loss=4.3484 gn=1.12 lr=1.07e-04 tps=20951 mem=5.5G lam=0.3119480072831114 Gm=None +[s_only s2] step 9760/12000 loss=4.2273 gn=0.98 lr=1.07e-04 tps=20951 mem=5.5G lam=0.312070540462931 Gm=None +[s_only s2] step 9780/12000 loss=4.3525 gn=1.13 lr=1.06e-04 tps=20951 mem=5.5G lam=0.3121681148186326 Gm=None +[s_only s2] step 9800/12000 loss=4.4060 gn=1.04 lr=1.05e-04 tps=20951 mem=5.5G lam=0.31227718697239953 Gm=None +[s_only s2] step 9820/12000 loss=4.4236 gn=1.11 lr=1.04e-04 tps=20951 mem=5.5G lam=0.3124190090845029 Gm=None +[s_only s2] step 9840/12000 loss=4.1631 gn=1.05 lr=1.03e-04 tps=20952 mem=5.5G lam=0.3125620245312651 Gm=None +[s_only s2] step 9860/12000 loss=4.4326 gn=1.01 lr=1.03e-04 tps=20952 mem=5.5G lam=0.3126934288690488 Gm=None +[s_only s2] step 9880/12000 loss=4.5091 gn=1.02 lr=1.02e-04 tps=20952 mem=5.5G lam=0.3128033395235737 Gm=None +[s_only s2] step 9900/12000 loss=4.3329 gn=1.04 lr=1.01e-04 tps=20952 mem=5.5G lam=0.3129155542701483 Gm=None +[s_only s2] step 9920/12000 loss=4.3940 gn=1.07 lr=1.00e-04 tps=20952 mem=5.5G lam=0.3130331278468172 Gm=None +[s_only s2] step 9940/12000 loss=4.3460 gn=1.01 lr=9.96e-05 tps=20952 mem=5.5G lam=0.31312812585383654 Gm=None +[s_only s2] step 9960/12000 loss=4.1908 gn=1.10 lr=9.89e-05 tps=20952 mem=5.5G lam=0.3132575511311491 Gm=None +[s_only s2] step 9980/12000 loss=4.3168 gn=1.03 lr=9.81e-05 tps=20952 mem=5.5G lam=0.313356245867908 Gm=None +[s_only s2] step 10000/12000 loss=4.2095 gn=1.12 lr=9.74e-05 tps=20952 mem=5.5G lam=0.31344156712293625 Gm=None +[s_only s2] EVAL step 10000: val_loss=4.4136 val_ppl=82.57 +[s_only s2] step 10020/12000 loss=4.4943 gn=1.08 lr=9.67e-05 tps=20944 mem=5.5G lam=0.3135682161276539 Gm=None +[s_only s2] step 10040/12000 loss=4.5404 gn=1.11 lr=9.59e-05 tps=20944 mem=5.5G lam=0.31365023273974657 Gm=None +[s_only s2] step 10060/12000 loss=4.5289 gn=1.03 lr=9.52e-05 tps=20944 mem=5.5G lam=0.3137767994776368 Gm=None +[s_only s2] step 10080/12000 loss=4.3640 gn=1.09 lr=9.45e-05 tps=20944 mem=5.5G lam=0.3139068717136979 Gm=None +[s_only s2] step 10100/12000 loss=4.5216 gn=1.08 lr=9.38e-05 tps=20944 mem=5.5G lam=0.31403287779539824 Gm=None +[s_only s2] step 10120/12000 loss=4.2964 gn=1.19 lr=9.31e-05 tps=20944 mem=5.5G lam=0.3141213372970621 Gm=None +[s_only s2] step 10140/12000 loss=4.2912 gn=1.11 lr=9.24e-05 tps=20944 mem=5.5G lam=0.31423222211500007 Gm=None +[s_only s2] step 10160/12000 loss=4.3864 gn=1.06 lr=9.18e-05 tps=20943 mem=5.5G lam=0.3143604143212239 Gm=None +[s_only s2] step 10180/12000 loss=4.3888 gn=1.16 lr=9.11e-05 tps=20944 mem=5.5G lam=0.31446348937849206 Gm=None +[s_only s2] step 10200/12000 loss=4.3001 gn=1.07 lr=9.04e-05 tps=20943 mem=5.5G lam=0.31457167211920023 Gm=None +[s_only s2] step 10220/12000 loss=4.3930 gn=1.04 lr=8.98e-05 tps=20943 mem=5.5G lam=0.314686903419594 Gm=None +[s_only s2] step 10240/12000 loss=4.2973 gn=1.10 lr=8.91e-05 tps=20943 mem=5.5G lam=0.31479004366944235 Gm=None +[s_only s2] step 10260/12000 loss=4.3161 gn=1.05 lr=8.85e-05 tps=20943 mem=5.5G lam=0.3149356059730053 Gm=None +[s_only s2] step 10280/12000 loss=4.4621 gn=1.07 lr=8.78e-05 tps=20943 mem=5.5G lam=0.315046984081467 Gm=None +[s_only s2] step 10300/12000 loss=4.3290 gn=1.13 lr=8.72e-05 tps=20943 mem=5.5G lam=0.31516568797330063 Gm=None +[s_only s2] step 10320/12000 loss=4.4829 gn=1.04 lr=8.66e-05 tps=20944 mem=5.5G lam=0.3152906059597929 Gm=None +[s_only s2] step 10340/12000 loss=4.3140 gn=1.12 lr=8.59e-05 tps=20944 mem=5.5G lam=0.315370158602794 Gm=None +[s_only s2] step 10360/12000 loss=4.3246 gn=1.05 lr=8.53e-05 tps=20944 mem=5.5G lam=0.3154366848369439 Gm=None +[s_only s2] step 10380/12000 loss=4.4606 gn=1.16 lr=8.47e-05 tps=20944 mem=5.5G lam=0.3155407576511304 Gm=None +[s_only s2] step 10400/12000 loss=4.4462 gn=1.03 lr=8.41e-05 tps=20944 mem=5.5G lam=0.31563942320644855 Gm=None +[s_only s2] step 10420/12000 loss=4.2734 gn=1.05 lr=8.35e-05 tps=20944 mem=5.5G lam=0.3157343876858552 Gm=None +[s_only s2] step 10440/12000 loss=4.2093 gn=1.04 lr=8.30e-05 tps=20944 mem=5.5G lam=0.3158433359737198 Gm=None +[s_only s2] step 10460/12000 loss=4.3770 gn=1.17 lr=8.24e-05 tps=20944 mem=5.5G lam=0.3159447442740202 Gm=None +[s_only s2] step 10480/12000 loss=4.3734 gn=1.07 lr=8.18e-05 tps=20943 mem=5.5G lam=0.3160390534127752 Gm=None +[s_only s2] step 10500/12000 loss=4.2808 gn=0.98 lr=8.12e-05 tps=20943 mem=5.5G lam=0.3161451180155079 Gm=None +[s_only s2] step 10520/12000 loss=4.2369 gn=1.00 lr=8.07e-05 tps=20942 mem=5.5G lam=0.3162609313925107 Gm=None +[s_only s2] step 10540/12000 loss=4.1131 gn=1.04 lr=8.01e-05 tps=20942 mem=5.5G lam=0.31634946446865797 Gm=None +[s_only s2] step 10560/12000 loss=4.3480 gn=1.07 lr=7.96e-05 tps=20942 mem=5.5G lam=0.31647471711039543 Gm=None +[s_only s2] step 10580/12000 loss=4.4044 gn=1.14 lr=7.91e-05 tps=20942 mem=5.5G lam=0.31658029463142157 Gm=None +[s_only s2] step 10600/12000 loss=4.4558 gn=1.14 lr=7.85e-05 tps=20942 mem=5.5G lam=0.31668085791170597 Gm=None +[s_only s2] step 10620/12000 loss=4.3825 gn=1.00 lr=7.80e-05 tps=20942 mem=5.5G lam=0.31675523798912764 Gm=None +[s_only s2] step 10640/12000 loss=4.3750 gn=1.06 lr=7.75e-05 tps=20942 mem=5.5G lam=0.3168450075512131 Gm=None +[s_only s2] step 10660/12000 loss=4.3486 gn=1.10 lr=7.70e-05 tps=20942 mem=5.5G lam=0.31692829572906095 Gm=None +[s_only s2] step 10680/12000 loss=4.4347 gn=1.05 lr=7.65e-05 tps=20942 mem=5.5G lam=0.3170183689023058 Gm=None +[s_only s2] step 10700/12000 loss=4.4570 gn=1.11 lr=7.60e-05 tps=20943 mem=5.5G lam=0.3171346504241228 Gm=None +[s_only s2] step 10720/12000 loss=4.3397 gn=1.02 lr=7.55e-05 tps=20943 mem=5.5G lam=0.3172450993830959 Gm=None +[s_only s2] step 10740/12000 loss=4.2111 gn=0.99 lr=7.50e-05 tps=20943 mem=5.5G lam=0.31733663318057853 Gm=None +[s_only s2] step 10760/12000 loss=4.3384 gn=1.09 lr=7.46e-05 tps=20943 mem=5.5G lam=0.3174064851676424 Gm=None +[s_only s2] step 10780/12000 loss=4.4516 gn=1.24 lr=7.41e-05 tps=20943 mem=5.5G lam=0.31751407341410715 Gm=None +[s_only s2] step 10800/12000 loss=4.3974 gn=1.16 lr=7.37e-05 tps=20943 mem=5.5G lam=0.3176001642520229 Gm=None +[s_only s2] step 10820/12000 loss=4.3170 gn=1.08 lr=7.32e-05 tps=20943 mem=5.5G lam=0.3176762890070677 Gm=None +[s_only s2] step 10840/12000 loss=4.3304 gn=1.20 lr=7.28e-05 tps=20943 mem=5.5G lam=0.31776675426711637 Gm=None +[s_only s2] step 10860/12000 loss=4.3026 gn=1.09 lr=7.23e-05 tps=20944 mem=5.5G lam=0.3178356569260359 Gm=None +[s_only s2] step 10880/12000 loss=4.4387 gn=1.08 lr=7.19e-05 tps=20944 mem=5.5G lam=0.31792027472207945 Gm=None +[s_only s2] step 10900/12000 loss=4.4938 gn=1.17 lr=7.15e-05 tps=20944 mem=5.5G lam=0.31800786467889947 Gm=None +[s_only s2] step 10920/12000 loss=4.5316 gn=1.36 lr=7.11e-05 tps=20944 mem=5.5G lam=0.3180863031496604 Gm=None +[s_only s2] step 10940/12000 loss=4.5354 gn=1.05 lr=7.07e-05 tps=20944 mem=5.5G lam=0.31818564763913554 Gm=None +[s_only s2] step 10960/12000 loss=4.2912 gn=1.08 lr=7.03e-05 tps=20944 mem=5.5G lam=0.318250703625381 Gm=None +[s_only s2] step 10980/12000 loss=4.1733 gn=1.09 lr=6.99e-05 tps=20944 mem=5.5G lam=0.31832778081297874 Gm=None +[s_only s2] step 11000/12000 loss=4.4134 gn=1.14 lr=6.95e-05 tps=20944 mem=5.5G lam=0.31840884933869046 Gm=None +[s_only s2] EVAL step 11000: val_loss=4.4170 val_ppl=82.84 +[s_only s2] step 11020/12000 loss=4.3584 gn=1.18 lr=6.91e-05 tps=20937 mem=5.5G lam=0.3184821304554741 Gm=None +[s_only s2] step 11040/12000 loss=4.2761 gn=1.17 lr=6.88e-05 tps=20937 mem=5.5G lam=0.31857701887687045 Gm=None +[s_only s2] step 11060/12000 loss=4.4017 gn=1.09 lr=6.84e-05 tps=20937 mem=5.5G lam=0.3186589435984691 Gm=None +[s_only s2] step 11080/12000 loss=4.3046 gn=1.13 lr=6.81e-05 tps=20937 mem=5.5G lam=0.3187442384660244 Gm=None +[s_only s2] step 11100/12000 loss=4.4506 gn=1.09 lr=6.77e-05 tps=20937 mem=5.5G lam=0.3188036906843384 Gm=None +[s_only s2] step 11120/12000 loss=4.1732 gn=1.11 lr=6.74e-05 tps=20937 mem=5.5G lam=0.31888922955840826 Gm=None +[s_only s2] step 11140/12000 loss=4.3649 gn=1.28 lr=6.70e-05 tps=20937 mem=5.5G lam=0.3189837612832586 Gm=None +[s_only s2] step 11160/12000 loss=4.3596 gn=1.11 lr=6.67e-05 tps=20937 mem=5.5G lam=0.3190742504472534 Gm=None +[s_only s2] step 11180/12000 loss=4.3524 gn=1.11 lr=6.64e-05 tps=20937 mem=5.5G lam=0.3191437888890505 Gm=None +[s_only s2] step 11200/12000 loss=4.1847 gn=1.06 lr=6.61e-05 tps=20937 mem=5.5G lam=0.3192397678891818 Gm=None +[s_only s2] step 11220/12000 loss=4.1324 gn=1.02 lr=6.58e-05 tps=20937 mem=5.5G lam=0.31933032255619764 Gm=None +[s_only s2] step 11240/12000 loss=4.3515 gn=1.24 lr=6.55e-05 tps=20937 mem=5.5G lam=0.3194219156478842 Gm=None +[s_only s2] step 11260/12000 loss=4.3744 gn=1.15 lr=6.52e-05 tps=20937 mem=5.5G lam=0.31950094488759834 Gm=None +[s_only s2] step 11280/12000 loss=4.3266 gn=1.15 lr=6.49e-05 tps=20937 mem=5.5G lam=0.319584546610713 Gm=None +[s_only s2] step 11300/12000 loss=4.3671 gn=1.06 lr=6.47e-05 tps=20937 mem=5.5G lam=0.319683985474209 Gm=None +[s_only s2] step 11320/12000 loss=4.2456 gn=1.07 lr=6.44e-05 tps=20937 mem=5.5G lam=0.319750570692122 Gm=None +[s_only s2] step 11340/12000 loss=4.4091 gn=1.26 lr=6.42e-05 tps=20937 mem=5.5G lam=0.3198445653542876 Gm=None +[s_only s2] step 11360/12000 loss=4.3223 gn=1.16 lr=6.39e-05 tps=20937 mem=5.5G lam=0.3199012711023291 Gm=None +[s_only s2] step 11380/12000 loss=4.3340 gn=1.14 lr=6.37e-05 tps=20937 mem=5.5G lam=0.3199791464333733 Gm=None +[s_only s2] step 11400/12000 loss=4.3759 gn=1.43 lr=6.34e-05 tps=20937 mem=5.5G lam=0.32003639452159405 Gm=None +[s_only s2] step 11420/12000 loss=4.3305 gn=1.03 lr=6.32e-05 tps=20937 mem=5.5G lam=0.32011103971550864 Gm=None +[s_only s2] step 11440/12000 loss=4.2018 gn=1.12 lr=6.30e-05 tps=20937 mem=5.5G lam=0.32020201285680133 Gm=None +[s_only s2] step 11460/12000 loss=4.2672 gn=1.09 lr=6.28e-05 tps=20938 mem=5.5G lam=0.3202570068339507 Gm=None +[s_only s2] step 11480/12000 loss=4.2477 gn=1.15 lr=6.26e-05 tps=20938 mem=5.5G lam=0.3203262286260724 Gm=None +[s_only s2] step 11500/12000 loss=4.3657 gn=1.14 lr=6.24e-05 tps=20938 mem=5.5G lam=0.3203818431744973 Gm=None +[s_only s2] step 11520/12000 loss=4.4691 gn=1.13 lr=6.22e-05 tps=20938 mem=5.5G lam=0.32046946107099455 Gm=None +[s_only s2] step 11540/12000 loss=4.3444 gn=1.21 lr=6.20e-05 tps=20938 mem=5.5G lam=0.3205560448889931 Gm=None +[s_only s2] step 11560/12000 loss=4.3037 gn=1.16 lr=6.19e-05 tps=20938 mem=5.5G lam=0.3206070481489102 Gm=None +[s_only s2] step 11580/12000 loss=4.4853 gn=1.30 lr=6.17e-05 tps=20938 mem=5.5G lam=0.3206480899825692 Gm=None +[s_only s2] step 11600/12000 loss=4.2072 gn=1.11 lr=6.15e-05 tps=20939 mem=5.5G lam=0.3207078240811825 Gm=None +[s_only s2] step 11620/12000 loss=4.3296 gn=1.07 lr=6.14e-05 tps=20939 mem=5.5G lam=0.32077139616012573 Gm=None +[s_only s2] step 11640/12000 loss=4.2504 gn=1.13 lr=6.12e-05 tps=20939 mem=5.5G lam=0.32085795638461906 Gm=None +[s_only s2] step 11660/12000 loss=4.2830 gn=1.26 lr=6.11e-05 tps=20939 mem=5.5G lam=0.32095415952305 Gm=None +[s_only s2] step 11680/12000 loss=4.3120 gn=1.08 lr=6.10e-05 tps=20939 mem=5.5G lam=0.3210279953976472 Gm=None +[s_only s2] step 11700/12000 loss=4.3523 gn=1.16 lr=6.09e-05 tps=20939 mem=5.5G lam=0.32109503261744976 Gm=None +[s_only s2] step 11720/12000 loss=4.3071 gn=1.16 lr=6.07e-05 tps=20939 mem=5.5G lam=0.3211668940881888 Gm=None +[s_only s2] step 11740/12000 loss=4.3488 gn=1.25 lr=6.06e-05 tps=20939 mem=5.5G lam=0.3212483761211236 Gm=None +[s_only s2] step 11760/12000 loss=4.2396 gn=1.18 lr=6.06e-05 tps=20939 mem=5.5G lam=0.3213210618123412 Gm=None +[s_only s2] step 11780/12000 loss=4.4185 gn=1.27 lr=6.05e-05 tps=20939 mem=5.5G lam=0.3213784700880448 Gm=None +[s_only s2] step 11800/12000 loss=4.3289 gn=1.15 lr=6.04e-05 tps=20938 mem=5.5G lam=0.3214435235907634 Gm=None +[s_only s2] step 11820/12000 loss=4.4963 gn=1.10 lr=6.03e-05 tps=20938 mem=5.5G lam=0.321516215801239 Gm=None +[s_only s2] step 11840/12000 loss=4.2640 gn=1.12 lr=6.02e-05 tps=20938 mem=5.5G lam=0.3215939151123166 Gm=None +[s_only s2] step 11860/12000 loss=4.2370 gn=1.20 lr=6.02e-05 tps=20938 mem=5.5G lam=0.32163970296581584 Gm=None +[s_only s2] step 11880/12000 loss=4.3461 gn=1.09 lr=6.01e-05 tps=20938 mem=5.5G lam=0.32171495134631795 Gm=None +[s_only s2] step 11900/12000 loss=4.3016 gn=1.16 lr=6.01e-05 tps=20939 mem=5.5G lam=0.32180316373705864 Gm=None +[s_only s2] step 11920/12000 loss=4.2151 gn=1.27 lr=6.01e-05 tps=20938 mem=5.5G lam=0.321872066706419 Gm=None +[s_only s2] step 11940/12000 loss=4.2026 gn=1.54 lr=6.00e-05 tps=20938 mem=5.5G lam=0.321935310959816 Gm=None +[s_only s2] step 11960/12000 loss=4.2968 gn=1.08 lr=6.00e-05 tps=20939 mem=5.5G lam=0.3220043744271 Gm=None +[s_only s2] step 11980/12000 loss=4.2947 gn=1.54 lr=6.00e-05 tps=20939 mem=5.5G lam=0.32208347382644814 Gm=None +[s_only s2] EVAL step 11999: val_loss=4.4076 val_ppl=82.07 +[init] variant=g_only seed=2 params=63.39M +[init] use_S=False use_G=True +[g_only s2] step 0/12000 loss=10.8905 gn=10.16 lr=3.00e-06 tps=6592 mem=5.0G lam=None Gm=0.5052897135416666 +[g_only s2] step 20/12000 loss=9.8805 gn=2.16 lr=6.30e-05 tps=20790 mem=5.5G lam=None Gm=0.436279296875 +[g_only s2] step 40/12000 loss=9.1124 gn=1.82 lr=1.23e-04 tps=19597 mem=5.5G lam=None Gm=0.427001953125 +[g_only s2] step 60/12000 loss=8.1915 gn=1.56 lr=1.83e-04 tps=20097 mem=5.5G lam=None Gm=0.4217122395833333 +[g_only s2] step 80/12000 loss=7.5614 gn=0.88 lr=2.43e-04 tps=20765 mem=5.5G lam=None Gm=0.4251302083333333 +[g_only s2] step 100/12000 loss=7.2424 gn=0.87 lr=3.03e-04 tps=21153 mem=5.5G lam=None Gm=0.4134114583333333 +[g_only s2] step 120/12000 loss=7.1390 gn=0.63 lr=3.63e-04 tps=21437 mem=5.5G lam=None Gm=0.4169108072916667 +[g_only s2] step 140/12000 loss=7.0135 gn=0.57 lr=4.23e-04 tps=21639 mem=5.5G lam=None Gm=0.40234375 +[g_only s2] step 160/12000 loss=6.7258 gn=0.53 lr=4.83e-04 tps=21793 mem=5.5G lam=None Gm=0.4013671875 +[g_only s2] step 180/12000 loss=6.8416 gn=0.52 lr=5.43e-04 tps=21906 mem=5.5G lam=None Gm=0.400634765625 +[g_only s2] step 200/12000 loss=6.6198 gn=0.55 lr=6.00e-04 tps=22002 mem=5.5G lam=None Gm=0.3798828125 +[g_only s2] step 220/12000 loss=6.6216 gn=0.58 lr=6.00e-04 tps=22081 mem=5.5G lam=None Gm=0.3935546875 +[g_only s2] step 240/12000 loss=6.5915 gn=0.52 lr=6.00e-04 tps=22141 mem=5.5G lam=None Gm=0.3861490885416667 +[g_only s2] step 260/12000 loss=6.4115 gn=0.44 lr=6.00e-04 tps=22189 mem=5.5G lam=None Gm=0.4021809895833333 +[g_only s2] step 280/12000 loss=6.5140 gn=0.48 lr=6.00e-04 tps=22234 mem=5.5G lam=None Gm=0.4115397135416667 +[g_only s2] step 300/12000 loss=6.6525 gn=0.47 lr=6.00e-04 tps=22276 mem=5.5G lam=None Gm=0.4124348958333333 +[g_only s2] step 320/12000 loss=6.6011 gn=0.49 lr=6.00e-04 tps=22312 mem=5.5G lam=None Gm=0.4034830729166667 +[g_only s2] step 340/12000 loss=6.4612 gn=0.46 lr=6.00e-04 tps=22342 mem=5.5G lam=None Gm=0.4058430989583333 +[g_only s2] step 360/12000 loss=6.4019 gn=0.54 lr=6.00e-04 tps=22371 mem=5.5G lam=None Gm=0.40625 +[g_only s2] step 380/12000 loss=6.2625 gn=0.49 lr=6.00e-04 tps=22399 mem=5.5G lam=None Gm=0.4348958333333333 +[g_only s2] step 400/12000 loss=6.4838 gn=0.49 lr=6.00e-04 tps=22421 mem=5.5G lam=None Gm=0.4185384114583333 +[g_only s2] step 420/12000 loss=6.2916 gn=0.45 lr=6.00e-04 tps=22440 mem=5.5G lam=None Gm=0.4432779947916667 +[g_only s2] step 440/12000 loss=6.3003 gn=0.47 lr=5.99e-04 tps=22459 mem=5.5G lam=None Gm=0.4318033854166667 +[g_only s2] step 460/12000 loss=6.2473 gn=0.46 lr=5.99e-04 tps=22470 mem=5.5G lam=None Gm=0.4419759114583333 +[g_only s2] step 480/12000 loss=6.4953 gn=0.50 lr=5.99e-04 tps=22489 mem=5.5G lam=None Gm=0.4442545572916667 +[g_only s2] step 500/12000 loss=6.3136 gn=0.48 lr=5.99e-04 tps=22504 mem=5.5G lam=None Gm=0.4357096354166667 +[g_only s2] step 520/12000 loss=6.3574 gn=0.53 lr=5.99e-04 tps=22517 mem=5.5G lam=None Gm=0.478515625 +[g_only s2] step 540/12000 loss=6.0854 gn=0.46 lr=5.99e-04 tps=22527 mem=5.5G lam=None Gm=0.4805501302083333 +[g_only s2] step 560/12000 loss=6.0611 gn=0.53 lr=5.99e-04 tps=22533 mem=5.5G lam=None Gm=0.5020345052083334 +[g_only s2] step 580/12000 loss=5.9147 gn=0.46 lr=5.99e-04 tps=22540 mem=5.5G lam=None Gm=0.4993489583333333 +[g_only s2] step 600/12000 loss=6.0519 gn=0.47 lr=5.98e-04 tps=22552 mem=5.5G lam=None Gm=0.513671875 +[g_only s2] step 620/12000 loss=6.2111 gn=0.50 lr=5.98e-04 tps=22561 mem=5.5G lam=None Gm=0.518310546875 +[g_only s2] step 640/12000 loss=6.1385 gn=0.47 lr=5.98e-04 tps=22572 mem=5.5G lam=None Gm=0.528076171875 +[g_only s2] step 660/12000 loss=6.1215 gn=0.49 lr=5.98e-04 tps=22584 mem=5.5G lam=None Gm=0.5537923177083334 +[g_only s2] step 680/12000 loss=6.0606 gn=0.44 lr=5.98e-04 tps=22591 mem=5.5G lam=None Gm=0.532470703125 +[g_only s2] step 700/12000 loss=5.9778 gn=0.51 lr=5.98e-04 tps=22600 mem=5.5G lam=None Gm=0.5603841145833334 +[g_only s2] step 720/12000 loss=6.0268 gn=0.60 lr=5.97e-04 tps=22605 mem=5.5G lam=None Gm=0.5616861979166666 +[g_only s2] step 740/12000 loss=6.0117 gn=0.52 lr=5.97e-04 tps=22614 mem=5.5G lam=None Gm=0.5357259114583334 +[g_only s2] step 760/12000 loss=6.0227 gn=0.48 lr=5.97e-04 tps=22620 mem=5.5G lam=None Gm=0.5782877604166666 +[g_only s2] step 780/12000 loss=5.7870 gn=0.48 lr=5.97e-04 tps=22628 mem=5.5G lam=None Gm=0.5655110677083334 +[g_only s2] step 800/12000 loss=5.8893 gn=0.50 lr=5.97e-04 tps=22632 mem=5.5G lam=None Gm=0.5841471354166666 +[g_only s2] step 820/12000 loss=5.8424 gn=0.46 lr=5.96e-04 tps=22637 mem=5.5G lam=None Gm=0.55810546875 +[g_only s2] step 840/12000 loss=5.9114 gn=0.49 lr=5.96e-04 tps=22641 mem=5.5G lam=None Gm=0.5636393229166666 +[g_only s2] step 860/12000 loss=5.8806 gn=0.50 lr=5.96e-04 tps=22646 mem=5.5G lam=None Gm=0.5511067708333334 +[g_only s2] step 880/12000 loss=5.9765 gn=0.57 lr=5.96e-04 tps=22649 mem=5.5G lam=None Gm=0.5885416666666666 +[g_only s2] step 900/12000 loss=5.7737 gn=0.50 lr=5.95e-04 tps=22647 mem=5.5G lam=None Gm=0.5504150390625 +[g_only s2] step 920/12000 loss=5.9124 gn=0.47 lr=5.95e-04 tps=22647 mem=5.5G lam=None Gm=0.5569661458333334 +[g_only s2] step 940/12000 loss=5.8328 gn=0.47 lr=5.95e-04 tps=22648 mem=5.5G lam=None Gm=0.5612386067708334 +[g_only s2] step 960/12000 loss=5.7309 gn=0.48 lr=5.94e-04 tps=22650 mem=5.5G lam=None Gm=0.572021484375 +[g_only s2] step 980/12000 loss=5.8104 gn=0.49 lr=5.94e-04 tps=22652 mem=5.5G lam=None Gm=0.56982421875 +[g_only s2] step 1000/12000 loss=5.7324 gn=0.50 lr=5.94e-04 tps=22653 mem=5.5G lam=None Gm=0.5630696614583334 +[g_only s2] EVAL step 1000: val_loss=5.7312 val_ppl=308.35 +[g_only s2] step 1020/12000 loss=5.6850 gn=0.49 lr=5.94e-04 tps=22579 mem=5.5G lam=None Gm=0.5563557942708334 +[g_only s2] step 1040/12000 loss=5.8191 gn=0.49 lr=5.93e-04 tps=22584 mem=5.5G lam=None Gm=0.5682373046875 +[g_only s2] step 1060/12000 loss=5.6883 gn=0.78 lr=5.93e-04 tps=22588 mem=5.5G lam=None Gm=0.5499267578125 +[g_only s2] step 1080/12000 loss=5.7648 gn=0.53 lr=5.93e-04 tps=22590 mem=5.5G lam=None Gm=0.5444742838541666 +[g_only s2] step 1100/12000 loss=5.5975 gn=0.49 lr=5.92e-04 tps=22593 mem=5.5G lam=None Gm=0.5621744791666666 +[g_only s2] step 1120/12000 loss=5.6899 gn=0.52 lr=5.92e-04 tps=22594 mem=5.5G lam=None Gm=0.5421142578125 +[g_only s2] step 1140/12000 loss=5.7412 gn=0.56 lr=5.92e-04 tps=22597 mem=5.5G lam=None Gm=0.5581461588541666 +[g_only s2] step 1160/12000 loss=5.6625 gn=0.50 lr=5.91e-04 tps=22600 mem=5.5G lam=None Gm=0.5555826822916666 +[g_only s2] step 1180/12000 loss=5.4883 gn=0.50 lr=5.91e-04 tps=22604 mem=5.5G lam=None Gm=0.5484212239583334 +[g_only s2] step 1200/12000 loss=5.5541 gn=0.48 lr=5.90e-04 tps=22603 mem=5.5G lam=None Gm=0.5351969401041666 +[g_only s2] step 1220/12000 loss=5.6044 gn=0.49 lr=5.90e-04 tps=22606 mem=5.5G lam=None Gm=0.5479329427083334 +[g_only s2] step 1240/12000 loss=5.5502 gn=0.58 lr=5.90e-04 tps=22610 mem=5.5G lam=None Gm=0.5462239583333334 +[g_only s2] step 1260/12000 loss=5.4804 gn=0.53 lr=5.89e-04 tps=22613 mem=5.5G lam=None Gm=0.5277913411458334 +[g_only s2] step 1280/12000 loss=5.4513 gn=0.53 lr=5.89e-04 tps=22615 mem=5.5G lam=None Gm=0.5511067708333334 +[g_only s2] step 1300/12000 loss=5.6064 gn=0.50 lr=5.89e-04 tps=22618 mem=5.5G lam=None Gm=0.5299072265625 +[g_only s2] step 1320/12000 loss=5.4722 gn=0.56 lr=5.88e-04 tps=22623 mem=5.5G lam=None Gm=0.54345703125 +[g_only s2] step 1340/12000 loss=5.6666 gn=0.55 lr=5.88e-04 tps=22627 mem=5.5G lam=None Gm=0.5423583984375 +[g_only s2] step 1360/12000 loss=5.4767 gn=0.52 lr=5.87e-04 tps=22630 mem=5.5G lam=None Gm=0.5489095052083334 +[g_only s2] step 1380/12000 loss=5.4949 gn=0.55 lr=5.87e-04 tps=22631 mem=5.5G lam=None Gm=0.5292561848958334 +[g_only s2] step 1400/12000 loss=5.3934 gn=0.53 lr=5.86e-04 tps=22634 mem=5.5G lam=None Gm=0.5227457682291666 +[g_only s2] step 1420/12000 loss=5.4659 gn=0.57 lr=5.86e-04 tps=22637 mem=5.5G lam=None Gm=0.5234375 +[g_only s2] step 1440/12000 loss=5.4999 gn=0.51 lr=5.85e-04 tps=22639 mem=5.5G lam=None Gm=0.5244954427083334 +[g_only s2] step 1460/12000 loss=5.4337 gn=0.59 lr=5.85e-04 tps=22640 mem=5.5G lam=None Gm=0.5108235677083334 +[g_only s2] step 1480/12000 loss=5.3577 gn=0.57 lr=5.84e-04 tps=22643 mem=5.5G lam=None Gm=0.5225626627604166 +[g_only s2] step 1500/12000 loss=5.6303 gn=0.60 lr=5.84e-04 tps=22646 mem=5.5G lam=None Gm=0.5278523763020834 +[g_only s2] step 1520/12000 loss=5.4162 gn=0.62 lr=5.83e-04 tps=22649 mem=5.5G lam=None Gm=0.5062459309895834 +[g_only s2] step 1540/12000 loss=5.3991 gn=0.51 lr=5.83e-04 tps=22650 mem=5.5G lam=None Gm=0.5170694986979166 +[g_only s2] step 1560/12000 loss=5.6102 gn=0.51 lr=5.82e-04 tps=22652 mem=5.5G lam=None Gm=0.5211385091145834 +[g_only s2] step 1580/12000 loss=5.5002 gn=0.56 lr=5.82e-04 tps=22654 mem=5.5G lam=None Gm=0.5106404622395834 +[g_only s2] step 1600/12000 loss=5.4035 gn=0.56 lr=5.81e-04 tps=22657 mem=5.5G lam=None Gm=0.5182291666666666 +[g_only s2] step 1620/12000 loss=5.3834 gn=0.53 lr=5.81e-04 tps=22658 mem=5.5G lam=None Gm=0.5196126302083334 +[g_only s2] step 1640/12000 loss=5.4492 gn=0.56 lr=5.80e-04 tps=22660 mem=5.5G lam=None Gm=0.5208943684895834 +[g_only s2] step 1660/12000 loss=5.1713 gn=0.55 lr=5.80e-04 tps=22662 mem=5.5G lam=None Gm=0.5033365885416666 +[g_only s2] step 1680/12000 loss=5.4410 gn=0.57 lr=5.79e-04 tps=22663 mem=5.5G lam=None Gm=0.5103352864583334 +[g_only s2] step 1700/12000 loss=5.5097 gn=0.59 lr=5.79e-04 tps=22665 mem=5.5G lam=None Gm=0.50445556640625 +[g_only s2] step 1720/12000 loss=5.3032 gn=0.57 lr=5.78e-04 tps=22667 mem=5.5G lam=None Gm=0.5204264322916666 +[g_only s2] step 1740/12000 loss=5.3685 gn=0.55 lr=5.78e-04 tps=22669 mem=5.5G lam=None Gm=0.5171101888020834 +[g_only s2] step 1760/12000 loss=5.2448 gn=0.55 lr=5.77e-04 tps=22670 mem=5.5G lam=None Gm=0.5015462239583334 +[g_only s2] step 1780/12000 loss=5.2823 gn=0.61 lr=5.76e-04 tps=22672 mem=5.5G lam=None Gm=0.50567626953125 +[g_only s2] step 1800/12000 loss=5.4544 gn=0.56 lr=5.76e-04 tps=22674 mem=5.5G lam=None Gm=0.50054931640625 +[g_only s2] step 1820/12000 loss=5.4790 gn=0.56 lr=5.75e-04 tps=22676 mem=5.5G lam=None Gm=0.4886474609375 +[g_only s2] step 1840/12000 loss=5.4174 gn=0.57 lr=5.75e-04 tps=22678 mem=5.5G lam=None Gm=0.5041097005208334 +[g_only s2] step 1860/12000 loss=5.4403 gn=0.62 lr=5.74e-04 tps=22679 mem=5.5G lam=None Gm=0.4883829752604167 +[g_only s2] step 1880/12000 loss=5.1817 gn=0.57 lr=5.73e-04 tps=22680 mem=5.5G lam=None Gm=0.4864908854166667 +[g_only s2] step 1900/12000 loss=5.2431 gn=0.65 lr=5.73e-04 tps=22682 mem=5.5G lam=None Gm=0.5016682942708334 +[g_only s2] step 1920/12000 loss=5.4238 gn=0.58 lr=5.72e-04 tps=22683 mem=5.5G lam=None Gm=0.4935302734375 +[g_only s2] step 1940/12000 loss=5.5611 gn=0.66 lr=5.72e-04 tps=22685 mem=5.5G lam=None Gm=0.4972330729166667 +[g_only s2] step 1960/12000 loss=5.1757 gn=0.57 lr=5.71e-04 tps=22686 mem=5.5G lam=None Gm=0.4930826822916667 +[g_only s2] step 1980/12000 loss=5.1623 gn=0.60 lr=5.70e-04 tps=22687 mem=5.5G lam=None Gm=0.4913533528645833 +[g_only s2] step 2000/12000 loss=5.1179 gn=0.61 lr=5.70e-04 tps=22689 mem=5.5G lam=None Gm=0.4912923177083333 +[g_only s2] EVAL step 2000: val_loss=5.2844 val_ppl=197.24 +[g_only s2] step 2020/12000 loss=5.1930 gn=0.59 lr=5.69e-04 tps=22652 mem=5.5G lam=None Gm=0.48016357421875 +[g_only s2] step 2040/12000 loss=5.3324 gn=0.80 lr=5.68e-04 tps=22653 mem=5.5G lam=None Gm=0.4796142578125 +[g_only s2] step 2060/12000 loss=5.3688 gn=0.70 lr=5.68e-04 tps=22656 mem=5.5G lam=None Gm=0.4782918294270833 +[g_only s2] step 2080/12000 loss=5.3927 gn=0.58 lr=5.67e-04 tps=22657 mem=5.5G lam=None Gm=0.4844970703125 +[g_only s2] step 2100/12000 loss=5.2067 gn=0.62 lr=5.66e-04 tps=22658 mem=5.5G lam=None Gm=0.4794514973958333 +[g_only s2] step 2120/12000 loss=5.3611 gn=0.68 lr=5.65e-04 tps=22658 mem=5.5G lam=None Gm=0.4737141927083333 +[g_only s2] step 2140/12000 loss=5.1343 gn=0.58 lr=5.65e-04 tps=22656 mem=5.5G lam=None Gm=0.4730021158854167 +[g_only s2] step 2160/12000 loss=5.1795 gn=0.60 lr=5.64e-04 tps=22657 mem=5.5G lam=None Gm=0.475738525390625 +[g_only s2] step 2180/12000 loss=5.2941 gn=0.61 lr=5.63e-04 tps=22659 mem=5.5G lam=None Gm=0.4795328776041667 +[g_only s2] step 2200/12000 loss=5.0680 gn=0.61 lr=5.63e-04 tps=22660 mem=5.5G lam=None Gm=0.4567667643229167 +[g_only s2] step 2220/12000 loss=5.1647 gn=0.61 lr=5.62e-04 tps=22661 mem=5.5G lam=None Gm=0.475311279296875 +[g_only s2] step 2240/12000 loss=5.2876 gn=0.59 lr=5.61e-04 tps=22662 mem=5.5G lam=None Gm=0.4803873697916667 +[g_only s2] step 2260/12000 loss=5.0723 gn=0.64 lr=5.60e-04 tps=22664 mem=5.5G lam=None Gm=0.4657999674479167 +[g_only s2] step 2280/12000 loss=5.1181 gn=0.65 lr=5.60e-04 tps=22663 mem=5.5G lam=None Gm=0.4778544108072917 +[g_only s2] step 2300/12000 loss=5.3242 gn=0.64 lr=5.59e-04 tps=22665 mem=5.5G lam=None Gm=0.4591878255208333 +[g_only s2] step 2320/12000 loss=5.2508 gn=0.65 lr=5.58e-04 tps=22665 mem=5.5G lam=None Gm=0.466827392578125 +[g_only s2] step 2340/12000 loss=5.1039 gn=0.61 lr=5.57e-04 tps=22666 mem=5.5G lam=None Gm=0.4684651692708333 +[g_only s2] step 2360/12000 loss=5.1176 gn=0.68 lr=5.57e-04 tps=22667 mem=5.5G lam=None Gm=0.4695231119791667 +[g_only s2] step 2380/12000 loss=5.0960 gn=0.66 lr=5.56e-04 tps=22669 mem=5.5G lam=None Gm=0.48406982421875 +[g_only s2] step 2400/12000 loss=5.0810 gn=0.61 lr=5.55e-04 tps=22670 mem=5.5G lam=None Gm=0.4647216796875 +[g_only s2] step 2420/12000 loss=5.2328 gn=0.64 lr=5.54e-04 tps=22672 mem=5.5G lam=None Gm=0.4826558430989583 +[g_only s2] step 2440/12000 loss=5.1062 gn=0.61 lr=5.53e-04 tps=22672 mem=5.5G lam=None Gm=0.4629007975260417 +[g_only s2] step 2460/12000 loss=5.0440 gn=0.62 lr=5.53e-04 tps=22673 mem=5.5G lam=None Gm=0.4675699869791667 +[g_only s2] step 2480/12000 loss=5.0550 gn=0.60 lr=5.52e-04 tps=22674 mem=5.5G lam=None Gm=0.4742838541666667 +[g_only s2] step 2500/12000 loss=5.1430 gn=0.64 lr=5.51e-04 tps=22676 mem=5.5G lam=None Gm=0.462432861328125 +[g_only s2] step 2520/12000 loss=4.9738 gn=0.61 lr=5.50e-04 tps=22677 mem=5.5G lam=None Gm=0.454742431640625 +[g_only s2] step 2540/12000 loss=5.0708 gn=0.64 lr=5.49e-04 tps=22678 mem=5.5G lam=None Gm=0.4590047200520833 +[g_only s2] step 2560/12000 loss=5.1622 gn=0.73 lr=5.48e-04 tps=22679 mem=5.5G lam=None Gm=0.47381591796875 +[g_only s2] step 2580/12000 loss=5.1506 gn=0.70 lr=5.48e-04 tps=22679 mem=5.5G lam=None Gm=0.4802144368489583 +[g_only s2] step 2600/12000 loss=5.0187 gn=0.59 lr=5.47e-04 tps=22681 mem=5.5G lam=None Gm=0.47052001953125 +[g_only s2] step 2620/12000 loss=4.9771 gn=0.66 lr=5.46e-04 tps=22682 mem=5.5G lam=None Gm=0.4625040690104167 +[g_only s2] step 2640/12000 loss=5.0932 gn=0.68 lr=5.45e-04 tps=22683 mem=5.5G lam=None Gm=0.4650675455729167 +[g_only s2] step 2660/12000 loss=5.1342 gn=0.64 lr=5.44e-04 tps=22685 mem=5.5G lam=None Gm=0.4666544596354167 +[g_only s2] step 2680/12000 loss=5.1061 gn=0.69 lr=5.43e-04 tps=22686 mem=5.5G lam=None Gm=0.4715983072916667 +[g_only s2] step 2700/12000 loss=5.1137 gn=0.64 lr=5.42e-04 tps=22687 mem=5.5G lam=None Gm=0.4589335123697917 +[g_only s2] step 2720/12000 loss=4.9872 gn=0.67 lr=5.41e-04 tps=22688 mem=5.5G lam=None Gm=0.4524943033854167 +[g_only s2] step 2740/12000 loss=5.0739 gn=0.59 lr=5.41e-04 tps=22688 mem=5.5G lam=None Gm=0.450042724609375 +[g_only s2] step 2760/12000 loss=5.0095 gn=0.63 lr=5.40e-04 tps=22689 mem=5.5G lam=None Gm=0.4508056640625 +[g_only s2] step 2780/12000 loss=4.9421 gn=0.63 lr=5.39e-04 tps=22690 mem=5.5G lam=None Gm=0.4373270670572917 +[g_only s2] step 2800/12000 loss=4.9860 gn=0.66 lr=5.38e-04 tps=22689 mem=5.5G lam=None Gm=0.4506022135416667 +[g_only s2] step 2820/12000 loss=4.9598 gn=0.66 lr=5.37e-04 tps=22690 mem=5.5G lam=None Gm=0.4437154134114583 +[g_only s2] step 2840/12000 loss=5.1060 gn=0.70 lr=5.36e-04 tps=22690 mem=5.5G lam=None Gm=0.4638570149739583 +[g_only s2] step 2860/12000 loss=4.8671 gn=0.69 lr=5.35e-04 tps=22691 mem=5.5G lam=None Gm=0.4420267740885417 +[g_only s2] step 2880/12000 loss=5.0353 gn=0.66 lr=5.34e-04 tps=22691 mem=5.5G lam=None Gm=0.4574686686197917 +[g_only s2] step 2900/12000 loss=4.9931 gn=0.69 lr=5.33e-04 tps=22691 mem=5.5G lam=None Gm=0.4514058430989583 +[g_only s2] step 2920/12000 loss=5.0424 gn=0.61 lr=5.32e-04 tps=22691 mem=5.5G lam=None Gm=0.447021484375 +[g_only s2] step 2940/12000 loss=5.1275 gn=0.66 lr=5.31e-04 tps=22692 mem=5.5G lam=None Gm=0.4496663411458333 +[g_only s2] step 2960/12000 loss=4.8846 gn=0.64 lr=5.30e-04 tps=22693 mem=5.5G lam=None Gm=0.4459431966145833 +[g_only s2] step 2980/12000 loss=4.9836 gn=0.66 lr=5.29e-04 tps=22693 mem=5.5G lam=None Gm=0.4384256998697917 +[g_only s2] step 3000/12000 loss=4.8805 gn=0.65 lr=5.28e-04 tps=22694 mem=5.5G lam=None Gm=0.4494120279947917 +[g_only s2] EVAL step 3000: val_loss=5.0398 val_ppl=154.44 +[g_only s2] step 3020/12000 loss=5.0617 gn=0.66 lr=5.27e-04 tps=22668 mem=5.5G lam=None Gm=0.4556986490885417 +[g_only s2] step 3040/12000 loss=5.0274 gn=0.70 lr=5.26e-04 tps=22669 mem=5.5G lam=None Gm=0.4429524739583333 +[g_only s2] step 3060/12000 loss=4.9518 gn=0.68 lr=5.25e-04 tps=22670 mem=5.5G lam=None Gm=0.4340108235677083 +[g_only s2] step 3080/12000 loss=4.8446 gn=0.64 lr=5.24e-04 tps=22669 mem=5.5G lam=None Gm=0.4388326009114583 +[g_only s2] step 3100/12000 loss=4.9757 gn=0.67 lr=5.23e-04 tps=22669 mem=5.5G lam=None Gm=0.4311014811197917 +[g_only s2] step 3120/12000 loss=4.7251 gn=0.71 lr=5.22e-04 tps=22671 mem=5.5G lam=None Gm=0.4437255859375 +[g_only s2] step 3140/12000 loss=4.8698 gn=0.64 lr=5.21e-04 tps=22671 mem=5.5G lam=None Gm=0.4434814453125 +[g_only s2] step 3160/12000 loss=4.8398 gn=0.63 lr=5.20e-04 tps=22672 mem=5.5G lam=None Gm=0.445220947265625 +[g_only s2] step 3180/12000 loss=4.9475 gn=0.62 lr=5.19e-04 tps=22673 mem=5.5G lam=None Gm=0.4402974446614583 +[g_only s2] step 3200/12000 loss=4.8823 gn=0.67 lr=5.18e-04 tps=22673 mem=5.5G lam=None Gm=0.4231160481770833 +[g_only s2] step 3220/12000 loss=4.9371 gn=0.69 lr=5.17e-04 tps=22674 mem=5.5G lam=None Gm=0.4395345052083333 +[g_only s2] step 3240/12000 loss=4.9884 gn=0.68 lr=5.16e-04 tps=22675 mem=5.5G lam=None Gm=0.4449666341145833 +[g_only s2] step 3260/12000 loss=5.0753 gn=0.67 lr=5.15e-04 tps=22676 mem=5.5G lam=None Gm=0.4401652018229167 +[g_only s2] step 3280/12000 loss=4.8406 gn=0.72 lr=5.14e-04 tps=22677 mem=5.5G lam=None Gm=0.4371134440104167 +[g_only s2] step 3300/12000 loss=4.7620 gn=0.69 lr=5.13e-04 tps=22679 mem=5.5G lam=None Gm=0.430328369140625 +[g_only s2] step 3320/12000 loss=4.8490 gn=0.74 lr=5.12e-04 tps=22679 mem=5.5G lam=None Gm=0.44268798828125 +[g_only s2] step 3340/12000 loss=4.8959 gn=0.66 lr=5.11e-04 tps=22679 mem=5.5G lam=None Gm=0.4423828125 +[g_only s2] step 3360/12000 loss=4.8525 gn=0.70 lr=5.10e-04 tps=22680 mem=5.5G lam=None Gm=0.4323628743489583 +[g_only s2] step 3380/12000 loss=4.9354 gn=0.66 lr=5.09e-04 tps=22681 mem=5.5G lam=None Gm=0.4411214192708333 +[g_only s2] step 3400/12000 loss=4.8699 gn=0.69 lr=5.08e-04 tps=22682 mem=5.5G lam=None Gm=0.4502461751302083 +[g_only s2] step 3420/12000 loss=4.9929 gn=0.66 lr=5.07e-04 tps=22683 mem=5.5G lam=None Gm=0.4383341471354167 +[g_only s2] step 3440/12000 loss=4.9184 gn=0.67 lr=5.06e-04 tps=22684 mem=5.5G lam=None Gm=0.4378763834635417 +[g_only s2] step 3460/12000 loss=4.9176 gn=0.77 lr=5.05e-04 tps=22685 mem=5.5G lam=None Gm=0.4361470540364583 +[g_only s2] step 3480/12000 loss=4.9490 gn=0.71 lr=5.03e-04 tps=22686 mem=5.5G lam=None Gm=0.4353129069010417 +[g_only s2] step 3500/12000 loss=4.8482 gn=0.69 lr=5.02e-04 tps=22686 mem=5.5G lam=None Gm=0.4349873860677083 +[g_only s2] step 3520/12000 loss=4.9036 gn=0.69 lr=5.01e-04 tps=22687 mem=5.5G lam=None Gm=0.4334055582682292 +[g_only s2] step 3540/12000 loss=4.8722 gn=0.67 lr=5.00e-04 tps=22687 mem=5.5G lam=None Gm=0.428955078125 +[g_only s2] step 3560/12000 loss=4.8013 gn=0.68 lr=4.99e-04 tps=22687 mem=5.5G lam=None Gm=0.4402364095052083 +[g_only s2] step 3580/12000 loss=4.8857 gn=0.75 lr=4.98e-04 tps=22688 mem=5.5G lam=None Gm=0.4342702229817708 +[g_only s2] step 3600/12000 loss=4.7443 gn=0.67 lr=4.97e-04 tps=22689 mem=5.5G lam=None Gm=0.4315745035807292 +[g_only s2] step 3620/12000 loss=4.8521 gn=0.72 lr=4.96e-04 tps=22689 mem=5.5G lam=None Gm=0.426422119140625 +[g_only s2] step 3640/12000 loss=4.7852 gn=0.70 lr=4.94e-04 tps=22690 mem=5.5G lam=None Gm=0.4347381591796875 +[g_only s2] step 3660/12000 loss=4.8483 gn=0.66 lr=4.93e-04 tps=22691 mem=5.5G lam=None Gm=0.4323628743489583 +[g_only s2] step 3680/12000 loss=4.8716 gn=0.66 lr=4.92e-04 tps=22691 mem=5.5G lam=None Gm=0.4336903889973958 +[g_only s2] step 3700/12000 loss=4.7929 gn=0.70 lr=4.91e-04 tps=22691 mem=5.5G lam=None Gm=0.4185384114583333 +[g_only s2] step 3720/12000 loss=4.9655 gn=0.70 lr=4.90e-04 tps=22692 mem=5.5G lam=None Gm=0.4273681640625 +[g_only s2] step 3740/12000 loss=4.8564 gn=0.71 lr=4.89e-04 tps=22692 mem=5.5G lam=None Gm=0.4201253255208333 +[g_only s2] step 3760/12000 loss=4.8122 gn=0.74 lr=4.88e-04 tps=22693 mem=5.5G lam=None Gm=0.4113413492838542 +[g_only s2] step 3780/12000 loss=4.7867 gn=0.71 lr=4.86e-04 tps=22694 mem=5.5G lam=None Gm=0.4171905517578125 +[g_only s2] step 3800/12000 loss=4.8924 gn=0.67 lr=4.85e-04 tps=22695 mem=5.5G lam=None Gm=0.4329630533854167 +[g_only s2] step 3820/12000 loss=4.8311 gn=0.68 lr=4.84e-04 tps=22696 mem=5.5G lam=None Gm=0.42279052734375 +[g_only s2] step 3840/12000 loss=4.9876 gn=0.69 lr=4.83e-04 tps=22696 mem=5.5G lam=None Gm=0.4304351806640625 +[g_only s2] step 3860/12000 loss=4.6406 gn=0.70 lr=4.82e-04 tps=22697 mem=5.5G lam=None Gm=0.41705322265625 +[g_only s2] step 3880/12000 loss=4.7462 gn=0.70 lr=4.80e-04 tps=22698 mem=5.5G lam=None Gm=0.4259745279947917 +[g_only s2] step 3900/12000 loss=4.7742 gn=0.65 lr=4.79e-04 tps=22698 mem=5.5G lam=None Gm=0.4114583333333333 +[g_only s2] step 3920/12000 loss=4.7932 gn=0.68 lr=4.78e-04 tps=22699 mem=5.5G lam=None Gm=0.4309844970703125 +[g_only s2] step 3940/12000 loss=4.9180 gn=0.71 lr=4.77e-04 tps=22700 mem=5.5G lam=None Gm=0.4263661702473958 +[g_only s2] step 3960/12000 loss=4.7004 gn=0.72 lr=4.76e-04 tps=22700 mem=5.5G lam=None Gm=0.421295166015625 +[g_only s2] step 3980/12000 loss=4.6387 gn=0.73 lr=4.74e-04 tps=22701 mem=5.5G lam=None Gm=0.4190572102864583 +[g_only s2] step 4000/12000 loss=4.8664 gn=0.70 lr=4.73e-04 tps=22702 mem=5.5G lam=None Gm=0.4163258870442708 +[g_only s2] EVAL step 4000: val_loss=4.8642 val_ppl=129.57 +[g_only s2] step 4020/12000 loss=4.8896 gn=0.73 lr=4.72e-04 tps=22683 mem=5.5G lam=None Gm=0.4218953450520833 +[g_only s2] step 4040/12000 loss=4.6737 gn=0.74 lr=4.71e-04 tps=22683 mem=5.5G lam=None Gm=0.4279937744140625 +[g_only s2] step 4060/12000 loss=4.7044 gn=0.75 lr=4.70e-04 tps=22684 mem=5.5G lam=None Gm=0.4234059651692708 +[g_only s2] step 4080/12000 loss=4.8238 gn=0.74 lr=4.68e-04 tps=22684 mem=5.5G lam=None Gm=0.4126180013020833 +[g_only s2] step 4100/12000 loss=4.7268 gn=0.73 lr=4.67e-04 tps=22684 mem=5.5G lam=None Gm=0.4149983723958333 +[g_only s2] step 4120/12000 loss=4.9097 gn=0.73 lr=4.66e-04 tps=22685 mem=5.5G lam=None Gm=0.4195810953776042 +[g_only s2] step 4140/12000 loss=4.8318 gn=0.73 lr=4.65e-04 tps=22686 mem=5.5G lam=None Gm=0.4228159586588542 +[g_only s2] step 4160/12000 loss=4.7492 gn=0.72 lr=4.63e-04 tps=22686 mem=5.5G lam=None Gm=0.4200541178385417 +[g_only s2] step 4180/12000 loss=4.8190 gn=0.67 lr=4.62e-04 tps=22686 mem=5.5G lam=None Gm=0.4147135416666667 +[g_only s2] step 4200/12000 loss=4.6574 gn=0.73 lr=4.61e-04 tps=22687 mem=5.5G lam=None Gm=0.4092508951822917 +[g_only s2] step 4220/12000 loss=4.7030 gn=0.69 lr=4.60e-04 tps=22688 mem=5.5G lam=None Gm=0.4293212890625 +[g_only s2] step 4240/12000 loss=4.7373 gn=0.72 lr=4.58e-04 tps=22689 mem=5.5G lam=None Gm=0.4128824869791667 +[g_only s2] step 4260/12000 loss=4.6321 gn=0.72 lr=4.57e-04 tps=22689 mem=5.5G lam=None Gm=0.413116455078125 +[g_only s2] step 4280/12000 loss=4.7187 gn=0.71 lr=4.56e-04 tps=22690 mem=5.5G lam=None Gm=0.4183451334635417 +[g_only s2] step 4300/12000 loss=4.7494 gn=0.82 lr=4.54e-04 tps=22691 mem=5.5G lam=None Gm=0.4118448893229167 +[g_only s2] step 4320/12000 loss=4.8459 gn=0.69 lr=4.53e-04 tps=22691 mem=5.5G lam=None Gm=0.4234466552734375 +[g_only s2] step 4340/12000 loss=4.7968 gn=0.73 lr=4.52e-04 tps=22692 mem=5.5G lam=None Gm=0.4102274576822917 +[g_only s2] step 4360/12000 loss=4.6781 gn=0.72 lr=4.51e-04 tps=22693 mem=5.5G lam=None Gm=0.4152170817057292 +[g_only s2] step 4380/12000 loss=4.6569 gn=0.72 lr=4.49e-04 tps=22693 mem=5.5G lam=None Gm=0.4015655517578125 +[g_only s2] step 4400/12000 loss=4.7196 gn=0.75 lr=4.48e-04 tps=22694 mem=5.5G lam=None Gm=0.4190521240234375 +[g_only s2] step 4420/12000 loss=4.7453 gn=0.72 lr=4.47e-04 tps=22694 mem=5.5G lam=None Gm=0.4155324300130208 +[g_only s2] step 4440/12000 loss=4.7420 gn=0.75 lr=4.45e-04 tps=22695 mem=5.5G lam=None Gm=0.4206492106119792 +[g_only s2] step 4460/12000 loss=4.8913 gn=0.74 lr=4.44e-04 tps=22696 mem=5.5G lam=None Gm=0.4086659749348958 +[g_only s2] step 4480/12000 loss=4.8320 gn=0.77 lr=4.43e-04 tps=22697 mem=5.5G lam=None Gm=0.416015625 +[g_only s2] step 4500/12000 loss=4.6918 gn=0.72 lr=4.42e-04 tps=22698 mem=5.5G lam=None Gm=0.412353515625 +[g_only s2] step 4520/12000 loss=4.8073 gn=0.75 lr=4.40e-04 tps=22698 mem=5.5G lam=None Gm=0.4203338623046875 +[g_only s2] step 4540/12000 loss=4.7654 gn=0.75 lr=4.39e-04 tps=22699 mem=5.5G lam=None Gm=0.4130605061848958 +[g_only s2] step 4560/12000 loss=4.7716 gn=0.76 lr=4.38e-04 tps=22700 mem=5.5G lam=None Gm=0.4023030598958333 +[g_only s2] step 4580/12000 loss=4.6046 gn=0.78 lr=4.36e-04 tps=22700 mem=5.5G lam=None Gm=0.4112294514973958 +[g_only s2] step 4600/12000 loss=4.6068 gn=0.71 lr=4.35e-04 tps=22700 mem=5.5G lam=None Gm=0.413970947265625 +[g_only s2] step 4620/12000 loss=4.6852 gn=0.72 lr=4.34e-04 tps=22701 mem=5.5G lam=None Gm=0.407440185546875 +[g_only s2] step 4640/12000 loss=4.6935 gn=0.72 lr=4.32e-04 tps=22702 mem=5.5G lam=None Gm=0.4008026123046875 +[g_only s2] step 4660/12000 loss=4.8501 gn=0.77 lr=4.31e-04 tps=22703 mem=5.5G lam=None Gm=0.4084421793619792 +[g_only s2] step 4680/12000 loss=4.8741 gn=0.76 lr=4.30e-04 tps=22704 mem=5.5G lam=None Gm=0.4175923665364583 +[g_only s2] step 4700/12000 loss=4.7029 gn=0.72 lr=4.28e-04 tps=22704 mem=5.5G lam=None Gm=0.4096221923828125 +[g_only s2] step 4720/12000 loss=4.5715 gn=0.79 lr=4.27e-04 tps=22705 mem=5.5G lam=None Gm=0.4029693603515625 +[g_only s2] step 4740/12000 loss=4.8803 gn=0.72 lr=4.26e-04 tps=22705 mem=5.5G lam=None Gm=0.4084320068359375 +[g_only s2] step 4760/12000 loss=4.5275 gn=0.75 lr=4.24e-04 tps=22706 mem=5.5G lam=None Gm=0.4013570149739583 +[g_only s2] step 4780/12000 loss=4.5136 gn=0.76 lr=4.23e-04 tps=22706 mem=5.5G lam=None Gm=0.404266357421875 +[g_only s2] step 4800/12000 loss=4.7774 gn=0.74 lr=4.22e-04 tps=22707 mem=5.5G lam=None Gm=0.4099934895833333 +[g_only s2] step 4820/12000 loss=4.6827 gn=0.74 lr=4.20e-04 tps=22708 mem=5.5G lam=None Gm=0.4095611572265625 +[g_only s2] step 4840/12000 loss=4.7501 gn=0.73 lr=4.19e-04 tps=22708 mem=5.5G lam=None Gm=0.3989461263020833 +[g_only s2] step 4860/12000 loss=4.7577 gn=0.76 lr=4.18e-04 tps=22708 mem=5.5G lam=None Gm=0.4147796630859375 +[g_only s2] step 4880/12000 loss=4.6304 gn=0.79 lr=4.16e-04 tps=22709 mem=5.5G lam=None Gm=0.4041035970052083 +[g_only s2] step 4900/12000 loss=4.6907 gn=0.74 lr=4.15e-04 tps=22710 mem=5.5G lam=None Gm=0.4107716878255208 +[g_only s2] step 4920/12000 loss=4.6693 gn=0.81 lr=4.13e-04 tps=22710 mem=5.5G lam=None Gm=0.4105733235677083 +[g_only s2] step 4940/12000 loss=4.6494 gn=0.70 lr=4.12e-04 tps=22710 mem=5.5G lam=None Gm=0.4005584716796875 +[g_only s2] step 4960/12000 loss=4.6317 gn=0.72 lr=4.11e-04 tps=22711 mem=5.5G lam=None Gm=0.4045155843098958 +[g_only s2] step 4980/12000 loss=4.6350 gn=0.77 lr=4.09e-04 tps=22711 mem=5.5G lam=None Gm=0.4083455403645833 +[g_only s2] step 5000/12000 loss=4.5454 gn=0.72 lr=4.08e-04 tps=22711 mem=5.5G lam=None Gm=0.3948872884114583 +[g_only s2] EVAL step 5000: val_loss=4.7388 val_ppl=114.29 +[g_only s2] step 5020/12000 loss=4.8083 gn=0.75 lr=4.07e-04 tps=22696 mem=5.5G lam=None Gm=0.4080098470052083 +[g_only s2] step 5040/12000 loss=4.6165 gn=0.80 lr=4.05e-04 tps=22696 mem=5.5G lam=None Gm=0.4147288004557292 +[g_only s2] step 5060/12000 loss=4.6361 gn=0.74 lr=4.04e-04 tps=22696 mem=5.5G lam=None Gm=0.4031219482421875 +[g_only s2] step 5080/12000 loss=4.5489 gn=0.74 lr=4.02e-04 tps=22694 mem=5.5G lam=None Gm=0.4016520182291667 +[g_only s2] step 5100/12000 loss=4.6541 gn=0.74 lr=4.01e-04 tps=22695 mem=5.5G lam=None Gm=0.4025115966796875 +[g_only s2] step 5120/12000 loss=4.4888 gn=0.76 lr=4.00e-04 tps=22694 mem=5.5G lam=None Gm=0.4011383056640625 +[g_only s2] step 5140/12000 loss=4.7935 gn=0.84 lr=3.98e-04 tps=22694 mem=5.5G lam=None Gm=0.4087371826171875 +[g_only s2] step 5160/12000 loss=4.8380 gn=0.82 lr=3.97e-04 tps=22694 mem=5.5G lam=None Gm=0.4083455403645833 +[g_only s2] step 5180/12000 loss=4.6244 gn=0.73 lr=3.95e-04 tps=22693 mem=5.5G lam=None Gm=0.4011891682942708 +[g_only s2] step 5200/12000 loss=4.6545 gn=0.79 lr=3.94e-04 tps=22692 mem=5.5G lam=None Gm=0.4001871744791667 +[g_only s2] step 5220/12000 loss=4.7301 gn=0.77 lr=3.93e-04 tps=22692 mem=5.5G lam=None Gm=0.4016164143880208 +[g_only s2] step 5240/12000 loss=4.5999 gn=0.76 lr=3.91e-04 tps=22692 mem=5.5G lam=None Gm=0.3951466878255208 +[g_only s2] step 5260/12000 loss=4.4573 gn=0.76 lr=3.90e-04 tps=22692 mem=5.5G lam=None Gm=0.4032948811848958 +[g_only s2] step 5280/12000 loss=4.8023 gn=0.77 lr=3.88e-04 tps=22692 mem=5.5G lam=None Gm=0.412322998046875 +[g_only s2] step 5300/12000 loss=4.6841 gn=0.78 lr=3.87e-04 tps=22692 mem=5.5G lam=None Gm=0.4116058349609375 +[g_only s2] step 5320/12000 loss=4.7252 gn=0.79 lr=3.86e-04 tps=22692 mem=5.5G lam=None Gm=0.39617919921875 +[g_only s2] step 5340/12000 loss=4.4718 gn=0.73 lr=3.84e-04 tps=22693 mem=5.5G lam=None Gm=0.3972371419270833 +[g_only s2] step 5360/12000 loss=4.7215 gn=0.76 lr=3.83e-04 tps=22693 mem=5.5G lam=None Gm=0.4118601481119792 +[g_only s2] step 5380/12000 loss=4.4875 gn=0.84 lr=3.81e-04 tps=22693 mem=5.5G lam=None Gm=0.4015909830729167 +[g_only s2] step 5400/12000 loss=4.5568 gn=0.78 lr=3.80e-04 tps=22693 mem=5.5G lam=None Gm=0.3969828287760417 +[g_only s2] step 5420/12000 loss=4.6936 gn=0.76 lr=3.79e-04 tps=22694 mem=5.5G lam=None Gm=0.4098307291666667 +[g_only s2] step 5440/12000 loss=4.5340 gn=0.73 lr=3.77e-04 tps=22694 mem=5.5G lam=None Gm=0.3980763753255208 +[g_only s2] step 5460/12000 loss=4.7729 gn=0.84 lr=3.76e-04 tps=22694 mem=5.5G lam=None Gm=0.3990987141927083 +[g_only s2] step 5480/12000 loss=4.5307 gn=0.78 lr=3.74e-04 tps=22693 mem=5.5G lam=None Gm=0.4024149576822917 +[g_only s2] step 5500/12000 loss=4.8875 gn=0.77 lr=3.73e-04 tps=22693 mem=5.5G lam=None Gm=0.4044138590494792 +[g_only s2] step 5520/12000 loss=4.6063 gn=0.84 lr=3.72e-04 tps=22693 mem=5.5G lam=None Gm=0.4057871500651042 +[g_only s2] step 5540/12000 loss=4.5803 gn=0.76 lr=3.70e-04 tps=22693 mem=5.5G lam=None Gm=0.4007008870442708 +[g_only s2] step 5560/12000 loss=4.6503 gn=0.73 lr=3.69e-04 tps=22692 mem=5.5G lam=None Gm=0.4074554443359375 +[g_only s2] step 5580/12000 loss=4.5975 gn=0.78 lr=3.67e-04 tps=22692 mem=5.5G lam=None Gm=0.3935648600260417 +[g_only s2] step 5600/12000 loss=4.7562 gn=0.76 lr=3.66e-04 tps=22691 mem=5.5G lam=None Gm=0.3907368977864583 +[g_only s2] step 5620/12000 loss=4.6563 gn=0.79 lr=3.64e-04 tps=22691 mem=5.5G lam=None Gm=0.3994038899739583 +[g_only s2] step 5640/12000 loss=4.5221 gn=0.79 lr=3.63e-04 tps=22691 mem=5.5G lam=None Gm=0.4011077880859375 +[g_only s2] step 5660/12000 loss=4.5570 gn=0.76 lr=3.62e-04 tps=22691 mem=5.5G lam=None Gm=0.4034372965494792 +[g_only s2] step 5680/12000 loss=4.6599 gn=0.80 lr=3.60e-04 tps=22691 mem=5.5G lam=None Gm=0.4017232259114583 +[g_only s2] step 5700/12000 loss=4.5695 gn=0.78 lr=3.59e-04 tps=22692 mem=5.5G lam=None Gm=0.3835652669270833 +[g_only s2] step 5720/12000 loss=4.5220 gn=0.76 lr=3.57e-04 tps=22692 mem=5.5G lam=None Gm=0.3983052571614583 +[g_only s2] step 5740/12000 loss=4.5552 gn=0.77 lr=3.56e-04 tps=22691 mem=5.5G lam=None Gm=0.4013519287109375 +[g_only s2] step 5760/12000 loss=4.3379 gn=0.74 lr=3.54e-04 tps=22692 mem=5.5G lam=None Gm=0.384979248046875 +[g_only s2] step 5780/12000 loss=4.5290 gn=0.73 lr=3.53e-04 tps=22691 mem=5.5G lam=None Gm=0.3918304443359375 +[g_only s2] step 5800/12000 loss=4.6298 gn=0.77 lr=3.52e-04 tps=22691 mem=5.5G lam=None Gm=0.3967437744140625 +[g_only s2] step 5820/12000 loss=4.5986 gn=0.80 lr=3.50e-04 tps=22692 mem=5.5G lam=None Gm=0.3949228922526042 +[g_only s2] step 5840/12000 loss=4.5989 gn=0.76 lr=3.49e-04 tps=22692 mem=5.5G lam=None Gm=0.3910115559895833 +[g_only s2] step 5860/12000 loss=4.6366 gn=0.80 lr=3.47e-04 tps=22692 mem=5.5G lam=None Gm=0.3939768473307292 +[g_only s2] step 5880/12000 loss=4.6339 gn=0.78 lr=3.46e-04 tps=22692 mem=5.5G lam=None Gm=0.398468017578125 +[g_only s2] step 5900/12000 loss=4.7860 gn=0.77 lr=3.44e-04 tps=22692 mem=5.5G lam=None Gm=0.4014078776041667 +[g_only s2] step 5920/12000 loss=4.6220 gn=0.81 lr=3.43e-04 tps=22692 mem=5.5G lam=None Gm=0.3944803873697917 +[g_only s2] step 5940/12000 loss=4.6033 gn=0.80 lr=3.41e-04 tps=22692 mem=5.5G lam=None Gm=0.3946787516276042 +[g_only s2] step 5960/12000 loss=4.6412 gn=0.74 lr=3.40e-04 tps=22692 mem=5.5G lam=None Gm=0.3865254720052083 +[g_only s2] step 5980/12000 loss=4.5086 gn=0.84 lr=3.39e-04 tps=22693 mem=5.5G lam=None Gm=0.3955434163411458 +[g_only s2] step 6000/12000 loss=4.5319 gn=0.81 lr=3.37e-04 tps=22693 mem=5.5G lam=None Gm=0.3934682210286458 +[g_only s2] EVAL step 6000: val_loss=4.6267 val_ppl=102.17 +[g_only s2] step 6020/12000 loss=4.4675 gn=0.77 lr=3.36e-04 tps=22680 mem=5.5G lam=None Gm=0.3888753255208333 +[g_only s2] step 6040/12000 loss=4.6073 gn=0.78 lr=3.34e-04 tps=22680 mem=5.5G lam=None Gm=0.3993123372395833 +[g_only s2] step 6060/12000 loss=4.5063 gn=0.80 lr=3.33e-04 tps=22680 mem=5.5G lam=None Gm=0.4021504720052083 +[g_only s2] step 6080/12000 loss=4.6006 gn=0.81 lr=3.31e-04 tps=22680 mem=5.5G lam=None Gm=0.3978627522786458 +[g_only s2] step 6100/12000 loss=4.5593 gn=0.81 lr=3.30e-04 tps=22680 mem=5.5G lam=None Gm=0.4018046061197917 +[g_only s2] step 6120/12000 loss=4.4538 gn=0.76 lr=3.29e-04 tps=22680 mem=5.5G lam=None Gm=0.3939971923828125 +[g_only s2] step 6140/12000 loss=4.5335 gn=0.84 lr=3.27e-04 tps=22681 mem=5.5G lam=None Gm=0.3853607177734375 +[g_only s2] step 6160/12000 loss=4.6238 gn=0.75 lr=3.26e-04 tps=22681 mem=5.5G lam=None Gm=0.4012400309244792 +[g_only s2] step 6180/12000 loss=4.5151 gn=0.78 lr=3.24e-04 tps=22681 mem=5.5G lam=None Gm=0.3979288736979167 +[g_only s2] step 6200/12000 loss=4.5917 gn=0.80 lr=3.23e-04 tps=22681 mem=5.5G lam=None Gm=0.3930409749348958 +[g_only s2] step 6220/12000 loss=4.4893 gn=0.80 lr=3.21e-04 tps=22681 mem=5.5G lam=None Gm=0.38916015625 +[g_only s2] step 6240/12000 loss=4.4353 gn=0.78 lr=3.20e-04 tps=22681 mem=5.5G lam=None Gm=0.3851267496744792 +[g_only s2] step 6260/12000 loss=4.4393 gn=0.83 lr=3.19e-04 tps=22681 mem=5.5G lam=None Gm=0.3953653971354167 +[g_only s2] step 6280/12000 loss=4.7339 gn=0.80 lr=3.17e-04 tps=22681 mem=5.5G lam=None Gm=0.3922932942708333 +[g_only s2] step 6300/12000 loss=4.7208 gn=0.82 lr=3.16e-04 tps=22681 mem=5.5G lam=None Gm=0.3865966796875 +[g_only s2] step 6320/12000 loss=4.4999 gn=0.82 lr=3.14e-04 tps=22681 mem=5.5G lam=None Gm=0.3899993896484375 +[g_only s2] step 6340/12000 loss=4.6183 gn=0.82 lr=3.13e-04 tps=22681 mem=5.5G lam=None Gm=0.3922627766927083 +[g_only s2] step 6360/12000 loss=4.4994 gn=0.76 lr=3.11e-04 tps=22681 mem=5.5G lam=None Gm=0.3840484619140625 +[g_only s2] step 6380/12000 loss=4.5370 gn=0.78 lr=3.10e-04 tps=22681 mem=5.5G lam=None Gm=0.3899383544921875 +[g_only s2] step 6400/12000 loss=4.3743 gn=0.81 lr=3.08e-04 tps=22681 mem=5.5G lam=None Gm=0.3812357584635417 +[g_only s2] step 6420/12000 loss=4.5722 gn=0.82 lr=3.07e-04 tps=22681 mem=5.5G lam=None Gm=0.4014180501302083 +[g_only s2] step 6440/12000 loss=4.7282 gn=0.83 lr=3.06e-04 tps=22681 mem=5.5G lam=None Gm=0.3953908284505208 +[g_only s2] step 6460/12000 loss=4.5161 gn=0.83 lr=3.04e-04 tps=22681 mem=5.5G lam=None Gm=0.39398193359375 +[g_only s2] step 6480/12000 loss=4.5898 gn=0.97 lr=3.03e-04 tps=22682 mem=5.5G lam=None Gm=0.390960693359375 +[g_only s2] step 6500/12000 loss=4.3882 gn=0.82 lr=3.01e-04 tps=22682 mem=5.5G lam=None Gm=0.389617919921875 +[g_only s2] step 6520/12000 loss=4.6593 gn=0.85 lr=3.00e-04 tps=22682 mem=5.5G lam=None Gm=0.3950093587239583 +[g_only s2] step 6540/12000 loss=4.5808 gn=0.87 lr=2.98e-04 tps=22682 mem=5.5G lam=None Gm=0.40057373046875 +[g_only s2] step 6560/12000 loss=4.5614 gn=0.83 lr=2.97e-04 tps=22683 mem=5.5G lam=None Gm=0.3989969889322917 +[g_only s2] step 6580/12000 loss=4.4359 gn=0.84 lr=2.96e-04 tps=22683 mem=5.5G lam=None Gm=0.3953501383463542 +[g_only s2] step 6600/12000 loss=4.6040 gn=0.79 lr=2.94e-04 tps=22683 mem=5.5G lam=None Gm=0.4027608235677083 +[g_only s2] step 6620/12000 loss=4.4638 gn=0.84 lr=2.93e-04 tps=22683 mem=5.5G lam=None Gm=0.3867085774739583 +[g_only s2] step 6640/12000 loss=4.5885 gn=0.84 lr=2.91e-04 tps=22683 mem=5.5G lam=None Gm=0.3845316569010417 +[g_only s2] step 6660/12000 loss=4.6772 gn=0.81 lr=2.90e-04 tps=22683 mem=5.5G lam=None Gm=0.3979136149088542 +[g_only s2] step 6680/12000 loss=4.5029 gn=0.82 lr=2.88e-04 tps=22683 mem=5.5G lam=None Gm=0.3922017415364583 +[g_only s2] step 6700/12000 loss=4.5926 gn=0.86 lr=2.87e-04 tps=22683 mem=5.5G lam=None Gm=0.3935597737630208 +[g_only s2] step 6720/12000 loss=4.6028 gn=0.81 lr=2.86e-04 tps=22683 mem=5.5G lam=None Gm=0.3894195556640625 +[g_only s2] step 6740/12000 loss=4.5699 gn=0.78 lr=2.84e-04 tps=22684 mem=5.5G lam=None Gm=0.3924001057942708 +[g_only s2] step 6760/12000 loss=4.5784 gn=0.76 lr=2.83e-04 tps=22684 mem=5.5G lam=None Gm=0.3881734212239583 +[g_only s2] step 6780/12000 loss=4.5420 gn=0.77 lr=2.81e-04 tps=22684 mem=5.5G lam=None Gm=0.3876953125 +[g_only s2] step 6800/12000 loss=4.5865 gn=0.86 lr=2.80e-04 tps=22684 mem=5.5G lam=None Gm=0.3860575358072917 +[g_only s2] step 6820/12000 loss=4.5462 gn=0.78 lr=2.79e-04 tps=22684 mem=5.5G lam=None Gm=0.3911692301432292 +[g_only s2] step 6840/12000 loss=4.6052 gn=0.78 lr=2.77e-04 tps=22684 mem=5.5G lam=None Gm=0.3895212809244792 +[g_only s2] step 6860/12000 loss=4.6050 gn=0.87 lr=2.76e-04 tps=22683 mem=5.5G lam=None Gm=0.3960622151692708 +[g_only s2] step 6880/12000 loss=4.4840 gn=0.82 lr=2.74e-04 tps=22683 mem=5.5G lam=None Gm=0.397918701171875 +[g_only s2] step 6900/12000 loss=4.5564 gn=0.85 lr=2.73e-04 tps=22683 mem=5.5G lam=None Gm=0.381988525390625 +[g_only s2] step 6920/12000 loss=4.4119 gn=0.82 lr=2.72e-04 tps=22684 mem=5.5G lam=None Gm=0.390167236328125 +[g_only s2] step 6940/12000 loss=4.7639 gn=0.84 lr=2.70e-04 tps=22684 mem=5.5G lam=None Gm=0.3911234537760417 +[g_only s2] step 6960/12000 loss=4.5231 gn=0.86 lr=2.69e-04 tps=22684 mem=5.5G lam=None Gm=0.3861185709635417 +[g_only s2] step 6980/12000 loss=4.4543 gn=0.81 lr=2.67e-04 tps=22684 mem=5.5G lam=None Gm=0.3850148518880208 +[g_only s2] step 7000/12000 loss=4.5679 gn=0.83 lr=2.66e-04 tps=22684 mem=5.5G lam=None Gm=0.3797607421875 +[g_only s2] EVAL step 7000: val_loss=4.5681 val_ppl=96.36 +[g_only s2] step 7020/12000 loss=4.4617 gn=0.82 lr=2.65e-04 tps=22673 mem=5.5G lam=None Gm=0.3901774088541667 +[g_only s2] step 7040/12000 loss=4.5697 gn=0.84 lr=2.63e-04 tps=22673 mem=5.5G lam=None Gm=0.3918406168619792 +[g_only s2] step 7060/12000 loss=4.6065 gn=0.83 lr=2.62e-04 tps=22673 mem=5.5G lam=None Gm=0.3917338053385417 +[g_only s2] step 7080/12000 loss=4.4696 gn=0.84 lr=2.60e-04 tps=22673 mem=5.5G lam=None Gm=0.3949228922526042 +[g_only s2] step 7100/12000 loss=4.4749 gn=0.82 lr=2.59e-04 tps=22673 mem=5.5G lam=None Gm=0.3844807942708333 +[g_only s2] step 7120/12000 loss=4.4300 gn=0.87 lr=2.58e-04 tps=22672 mem=5.5G lam=None Gm=0.3865102132161458 +[g_only s2] step 7140/12000 loss=4.5677 gn=0.82 lr=2.56e-04 tps=22672 mem=5.5G lam=None Gm=0.3775227864583333 +[g_only s2] step 7160/12000 loss=4.4268 gn=0.82 lr=2.55e-04 tps=22672 mem=5.5G lam=None Gm=0.3889516194661458 +[g_only s2] step 7180/12000 loss=4.5047 gn=0.85 lr=2.53e-04 tps=22672 mem=5.5G lam=None Gm=0.38446044921875 +[g_only s2] step 7200/12000 loss=4.5477 gn=0.88 lr=2.52e-04 tps=22672 mem=5.5G lam=None Gm=0.3812204996744792 +[g_only s2] step 7220/12000 loss=4.5741 gn=0.84 lr=2.51e-04 tps=22672 mem=5.5G lam=None Gm=0.3904876708984375 +[g_only s2] step 7240/12000 loss=4.3680 gn=0.86 lr=2.49e-04 tps=22673 mem=5.5G lam=None Gm=0.38714599609375 +[g_only s2] step 7260/12000 loss=4.5805 gn=0.91 lr=2.48e-04 tps=22673 mem=5.5G lam=None Gm=0.3844451904296875 +[g_only s2] step 7280/12000 loss=4.4715 gn=0.88 lr=2.47e-04 tps=22673 mem=5.5G lam=None Gm=0.3826904296875 +[g_only s2] step 7300/12000 loss=4.4653 gn=0.84 lr=2.45e-04 tps=22673 mem=5.5G lam=None Gm=0.3891042073567708 +[g_only s2] step 7320/12000 loss=4.6209 gn=0.86 lr=2.44e-04 tps=22674 mem=5.5G lam=None Gm=0.3962148030598958 +[g_only s2] step 7340/12000 loss=4.4030 gn=0.81 lr=2.42e-04 tps=22674 mem=5.5G lam=None Gm=0.3846689860026042 +[g_only s2] step 7360/12000 loss=4.3822 gn=0.90 lr=2.41e-04 tps=22674 mem=5.5G lam=None Gm=0.3853556315104167 +[g_only s2] step 7380/12000 loss=4.6003 gn=0.83 lr=2.40e-04 tps=22674 mem=5.5G lam=None Gm=0.3907267252604167 +[g_only s2] step 7400/12000 loss=4.4286 gn=0.83 lr=2.38e-04 tps=22674 mem=5.5G lam=None Gm=0.3799641927083333 +[g_only s2] step 7420/12000 loss=4.6577 gn=0.83 lr=2.37e-04 tps=22675 mem=5.5G lam=None Gm=0.3931121826171875 +[g_only s2] step 7440/12000 loss=4.3962 gn=0.80 lr=2.36e-04 tps=22675 mem=5.5G lam=None Gm=0.3898061116536458 +[g_only s2] step 7460/12000 loss=4.5322 gn=0.87 lr=2.34e-04 tps=22675 mem=5.5G lam=None Gm=0.376434326171875 +[g_only s2] step 7480/12000 loss=4.5091 gn=0.81 lr=2.33e-04 tps=22675 mem=5.5G lam=None Gm=0.3774363199869792 +[g_only s2] step 7500/12000 loss=4.4626 gn=0.82 lr=2.32e-04 tps=22675 mem=5.5G lam=None Gm=0.3787078857421875 +[g_only s2] step 7520/12000 loss=4.3465 gn=0.79 lr=2.30e-04 tps=22675 mem=5.5G lam=None Gm=0.3869578043619792 +[g_only s2] step 7540/12000 loss=4.4471 gn=0.86 lr=2.29e-04 tps=22675 mem=5.5G lam=None Gm=0.3838399251302083 +[g_only s2] step 7560/12000 loss=4.5947 gn=0.81 lr=2.28e-04 tps=22675 mem=5.5G lam=None Gm=0.3857574462890625 +[g_only s2] step 7580/12000 loss=4.4681 gn=0.85 lr=2.26e-04 tps=22676 mem=5.5G lam=None Gm=0.3809916178385417 +[g_only s2] step 7600/12000 loss=4.4570 gn=0.81 lr=2.25e-04 tps=22675 mem=5.5G lam=None Gm=0.3815409342447917 +[g_only s2] step 7620/12000 loss=4.7112 gn=0.93 lr=2.24e-04 tps=22675 mem=5.5G lam=None Gm=0.3878224690755208 +[g_only s2] step 7640/12000 loss=4.4253 gn=0.87 lr=2.22e-04 tps=22676 mem=5.5G lam=None Gm=0.3770090738932292 +[g_only s2] step 7660/12000 loss=4.3538 gn=0.80 lr=2.21e-04 tps=22676 mem=5.5G lam=None Gm=0.3863983154296875 +[g_only s2] step 7680/12000 loss=4.3220 gn=0.83 lr=2.20e-04 tps=22676 mem=5.5G lam=None Gm=0.3846537272135417 +[g_only s2] step 7700/12000 loss=4.4808 gn=0.80 lr=2.18e-04 tps=22676 mem=5.5G lam=None Gm=0.3836619059244792 +[g_only s2] step 7720/12000 loss=4.5314 gn=0.89 lr=2.17e-04 tps=22675 mem=5.5G lam=None Gm=0.38641357421875 +[g_only s2] step 7740/12000 loss=4.4207 gn=0.90 lr=2.16e-04 tps=22675 mem=5.5G lam=None Gm=0.3866526285807292 +[g_only s2] step 7760/12000 loss=4.5504 gn=0.89 lr=2.15e-04 tps=22675 mem=5.5G lam=None Gm=0.390777587890625 +[g_only s2] step 7780/12000 loss=4.6005 gn=0.87 lr=2.13e-04 tps=22675 mem=5.5G lam=None Gm=0.3752288818359375 +[g_only s2] step 7800/12000 loss=4.4858 gn=0.87 lr=2.12e-04 tps=22675 mem=5.5G lam=None Gm=0.3849538167317708 +[g_only s2] step 7820/12000 loss=4.4475 gn=0.84 lr=2.11e-04 tps=22675 mem=5.5G lam=None Gm=0.388336181640625 +[g_only s2] step 7840/12000 loss=4.3501 gn=0.84 lr=2.09e-04 tps=22675 mem=5.5G lam=None Gm=0.37945556640625 +[g_only s2] step 7860/12000 loss=4.4040 gn=0.83 lr=2.08e-04 tps=22675 mem=5.5G lam=None Gm=0.3763427734375 +[g_only s2] step 7880/12000 loss=4.4709 gn=0.91 lr=2.07e-04 tps=22675 mem=5.5G lam=None Gm=0.3832346598307292 +[g_only s2] step 7900/12000 loss=4.4626 gn=0.85 lr=2.06e-04 tps=22675 mem=5.5G lam=None Gm=0.3781483968098958 +[g_only s2] step 7920/12000 loss=4.5341 gn=0.85 lr=2.04e-04 tps=22675 mem=5.5G lam=None Gm=0.3912760416666667 +[g_only s2] step 7940/12000 loss=4.4412 gn=0.90 lr=2.03e-04 tps=22675 mem=5.5G lam=None Gm=0.3818511962890625 +[g_only s2] step 7960/12000 loss=4.5028 gn=0.90 lr=2.02e-04 tps=22675 mem=5.5G lam=None Gm=0.3751118977864583 +[g_only s2] step 7980/12000 loss=4.4686 gn=0.89 lr=2.00e-04 tps=22676 mem=5.5G lam=None Gm=0.3753611246744792 +[g_only s2] step 8000/12000 loss=4.4229 gn=0.85 lr=1.99e-04 tps=22676 mem=5.5G lam=None Gm=0.3785858154296875 +[g_only s2] EVAL step 8000: val_loss=4.4698 val_ppl=87.34 +[g_only s2] step 8020/12000 loss=4.4877 gn=0.86 lr=1.98e-04 tps=22666 mem=5.5G lam=None Gm=0.38470458984375 +[g_only s2] step 8040/12000 loss=4.5054 gn=0.88 lr=1.97e-04 tps=22667 mem=5.5G lam=None Gm=0.3830464680989583 +[g_only s2] step 8060/12000 loss=4.3751 gn=0.89 lr=1.95e-04 tps=22667 mem=5.5G lam=None Gm=0.3803151448567708 +[g_only s2] step 8080/12000 loss=4.4431 gn=0.83 lr=1.94e-04 tps=22667 mem=5.5G lam=None Gm=0.3859507242838542 +[g_only s2] step 8100/12000 loss=4.4979 gn=0.88 lr=1.93e-04 tps=22662 mem=5.5G lam=None Gm=0.3805898030598958 +[g_only s2] step 8120/12000 loss=4.5184 gn=0.86 lr=1.92e-04 tps=22660 mem=5.5G lam=None Gm=0.3861033121744792 +[g_only s2] step 8140/12000 loss=4.4382 gn=0.86 lr=1.90e-04 tps=22661 mem=5.5G lam=None Gm=0.38421630859375 +[g_only s2] step 8160/12000 loss=4.5449 gn=0.84 lr=1.89e-04 tps=22660 mem=5.5G lam=None Gm=0.3787587483723958 +[g_only s2] step 8180/12000 loss=4.5309 gn=0.95 lr=1.88e-04 tps=22661 mem=5.5G lam=None Gm=0.3836568196614583 +[g_only s2] step 8200/12000 loss=4.4948 gn=0.84 lr=1.87e-04 tps=22661 mem=5.5G lam=None Gm=0.3839314778645833 +[g_only s2] step 8220/12000 loss=4.4949 gn=0.89 lr=1.86e-04 tps=22661 mem=5.5G lam=None Gm=0.3905893961588542 +[g_only s2] step 8240/12000 loss=4.4152 gn=0.91 lr=1.84e-04 tps=22661 mem=5.5G lam=None Gm=0.3793131510416667 +[g_only s2] step 8260/12000 loss=4.4631 gn=0.88 lr=1.83e-04 tps=22661 mem=5.5G lam=None Gm=0.3801625569661458 +[g_only s2] step 8280/12000 loss=4.4381 gn=0.85 lr=1.82e-04 tps=22661 mem=5.5G lam=None Gm=0.3855387369791667 +[g_only s2] step 8300/12000 loss=4.4890 gn=0.91 lr=1.81e-04 tps=22661 mem=5.5G lam=None Gm=0.3873291015625 +[g_only s2] step 8320/12000 loss=4.4158 gn=0.88 lr=1.80e-04 tps=22661 mem=5.5G lam=None Gm=0.3872578938802083 +[g_only s2] step 8340/12000 loss=4.4602 gn=0.88 lr=1.78e-04 tps=22661 mem=5.5G lam=None Gm=0.3767547607421875 +[g_only s2] step 8360/12000 loss=4.4705 gn=0.85 lr=1.77e-04 tps=22661 mem=5.5G lam=None Gm=0.3802388509114583 +[g_only s2] step 8380/12000 loss=4.4303 gn=0.88 lr=1.76e-04 tps=22661 mem=5.5G lam=None Gm=0.3799794514973958 +[g_only s2] step 8400/12000 loss=4.3810 gn=0.93 lr=1.75e-04 tps=22661 mem=5.5G lam=None Gm=0.38995361328125 +[g_only s2] step 8420/12000 loss=4.5770 gn=0.91 lr=1.74e-04 tps=22661 mem=5.5G lam=None Gm=0.3868509928385417 +[g_only s2] step 8440/12000 loss=4.4606 gn=0.95 lr=1.72e-04 tps=22661 mem=5.5G lam=None Gm=0.3803354899088542 +[g_only s2] step 8460/12000 loss=4.4007 gn=0.91 lr=1.71e-04 tps=22662 mem=5.5G lam=None Gm=0.3866068522135417 +[g_only s2] step 8480/12000 loss=4.3583 gn=0.83 lr=1.70e-04 tps=22661 mem=5.5G lam=None Gm=0.3737691243489583 +[g_only s2] step 8500/12000 loss=4.3820 gn=0.85 lr=1.69e-04 tps=22661 mem=5.5G lam=None Gm=0.3739013671875 +[g_only s2] step 8520/12000 loss=4.5145 gn=0.90 lr=1.68e-04 tps=22661 mem=5.5G lam=None Gm=0.3845977783203125 +[g_only s2] step 8540/12000 loss=4.5828 gn=1.00 lr=1.67e-04 tps=22660 mem=5.5G lam=None Gm=0.3956095377604167 +[g_only s2] step 8560/12000 loss=4.3139 gn=0.94 lr=1.66e-04 tps=22658 mem=5.5G lam=None Gm=0.3802235921223958 +[g_only s2] step 8580/12000 loss=4.4689 gn=0.87 lr=1.64e-04 tps=22658 mem=5.5G lam=None Gm=0.3821665445963542 +[g_only s2] step 8600/12000 loss=4.3586 gn=0.92 lr=1.63e-04 tps=22657 mem=5.5G lam=None Gm=0.3863169352213542 +[g_only s2] step 8620/12000 loss=4.3806 gn=0.87 lr=1.62e-04 tps=22657 mem=5.5G lam=None Gm=0.3836313883463542 +[g_only s2] step 8640/12000 loss=4.3791 gn=0.85 lr=1.61e-04 tps=22655 mem=5.5G lam=None Gm=0.3756815592447917 +[g_only s2] step 8660/12000 loss=4.4413 gn=0.91 lr=1.60e-04 tps=22653 mem=5.5G lam=None Gm=0.3820648193359375 +[g_only s2] step 8680/12000 loss=4.3190 gn=0.94 lr=1.59e-04 tps=22652 mem=5.5G lam=None Gm=0.3899332682291667 +[g_only s2] step 8700/12000 loss=4.1986 gn=0.84 lr=1.58e-04 tps=22649 mem=5.5G lam=None Gm=0.3725077311197917 +[g_only s2] step 8720/12000 loss=4.3783 gn=0.91 lr=1.57e-04 tps=22648 mem=5.5G lam=None Gm=0.3720347086588542 +[g_only s2] step 8740/12000 loss=4.4178 gn=0.88 lr=1.55e-04 tps=22645 mem=5.5G lam=None Gm=0.3832143147786458 +[g_only s2] step 8760/12000 loss=4.5165 gn=0.96 lr=1.54e-04 tps=22643 mem=5.5G lam=None Gm=0.390869140625 +[g_only s2] step 8780/12000 loss=4.5298 gn=0.88 lr=1.53e-04 tps=22641 mem=5.5G lam=None Gm=0.3752237955729167 +[g_only s2] step 8800/12000 loss=4.3576 gn=0.89 lr=1.52e-04 tps=22641 mem=5.5G lam=None Gm=0.3797251383463542 +[g_only s2] step 8820/12000 loss=4.3733 gn=0.91 lr=1.51e-04 tps=22640 mem=5.5G lam=None Gm=0.3725382486979167 +[g_only s2] step 8840/12000 loss=4.5179 gn=0.87 lr=1.50e-04 tps=22640 mem=5.5G lam=None Gm=0.3775227864583333 +[g_only s2] step 8860/12000 loss=4.3843 gn=0.91 lr=1.49e-04 tps=22640 mem=5.5G lam=None Gm=0.3813730875651042 +[g_only s2] step 8880/12000 loss=4.3821 gn=0.88 lr=1.48e-04 tps=22640 mem=5.5G lam=None Gm=0.3733876546223958 +[g_only s2] step 8900/12000 loss=4.4782 gn=0.88 lr=1.47e-04 tps=22640 mem=5.5G lam=None Gm=0.3791859944661458 +[g_only s2] step 8920/12000 loss=4.3601 gn=0.92 lr=1.46e-04 tps=22639 mem=5.5G lam=None Gm=0.3771158854166667 +[g_only s2] step 8940/12000 loss=4.4431 gn=0.94 lr=1.45e-04 tps=22639 mem=5.5G lam=None Gm=0.3790435791015625 +[g_only s2] step 8960/12000 loss=4.4145 gn=0.88 lr=1.44e-04 tps=22638 mem=5.5G lam=None Gm=0.3790842692057292 +[g_only s2] step 8980/12000 loss=4.3136 gn=0.97 lr=1.43e-04 tps=22638 mem=5.5G lam=None Gm=0.3834279378255208 +[g_only s2] step 9000/12000 loss=4.3526 gn=0.87 lr=1.42e-04 tps=22638 mem=5.5G lam=None Gm=0.3805287679036458 +[g_only s2] EVAL step 9000: val_loss=4.4928 val_ppl=89.37 +[g_only s2] step 9020/12000 loss=4.3761 gn=0.93 lr=1.41e-04 tps=22629 mem=5.5G lam=None Gm=0.3788604736328125 +[g_only s2] step 9040/12000 loss=4.5242 gn=0.90 lr=1.40e-04 tps=22629 mem=5.5G lam=None Gm=0.378875732421875 +[g_only s2] step 9060/12000 loss=4.4010 gn=0.95 lr=1.39e-04 tps=22629 mem=5.5G lam=None Gm=0.3785756429036458 +[g_only s2] step 9080/12000 loss=4.3047 gn=0.92 lr=1.38e-04 tps=22629 mem=5.5G lam=None Gm=0.3785349527994792 +[g_only s2] step 9100/12000 loss=4.4362 gn=0.94 lr=1.37e-04 tps=22630 mem=5.5G lam=None Gm=0.3842315673828125 +[g_only s2] step 9120/12000 loss=4.3660 gn=0.92 lr=1.36e-04 tps=22630 mem=5.5G lam=None Gm=0.3827972412109375 +[g_only s2] step 9140/12000 loss=4.3053 gn=0.93 lr=1.35e-04 tps=22630 mem=5.5G lam=None Gm=0.3722381591796875 +[g_only s2] step 9160/12000 loss=4.4296 gn=0.99 lr=1.34e-04 tps=22629 mem=5.5G lam=None Gm=0.3784230550130208 +[g_only s2] step 9180/12000 loss=4.3543 gn=0.95 lr=1.33e-04 tps=22630 mem=5.5G lam=None Gm=0.3857930501302083 +[g_only s2] step 9200/12000 loss=4.4738 gn=0.97 lr=1.32e-04 tps=22630 mem=5.5G lam=None Gm=0.3816121419270833 +[g_only s2] step 9220/12000 loss=4.3266 gn=0.98 lr=1.31e-04 tps=22630 mem=5.5G lam=None Gm=0.3791707356770833 +[g_only s2] step 9240/12000 loss=4.3585 gn=0.90 lr=1.30e-04 tps=22630 mem=5.5G lam=None Gm=0.3759206136067708 +[g_only s2] step 9260/12000 loss=4.3506 gn=0.92 lr=1.29e-04 tps=22630 mem=5.5G lam=None Gm=0.373138427734375 +[g_only s2] step 9280/12000 loss=4.2280 gn=0.94 lr=1.28e-04 tps=22630 mem=5.5G lam=None Gm=0.3821868896484375 +[g_only s2] step 9300/12000 loss=4.4802 gn=0.97 lr=1.27e-04 tps=22630 mem=5.5G lam=None Gm=0.3853505452473958 +[g_only s2] step 9320/12000 loss=4.3461 gn=0.95 lr=1.26e-04 tps=22631 mem=5.5G lam=None Gm=0.3818206787109375 +[g_only s2] step 9340/12000 loss=4.3585 gn=0.88 lr=1.25e-04 tps=22631 mem=5.5G lam=None Gm=0.3750356038411458 +[g_only s2] step 9360/12000 loss=4.1271 gn=0.88 lr=1.24e-04 tps=22631 mem=5.5G lam=None Gm=0.3682403564453125 +[g_only s2] step 9380/12000 loss=4.4951 gn=0.98 lr=1.23e-04 tps=22631 mem=5.5G lam=None Gm=0.3883056640625 +[g_only s2] step 9400/12000 loss=4.4123 gn=0.89 lr=1.22e-04 tps=22631 mem=5.5G lam=None Gm=0.3844655354817708 +[g_only s2] step 9420/12000 loss=4.3731 gn=0.96 lr=1.21e-04 tps=22631 mem=5.5G lam=None Gm=0.3857981363932292 +[g_only s2] step 9440/12000 loss=4.2772 gn=0.88 lr=1.20e-04 tps=22631 mem=5.5G lam=None Gm=0.3730214436848958 +[g_only s2] step 9460/12000 loss=4.4363 gn=0.88 lr=1.19e-04 tps=22631 mem=5.5G lam=None Gm=0.377685546875 +[g_only s2] step 9480/12000 loss=4.2511 gn=1.02 lr=1.19e-04 tps=22631 mem=5.5G lam=None Gm=0.3754425048828125 +[g_only s2] step 9500/12000 loss=4.4106 gn=0.93 lr=1.18e-04 tps=22631 mem=5.5G lam=None Gm=0.381561279296875 +[g_only s2] step 9520/12000 loss=4.2818 gn=0.91 lr=1.17e-04 tps=22631 mem=5.5G lam=None Gm=0.3802490234375 +[g_only s2] step 9540/12000 loss=4.2377 gn=0.93 lr=1.16e-04 tps=22631 mem=5.5G lam=None Gm=0.3779703776041667 +[g_only s2] step 9560/12000 loss=4.3055 gn=0.90 lr=1.15e-04 tps=22631 mem=5.5G lam=None Gm=0.3780008951822917 +[g_only s2] step 9580/12000 loss=4.2218 gn=0.89 lr=1.14e-04 tps=22631 mem=5.5G lam=None Gm=0.3693491617838542 +[g_only s2] step 9600/12000 loss=4.3734 gn=0.98 lr=1.13e-04 tps=22631 mem=5.5G lam=None Gm=0.3765970865885417 +[g_only s2] step 9620/12000 loss=4.4704 gn=0.88 lr=1.12e-04 tps=22631 mem=5.5G lam=None Gm=0.377899169921875 +[g_only s2] step 9640/12000 loss=4.3871 gn=0.90 lr=1.12e-04 tps=22632 mem=5.5G lam=None Gm=0.3731689453125 +[g_only s2] step 9660/12000 loss=4.2317 gn=0.89 lr=1.11e-04 tps=22632 mem=5.5G lam=None Gm=0.374786376953125 +[g_only s2] step 9680/12000 loss=4.2010 gn=0.93 lr=1.10e-04 tps=22632 mem=5.5G lam=None Gm=0.3736979166666667 +[g_only s2] step 9700/12000 loss=4.2449 gn=0.94 lr=1.09e-04 tps=22632 mem=5.5G lam=None Gm=0.3732859293619792 +[g_only s2] step 9720/12000 loss=4.2407 gn=0.91 lr=1.08e-04 tps=22632 mem=5.5G lam=None Gm=0.3774973551432292 +[g_only s2] step 9740/12000 loss=4.3421 gn=0.97 lr=1.07e-04 tps=22632 mem=5.5G lam=None Gm=0.3720804850260417 +[g_only s2] step 9760/12000 loss=4.2041 gn=0.88 lr=1.07e-04 tps=22631 mem=5.5G lam=None Gm=0.3719228108723958 +[g_only s2] step 9780/12000 loss=4.3522 gn=0.92 lr=1.06e-04 tps=22631 mem=5.5G lam=None Gm=0.3690897623697917 +[g_only s2] step 9800/12000 loss=4.3923 gn=0.93 lr=1.05e-04 tps=22631 mem=5.5G lam=None Gm=0.3722635904947917 +[g_only s2] step 9820/12000 loss=4.4281 gn=0.95 lr=1.04e-04 tps=22631 mem=5.5G lam=None Gm=0.3753712972005208 +[g_only s2] step 9840/12000 loss=4.1713 gn=0.96 lr=1.03e-04 tps=22632 mem=5.5G lam=None Gm=0.3708038330078125 +[g_only s2] step 9860/12000 loss=4.4320 gn=0.95 lr=1.03e-04 tps=22632 mem=5.5G lam=None Gm=0.3753306070963542 +[g_only s2] step 9880/12000 loss=4.4980 gn=0.92 lr=1.02e-04 tps=22632 mem=5.5G lam=None Gm=0.377899169921875 +[g_only s2] step 9900/12000 loss=4.3312 gn=0.99 lr=1.01e-04 tps=22633 mem=5.5G lam=None Gm=0.3768157958984375 +[g_only s2] step 9920/12000 loss=4.4004 gn=0.93 lr=1.00e-04 tps=22632 mem=5.5G lam=None Gm=0.3730417887369792 +[g_only s2] step 9940/12000 loss=4.3372 gn=0.89 lr=9.96e-05 tps=22633 mem=5.5G lam=None Gm=0.3732503255208333 +[g_only s2] step 9960/12000 loss=4.1768 gn=0.87 lr=9.89e-05 tps=22633 mem=5.5G lam=None Gm=0.3719126383463542 +[g_only s2] step 9980/12000 loss=4.3172 gn=0.93 lr=9.81e-05 tps=22633 mem=5.5G lam=None Gm=0.3733418782552083 +[g_only s2] step 10000/12000 loss=4.2138 gn=0.90 lr=9.74e-05 tps=22633 mem=5.5G lam=None Gm=0.3727671305338542 +[g_only s2] EVAL step 10000: val_loss=4.4047 val_ppl=81.84 +[g_only s2] step 10020/12000 loss=4.4931 gn=0.95 lr=9.67e-05 tps=22625 mem=5.5G lam=None Gm=0.375335693359375 +[g_only s2] step 10040/12000 loss=4.5289 gn=0.99 lr=9.59e-05 tps=22625 mem=5.5G lam=None Gm=0.3736521402994792 +[g_only s2] step 10060/12000 loss=4.5119 gn=0.94 lr=9.52e-05 tps=22625 mem=5.5G lam=None Gm=0.3796844482421875 +[g_only s2] step 10080/12000 loss=4.3691 gn=0.94 lr=9.45e-05 tps=22625 mem=5.5G lam=None Gm=0.3678690592447917 +[g_only s2] step 10100/12000 loss=4.4869 gn=0.92 lr=9.38e-05 tps=22625 mem=5.5G lam=None Gm=0.3781178792317708 +[g_only s2] step 10120/12000 loss=4.2805 gn=1.00 lr=9.31e-05 tps=22625 mem=5.5G lam=None Gm=0.3739827473958333 +[g_only s2] step 10140/12000 loss=4.2794 gn=0.90 lr=9.24e-05 tps=22625 mem=5.5G lam=None Gm=0.3681538899739583 +[g_only s2] step 10160/12000 loss=4.3776 gn=0.93 lr=9.18e-05 tps=22625 mem=5.5G lam=None Gm=0.3758087158203125 +[g_only s2] step 10180/12000 loss=4.3889 gn=0.94 lr=9.11e-05 tps=22625 mem=5.5G lam=None Gm=0.3707529703776042 +[g_only s2] step 10200/12000 loss=4.2895 gn=0.96 lr=9.04e-05 tps=22625 mem=5.5G lam=None Gm=0.3681691487630208 +[g_only s2] step 10220/12000 loss=4.3753 gn=0.91 lr=8.98e-05 tps=22625 mem=5.5G lam=None Gm=0.3725179036458333 +[g_only s2] step 10240/12000 loss=4.3018 gn=0.93 lr=8.91e-05 tps=22625 mem=5.5G lam=None Gm=0.3730112711588542 +[g_only s2] step 10260/12000 loss=4.3143 gn=0.93 lr=8.85e-05 tps=22625 mem=5.5G lam=None Gm=0.3791656494140625 +[g_only s2] step 10280/12000 loss=4.4540 gn=0.92 lr=8.78e-05 tps=22625 mem=5.5G lam=None Gm=0.3731435139973958 +[g_only s2] step 10300/12000 loss=4.2939 gn=0.93 lr=8.72e-05 tps=22625 mem=5.5G lam=None Gm=0.3811492919921875 +[g_only s2] step 10320/12000 loss=4.4800 gn=1.02 lr=8.66e-05 tps=22625 mem=5.5G lam=None Gm=0.3787587483723958 +[g_only s2] step 10340/12000 loss=4.2808 gn=0.92 lr=8.59e-05 tps=22626 mem=5.5G lam=None Gm=0.3708597819010417 +[g_only s2] step 10360/12000 loss=4.3309 gn=0.93 lr=8.53e-05 tps=22626 mem=5.5G lam=None Gm=0.372314453125 +[g_only s2] step 10380/12000 loss=4.4381 gn=1.03 lr=8.47e-05 tps=22626 mem=5.5G lam=None Gm=0.3773701985677083 +[g_only s2] step 10400/12000 loss=4.4251 gn=0.97 lr=8.41e-05 tps=22626 mem=5.5G lam=None Gm=0.3853047688802083 +[g_only s2] step 10420/12000 loss=4.2637 gn=0.98 lr=8.35e-05 tps=22626 mem=5.5G lam=None Gm=0.3735758463541667 +[g_only s2] step 10440/12000 loss=4.2115 gn=0.94 lr=8.30e-05 tps=22626 mem=5.5G lam=None Gm=0.3761240641276042 +[g_only s2] step 10460/12000 loss=4.3606 gn=0.98 lr=8.24e-05 tps=22626 mem=5.5G lam=None Gm=0.3809000651041667 +[g_only s2] step 10480/12000 loss=4.3831 gn=0.99 lr=8.18e-05 tps=22626 mem=5.5G lam=None Gm=0.375732421875 +[g_only s2] step 10500/12000 loss=4.2728 gn=0.90 lr=8.12e-05 tps=22626 mem=5.5G lam=None Gm=0.3720448811848958 +[g_only s2] step 10520/12000 loss=4.2335 gn=0.89 lr=8.07e-05 tps=22627 mem=5.5G lam=None Gm=0.3734792073567708 +[g_only s2] step 10540/12000 loss=4.1016 gn=1.00 lr=8.01e-05 tps=22627 mem=5.5G lam=None Gm=0.3683420817057292 +[g_only s2] step 10560/12000 loss=4.3457 gn=0.92 lr=7.96e-05 tps=22627 mem=5.5G lam=None Gm=0.3748931884765625 +[g_only s2] step 10580/12000 loss=4.3925 gn=0.98 lr=7.91e-05 tps=22627 mem=5.5G lam=None Gm=0.3796590169270833 +[g_only s2] step 10600/12000 loss=4.4371 gn=1.02 lr=7.85e-05 tps=22627 mem=5.5G lam=None Gm=0.3749237060546875 +[g_only s2] step 10620/12000 loss=4.3808 gn=0.97 lr=7.80e-05 tps=22627 mem=5.5G lam=None Gm=0.3728586832682292 +[g_only s2] step 10640/12000 loss=4.3684 gn=0.99 lr=7.75e-05 tps=22627 mem=5.5G lam=None Gm=0.3742624918619792 +[g_only s2] step 10660/12000 loss=4.3403 gn=0.99 lr=7.70e-05 tps=22628 mem=5.5G lam=None Gm=0.373870849609375 +[g_only s2] step 10680/12000 loss=4.4234 gn=0.95 lr=7.65e-05 tps=22628 mem=5.5G lam=None Gm=0.3711649576822917 +[g_only s2] step 10700/12000 loss=4.4433 gn=0.94 lr=7.60e-05 tps=22628 mem=5.5G lam=None Gm=0.3749440511067708 +[g_only s2] step 10720/12000 loss=4.3283 gn=0.88 lr=7.55e-05 tps=22628 mem=5.5G lam=None Gm=0.3688812255859375 +[g_only s2] step 10740/12000 loss=4.2055 gn=0.89 lr=7.50e-05 tps=22628 mem=5.5G lam=None Gm=0.3717600504557292 +[g_only s2] step 10760/12000 loss=4.3187 gn=0.94 lr=7.46e-05 tps=22628 mem=5.5G lam=None Gm=0.3761342366536458 +[g_only s2] step 10780/12000 loss=4.4319 gn=0.95 lr=7.41e-05 tps=22629 mem=5.5G lam=None Gm=0.3846893310546875 +[g_only s2] step 10800/12000 loss=4.3884 gn=1.05 lr=7.37e-05 tps=22629 mem=5.5G lam=None Gm=0.3733927408854167 +[g_only s2] step 10820/12000 loss=4.3058 gn=0.95 lr=7.32e-05 tps=22629 mem=5.5G lam=None Gm=0.378936767578125 +[g_only s2] step 10840/12000 loss=4.3084 gn=0.98 lr=7.28e-05 tps=22629 mem=5.5G lam=None Gm=0.3789825439453125 +[g_only s2] step 10860/12000 loss=4.3008 gn=0.97 lr=7.23e-05 tps=22629 mem=5.5G lam=None Gm=0.3772939046223958 +[g_only s2] step 10880/12000 loss=4.4311 gn=0.97 lr=7.19e-05 tps=22629 mem=5.5G lam=None Gm=0.372955322265625 +[g_only s2] step 10900/12000 loss=4.4850 gn=1.05 lr=7.15e-05 tps=22629 mem=5.5G lam=None Gm=0.3794504801432292 +[g_only s2] step 10920/12000 loss=4.5226 gn=1.26 lr=7.11e-05 tps=22629 mem=5.5G lam=None Gm=0.373138427734375 +[g_only s2] step 10940/12000 loss=4.5196 gn=0.95 lr=7.07e-05 tps=22629 mem=5.5G lam=None Gm=0.3744303385416667 +[g_only s2] step 10960/12000 loss=4.2891 gn=0.97 lr=7.03e-05 tps=22629 mem=5.5G lam=None Gm=0.3699849446614583 +[g_only s2] step 10980/12000 loss=4.1632 gn=0.91 lr=6.99e-05 tps=22629 mem=5.5G lam=None Gm=0.3722279866536458 +[g_only s2] step 11000/12000 loss=4.3970 gn=0.99 lr=6.95e-05 tps=22629 mem=5.5G lam=None Gm=0.3775583902994792 +[g_only s2] EVAL step 11000: val_loss=4.4030 val_ppl=81.70 +[g_only s2] step 11020/12000 loss=4.3389 gn=1.04 lr=6.91e-05 tps=22622 mem=5.5G lam=None Gm=0.37542724609375 +[g_only s2] step 11040/12000 loss=4.2769 gn=0.97 lr=6.88e-05 tps=22622 mem=5.5G lam=None Gm=0.3743336995442708 +[g_only s2] step 11060/12000 loss=4.3864 gn=1.04 lr=6.84e-05 tps=22622 mem=5.5G lam=None Gm=0.3834177652994792 +[g_only s2] step 11080/12000 loss=4.2964 gn=1.04 lr=6.81e-05 tps=22622 mem=5.5G lam=None Gm=0.3700358072916667 +[g_only s2] step 11100/12000 loss=4.4284 gn=0.96 lr=6.77e-05 tps=22622 mem=5.5G lam=None Gm=0.3770650227864583 +[g_only s2] step 11120/12000 loss=4.1675 gn=0.93 lr=6.74e-05 tps=22622 mem=5.5G lam=None Gm=0.3697662353515625 +[g_only s2] step 11140/12000 loss=4.3310 gn=0.96 lr=6.70e-05 tps=22622 mem=5.5G lam=None Gm=0.3728383382161458 +[g_only s2] step 11160/12000 loss=4.3456 gn=0.96 lr=6.67e-05 tps=22622 mem=5.5G lam=None Gm=0.371063232421875 +[g_only s2] step 11180/12000 loss=4.3330 gn=0.94 lr=6.64e-05 tps=22622 mem=5.5G lam=None Gm=0.3712056477864583 +[g_only s2] step 11200/12000 loss=4.1793 gn=1.00 lr=6.61e-05 tps=22622 mem=5.5G lam=None Gm=0.3695220947265625 +[g_only s2] step 11220/12000 loss=4.1199 gn=0.97 lr=6.58e-05 tps=22622 mem=5.5G lam=None Gm=0.3672993977864583 +[g_only s2] step 11240/12000 loss=4.3324 gn=1.03 lr=6.55e-05 tps=22622 mem=5.5G lam=None Gm=0.3761240641276042 +[g_only s2] step 11260/12000 loss=4.3626 gn=1.01 lr=6.52e-05 tps=22622 mem=5.5G lam=None Gm=0.3727060953776042 +[g_only s2] step 11280/12000 loss=4.3203 gn=1.00 lr=6.49e-05 tps=22622 mem=5.5G lam=None Gm=0.3746999104817708 +[g_only s2] step 11300/12000 loss=4.3607 gn=1.05 lr=6.47e-05 tps=22622 mem=5.5G lam=None Gm=0.3670857747395833 +[g_only s2] step 11320/12000 loss=4.2294 gn=1.09 lr=6.44e-05 tps=22622 mem=5.5G lam=None Gm=0.3720855712890625 +[g_only s2] step 11340/12000 loss=4.4051 gn=1.02 lr=6.42e-05 tps=22622 mem=5.5G lam=None Gm=0.3773651123046875 +[g_only s2] step 11360/12000 loss=4.3279 gn=1.01 lr=6.39e-05 tps=22621 mem=5.5G lam=None Gm=0.3742472330729167 +[g_only s2] step 11380/12000 loss=4.3338 gn=1.00 lr=6.37e-05 tps=22621 mem=5.5G lam=None Gm=0.37176513671875 +[g_only s2] step 11400/12000 loss=4.3554 gn=1.16 lr=6.34e-05 tps=22621 mem=5.5G lam=None Gm=0.3716227213541667 +[g_only s2] step 11420/12000 loss=4.3283 gn=1.00 lr=6.32e-05 tps=22621 mem=5.5G lam=None Gm=0.3755900065104167 +[g_only s2] step 11440/12000 loss=4.1997 gn=0.96 lr=6.30e-05 tps=22621 mem=5.5G lam=None Gm=0.3776295979817708 +[g_only s2] step 11460/12000 loss=4.2585 gn=1.04 lr=6.28e-05 tps=22621 mem=5.5G lam=None Gm=0.3709971110026042 +[g_only s2] step 11480/12000 loss=4.2338 gn=1.04 lr=6.26e-05 tps=22621 mem=5.5G lam=None Gm=0.3670196533203125 +[g_only s2] step 11500/12000 loss=4.3549 gn=0.96 lr=6.24e-05 tps=22620 mem=5.5G lam=None Gm=0.3767038981119792 +[g_only s2] step 11520/12000 loss=4.4486 gn=0.99 lr=6.22e-05 tps=22616 mem=5.5G lam=None Gm=0.3751271565755208 +[g_only s2] step 11540/12000 loss=4.3157 gn=1.01 lr=6.20e-05 tps=22610 mem=5.5G lam=None Gm=0.37774658203125 +[g_only s2] step 11560/12000 loss=4.2876 gn=1.05 lr=6.19e-05 tps=22602 mem=5.5G lam=None Gm=0.3789265950520833 +[g_only s2] step 11580/12000 loss=4.4789 gn=1.18 lr=6.17e-05 tps=22597 mem=5.5G lam=None Gm=0.3744099934895833 +[g_only s2] step 11600/12000 loss=4.1962 gn=0.99 lr=6.15e-05 tps=22597 mem=5.5G lam=None Gm=0.3798319498697917 +[g_only s2] step 11620/12000 loss=4.3171 gn=1.02 lr=6.14e-05 tps=22597 mem=5.5G lam=None Gm=0.3704681396484375 +[g_only s2] step 11640/12000 loss=4.2416 gn=0.89 lr=6.12e-05 tps=22597 mem=5.5G lam=None Gm=0.3709004720052083 +[g_only s2] step 11660/12000 loss=4.2890 gn=1.05 lr=6.11e-05 tps=22597 mem=5.5G lam=None Gm=0.3713785807291667 +[g_only s2] step 11680/12000 loss=4.3083 gn=0.94 lr=6.10e-05 tps=22597 mem=5.5G lam=None Gm=0.3693288167317708 +[g_only s2] step 11700/12000 loss=4.3434 gn=0.97 lr=6.09e-05 tps=22597 mem=5.5G lam=None Gm=0.3730926513671875 +[g_only s2] step 11720/12000 loss=4.2888 gn=1.02 lr=6.07e-05 tps=22598 mem=5.5G lam=None Gm=0.3698527018229167 +[g_only s2] step 11740/12000 loss=4.3491 gn=1.03 lr=6.06e-05 tps=22597 mem=5.5G lam=None Gm=0.3814493815104167 +[g_only s2] step 11760/12000 loss=4.2133 gn=0.99 lr=6.06e-05 tps=22598 mem=5.5G lam=None Gm=0.3657735188802083 +[g_only s2] step 11780/12000 loss=4.4250 gn=1.07 lr=6.05e-05 tps=22598 mem=5.5G lam=None Gm=0.37506103515625 +[g_only s2] step 11800/12000 loss=4.3204 gn=1.11 lr=6.04e-05 tps=22598 mem=5.5G lam=None Gm=0.3726603190104167 +[g_only s2] step 11820/12000 loss=4.4857 gn=0.93 lr=6.03e-05 tps=22598 mem=5.5G lam=None Gm=0.3721923828125 +[g_only s2] step 11840/12000 loss=4.2489 gn=0.97 lr=6.02e-05 tps=22598 mem=5.5G lam=None Gm=0.3699798583984375 +[g_only s2] step 11860/12000 loss=4.2438 gn=1.07 lr=6.02e-05 tps=22598 mem=5.5G lam=None Gm=0.3698832194010417 +[g_only s2] step 11880/12000 loss=4.3410 gn=1.08 lr=6.01e-05 tps=22598 mem=5.5G lam=None Gm=0.3696339925130208 +[g_only s2] step 11900/12000 loss=4.2947 gn=1.00 lr=6.01e-05 tps=22598 mem=5.5G lam=None Gm=0.3719329833984375 +[g_only s2] step 11920/12000 loss=4.2152 gn=1.06 lr=6.01e-05 tps=22598 mem=5.5G lam=None Gm=0.3739369710286458 +[g_only s2] step 11940/12000 loss=4.1800 gn=1.00 lr=6.00e-05 tps=22598 mem=5.5G lam=None Gm=0.367034912109375 +[g_only s2] step 11960/12000 loss=4.3057 gn=1.00 lr=6.00e-05 tps=22598 mem=5.5G lam=None Gm=0.369171142578125 +[g_only s2] step 11980/12000 loss=4.2727 gn=1.05 lr=6.00e-05 tps=22598 mem=5.5G lam=None Gm=0.3676300048828125 +[g_only s2] EVAL step 11999: val_loss=4.3986 val_ppl=81.34 +[init] variant=full seed=2 params=63.98M +[init] use_S=True use_G=True +[full s2] step 0/12000 loss=10.8549 gn=9.82 lr=3.00e-06 tps=7949 mem=5.2G lam=0.12692801654338837 Gm=0.48413818205396336 +[full s2] step 20/12000 loss=9.8600 gn=1.92 lr=6.30e-05 tps=16895 mem=5.7G lam=0.12692545354366302 Gm=0.4522758461534977 +[full s2] step 40/12000 loss=9.0901 gn=1.79 lr=1.23e-04 tps=17467 mem=5.7G lam=0.1268333693345388 Gm=0.43023352945844334 +[full s2] step 60/12000 loss=8.1711 gn=1.49 lr=1.83e-04 tps=17765 mem=5.7G lam=0.1267441939562559 Gm=0.42800211906433105 +[full s2] step 80/12000 loss=7.5458 gn=0.84 lr=2.43e-04 tps=17918 mem=5.7G lam=0.12663992308080196 Gm=0.4271373637020588 +[full s2] step 100/12000 loss=7.2317 gn=0.72 lr=3.03e-04 tps=18036 mem=5.7G lam=0.12654135872920355 Gm=0.415961633125941 +[full s2] step 120/12000 loss=7.1380 gn=0.67 lr=3.63e-04 tps=18104 mem=5.7G lam=0.1264414396136999 Gm=0.41266874223947525 +[full s2] step 140/12000 loss=7.0255 gn=0.68 lr=4.23e-04 tps=18150 mem=5.7G lam=0.12638316924373308 Gm=0.419362660497427 +[full s2] step 160/12000 loss=6.7394 gn=0.62 lr=4.83e-04 tps=18177 mem=5.7G lam=0.12631283203760782 Gm=0.40420519933104515 +[full s2] step 180/12000 loss=6.8471 gn=0.50 lr=5.43e-04 tps=18192 mem=5.7G lam=0.12634840918083987 Gm=0.40053849294781685 +[full s2] step 200/12000 loss=6.6233 gn=0.56 lr=6.00e-04 tps=18209 mem=5.7G lam=0.12632321938872337 Gm=0.3915938114126523 +[full s2] step 220/12000 loss=6.6334 gn=0.60 lr=6.00e-04 tps=18237 mem=5.7G lam=0.1262961132451892 Gm=0.39850909635424614 +[full s2] step 240/12000 loss=6.5848 gn=0.52 lr=6.00e-04 tps=18261 mem=5.7G lam=0.12628011809041104 Gm=0.38567479078968364 +[full s2] step 260/12000 loss=6.4205 gn=0.49 lr=6.00e-04 tps=18285 mem=5.7G lam=0.1262688816835483 Gm=0.40669455751776695 +[full s2] step 280/12000 loss=6.5054 gn=0.49 lr=6.00e-04 tps=18305 mem=5.7G lam=0.12628869184603295 Gm=0.4135252746442954 +[full s2] step 300/12000 loss=6.6429 gn=0.46 lr=6.00e-04 tps=18321 mem=5.7G lam=0.12627174612134695 Gm=0.41215543697277707 +[full s2] step 320/12000 loss=6.5922 gn=0.48 lr=6.00e-04 tps=18328 mem=5.7G lam=0.12624872537950674 Gm=0.4080927607913812 +[full s2] step 340/12000 loss=6.4547 gn=0.47 lr=6.00e-04 tps=18349 mem=5.7G lam=0.12630452867597342 Gm=0.41941652198632556 +[full s2] step 360/12000 loss=6.4022 gn=0.51 lr=6.00e-04 tps=18358 mem=5.7G lam=0.12636436987668276 Gm=0.41444091126322746 +[full s2] step 380/12000 loss=6.2693 gn=0.49 lr=6.00e-04 tps=18367 mem=5.7G lam=0.12644809919099012 Gm=0.44231489673256874 +[full s2] step 400/12000 loss=6.4634 gn=0.49 lr=6.00e-04 tps=18360 mem=5.7G lam=0.1264700721949339 Gm=0.4171920989950498 +[full s2] step 420/12000 loss=6.2934 gn=0.47 lr=6.00e-04 tps=18353 mem=5.7G lam=0.12653880566358566 Gm=0.44463082775473595 +[full s2] step 440/12000 loss=6.2999 gn=0.48 lr=5.99e-04 tps=18369 mem=5.7G lam=0.12665680578599373 Gm=0.4387928210198879 +[full s2] step 460/12000 loss=6.2429 gn=0.47 lr=5.99e-04 tps=18383 mem=5.7G lam=0.1267462329318126 Gm=0.442952507485946 +[full s2] step 480/12000 loss=6.4837 gn=0.51 lr=5.99e-04 tps=18387 mem=5.7G lam=0.12689094307521978 Gm=0.4348972775042057 +[full s2] step 500/12000 loss=6.3186 gn=0.47 lr=5.99e-04 tps=18394 mem=5.7G lam=0.12710919479529062 Gm=0.4438770276804765 +[full s2] step 520/12000 loss=6.3432 gn=0.47 lr=5.99e-04 tps=18390 mem=5.7G lam=0.1273205168545246 Gm=0.46138598769903183 +[full s2] step 540/12000 loss=6.0884 gn=0.46 lr=5.99e-04 tps=18396 mem=5.7G lam=0.12752549350261688 Gm=0.49114174271623295 +[full s2] step 560/12000 loss=6.0521 gn=0.50 lr=5.99e-04 tps=18402 mem=5.7G lam=0.12772539630532265 Gm=0.4747086713711421 +[full s2] step 580/12000 loss=5.9166 gn=0.51 lr=5.99e-04 tps=18407 mem=5.7G lam=0.1279321194936832 Gm=0.48434366037448245 +[full s2] step 600/12000 loss=6.0612 gn=0.51 lr=5.98e-04 tps=18416 mem=5.7G lam=0.1281968541443348 Gm=0.5085552694896857 +[full s2] step 620/12000 loss=6.2218 gn=0.47 lr=5.98e-04 tps=18420 mem=5.7G lam=0.12840498921771845 Gm=0.5146757401525974 +[full s2] step 640/12000 loss=6.1311 gn=0.46 lr=5.98e-04 tps=18419 mem=5.7G lam=0.12866881676018238 Gm=0.4856528143088023 +[full s2] step 660/12000 loss=6.1160 gn=0.49 lr=5.98e-04 tps=18424 mem=5.7G lam=0.12898446805775166 Gm=0.525129958987236 +[full s2] step 680/12000 loss=6.0618 gn=0.43 lr=5.98e-04 tps=18423 mem=5.7G lam=0.12929840572178364 Gm=0.5323292103906473 +[full s2] step 700/12000 loss=5.9658 gn=0.48 lr=5.98e-04 tps=18424 mem=5.7G lam=0.12961199941734472 Gm=0.5174335166811943 +[full s2] step 720/12000 loss=6.0338 gn=0.59 lr=5.97e-04 tps=18428 mem=5.7G lam=0.13001847142974535 Gm=0.5206350771089395 +[full s2] step 740/12000 loss=6.0100 gn=0.51 lr=5.97e-04 tps=18424 mem=5.7G lam=0.1303439848124981 Gm=0.5434011556208134 +[full s2] step 760/12000 loss=6.0204 gn=0.49 lr=5.97e-04 tps=18422 mem=5.7G lam=0.13071177092691263 Gm=0.5497094007829825 +[full s2] step 780/12000 loss=5.7890 gn=0.45 lr=5.97e-04 tps=18426 mem=5.7G lam=0.1311031791071097 Gm=0.5489365843435129 +[full s2] step 800/12000 loss=5.8960 gn=0.52 lr=5.97e-04 tps=18429 mem=5.7G lam=0.13150799833238125 Gm=0.5431501778463522 +[full s2] step 820/12000 loss=5.8409 gn=0.46 lr=5.96e-04 tps=18434 mem=5.7G lam=0.13191263501842818 Gm=0.5251021397610506 +[full s2] step 840/12000 loss=5.9160 gn=0.51 lr=5.96e-04 tps=18433 mem=5.7G lam=0.13229165226221085 Gm=0.5420791395008564 +[full s2] step 860/12000 loss=5.8857 gn=0.48 lr=5.96e-04 tps=18436 mem=5.7G lam=0.13265409568945566 Gm=0.5364958656330904 +[full s2] step 880/12000 loss=5.9824 gn=0.56 lr=5.96e-04 tps=18439 mem=5.7G lam=0.13300994535287222 Gm=0.5545470807701349 +[full s2] step 900/12000 loss=5.7609 gn=0.48 lr=5.95e-04 tps=18442 mem=5.7G lam=0.13343547781308493 Gm=0.5333253517746925 +[full s2] step 920/12000 loss=5.9207 gn=0.48 lr=5.95e-04 tps=18445 mem=5.7G lam=0.13383407766620317 Gm=0.5261904081950585 +[full s2] step 940/12000 loss=5.8397 gn=0.49 lr=5.95e-04 tps=18449 mem=5.7G lam=0.1342735110471646 Gm=0.5237048634638389 +[full s2] step 960/12000 loss=5.7283 gn=0.53 lr=5.94e-04 tps=18453 mem=5.7G lam=0.13469342483828464 Gm=0.5491224192082882 +[full s2] step 980/12000 loss=5.8160 gn=0.49 lr=5.94e-04 tps=18457 mem=5.7G lam=0.13506792889287075 Gm=0.5385550862799088 +[full s2] step 1000/12000 loss=5.7309 gn=0.52 lr=5.94e-04 tps=18458 mem=5.7G lam=0.13545610196888447 Gm=0.5289684981107712 +[full s2] EVAL step 1000: val_loss=5.7257 val_ppl=306.65 +[full s2] step 1020/12000 loss=5.6772 gn=0.52 lr=5.94e-04 tps=18393 mem=5.7G lam=0.13589525253822407 Gm=0.5185140880445639 +[full s2] step 1040/12000 loss=5.8224 gn=0.52 lr=5.93e-04 tps=18389 mem=5.7G lam=0.13630167953670025 Gm=0.5284766412029663 +[full s2] step 1060/12000 loss=5.7123 gn=0.85 lr=5.93e-04 tps=18391 mem=5.7G lam=0.13668747867147127 Gm=0.5174616674582163 +[full s2] step 1080/12000 loss=5.7744 gn=0.52 lr=5.93e-04 tps=18392 mem=5.7G lam=0.1370680301139752 Gm=0.5161223740627369 +[full s2] step 1100/12000 loss=5.5989 gn=0.52 lr=5.92e-04 tps=18394 mem=5.7G lam=0.137446830359598 Gm=0.5228482174376646 +[full s2] step 1120/12000 loss=5.7034 gn=0.57 lr=5.92e-04 tps=18398 mem=5.7G lam=0.13789547079553208 Gm=0.507317022110025 +[full s2] step 1140/12000 loss=5.7423 gn=0.57 lr=5.92e-04 tps=18398 mem=5.7G lam=0.13832512373725572 Gm=0.5107714664191008 +[full s2] step 1160/12000 loss=5.6599 gn=0.52 lr=5.91e-04 tps=18399 mem=5.7G lam=0.1387224905192852 Gm=0.5247598265608152 +[full s2] step 1180/12000 loss=5.5033 gn=0.51 lr=5.91e-04 tps=18398 mem=5.7G lam=0.13918971673895916 Gm=0.5210008726765712 +[full s2] step 1200/12000 loss=5.5467 gn=0.48 lr=5.90e-04 tps=18398 mem=5.7G lam=0.13964535668492317 Gm=0.500509966785709 +[full s2] step 1220/12000 loss=5.5963 gn=0.49 lr=5.90e-04 tps=18400 mem=5.7G lam=0.14003067122151455 Gm=0.5094861071556807 +[full s2] step 1240/12000 loss=5.5315 gn=0.52 lr=5.90e-04 tps=18399 mem=5.7G lam=0.14037068064014116 Gm=0.49680651413897675 +[full s2] step 1260/12000 loss=5.4802 gn=0.56 lr=5.89e-04 tps=18402 mem=5.7G lam=0.14069891100128493 Gm=0.49864096318682033 +[full s2] step 1280/12000 loss=5.4696 gn=0.54 lr=5.89e-04 tps=18402 mem=5.7G lam=0.14109345184018215 Gm=0.49228996845583123 +[full s2] step 1300/12000 loss=5.5974 gn=0.51 lr=5.89e-04 tps=18403 mem=5.7G lam=0.14156868805487952 Gm=0.4901578289767106 +[full s2] step 1320/12000 loss=5.4460 gn=0.53 lr=5.88e-04 tps=18405 mem=5.7G lam=0.14196639135479927 Gm=0.4994811595728 +[full s2] step 1340/12000 loss=5.6604 gn=0.59 lr=5.88e-04 tps=18407 mem=5.7G lam=0.1423983027537664 Gm=0.48677730311950046 +[full s2] step 1360/12000 loss=5.4658 gn=0.57 lr=5.87e-04 tps=18409 mem=5.7G lam=0.14282935267935196 Gm=0.4950997866690159 +[full s2] step 1380/12000 loss=5.4970 gn=0.62 lr=5.87e-04 tps=18411 mem=5.7G lam=0.14321468956768513 Gm=0.49573741666972637 +[full s2] step 1400/12000 loss=5.3893 gn=0.56 lr=5.86e-04 tps=18413 mem=5.7G lam=0.14363887626677752 Gm=0.49526443021992844 +[full s2] step 1420/12000 loss=5.4517 gn=0.54 lr=5.86e-04 tps=18415 mem=5.7G lam=0.1440575054536263 Gm=0.4655254849543174 +[full s2] step 1440/12000 loss=5.5097 gn=0.54 lr=5.85e-04 tps=18416 mem=5.7G lam=0.1445297353590528 Gm=0.47922661527991295 +[full s2] step 1460/12000 loss=5.4204 gn=0.57 lr=5.85e-04 tps=18420 mem=5.7G lam=0.14494937906662622 Gm=0.47419710208972293 +[full s2] step 1480/12000 loss=5.3641 gn=0.53 lr=5.84e-04 tps=18423 mem=5.7G lam=0.1453350962450107 Gm=0.47464812360703945 +[full s2] step 1500/12000 loss=5.6162 gn=0.60 lr=5.84e-04 tps=18426 mem=5.7G lam=0.14579517673701048 Gm=0.47905650983254117 +[full s2] step 1520/12000 loss=5.3954 gn=0.56 lr=5.83e-04 tps=18428 mem=5.7G lam=0.1461938265711069 Gm=0.46732063715656597 +[full s2] step 1540/12000 loss=5.3966 gn=0.53 lr=5.83e-04 tps=18431 mem=5.7G lam=0.14668183152874312 Gm=0.4651111603404085 +[full s2] step 1560/12000 loss=5.6022 gn=0.57 lr=5.82e-04 tps=18433 mem=5.7G lam=0.14709010782341161 Gm=0.47138171053181094 +[full s2] step 1580/12000 loss=5.4933 gn=0.55 lr=5.82e-04 tps=18429 mem=5.7G lam=0.1475543569152554 Gm=0.4618686428293586 +[full s2] step 1600/12000 loss=5.4038 gn=0.57 lr=5.81e-04 tps=18431 mem=5.7G lam=0.14800871443003416 Gm=0.46421256692459184 +[full s2] step 1620/12000 loss=5.3787 gn=0.56 lr=5.81e-04 tps=18430 mem=5.7G lam=0.14845969714224339 Gm=0.4632339822128415 +[full s2] step 1640/12000 loss=5.4404 gn=0.58 lr=5.80e-04 tps=18431 mem=5.7G lam=0.14885643031448126 Gm=0.47566917911171913 +[full s2] step 1660/12000 loss=5.1687 gn=0.54 lr=5.80e-04 tps=18431 mem=5.7G lam=0.14929607852051655 Gm=0.44603611901402473 +[full s2] step 1680/12000 loss=5.4398 gn=0.57 lr=5.79e-04 tps=18431 mem=5.7G lam=0.14969619922339916 Gm=0.4659826283653577 +[full s2] step 1700/12000 loss=5.5159 gn=0.60 lr=5.79e-04 tps=18432 mem=5.7G lam=0.15010455095519623 Gm=0.47073886419336003 +[full s2] step 1720/12000 loss=5.2982 gn=0.55 lr=5.78e-04 tps=18431 mem=5.7G lam=0.15055705265452465 Gm=0.47026906193544465 +[full s2] step 1740/12000 loss=5.3695 gn=0.58 lr=5.78e-04 tps=18434 mem=5.7G lam=0.15100570799161991 Gm=0.4625313753883044 +[full s2] step 1760/12000 loss=5.2400 gn=0.53 lr=5.77e-04 tps=18435 mem=5.7G lam=0.15146891369173923 Gm=0.4495198515554269 +[full s2] step 1780/12000 loss=5.2724 gn=0.57 lr=5.76e-04 tps=18436 mem=5.7G lam=0.1519279001901547 Gm=0.45763474112997454 +[full s2] step 1800/12000 loss=5.4397 gn=0.58 lr=5.76e-04 tps=18438 mem=5.7G lam=0.15239717345684767 Gm=0.45396521128714085 +[full s2] step 1820/12000 loss=5.4899 gn=0.58 lr=5.75e-04 tps=18438 mem=5.7G lam=0.15286842888842025 Gm=0.4512081081047654 +[full s2] step 1840/12000 loss=5.4258 gn=0.58 lr=5.75e-04 tps=18440 mem=5.7G lam=0.15337270312011242 Gm=0.4454848974322279 +[full s2] step 1860/12000 loss=5.4309 gn=0.59 lr=5.74e-04 tps=18442 mem=5.7G lam=0.15397522784769535 Gm=0.4515508736173312 +[full s2] step 1880/12000 loss=5.1769 gn=0.57 lr=5.73e-04 tps=18442 mem=5.7G lam=0.15446000453084707 Gm=0.45178980721781653 +[full s2] step 1900/12000 loss=5.2322 gn=0.59 lr=5.73e-04 tps=18444 mem=5.7G lam=0.15490084451933703 Gm=0.4526829474295179 +[full s2] step 1920/12000 loss=5.4306 gn=0.60 lr=5.72e-04 tps=18446 mem=5.7G lam=0.15538312184313932 Gm=0.44620835439612466 +[full s2] step 1940/12000 loss=5.5489 gn=0.67 lr=5.72e-04 tps=18447 mem=5.7G lam=0.15589597324530283 Gm=0.4416217254474759 +[full s2] step 1960/12000 loss=5.1733 gn=0.58 lr=5.71e-04 tps=18449 mem=5.7G lam=0.15629517752677202 Gm=0.4462760289510091 +[full s2] step 1980/12000 loss=5.1568 gn=0.60 lr=5.70e-04 tps=18450 mem=5.7G lam=0.1567376066620151 Gm=0.43893749949832755 +[full s2] step 2000/12000 loss=5.1092 gn=0.58 lr=5.70e-04 tps=18451 mem=5.7G lam=0.15720953016231456 Gm=0.4413282200694084 +[full s2] EVAL step 2000: val_loss=5.2818 val_ppl=196.73 +[full s2] step 2020/12000 loss=5.1948 gn=0.68 lr=5.69e-04 tps=18417 mem=5.7G lam=0.1577280911927422 Gm=0.4364671589185794 +[full s2] step 2040/12000 loss=5.2990 gn=0.57 lr=5.68e-04 tps=18420 mem=5.7G lam=0.1582211727897326 Gm=0.4461989753569166 +[full s2] step 2060/12000 loss=5.3466 gn=0.61 lr=5.68e-04 tps=18421 mem=5.7G lam=0.15872606386741003 Gm=0.434550487746795 +[full s2] step 2080/12000 loss=5.3829 gn=0.60 lr=5.67e-04 tps=18423 mem=5.7G lam=0.15926411903152862 Gm=0.4344435107583801 +[full s2] step 2100/12000 loss=5.1996 gn=0.61 lr=5.66e-04 tps=18425 mem=5.7G lam=0.15979217582692704 Gm=0.4274547640234232 +[full s2] step 2120/12000 loss=5.3441 gn=0.61 lr=5.65e-04 tps=18426 mem=5.7G lam=0.16027451865375042 Gm=0.42834157838175696 +[full s2] step 2140/12000 loss=5.1312 gn=0.56 lr=5.65e-04 tps=18427 mem=5.7G lam=0.1607913759847482 Gm=0.421819521424671 +[full s2] step 2160/12000 loss=5.1735 gn=0.57 lr=5.64e-04 tps=18428 mem=5.7G lam=0.16130892280489206 Gm=0.4264714699238539 +[full s2] step 2180/12000 loss=5.2957 gn=0.63 lr=5.63e-04 tps=18430 mem=5.7G lam=0.16189563553780317 Gm=0.4254876160994172 +[full s2] step 2200/12000 loss=5.0705 gn=0.68 lr=5.63e-04 tps=18432 mem=5.7G lam=0.16239047547181448 Gm=0.4118834890735646 +[full s2] step 2220/12000 loss=5.1503 gn=0.56 lr=5.62e-04 tps=18432 mem=5.7G lam=0.16283243149518967 Gm=0.4342050813138485 +[full s2] step 2240/12000 loss=5.2929 gn=0.67 lr=5.61e-04 tps=18433 mem=5.7G lam=0.16335143117854992 Gm=0.4264349148919185 +[full s2] step 2260/12000 loss=5.0786 gn=0.64 lr=5.60e-04 tps=18433 mem=5.7G lam=0.16392874903976917 Gm=0.42783731361851096 +[full s2] step 2280/12000 loss=5.1173 gn=0.62 lr=5.60e-04 tps=18434 mem=5.7G lam=0.16449401434510946 Gm=0.430241489627709 +[full s2] step 2300/12000 loss=5.3252 gn=0.61 lr=5.59e-04 tps=18435 mem=5.7G lam=0.1649830381696423 Gm=0.41450281891350943 +[full s2] step 2320/12000 loss=5.2681 gn=0.64 lr=5.58e-04 tps=18435 mem=5.7G lam=0.16551449801772833 Gm=0.4235620096636315 +[full s2] step 2340/12000 loss=5.1128 gn=0.63 lr=5.57e-04 tps=18436 mem=5.7G lam=0.16596304842581353 Gm=0.4244716231090327 +[full s2] step 2360/12000 loss=5.1075 gn=0.60 lr=5.57e-04 tps=18439 mem=5.7G lam=0.16649243297676244 Gm=0.4092130420419077 +[full s2] step 2380/12000 loss=5.1006 gn=0.64 lr=5.56e-04 tps=18441 mem=5.7G lam=0.16700528499980768 Gm=0.42368073513110477 +[full s2] step 2400/12000 loss=5.0811 gn=0.61 lr=5.55e-04 tps=18439 mem=5.7G lam=0.16756803635507822 Gm=0.4141194128120939 +[full s2] step 2420/12000 loss=5.2364 gn=0.67 lr=5.54e-04 tps=18437 mem=5.7G lam=0.168087774887681 Gm=0.4349857857450843 +[full s2] step 2440/12000 loss=5.1181 gn=0.63 lr=5.53e-04 tps=18436 mem=5.7G lam=0.16862649905184904 Gm=0.4139840779826045 +[full s2] step 2460/12000 loss=5.0357 gn=0.61 lr=5.53e-04 tps=18435 mem=5.7G lam=0.1691224565729499 Gm=0.41543683502823114 +[full s2] step 2480/12000 loss=5.0441 gn=0.66 lr=5.52e-04 tps=18434 mem=5.7G lam=0.16971852785597244 Gm=0.42961839788282913 +[full s2] step 2500/12000 loss=5.1427 gn=0.60 lr=5.51e-04 tps=18432 mem=5.7G lam=0.17016773981352648 Gm=0.41371549883236486 +[full s2] step 2520/12000 loss=4.9654 gn=0.61 lr=5.50e-04 tps=18431 mem=5.7G lam=0.17066154287507138 Gm=0.4112963353594144 +[full s2] step 2540/12000 loss=5.0562 gn=0.66 lr=5.49e-04 tps=18431 mem=5.7G lam=0.17125032128145298 Gm=0.4135508197359741 +[full s2] step 2560/12000 loss=5.1571 gn=0.77 lr=5.48e-04 tps=18429 mem=5.7G lam=0.1718167985479037 Gm=0.41368107000986737 +[full s2] step 2580/12000 loss=5.1532 gn=0.69 lr=5.48e-04 tps=18428 mem=5.7G lam=0.17230215078840652 Gm=0.42312095252176124 +[full s2] step 2600/12000 loss=5.0284 gn=0.62 lr=5.47e-04 tps=18428 mem=5.7G lam=0.1728131820758184 Gm=0.4155926622139911 +[full s2] step 2620/12000 loss=4.9642 gn=0.67 lr=5.46e-04 tps=18427 mem=5.7G lam=0.17337300473203263 Gm=0.4151487424969673 +[full s2] step 2640/12000 loss=5.0885 gn=0.68 lr=5.45e-04 tps=18427 mem=5.7G lam=0.17382766089091697 Gm=0.4139624477053682 +[full s2] step 2660/12000 loss=5.1452 gn=0.67 lr=5.44e-04 tps=18426 mem=5.7G lam=0.17434489292403063 Gm=0.40837875303501886 +[full s2] step 2680/12000 loss=5.0928 gn=0.64 lr=5.43e-04 tps=18425 mem=5.7G lam=0.17484164300064245 Gm=0.40649866793925565 +[full s2] step 2700/12000 loss=5.1179 gn=0.66 lr=5.42e-04 tps=18424 mem=5.7G lam=0.17536256772776446 Gm=0.41110881650820374 +[full s2] step 2720/12000 loss=4.9922 gn=0.66 lr=5.41e-04 tps=18423 mem=5.7G lam=0.17586318496614695 Gm=0.4039522058640917 +[full s2] step 2740/12000 loss=5.0633 gn=0.63 lr=5.41e-04 tps=18422 mem=5.7G lam=0.17638398396472135 Gm=0.40223020532478887 +[full s2] step 2760/12000 loss=5.0013 gn=0.64 lr=5.40e-04 tps=18422 mem=5.7G lam=0.17691764483849207 Gm=0.4051111680455506 +[full s2] step 2780/12000 loss=4.9417 gn=0.64 lr=5.39e-04 tps=18421 mem=5.7G lam=0.1773693005864819 Gm=0.39169166640688974 +[full s2] step 2800/12000 loss=4.9872 gn=0.69 lr=5.38e-04 tps=18419 mem=5.7G lam=0.17784373989949623 Gm=0.40321701345965266 +[full s2] step 2820/12000 loss=4.9635 gn=0.66 lr=5.37e-04 tps=18417 mem=5.7G lam=0.1784131700793902 Gm=0.39375390236576396 +[full s2] step 2840/12000 loss=5.1185 gn=0.66 lr=5.36e-04 tps=18415 mem=5.7G lam=0.17893077867726484 Gm=0.4047230863943696 +[full s2] step 2860/12000 loss=4.8520 gn=0.65 lr=5.35e-04 tps=18412 mem=5.7G lam=0.17937469203025103 Gm=0.3962149474148949 +[full s2] step 2880/12000 loss=5.0210 gn=0.63 lr=5.34e-04 tps=18412 mem=5.7G lam=0.1798654356971383 Gm=0.4030799643757443 +[full s2] step 2900/12000 loss=5.0084 gn=0.69 lr=5.33e-04 tps=18410 mem=5.7G lam=0.18034114222973585 Gm=0.4049566965550184 +[full s2] step 2920/12000 loss=5.0510 gn=0.67 lr=5.32e-04 tps=18410 mem=5.7G lam=0.18082317368437847 Gm=0.39658166623363894 +[full s2] step 2940/12000 loss=5.1431 gn=0.74 lr=5.31e-04 tps=18409 mem=5.7G lam=0.18131747903923193 Gm=0.40029849562173087 +[full s2] step 2960/12000 loss=4.8792 gn=0.65 lr=5.30e-04 tps=18408 mem=5.7G lam=0.18177484907209873 Gm=0.3961283323975901 +[full s2] step 2980/12000 loss=4.9670 gn=0.64 lr=5.29e-04 tps=18407 mem=5.7G lam=0.1822403014327089 Gm=0.393532975576818 +[full s2] step 3000/12000 loss=4.8794 gn=0.64 lr=5.28e-04 tps=18406 mem=5.7G lam=0.18269335757941008 Gm=0.4037764500826597 +[full s2] EVAL step 3000: val_loss=5.0354 val_ppl=153.76 +[full s2] step 3020/12000 loss=5.0466 gn=0.70 lr=5.27e-04 tps=18382 mem=5.7G lam=0.1831766627728939 Gm=0.41135511733591557 +[full s2] step 3040/12000 loss=5.0272 gn=0.70 lr=5.26e-04 tps=18380 mem=5.7G lam=0.1837120500082771 Gm=0.3988512436238428 +[full s2] step 3060/12000 loss=4.9663 gn=0.71 lr=5.25e-04 tps=18380 mem=5.7G lam=0.18421843368560076 Gm=0.38992744653175276 +[full s2] step 3080/12000 loss=4.8410 gn=0.63 lr=5.24e-04 tps=18381 mem=5.7G lam=0.1848160351316134 Gm=0.3933831875522931 +[full s2] step 3100/12000 loss=4.9848 gn=0.68 lr=5.23e-04 tps=18381 mem=5.7G lam=0.18530254159122705 Gm=0.38876636264224845 +[full s2] step 3120/12000 loss=4.7244 gn=0.67 lr=5.22e-04 tps=18378 mem=5.7G lam=0.1858147345483303 Gm=0.39413539599627256 +[full s2] step 3140/12000 loss=4.8798 gn=0.63 lr=5.21e-04 tps=18378 mem=5.7G lam=0.18633918619404236 Gm=0.3939918363466859 +[full s2] step 3160/12000 loss=4.8428 gn=0.69 lr=5.20e-04 tps=18379 mem=5.7G lam=0.18685126273582378 Gm=0.39178433222696185 +[full s2] step 3180/12000 loss=4.9391 gn=0.62 lr=5.19e-04 tps=18378 mem=5.7G lam=0.1873847550402085 Gm=0.3922201964693765 +[full s2] step 3200/12000 loss=4.8913 gn=0.66 lr=5.18e-04 tps=18378 mem=5.7G lam=0.18788201113541922 Gm=0.3853043174992005 +[full s2] step 3220/12000 loss=4.9277 gn=0.72 lr=5.17e-04 tps=18376 mem=5.7G lam=0.18842553750922283 Gm=0.39062416056791943 +[full s2] step 3240/12000 loss=4.9800 gn=0.72 lr=5.16e-04 tps=18376 mem=5.7G lam=0.18892436133076748 Gm=0.3880583730836709 +[full s2] step 3260/12000 loss=5.0697 gn=0.73 lr=5.15e-04 tps=18375 mem=5.7G lam=0.1894298124437531 Gm=0.4001246471889317 +[full s2] step 3280/12000 loss=4.8568 gn=0.93 lr=5.14e-04 tps=18375 mem=5.7G lam=0.1899496034408609 Gm=0.38592807451883954 +[full s2] step 3300/12000 loss=4.7542 gn=0.67 lr=5.13e-04 tps=18375 mem=5.7G lam=0.19044240284711123 Gm=0.38255268211166066 +[full s2] step 3320/12000 loss=4.8663 gn=0.82 lr=5.12e-04 tps=18374 mem=5.7G lam=0.19088284516086182 Gm=0.38907974772155285 +[full s2] step 3340/12000 loss=4.8951 gn=0.70 lr=5.11e-04 tps=18373 mem=5.7G lam=0.1913753580302 Gm=0.394565490540117 +[full s2] step 3360/12000 loss=4.8391 gn=0.72 lr=5.10e-04 tps=18373 mem=5.7G lam=0.19190064569314322 Gm=0.3847434848236541 +[full s2] step 3380/12000 loss=4.9555 gn=0.72 lr=5.09e-04 tps=18373 mem=5.7G lam=0.19236861417690912 Gm=0.3877182236562173 +[full s2] step 3400/12000 loss=4.8665 gn=0.69 lr=5.08e-04 tps=18372 mem=5.7G lam=0.1928472894554337 Gm=0.39847093882660073 +[full s2] step 3420/12000 loss=4.9917 gn=0.69 lr=5.07e-04 tps=18372 mem=5.7G lam=0.19331851601600647 Gm=0.3863439983688295 +[full s2] step 3440/12000 loss=4.9193 gn=0.73 lr=5.06e-04 tps=18371 mem=5.7G lam=0.19383815148224434 Gm=0.3923082761466503 +[full s2] step 3460/12000 loss=4.9070 gn=0.70 lr=5.05e-04 tps=18371 mem=5.7G lam=0.19435821970303854 Gm=0.3862011462139587 +[full s2] step 3480/12000 loss=4.9366 gn=0.73 lr=5.03e-04 tps=18370 mem=5.7G lam=0.1948233644167582 Gm=0.3873302536085248 +[full s2] step 3500/12000 loss=4.8354 gn=0.69 lr=5.02e-04 tps=18369 mem=5.7G lam=0.1953278916577498 Gm=0.38518177702402073 +[full s2] step 3520/12000 loss=4.9114 gn=0.71 lr=5.01e-04 tps=18367 mem=5.7G lam=0.19589741869519153 Gm=0.38527073928465444 +[full s2] step 3540/12000 loss=4.8637 gn=0.66 lr=5.00e-04 tps=18367 mem=5.7G lam=0.19641887893279394 Gm=0.3838514550589025 +[full s2] step 3560/12000 loss=4.8009 gn=0.75 lr=4.99e-04 tps=18367 mem=5.7G lam=0.19689542582879463 Gm=0.3865942684933543 +[full s2] step 3580/12000 loss=4.8777 gn=0.68 lr=4.98e-04 tps=18366 mem=5.7G lam=0.19738874894877276 Gm=0.38344471994787455 +[full s2] step 3600/12000 loss=4.7430 gn=0.80 lr=4.97e-04 tps=18366 mem=5.7G lam=0.197815821506083 Gm=0.3862852974173923 +[full s2] step 3620/12000 loss=4.8467 gn=0.80 lr=4.96e-04 tps=18366 mem=5.7G lam=0.19834799381593862 Gm=0.3789879215570788 +[full s2] step 3640/12000 loss=4.7840 gn=0.80 lr=4.94e-04 tps=18366 mem=5.7G lam=0.198832582992812 Gm=0.38778681276987 +[full s2] step 3660/12000 loss=4.8440 gn=0.71 lr=4.93e-04 tps=18367 mem=5.7G lam=0.19932392860452333 Gm=0.3850980217879017 +[full s2] step 3680/12000 loss=4.8846 gn=0.71 lr=4.92e-04 tps=18367 mem=5.7G lam=0.19980428367853165 Gm=0.3861343474127352 +[full s2] step 3700/12000 loss=4.7966 gn=0.70 lr=4.91e-04 tps=18368 mem=5.7G lam=0.20031365348647037 Gm=0.38018740294501185 +[full s2] step 3720/12000 loss=4.9615 gn=0.71 lr=4.90e-04 tps=18368 mem=5.7G lam=0.20072645942370096 Gm=0.37767988350242376 +[full s2] step 3740/12000 loss=4.8553 gn=0.71 lr=4.89e-04 tps=18369 mem=5.7G lam=0.20118917804211378 Gm=0.37415416243796545 +[full s2] step 3760/12000 loss=4.8041 gn=0.67 lr=4.88e-04 tps=18369 mem=5.7G lam=0.20177840751906237 Gm=0.3798754387535155 +[full s2] step 3780/12000 loss=4.7846 gn=0.71 lr=4.86e-04 tps=18369 mem=5.7G lam=0.2023206309725841 Gm=0.37871278449893 +[full s2] step 3800/12000 loss=4.8914 gn=0.69 lr=4.85e-04 tps=18369 mem=5.7G lam=0.20276674100508293 Gm=0.3825483511512478 +[full s2] step 3820/12000 loss=4.8381 gn=0.65 lr=4.84e-04 tps=18370 mem=5.7G lam=0.2032441608607769 Gm=0.37459241412580013 +[full s2] step 3840/12000 loss=5.0072 gn=0.70 lr=4.83e-04 tps=18369 mem=5.7G lam=0.2037030402570963 Gm=0.3785966210222493 +[full s2] step 3860/12000 loss=4.6479 gn=0.71 lr=4.82e-04 tps=18369 mem=5.7G lam=0.20427408752342066 Gm=0.37602947598012787 +[full s2] step 3880/12000 loss=4.7572 gn=0.78 lr=4.80e-04 tps=18368 mem=5.7G lam=0.20472345563272634 Gm=0.3808244798177232 +[full s2] step 3900/12000 loss=4.7683 gn=0.70 lr=4.79e-04 tps=18367 mem=5.7G lam=0.20519114627192417 Gm=0.3668788247741759 +[full s2] step 3920/12000 loss=4.7967 gn=0.69 lr=4.78e-04 tps=18367 mem=5.7G lam=0.20564960315823555 Gm=0.3729277398282041 +[full s2] step 3940/12000 loss=4.9316 gn=0.71 lr=4.77e-04 tps=18367 mem=5.7G lam=0.20609594322741032 Gm=0.3854743697059651 +[full s2] step 3960/12000 loss=4.7196 gn=0.75 lr=4.76e-04 tps=18367 mem=5.7G lam=0.20658881527682146 Gm=0.379374358492593 +[full s2] step 3980/12000 loss=4.6355 gn=0.70 lr=4.74e-04 tps=18367 mem=5.7G lam=0.2070237503697475 Gm=0.3729872376813243 +[full s2] step 4000/12000 loss=4.8703 gn=0.71 lr=4.73e-04 tps=18367 mem=5.7G lam=0.20749107841402292 Gm=0.3746379624741773 +[full s2] EVAL step 4000: val_loss=4.8564 val_ppl=128.56 +[full s2] step 4020/12000 loss=4.8899 gn=0.72 lr=4.72e-04 tps=18349 mem=5.7G lam=0.20798185374587774 Gm=0.37987362042379874 +[full s2] step 4040/12000 loss=4.6569 gn=0.70 lr=4.71e-04 tps=18348 mem=5.7G lam=0.2084938862050573 Gm=0.37919288019960123 +[full s2] step 4060/12000 loss=4.7079 gn=0.74 lr=4.70e-04 tps=18344 mem=5.7G lam=0.2089303166915973 Gm=0.37904641567729414 +[full s2] step 4080/12000 loss=4.8316 gn=0.72 lr=4.68e-04 tps=18344 mem=5.7G lam=0.20939962783207497 Gm=0.3734715984513362 +[full s2] step 4100/12000 loss=4.7223 gn=0.73 lr=4.67e-04 tps=18344 mem=5.7G lam=0.20991316996514797 Gm=0.36823732778429985 +[full s2] step 4120/12000 loss=4.9089 gn=0.74 lr=4.66e-04 tps=18344 mem=5.7G lam=0.21041056948403516 Gm=0.3751599001698196 +[full s2] step 4140/12000 loss=4.8168 gn=0.74 lr=4.65e-04 tps=18344 mem=5.7G lam=0.21092920874555907 Gm=0.3734304079165061 +[full s2] step 4160/12000 loss=4.7469 gn=0.68 lr=4.63e-04 tps=18345 mem=5.7G lam=0.21138408749053875 Gm=0.3752025291323662 +[full s2] step 4180/12000 loss=4.8220 gn=0.69 lr=4.62e-04 tps=18344 mem=5.7G lam=0.2118880252043406 Gm=0.36622643082713086 +[full s2] step 4200/12000 loss=4.6627 gn=0.72 lr=4.61e-04 tps=18344 mem=5.7G lam=0.21241609131296477 Gm=0.36605495602513355 +[full s2] step 4220/12000 loss=4.7038 gn=0.68 lr=4.60e-04 tps=18344 mem=5.7G lam=0.2128771562129259 Gm=0.3791825131823619 +[full s2] step 4240/12000 loss=4.7370 gn=0.72 lr=4.58e-04 tps=18344 mem=5.7G lam=0.21334155400594076 Gm=0.3726536488781373 +[full s2] step 4260/12000 loss=4.6196 gn=0.74 lr=4.57e-04 tps=18344 mem=5.7G lam=0.21385138357679048 Gm=0.37003951938822865 +[full s2] step 4280/12000 loss=4.7041 gn=0.72 lr=4.56e-04 tps=18344 mem=5.7G lam=0.21429701211551824 Gm=0.3701711723891397 +[full s2] step 4300/12000 loss=4.7516 gn=0.80 lr=4.54e-04 tps=18344 mem=5.7G lam=0.21476611432929835 Gm=0.3680025717864434 +[full s2] step 4320/12000 loss=4.8398 gn=0.77 lr=4.53e-04 tps=18344 mem=5.7G lam=0.2153128059580922 Gm=0.37151685915887356 +[full s2] step 4340/12000 loss=4.7799 gn=0.71 lr=4.52e-04 tps=18344 mem=5.7G lam=0.21580229544391236 Gm=0.3638923742497961 +[full s2] step 4360/12000 loss=4.6910 gn=0.77 lr=4.51e-04 tps=18344 mem=5.7G lam=0.21623171865940094 Gm=0.371763720177114 +[full s2] step 4380/12000 loss=4.6486 gn=0.69 lr=4.49e-04 tps=18344 mem=5.7G lam=0.21671825492133698 Gm=0.36193791606153053 +[full s2] step 4400/12000 loss=4.7203 gn=0.76 lr=4.48e-04 tps=18344 mem=5.7G lam=0.21719104579339424 Gm=0.3707248494805147 +[full s2] step 4420/12000 loss=4.7377 gn=0.72 lr=4.47e-04 tps=18344 mem=5.7G lam=0.21760284279783568 Gm=0.37194110996400315 +[full s2] step 4440/12000 loss=4.7360 gn=0.75 lr=4.45e-04 tps=18343 mem=5.7G lam=0.21803683818628392 Gm=0.3762660894232492 +[full s2] step 4460/12000 loss=4.8833 gn=0.80 lr=4.44e-04 tps=18343 mem=5.7G lam=0.21844663905600706 Gm=0.36829678419356543 +[full s2] step 4480/12000 loss=4.8281 gn=0.73 lr=4.43e-04 tps=18343 mem=5.7G lam=0.21886573328326145 Gm=0.373716742809241 +[full s2] step 4500/12000 loss=4.6800 gn=0.75 lr=4.42e-04 tps=18343 mem=5.7G lam=0.2192375414694349 Gm=0.3700681682676077 +[full s2] step 4520/12000 loss=4.7962 gn=0.75 lr=4.40e-04 tps=18343 mem=5.7G lam=0.21969680146624646 Gm=0.37437101911442977 +[full s2] step 4540/12000 loss=4.7517 gn=0.71 lr=4.39e-04 tps=18343 mem=5.7G lam=0.22013943828642368 Gm=0.36656052983986837 +[full s2] step 4560/12000 loss=4.7635 gn=0.73 lr=4.38e-04 tps=18342 mem=5.7G lam=0.22059112321585417 Gm=0.36933529990104336 +[full s2] step 4580/12000 loss=4.5918 gn=0.73 lr=4.36e-04 tps=18342 mem=5.7G lam=0.22097623565544686 Gm=0.3670616849946479 +[full s2] step 4600/12000 loss=4.6058 gn=0.73 lr=4.35e-04 tps=18342 mem=5.7G lam=0.22144688945263624 Gm=0.37005076456504565 +[full s2] step 4620/12000 loss=4.6723 gn=0.74 lr=4.34e-04 tps=18341 mem=5.7G lam=0.2219452199836572 Gm=0.3624509124395748 +[full s2] step 4640/12000 loss=4.7003 gn=0.77 lr=4.32e-04 tps=18341 mem=5.7G lam=0.22240000311285257 Gm=0.355315966764465 +[full s2] step 4660/12000 loss=4.8718 gn=0.73 lr=4.31e-04 tps=18341 mem=5.7G lam=0.22276649562021097 Gm=0.3647414025229712 +[full s2] step 4680/12000 loss=4.8657 gn=0.78 lr=4.30e-04 tps=18342 mem=5.7G lam=0.22325715453674397 Gm=0.36839777169128257 +[full s2] step 4700/12000 loss=4.7021 gn=0.77 lr=4.28e-04 tps=18343 mem=5.7G lam=0.22376455211391053 Gm=0.3655489800342669 +[full s2] step 4720/12000 loss=4.5634 gn=0.81 lr=4.27e-04 tps=18343 mem=5.7G lam=0.22411862884958586 Gm=0.36582632541346055 +[full s2] step 4740/12000 loss=4.8844 gn=0.74 lr=4.26e-04 tps=18344 mem=5.7G lam=0.2245661892617742 Gm=0.36656110376740497 +[full s2] step 4760/12000 loss=4.5254 gn=0.80 lr=4.24e-04 tps=18344 mem=5.7G lam=0.22499683126807213 Gm=0.3614852003908406 +[full s2] step 4780/12000 loss=4.5047 gn=0.74 lr=4.23e-04 tps=18345 mem=5.7G lam=0.22545017457256714 Gm=0.36245282537614304 +[full s2] step 4800/12000 loss=4.7722 gn=0.73 lr=4.22e-04 tps=18344 mem=5.7G lam=0.22590030543506145 Gm=0.36767158440003794 +[full s2] step 4820/12000 loss=4.6723 gn=0.76 lr=4.20e-04 tps=18344 mem=5.7G lam=0.22634083156784376 Gm=0.3642715894772361 +[full s2] step 4840/12000 loss=4.7346 gn=0.71 lr=4.19e-04 tps=18344 mem=5.7G lam=0.22682149770359197 Gm=0.35907149376968545 +[full s2] step 4860/12000 loss=4.7640 gn=0.78 lr=4.18e-04 tps=18344 mem=5.7G lam=0.2272549153616031 Gm=0.37219169611732167 +[full s2] step 4880/12000 loss=4.6199 gn=0.77 lr=4.16e-04 tps=18344 mem=5.7G lam=0.22768030098328987 Gm=0.3636318159600099 +[full s2] step 4900/12000 loss=4.6797 gn=0.77 lr=4.15e-04 tps=18344 mem=5.7G lam=0.22808641909311214 Gm=0.3708611101222535 +[full s2] step 4920/12000 loss=4.6380 gn=0.75 lr=4.13e-04 tps=18344 mem=5.7G lam=0.22853189085920653 Gm=0.3683882237722476 +[full s2] step 4940/12000 loss=4.6538 gn=0.76 lr=4.12e-04 tps=18344 mem=5.7G lam=0.2289446129774054 Gm=0.3626647622634967 +[full s2] step 4960/12000 loss=4.6401 gn=0.73 lr=4.11e-04 tps=18344 mem=5.7G lam=0.22938456603636345 Gm=0.3609300156434377 +[full s2] step 4980/12000 loss=4.6223 gn=0.76 lr=4.09e-04 tps=18344 mem=5.7G lam=0.2299287881081303 Gm=0.3685564718519648 +[full s2] step 5000/12000 loss=4.5271 gn=0.72 lr=4.08e-04 tps=18344 mem=5.7G lam=0.23039571971942982 Gm=0.3614051959011704 +[full s2] EVAL step 5000: val_loss=4.7313 val_ppl=113.45 +[full s2] step 5020/12000 loss=4.8063 gn=0.78 lr=4.07e-04 tps=18329 mem=5.7G lam=0.23078136704862118 Gm=0.36648691119626164 +[full s2] step 5040/12000 loss=4.6118 gn=0.80 lr=4.05e-04 tps=18329 mem=5.7G lam=0.23125379936148724 Gm=0.36731131731842953 +[full s2] step 5060/12000 loss=4.6353 gn=0.77 lr=4.04e-04 tps=18328 mem=5.7G lam=0.23172950775672993 Gm=0.35900086257606745 +[full s2] step 5080/12000 loss=4.5414 gn=0.76 lr=4.02e-04 tps=18328 mem=5.7G lam=0.23209853656589985 Gm=0.35634246841073036 +[full s2] step 5100/12000 loss=4.6732 gn=0.80 lr=4.01e-04 tps=18326 mem=5.7G lam=0.23248967000593743 Gm=0.36003898790416616 +[full s2] step 5120/12000 loss=4.5076 gn=0.79 lr=4.00e-04 tps=18325 mem=5.7G lam=0.23290867265313864 Gm=0.36572482793902356 +[full s2] step 5140/12000 loss=4.7718 gn=0.85 lr=3.98e-04 tps=18325 mem=5.7G lam=0.23327579628676176 Gm=0.3646634053438902 +[full s2] step 5160/12000 loss=4.8395 gn=0.87 lr=3.97e-04 tps=18325 mem=5.7G lam=0.23367973789572716 Gm=0.3683308439018826 +[full s2] step 5180/12000 loss=4.6176 gn=0.72 lr=3.95e-04 tps=18324 mem=5.7G lam=0.23416248646875223 Gm=0.35976670884216827 +[full s2] step 5200/12000 loss=4.6424 gn=0.79 lr=3.94e-04 tps=18323 mem=5.7G lam=0.23463715302447477 Gm=0.35749017881850403 +[full s2] step 5220/12000 loss=4.7180 gn=0.82 lr=3.93e-04 tps=18324 mem=5.7G lam=0.23508761016031107 Gm=0.36218861141242087 +[full s2] step 5240/12000 loss=4.5908 gn=0.74 lr=3.91e-04 tps=18324 mem=5.7G lam=0.23551624920219183 Gm=0.3550149424312015 +[full s2] step 5260/12000 loss=4.4556 gn=0.80 lr=3.90e-04 tps=18324 mem=5.7G lam=0.2359797777608037 Gm=0.3588457959704101 +[full s2] step 5280/12000 loss=4.7925 gn=0.81 lr=3.88e-04 tps=18324 mem=5.7G lam=0.23636363012095293 Gm=0.37189727183431387 +[full s2] step 5300/12000 loss=4.6837 gn=0.77 lr=3.87e-04 tps=18325 mem=5.7G lam=0.2366569982841611 Gm=0.3652012792105476 +[full s2] step 5320/12000 loss=4.7220 gn=0.79 lr=3.86e-04 tps=18324 mem=5.7G lam=0.2370383289332191 Gm=0.35622096558411914 +[full s2] step 5340/12000 loss=4.4667 gn=0.75 lr=3.84e-04 tps=18325 mem=5.7G lam=0.2374259065836668 Gm=0.3587189270959546 +[full s2] step 5360/12000 loss=4.7094 gn=0.82 lr=3.83e-04 tps=18324 mem=5.7G lam=0.2378961956128478 Gm=0.36667414599408704 +[full s2] step 5380/12000 loss=4.4924 gn=0.83 lr=3.81e-04 tps=18325 mem=5.7G lam=0.23831748279432455 Gm=0.3562206371376912 +[full s2] step 5400/12000 loss=4.5411 gn=0.79 lr=3.80e-04 tps=18324 mem=5.7G lam=0.2387167252600193 Gm=0.3605493443707625 +[full s2] step 5420/12000 loss=4.6958 gn=0.80 lr=3.79e-04 tps=18325 mem=5.7G lam=0.23914241635551056 Gm=0.3691505810711533 +[full s2] step 5440/12000 loss=4.5360 gn=0.76 lr=3.77e-04 tps=18325 mem=5.7G lam=0.23948462773114443 Gm=0.3569627126368384 +[full s2] step 5460/12000 loss=4.7502 gn=0.79 lr=3.76e-04 tps=18325 mem=5.7G lam=0.23992769109706083 Gm=0.3621095049505432 +[full s2] step 5480/12000 loss=4.5312 gn=0.74 lr=3.74e-04 tps=18325 mem=5.7G lam=0.24027853272855282 Gm=0.3600333489788075 +[full s2] step 5500/12000 loss=4.8859 gn=0.80 lr=3.73e-04 tps=18325 mem=5.7G lam=0.24063249056537947 Gm=0.3626681532866011 +[full s2] step 5520/12000 loss=4.5877 gn=0.84 lr=3.72e-04 tps=18325 mem=5.7G lam=0.24102115258574486 Gm=0.36627815337851644 +[full s2] step 5540/12000 loss=4.5795 gn=0.81 lr=3.70e-04 tps=18325 mem=5.7G lam=0.2414532263452808 Gm=0.35865988675504923 +[full s2] step 5560/12000 loss=4.6303 gn=0.79 lr=3.69e-04 tps=18325 mem=5.7G lam=0.2418029671534896 Gm=0.36070890026167035 +[full s2] step 5580/12000 loss=4.5757 gn=0.76 lr=3.67e-04 tps=18324 mem=5.7G lam=0.2422080272808671 Gm=0.35591344147299725 +[full s2] step 5600/12000 loss=4.7499 gn=0.76 lr=3.66e-04 tps=18325 mem=5.7G lam=0.24267926843216023 Gm=0.35002347896806896 +[full s2] step 5620/12000 loss=4.6610 gn=0.84 lr=3.64e-04 tps=18325 mem=5.7G lam=0.2430457528680563 Gm=0.3599376818941285 +[full s2] step 5640/12000 loss=4.5079 gn=0.76 lr=3.63e-04 tps=18323 mem=5.7G lam=0.24339958435545364 Gm=0.3587719136849046 +[full s2] step 5660/12000 loss=4.5346 gn=0.76 lr=3.62e-04 tps=18323 mem=5.7G lam=0.24386021650085846 Gm=0.3577356638076405 +[full s2] step 5680/12000 loss=4.6379 gn=0.80 lr=3.60e-04 tps=18323 mem=5.7G lam=0.24425046953062216 Gm=0.3595401063406219 +[full s2] step 5700/12000 loss=4.5525 gn=0.81 lr=3.59e-04 tps=18323 mem=5.7G lam=0.2446083746229609 Gm=0.3507873003060619 +[full s2] step 5720/12000 loss=4.5219 gn=0.81 lr=3.57e-04 tps=18324 mem=5.7G lam=0.24502059817314148 Gm=0.3544035234954208 +[full s2] step 5740/12000 loss=4.5472 gn=0.76 lr=3.56e-04 tps=18324 mem=5.7G lam=0.24539149397363266 Gm=0.35764799538689357 +[full s2] step 5760/12000 loss=4.3295 gn=0.75 lr=3.54e-04 tps=18324 mem=5.7G lam=0.24579982490589222 Gm=0.34527440726136166 +[full s2] step 5780/12000 loss=4.5479 gn=0.83 lr=3.53e-04 tps=18325 mem=5.7G lam=0.24613815701256195 Gm=0.3523742681524406 +[full s2] step 5800/12000 loss=4.6207 gn=0.78 lr=3.52e-04 tps=18325 mem=5.7G lam=0.24657330506791672 Gm=0.3569521813187748 +[full s2] step 5820/12000 loss=4.5724 gn=0.81 lr=3.50e-04 tps=18325 mem=5.7G lam=0.2470536750430862 Gm=0.35477494083655375 +[full s2] step 5840/12000 loss=4.5916 gn=0.77 lr=3.49e-04 tps=18325 mem=5.7G lam=0.2474379101768136 Gm=0.35730172010759514 +[full s2] step 5860/12000 loss=4.6233 gn=0.80 lr=3.47e-04 tps=18326 mem=5.7G lam=0.24784611475964388 Gm=0.35157510071682435 +[full s2] step 5880/12000 loss=4.6421 gn=0.86 lr=3.46e-04 tps=18326 mem=5.7G lam=0.24823339097201824 Gm=0.3571040197275579 +[full s2] step 5900/12000 loss=4.7835 gn=0.81 lr=3.44e-04 tps=18326 mem=5.7G lam=0.24857573583722115 Gm=0.3591122531797737 +[full s2] step 5920/12000 loss=4.6194 gn=0.82 lr=3.43e-04 tps=18326 mem=5.7G lam=0.24897401966154575 Gm=0.35313502644809586 +[full s2] step 5940/12000 loss=4.5860 gn=0.82 lr=3.41e-04 tps=18326 mem=5.7G lam=0.2493547728906075 Gm=0.35611751636800665 +[full s2] step 5960/12000 loss=4.6417 gn=0.74 lr=3.40e-04 tps=18326 mem=5.7G lam=0.2497344557195902 Gm=0.35023068031296134 +[full s2] step 5980/12000 loss=4.4890 gn=0.80 lr=3.39e-04 tps=18326 mem=5.7G lam=0.2501613786444068 Gm=0.35462310118600726 +[full s2] step 6000/12000 loss=4.5148 gn=0.76 lr=3.37e-04 tps=18327 mem=5.7G lam=0.25054719516386587 Gm=0.3519288348034024 +[full s2] EVAL step 6000: val_loss=4.6151 val_ppl=101.00 +[full s2] step 6020/12000 loss=4.4491 gn=0.79 lr=3.36e-04 tps=18314 mem=5.7G lam=0.2508945145333807 Gm=0.34866391378454864 +[full s2] step 6040/12000 loss=4.5964 gn=0.78 lr=3.34e-04 tps=18315 mem=5.7G lam=0.2513109513868888 Gm=0.35723599046468735 +[full s2] step 6060/12000 loss=4.5043 gn=0.82 lr=3.33e-04 tps=18314 mem=5.7G lam=0.25163984516014654 Gm=0.36254176722529036 +[full s2] step 6080/12000 loss=4.5953 gn=0.81 lr=3.31e-04 tps=18314 mem=5.7G lam=0.2519926804428299 Gm=0.35580452958432335 +[full s2] step 6100/12000 loss=4.5521 gn=0.82 lr=3.30e-04 tps=18314 mem=5.7G lam=0.2524294729034106 Gm=0.35547514143399894 +[full s2] step 6120/12000 loss=4.4517 gn=0.76 lr=3.29e-04 tps=18314 mem=5.7G lam=0.2528292102118333 Gm=0.35154489598547417 +[full s2] step 6140/12000 loss=4.5132 gn=0.83 lr=3.27e-04 tps=18314 mem=5.7G lam=0.2531834427888195 Gm=0.3497035066441943 +[full s2] step 6160/12000 loss=4.6301 gn=0.85 lr=3.26e-04 tps=18314 mem=5.7G lam=0.2535893510406216 Gm=0.3611762940417975 +[full s2] step 6180/12000 loss=4.5106 gn=0.84 lr=3.24e-04 tps=18314 mem=5.7G lam=0.2539746171484391 Gm=0.3534837254943947 +[full s2] step 6200/12000 loss=4.5873 gn=0.81 lr=3.23e-04 tps=18314 mem=5.7G lam=0.25430611707270145 Gm=0.35488781162227195 +[full s2] step 6220/12000 loss=4.4781 gn=0.81 lr=3.21e-04 tps=18314 mem=5.7G lam=0.25466801784932613 Gm=0.35047577368095517 +[full s2] step 6240/12000 loss=4.4461 gn=0.80 lr=3.20e-04 tps=18314 mem=5.7G lam=0.25506712806721527 Gm=0.3448609738455464 +[full s2] step 6260/12000 loss=4.4310 gn=0.93 lr=3.19e-04 tps=18314 mem=5.7G lam=0.25548212602734566 Gm=0.3521242570908119 +[full s2] step 6280/12000 loss=4.7337 gn=0.81 lr=3.17e-04 tps=18315 mem=5.7G lam=0.25578777057429153 Gm=0.35100746310005587 +[full s2] step 6300/12000 loss=4.7230 gn=0.85 lr=3.16e-04 tps=18315 mem=5.7G lam=0.2561467581739028 Gm=0.34998311459397274 +[full s2] step 6320/12000 loss=4.4769 gn=0.82 lr=3.14e-04 tps=18316 mem=5.7G lam=0.25649868107090396 Gm=0.3466288612689823 +[full s2] step 6340/12000 loss=4.6287 gn=0.81 lr=3.13e-04 tps=18316 mem=5.7G lam=0.2568798279389739 Gm=0.34797933565763134 +[full s2] step 6360/12000 loss=4.5012 gn=0.78 lr=3.11e-04 tps=18316 mem=5.7G lam=0.2572236234943072 Gm=0.3391026667474459 +[full s2] step 6380/12000 loss=4.5289 gn=0.77 lr=3.10e-04 tps=18317 mem=5.7G lam=0.257581099246939 Gm=0.34649092812711996 +[full s2] step 6400/12000 loss=4.3637 gn=0.89 lr=3.08e-04 tps=18317 mem=5.7G lam=0.25803817994892597 Gm=0.3426804624808331 +[full s2] step 6420/12000 loss=4.5723 gn=0.84 lr=3.07e-04 tps=18317 mem=5.7G lam=0.25837179242322844 Gm=0.35516571734721464 +[full s2] step 6440/12000 loss=4.7193 gn=0.89 lr=3.06e-04 tps=18317 mem=5.7G lam=0.2586974569906791 Gm=0.34903534354331595 +[full s2] step 6460/12000 loss=4.5144 gn=0.82 lr=3.04e-04 tps=18317 mem=5.7G lam=0.25904644653201103 Gm=0.3505143219760309 +[full s2] step 6480/12000 loss=4.5650 gn=0.95 lr=3.03e-04 tps=18317 mem=5.7G lam=0.2594121241321166 Gm=0.3448467819641034 +[full s2] step 6500/12000 loss=4.3932 gn=0.80 lr=3.01e-04 tps=18317 mem=5.7G lam=0.25970530106375617 Gm=0.3492836563382298 +[full s2] step 6520/12000 loss=4.6475 gn=0.85 lr=3.00e-04 tps=18317 mem=5.7G lam=0.2600031749655803 Gm=0.3541179499588907 +[full s2] step 6540/12000 loss=4.5583 gn=0.85 lr=2.98e-04 tps=18317 mem=5.7G lam=0.2603443854798873 Gm=0.35399328971592087 +[full s2] step 6560/12000 loss=4.5603 gn=0.87 lr=2.97e-04 tps=18316 mem=5.7G lam=0.26064069972683984 Gm=0.3536200257949531 +[full s2] step 6580/12000 loss=4.4335 gn=0.81 lr=2.96e-04 tps=18316 mem=5.7G lam=0.2609589636946718 Gm=0.35654911228145164 +[full s2] step 6600/12000 loss=4.6110 gn=0.83 lr=2.94e-04 tps=18316 mem=5.7G lam=0.2612045304849744 Gm=0.3551192869587491 +[full s2] step 6620/12000 loss=4.4582 gn=0.79 lr=2.93e-04 tps=18316 mem=5.7G lam=0.26152102028330165 Gm=0.34749683861931163 +[full s2] step 6640/12000 loss=4.5684 gn=0.82 lr=2.91e-04 tps=18316 mem=5.7G lam=0.26186503004282713 Gm=0.3510279416417082 +[full s2] step 6660/12000 loss=4.6608 gn=0.84 lr=2.90e-04 tps=18316 mem=5.7G lam=0.2621956942602992 Gm=0.35527290791894 +[full s2] step 6680/12000 loss=4.4864 gn=0.83 lr=2.88e-04 tps=18316 mem=5.7G lam=0.26259634271264076 Gm=0.3510572847444564 +[full s2] step 6700/12000 loss=4.5880 gn=0.81 lr=2.87e-04 tps=18316 mem=5.7G lam=0.26286860182881355 Gm=0.3537772623045991 +[full s2] step 6720/12000 loss=4.5867 gn=0.81 lr=2.86e-04 tps=18315 mem=5.7G lam=0.26317142850408953 Gm=0.3513659493376811 +[full s2] step 6740/12000 loss=4.5622 gn=0.78 lr=2.84e-04 tps=18315 mem=5.7G lam=0.2634890132273237 Gm=0.34843026466357213 +[full s2] step 6760/12000 loss=4.5906 gn=0.81 lr=2.83e-04 tps=18315 mem=5.7G lam=0.2638515178114176 Gm=0.34799542735951644 +[full s2] step 6780/12000 loss=4.5395 gn=0.79 lr=2.81e-04 tps=18315 mem=5.7G lam=0.26413755770772696 Gm=0.35010140396965045 +[full s2] step 6800/12000 loss=4.5679 gn=0.87 lr=2.80e-04 tps=18315 mem=5.7G lam=0.264411261305213 Gm=0.34468358017814654 +[full s2] step 6820/12000 loss=4.5252 gn=0.82 lr=2.79e-04 tps=18316 mem=5.7G lam=0.26471197543044883 Gm=0.35014793323352933 +[full s2] step 6840/12000 loss=4.5943 gn=0.83 lr=2.77e-04 tps=18316 mem=5.7G lam=0.2649932544057568 Gm=0.3480816565764447 +[full s2] step 6860/12000 loss=4.5718 gn=0.87 lr=2.76e-04 tps=18316 mem=5.7G lam=0.2653119992464781 Gm=0.35150448031102616 +[full s2] step 6880/12000 loss=4.4638 gn=0.88 lr=2.74e-04 tps=18316 mem=5.7G lam=0.26557062411059934 Gm=0.3512707483799507 +[full s2] step 6900/12000 loss=4.5582 gn=0.86 lr=2.73e-04 tps=18317 mem=5.7G lam=0.2658611920972665 Gm=0.3417081370328863 +[full s2] step 6920/12000 loss=4.3920 gn=0.81 lr=2.72e-04 tps=18317 mem=5.7G lam=0.2661539052302639 Gm=0.34680186902793747 +[full s2] step 6940/12000 loss=4.7371 gn=0.89 lr=2.70e-04 tps=18317 mem=5.7G lam=0.26639222415784997 Gm=0.3537677451968193 +[full s2] step 6960/12000 loss=4.5053 gn=0.83 lr=2.69e-04 tps=18318 mem=5.7G lam=0.2667310054724415 Gm=0.3496264221612364 +[full s2] step 6980/12000 loss=4.4456 gn=0.83 lr=2.67e-04 tps=18317 mem=5.7G lam=0.2670760548984011 Gm=0.3417915853206068 +[full s2] step 7000/12000 loss=4.5586 gn=0.83 lr=2.66e-04 tps=18318 mem=5.7G lam=0.2673562038689852 Gm=0.3417685638802747 +[full s2] EVAL step 7000: val_loss=4.5634 val_ppl=95.91 +[full s2] step 7020/12000 loss=4.4617 gn=0.86 lr=2.65e-04 tps=18308 mem=5.7G lam=0.26773539930582047 Gm=0.3509233488390843 +[full s2] step 7040/12000 loss=4.5660 gn=0.90 lr=2.63e-04 tps=18308 mem=5.7G lam=0.267977526721855 Gm=0.35230124679704505 +[full s2] step 7060/12000 loss=4.5962 gn=0.90 lr=2.62e-04 tps=18307 mem=5.7G lam=0.2681980977455775 Gm=0.3519947857906421 +[full s2] step 7080/12000 loss=4.4650 gn=0.85 lr=2.60e-04 tps=18308 mem=5.7G lam=0.2684271289035678 Gm=0.34930820561324555 +[full s2] step 7100/12000 loss=4.4679 gn=0.83 lr=2.59e-04 tps=18308 mem=5.7G lam=0.2687070118263364 Gm=0.34349508156689507 +[full s2] step 7120/12000 loss=4.4218 gn=0.88 lr=2.58e-04 tps=18308 mem=5.7G lam=0.26900801062583923 Gm=0.3450092206088205 +[full s2] step 7140/12000 loss=4.5450 gn=0.82 lr=2.56e-04 tps=18308 mem=5.7G lam=0.26931104219208163 Gm=0.34349807801966864 +[full s2] step 7160/12000 loss=4.4232 gn=0.81 lr=2.55e-04 tps=18308 mem=5.7G lam=0.269560053323706 Gm=0.345428672619164 +[full s2] step 7180/12000 loss=4.4903 gn=0.86 lr=2.53e-04 tps=18308 mem=5.7G lam=0.26989066414535046 Gm=0.34445443594207364 +[full s2] step 7200/12000 loss=4.5330 gn=0.86 lr=2.52e-04 tps=18308 mem=5.7G lam=0.270141807384789 Gm=0.34280206814097863 +[full s2] step 7220/12000 loss=4.5741 gn=0.87 lr=2.51e-04 tps=18308 mem=5.7G lam=0.2704648192351063 Gm=0.3518008942094942 +[full s2] step 7240/12000 loss=4.3837 gn=0.88 lr=2.49e-04 tps=18308 mem=5.7G lam=0.27075411037852365 Gm=0.3458780182215075 +[full s2] step 7260/12000 loss=4.5761 gn=0.91 lr=2.48e-04 tps=18308 mem=5.7G lam=0.2710619615390897 Gm=0.34576650025943917 +[full s2] step 7280/12000 loss=4.4439 gn=0.88 lr=2.47e-04 tps=18308 mem=5.7G lam=0.2713607791811228 Gm=0.3417631190580626 +[full s2] step 7300/12000 loss=4.4499 gn=0.87 lr=2.45e-04 tps=18308 mem=5.7G lam=0.2716145838300387 Gm=0.3456472070732464 +[full s2] step 7320/12000 loss=4.5973 gn=0.87 lr=2.44e-04 tps=18308 mem=5.7G lam=0.27185906500866014 Gm=0.35261366454263526 +[full s2] step 7340/12000 loss=4.3823 gn=0.81 lr=2.42e-04 tps=18308 mem=5.7G lam=0.2721202426279585 Gm=0.34390215047945577 +[full s2] step 7360/12000 loss=4.3431 gn=0.87 lr=2.41e-04 tps=18308 mem=5.7G lam=0.27236748735109967 Gm=0.3472267137064288 +[full s2] step 7380/12000 loss=4.5738 gn=0.83 lr=2.40e-04 tps=18308 mem=5.7G lam=0.2726407693699002 Gm=0.3504553808985899 +[full s2] step 7400/12000 loss=4.4419 gn=0.84 lr=2.38e-04 tps=18309 mem=5.7G lam=0.27289222956945497 Gm=0.3411280196160078 +[full s2] step 7420/12000 loss=4.6553 gn=0.84 lr=2.37e-04 tps=18309 mem=5.7G lam=0.2731316403175394 Gm=0.3511813074971239 +[full s2] step 7440/12000 loss=4.4277 gn=0.85 lr=2.36e-04 tps=18309 mem=5.7G lam=0.27346060518175364 Gm=0.3474676991657664 +[full s2] step 7460/12000 loss=4.5304 gn=0.89 lr=2.34e-04 tps=18309 mem=5.7G lam=0.27370642808576423 Gm=0.3403428123953442 +[full s2] step 7480/12000 loss=4.5128 gn=0.85 lr=2.33e-04 tps=18309 mem=5.7G lam=0.273967313580215 Gm=0.3409225351642817 +[full s2] step 7500/12000 loss=4.4574 gn=0.86 lr=2.32e-04 tps=18309 mem=5.7G lam=0.27417553402483463 Gm=0.34461744587558013 +[full s2] step 7520/12000 loss=4.3435 gn=0.87 lr=2.30e-04 tps=18309 mem=5.7G lam=0.27432938509931165 Gm=0.34491568446780246 +[full s2] step 7540/12000 loss=4.4453 gn=0.87 lr=2.29e-04 tps=18309 mem=5.7G lam=0.2745937841633956 Gm=0.34407577019495267 +[full s2] step 7560/12000 loss=4.5801 gn=0.87 lr=2.28e-04 tps=18310 mem=5.7G lam=0.27488923724740744 Gm=0.34745575363437337 +[full s2] step 7580/12000 loss=4.4852 gn=0.87 lr=2.26e-04 tps=18310 mem=5.7G lam=0.275138265453279 Gm=0.34007783024571836 +[full s2] step 7600/12000 loss=4.4722 gn=0.84 lr=2.25e-04 tps=18310 mem=5.7G lam=0.2753533609211445 Gm=0.34357141936197877 +[full s2] step 7620/12000 loss=4.6985 gn=0.95 lr=2.24e-04 tps=18310 mem=5.7G lam=0.27554303128272295 Gm=0.3465550330777963 +[full s2] step 7640/12000 loss=4.4269 gn=0.91 lr=2.22e-04 tps=18310 mem=5.7G lam=0.27578023355454206 Gm=0.34294593582550686 +[full s2] step 7660/12000 loss=4.3693 gn=0.91 lr=2.21e-04 tps=18310 mem=5.7G lam=0.27607018190125626 Gm=0.34929618790435296 +[full s2] step 7680/12000 loss=4.3235 gn=0.86 lr=2.20e-04 tps=18310 mem=5.7G lam=0.2763404094924529 Gm=0.3509070803411305 +[full s2] step 7700/12000 loss=4.4607 gn=0.86 lr=2.18e-04 tps=18310 mem=5.7G lam=0.276559518960615 Gm=0.34671878232620656 +[full s2] step 7720/12000 loss=4.5134 gn=0.86 lr=2.17e-04 tps=18311 mem=5.7G lam=0.27682762779295444 Gm=0.3445802017425497 +[full s2] step 7740/12000 loss=4.4098 gn=0.91 lr=2.16e-04 tps=18311 mem=5.7G lam=0.27704560042669374 Gm=0.34736425064814586 +[full s2] step 7760/12000 loss=4.5452 gn=0.94 lr=2.15e-04 tps=18311 mem=5.7G lam=0.27727884457757074 Gm=0.3515725485049188 +[full s2] step 7780/12000 loss=4.5954 gn=0.90 lr=2.13e-04 tps=18311 mem=5.7G lam=0.2775564383094509 Gm=0.33264025393873453 +[full s2] step 7800/12000 loss=4.4886 gn=0.88 lr=2.12e-04 tps=18311 mem=5.7G lam=0.27780077420175076 Gm=0.3463479704999675 +[full s2] step 7820/12000 loss=4.4318 gn=0.85 lr=2.11e-04 tps=18311 mem=5.7G lam=0.27803632368644077 Gm=0.3506323369219899 +[full s2] step 7840/12000 loss=4.3344 gn=0.82 lr=2.09e-04 tps=18311 mem=5.7G lam=0.278230636070172 Gm=0.34209092054516077 +[full s2] step 7860/12000 loss=4.4086 gn=0.91 lr=2.08e-04 tps=18311 mem=5.7G lam=0.27840364755441743 Gm=0.34498072632898885 +[full s2] step 7880/12000 loss=4.4746 gn=0.95 lr=2.07e-04 tps=18311 mem=5.7G lam=0.2786326926822464 Gm=0.3454666421748698 +[full s2] step 7900/12000 loss=4.4444 gn=0.85 lr=2.06e-04 tps=18311 mem=5.7G lam=0.27885624797393876 Gm=0.3414739913617571 +[full s2] step 7920/12000 loss=4.5235 gn=0.87 lr=2.04e-04 tps=18311 mem=5.7G lam=0.2790412101894617 Gm=0.34968565109496313 +[full s2] step 7940/12000 loss=4.4336 gn=0.93 lr=2.03e-04 tps=18312 mem=5.7G lam=0.2792527923981349 Gm=0.346094433683902 +[full s2] step 7960/12000 loss=4.4926 gn=0.87 lr=2.02e-04 tps=18312 mem=5.7G lam=0.27943616081029177 Gm=0.3391097190324217 +[full s2] step 7980/12000 loss=4.4708 gn=0.86 lr=2.00e-04 tps=18312 mem=5.7G lam=0.27962516620755196 Gm=0.33773344111007947 +[full s2] step 8000/12000 loss=4.4232 gn=0.88 lr=1.99e-04 tps=18312 mem=5.7G lam=0.2798181405911843 Gm=0.341323188583677 +[full s2] EVAL step 8000: val_loss=4.4613 val_ppl=86.60 +[full s2] step 8020/12000 loss=4.4831 gn=0.88 lr=1.98e-04 tps=18304 mem=5.7G lam=0.2800451160098116 Gm=0.3455191315151751 +[full s2] step 8040/12000 loss=4.5102 gn=0.88 lr=1.97e-04 tps=18304 mem=5.7G lam=0.2802237846578161 Gm=0.33901863413242 +[full s2] step 8060/12000 loss=4.3732 gn=0.86 lr=1.95e-04 tps=18304 mem=5.7G lam=0.2804586937030156 Gm=0.34125416663785774 +[full s2] step 8080/12000 loss=4.4435 gn=0.85 lr=1.94e-04 tps=18302 mem=5.7G lam=0.2807151038820545 Gm=0.34719248100494343 +[full s2] step 8100/12000 loss=4.4747 gn=0.89 lr=1.93e-04 tps=18302 mem=5.7G lam=0.2809012910972039 Gm=0.34576702358511585 +[full s2] step 8120/12000 loss=4.5229 gn=0.88 lr=1.92e-04 tps=18301 mem=5.7G lam=0.2811124635239442 Gm=0.3453328316099942 +[full s2] step 8140/12000 loss=4.4180 gn=0.83 lr=1.90e-04 tps=18302 mem=5.7G lam=0.28133445885032415 Gm=0.34428204378734034 +[full s2] step 8160/12000 loss=4.5540 gn=0.86 lr=1.89e-04 tps=18301 mem=5.7G lam=0.28156424841533106 Gm=0.3436093144118786 +[full s2] step 8180/12000 loss=4.5321 gn=0.94 lr=1.88e-04 tps=18302 mem=5.7G lam=0.2817673208191991 Gm=0.3430606619610141 +[full s2] step 8200/12000 loss=4.4874 gn=0.86 lr=1.87e-04 tps=18302 mem=5.7G lam=0.2819822688276569 Gm=0.34271760533253354 +[full s2] step 8220/12000 loss=4.5120 gn=0.99 lr=1.86e-04 tps=18302 mem=5.7G lam=0.2822049530223012 Gm=0.34834318530435365 +[full s2] step 8240/12000 loss=4.4097 gn=0.95 lr=1.84e-04 tps=18302 mem=5.7G lam=0.28237522145112354 Gm=0.3425897490233183 +[full s2] step 8260/12000 loss=4.4494 gn=0.88 lr=1.83e-04 tps=18302 mem=5.7G lam=0.2825614387790362 Gm=0.34251963269586366 +[full s2] step 8280/12000 loss=4.4421 gn=0.88 lr=1.82e-04 tps=18303 mem=5.7G lam=0.2827739864587784 Gm=0.3436471870324264 +[full s2] step 8300/12000 loss=4.4927 gn=0.94 lr=1.81e-04 tps=18302 mem=5.7G lam=0.2829879376416405 Gm=0.34521379228681326 +[full s2] step 8320/12000 loss=4.4156 gn=0.91 lr=1.80e-04 tps=18302 mem=5.7G lam=0.2831706767901778 Gm=0.3452549030383428 +[full s2] step 8340/12000 loss=4.4694 gn=0.87 lr=1.78e-04 tps=18303 mem=5.7G lam=0.2833177875727415 Gm=0.33958334289491177 +[full s2] step 8360/12000 loss=4.4762 gn=0.87 lr=1.77e-04 tps=18303 mem=5.7G lam=0.28351906811197597 Gm=0.34123490176474053 +[full s2] step 8380/12000 loss=4.4158 gn=0.87 lr=1.76e-04 tps=18303 mem=5.7G lam=0.28374673519283533 Gm=0.3427600240490089 +[full s2] step 8400/12000 loss=4.3812 gn=0.92 lr=1.75e-04 tps=18303 mem=5.7G lam=0.28394926711916924 Gm=0.34632193025512 +[full s2] step 8420/12000 loss=4.5570 gn=0.91 lr=1.74e-04 tps=18303 mem=5.7G lam=0.2841445741554101 Gm=0.3454383021841447 +[full s2] step 8440/12000 loss=4.4496 gn=0.92 lr=1.72e-04 tps=18303 mem=5.7G lam=0.28432086296379566 Gm=0.3401517115222911 +[full s2] step 8460/12000 loss=4.4003 gn=0.90 lr=1.71e-04 tps=18303 mem=5.7G lam=0.28454557651033 Gm=0.3425490638862054 +[full s2] step 8480/12000 loss=4.3573 gn=0.85 lr=1.70e-04 tps=18304 mem=5.7G lam=0.2847661593308051 Gm=0.33732015592977405 +[full s2] step 8500/12000 loss=4.3747 gn=0.86 lr=1.69e-04 tps=18304 mem=5.7G lam=0.2849052483215928 Gm=0.3368231837327282 +[full s2] step 8520/12000 loss=4.5093 gn=0.90 lr=1.68e-04 tps=18304 mem=5.7G lam=0.2850850997492671 Gm=0.3454651453066617 +[full s2] step 8540/12000 loss=4.5737 gn=1.08 lr=1.67e-04 tps=18304 mem=5.7G lam=0.28532285677889985 Gm=0.3492842603009194 +[full s2] step 8560/12000 loss=4.3011 gn=0.94 lr=1.66e-04 tps=18304 mem=5.7G lam=0.28551208848754567 Gm=0.3420008389900128 +[full s2] step 8580/12000 loss=4.4717 gn=0.87 lr=1.64e-04 tps=18304 mem=5.7G lam=0.28568725877751905 Gm=0.3427431220188737 +[full s2] step 8600/12000 loss=4.3637 gn=0.90 lr=1.63e-04 tps=18304 mem=5.7G lam=0.28584979691853124 Gm=0.34264243106978637 +[full s2] step 8620/12000 loss=4.3837 gn=0.88 lr=1.62e-04 tps=18304 mem=5.7G lam=0.28602315795918304 Gm=0.3388632466085255 +[full s2] step 8640/12000 loss=4.3757 gn=0.91 lr=1.61e-04 tps=18304 mem=5.7G lam=0.2862177149703105 Gm=0.3360674226035674 +[full s2] step 8660/12000 loss=4.4359 gn=0.91 lr=1.60e-04 tps=18304 mem=5.7G lam=0.28639540790269774 Gm=0.34096252320644754 +[full s2] step 8680/12000 loss=4.3073 gn=0.92 lr=1.59e-04 tps=18304 mem=5.7G lam=0.28655648169418174 Gm=0.3462958197730283 +[full s2] step 8700/12000 loss=4.1872 gn=0.87 lr=1.58e-04 tps=18304 mem=5.7G lam=0.2867042403668165 Gm=0.3338698341976851 +[full s2] step 8720/12000 loss=4.3663 gn=0.93 lr=1.57e-04 tps=18304 mem=5.7G lam=0.28689033165574074 Gm=0.33544097864069045 +[full s2] step 8740/12000 loss=4.4253 gn=0.89 lr=1.55e-04 tps=18304 mem=5.7G lam=0.2870753730336825 Gm=0.3406755831092596 +[full s2] step 8760/12000 loss=4.5214 gn=0.91 lr=1.54e-04 tps=18305 mem=5.7G lam=0.2872346993535757 Gm=0.3465053917995344 +[full s2] step 8780/12000 loss=4.5311 gn=0.93 lr=1.53e-04 tps=18305 mem=5.7G lam=0.2873885929584503 Gm=0.33731718532120186 +[full s2] step 8800/12000 loss=4.3403 gn=0.89 lr=1.52e-04 tps=18305 mem=5.7G lam=0.28755657747387886 Gm=0.33812927291728556 +[full s2] step 8820/12000 loss=4.3749 gn=0.98 lr=1.51e-04 tps=18305 mem=5.7G lam=0.2877129102125764 Gm=0.3362420320821305 +[full s2] step 8840/12000 loss=4.5028 gn=0.87 lr=1.50e-04 tps=18305 mem=5.7G lam=0.287870019984742 Gm=0.335821316887935 +[full s2] step 8860/12000 loss=4.3806 gn=0.89 lr=1.49e-04 tps=18306 mem=5.7G lam=0.28801581387718517 Gm=0.3378468573403855 +[full s2] step 8880/12000 loss=4.3951 gn=0.90 lr=1.48e-04 tps=18306 mem=5.7G lam=0.2881877450272441 Gm=0.33469846736018855 +[full s2] step 8900/12000 loss=4.4586 gn=0.91 lr=1.47e-04 tps=18306 mem=5.7G lam=0.28834663207332295 Gm=0.3401527948056658 +[full s2] step 8920/12000 loss=4.3641 gn=0.92 lr=1.46e-04 tps=18306 mem=5.7G lam=0.28851402923464775 Gm=0.33773778681643307 +[full s2] step 8940/12000 loss=4.4373 gn=0.88 lr=1.45e-04 tps=18306 mem=5.7G lam=0.2887246481453379 Gm=0.33842016554748017 +[full s2] step 8960/12000 loss=4.4077 gn=0.93 lr=1.44e-04 tps=18306 mem=5.7G lam=0.28890900282810134 Gm=0.34036789066158235 +[full s2] step 8980/12000 loss=4.3170 gn=0.89 lr=1.43e-04 tps=18307 mem=5.7G lam=0.28903149751325447 Gm=0.34201516350731254 +[full s2] step 9000/12000 loss=4.3509 gn=0.93 lr=1.42e-04 tps=18307 mem=5.7G lam=0.28915111379077035 Gm=0.3391826833443095 +[full s2] EVAL step 9000: val_loss=4.4914 val_ppl=89.24 +[full s2] step 9020/12000 loss=4.3672 gn=0.91 lr=1.41e-04 tps=18300 mem=5.7G lam=0.28927144470314187 Gm=0.33862423004272085 +[full s2] step 9040/12000 loss=4.5250 gn=0.95 lr=1.40e-04 tps=18301 mem=5.7G lam=0.2893817185734709 Gm=0.34088653915872175 +[full s2] step 9060/12000 loss=4.3943 gn=0.92 lr=1.39e-04 tps=18301 mem=5.7G lam=0.2895372382675608 Gm=0.34071164710136753 +[full s2] step 9080/12000 loss=4.3073 gn=0.87 lr=1.38e-04 tps=18301 mem=5.7G lam=0.28968385327607393 Gm=0.3396356809729089 +[full s2] step 9100/12000 loss=4.4477 gn=1.01 lr=1.37e-04 tps=18301 mem=5.7G lam=0.28984434126565856 Gm=0.3415426362771541 +[full s2] step 9120/12000 loss=4.3634 gn=0.92 lr=1.36e-04 tps=18301 mem=5.7G lam=0.28998504144450027 Gm=0.34391885142152506 +[full s2] step 9140/12000 loss=4.2923 gn=0.91 lr=1.35e-04 tps=18301 mem=5.7G lam=0.2901645516976714 Gm=0.33488942504239577 +[full s2] step 9160/12000 loss=4.4389 gn=0.94 lr=1.34e-04 tps=18301 mem=5.7G lam=0.2903410258392493 Gm=0.3386434703910102 +[full s2] step 9180/12000 loss=4.3573 gn=1.03 lr=1.33e-04 tps=18301 mem=5.7G lam=0.29047109310825664 Gm=0.3450638285527627 +[full s2] step 9200/12000 loss=4.4686 gn=0.95 lr=1.32e-04 tps=18301 mem=5.7G lam=0.29060574527829885 Gm=0.3408366416115314 +[full s2] step 9220/12000 loss=4.3143 gn=0.97 lr=1.31e-04 tps=18301 mem=5.7G lam=0.290707743105789 Gm=0.33629184992363054 +[full s2] step 9240/12000 loss=4.3555 gn=0.96 lr=1.30e-04 tps=18301 mem=5.7G lam=0.29084125130126876 Gm=0.3358357850617419 +[full s2] step 9260/12000 loss=4.3491 gn=0.88 lr=1.29e-04 tps=18301 mem=5.7G lam=0.29100487505396205 Gm=0.3334297842035691 +[full s2] step 9280/12000 loss=4.2185 gn=0.93 lr=1.28e-04 tps=18301 mem=5.7G lam=0.29116071357081336 Gm=0.3412368368978302 +[full s2] step 9300/12000 loss=4.4581 gn=1.01 lr=1.27e-04 tps=18301 mem=5.7G lam=0.29130803669492406 Gm=0.3452843878573428 +[full s2] step 9320/12000 loss=4.3303 gn=0.94 lr=1.26e-04 tps=18301 mem=5.7G lam=0.2914233735452096 Gm=0.3435021953191608 +[full s2] step 9340/12000 loss=4.3509 gn=0.92 lr=1.25e-04 tps=18301 mem=5.7G lam=0.2915653583283226 Gm=0.33519791966925067 +[full s2] step 9360/12000 loss=4.1000 gn=0.90 lr=1.24e-04 tps=18301 mem=5.7G lam=0.2916759125267466 Gm=0.3326109075763573 +[full s2] step 9380/12000 loss=4.4985 gn=1.00 lr=1.23e-04 tps=18301 mem=5.7G lam=0.2918123643224438 Gm=0.3454054951046904 +[full s2] step 9400/12000 loss=4.4234 gn=0.91 lr=1.22e-04 tps=18301 mem=5.7G lam=0.29192141722887754 Gm=0.3417293028129886 +[full s2] step 9420/12000 loss=4.3565 gn=0.98 lr=1.21e-04 tps=18301 mem=5.7G lam=0.29202832467854023 Gm=0.34317640005610883 +[full s2] step 9440/12000 loss=4.2715 gn=0.92 lr=1.20e-04 tps=18301 mem=5.7G lam=0.2921571529780825 Gm=0.33100863088232774 +[full s2] step 9460/12000 loss=4.4359 gn=0.87 lr=1.19e-04 tps=18302 mem=5.7G lam=0.2922652779767911 Gm=0.33791570202447474 +[full s2] step 9480/12000 loss=4.2528 gn=1.00 lr=1.19e-04 tps=18302 mem=5.7G lam=0.29236923065036535 Gm=0.3376257074220727 +[full s2] step 9500/12000 loss=4.4217 gn=0.94 lr=1.18e-04 tps=18302 mem=5.7G lam=0.2925151623785496 Gm=0.3401383471985658 +[full s2] step 9520/12000 loss=4.2762 gn=0.91 lr=1.17e-04 tps=18302 mem=5.7G lam=0.29266442855199176 Gm=0.3382982169277966 +[full s2] step 9540/12000 loss=4.2537 gn=0.94 lr=1.16e-04 tps=18302 mem=5.7G lam=0.2928137170771758 Gm=0.33977019755790633 +[full s2] step 9560/12000 loss=4.2919 gn=0.91 lr=1.15e-04 tps=18302 mem=5.7G lam=0.2929294751957059 Gm=0.33709967974573374 +[full s2] step 9580/12000 loss=4.2067 gn=0.88 lr=1.14e-04 tps=18303 mem=5.7G lam=0.29305023948351544 Gm=0.3303916146978736 +[full s2] step 9600/12000 loss=4.3628 gn=0.96 lr=1.13e-04 tps=18303 mem=5.7G lam=0.29315228822330636 Gm=0.3390725862700492 +[full s2] step 9620/12000 loss=4.4498 gn=0.90 lr=1.12e-04 tps=18303 mem=5.7G lam=0.29326609211663407 Gm=0.33715248891773325 +[full s2] step 9640/12000 loss=4.3852 gn=0.90 lr=1.12e-04 tps=18303 mem=5.7G lam=0.293414739271005 Gm=0.3335470393455277 +[full s2] step 9660/12000 loss=4.2443 gn=0.97 lr=1.11e-04 tps=18303 mem=5.7G lam=0.2935737905402978 Gm=0.3376707124213378 +[full s2] step 9680/12000 loss=4.2084 gn=0.96 lr=1.10e-04 tps=18303 mem=5.7G lam=0.29369101269791525 Gm=0.33507958784078556 +[full s2] step 9700/12000 loss=4.2276 gn=0.97 lr=1.09e-04 tps=18303 mem=5.7G lam=0.2938101164375742 Gm=0.33479424302155775 +[full s2] step 9720/12000 loss=4.2321 gn=0.96 lr=1.08e-04 tps=18303 mem=5.7G lam=0.29392986247936886 Gm=0.3371958912660678 +[full s2] step 9740/12000 loss=4.3465 gn=0.98 lr=1.07e-04 tps=18303 mem=5.7G lam=0.29402144129077595 Gm=0.3368426085604976 +[full s2] step 9760/12000 loss=4.2040 gn=0.89 lr=1.07e-04 tps=18303 mem=5.7G lam=0.2941578694929679 Gm=0.33265756047330797 +[full s2] step 9780/12000 loss=4.3503 gn=0.97 lr=1.06e-04 tps=18303 mem=5.7G lam=0.2942352027942737 Gm=0.3320882262196392 +[full s2] step 9800/12000 loss=4.3884 gn=0.96 lr=1.05e-04 tps=18303 mem=5.7G lam=0.29434534596900147 Gm=0.33472171099856496 +[full s2] step 9820/12000 loss=4.4263 gn=0.94 lr=1.04e-04 tps=18303 mem=5.7G lam=0.2944585792720318 Gm=0.33602952988197404 +[full s2] step 9840/12000 loss=4.1682 gn=0.97 lr=1.03e-04 tps=18303 mem=5.7G lam=0.2945928235227863 Gm=0.3351038912466417 +[full s2] step 9860/12000 loss=4.4296 gn=0.96 lr=1.03e-04 tps=18303 mem=5.7G lam=0.29472578534235555 Gm=0.33624815203559894 +[full s2] step 9880/12000 loss=4.5135 gn=0.95 lr=1.02e-04 tps=18303 mem=5.7G lam=0.29482037015259266 Gm=0.33789721317589283 +[full s2] step 9900/12000 loss=4.3128 gn=0.97 lr=1.01e-04 tps=18303 mem=5.7G lam=0.2949146314834555 Gm=0.33645067488153774 +[full s2] step 9920/12000 loss=4.3972 gn=0.94 lr=1.00e-04 tps=18303 mem=5.7G lam=0.2950349214176337 Gm=0.33672376939405996 +[full s2] step 9940/12000 loss=4.3311 gn=0.88 lr=9.96e-05 tps=18303 mem=5.7G lam=0.2951022122676174 Gm=0.33262418761538964 +[full s2] step 9960/12000 loss=4.1846 gn=0.91 lr=9.89e-05 tps=18303 mem=5.7G lam=0.29520911288758117 Gm=0.33283226176475483 +[full s2] step 9980/12000 loss=4.3211 gn=0.94 lr=9.81e-05 tps=18304 mem=5.7G lam=0.2953006786604722 Gm=0.3369789677672088 +[full s2] step 10000/12000 loss=4.2114 gn=0.88 lr=9.74e-05 tps=18304 mem=5.7G lam=0.2954214770967762 Gm=0.3333827465151747 +[full s2] EVAL step 10000: val_loss=4.4008 val_ppl=81.52 +[full s2] step 10020/12000 loss=4.4945 gn=0.94 lr=9.67e-05 tps=18297 mem=5.7G lam=0.29554028374453384 Gm=0.33878051637051004 +[full s2] step 10040/12000 loss=4.5187 gn=1.02 lr=9.59e-05 tps=18297 mem=5.7G lam=0.2956211045384407 Gm=0.335706892888993 +[full s2] step 10060/12000 loss=4.5073 gn=0.99 lr=9.52e-05 tps=18297 mem=5.7G lam=0.295744691354533 Gm=0.3379207417989771 +[full s2] step 10080/12000 loss=4.3479 gn=0.95 lr=9.45e-05 tps=18297 mem=5.7G lam=0.2958727565904458 Gm=0.32970282970927656 +[full s2] step 10100/12000 loss=4.4742 gn=0.98 lr=9.38e-05 tps=18298 mem=5.7G lam=0.29597582078228396 Gm=0.33863294854139286 +[full s2] step 10120/12000 loss=4.2817 gn=0.97 lr=9.31e-05 tps=18298 mem=5.7G lam=0.2960841543972492 Gm=0.33341941563412547 +[full s2] step 10140/12000 loss=4.2789 gn=0.94 lr=9.24e-05 tps=18298 mem=5.7G lam=0.2961673429235816 Gm=0.33104372140951455 +[full s2] step 10160/12000 loss=4.3724 gn=1.00 lr=9.18e-05 tps=18298 mem=5.7G lam=0.296258218276004 Gm=0.334837014398848 +[full s2] step 10180/12000 loss=4.3798 gn=1.02 lr=9.11e-05 tps=18298 mem=5.7G lam=0.2963631597037117 Gm=0.33466524886898696 +[full s2] step 10200/12000 loss=4.2829 gn=0.96 lr=9.04e-05 tps=18299 mem=5.7G lam=0.2964528252681096 Gm=0.3319472138925145 +[full s2] step 10220/12000 loss=4.3792 gn=0.94 lr=8.98e-05 tps=18298 mem=5.7G lam=0.29653269456078607 Gm=0.3360574174827586 +[full s2] step 10240/12000 loss=4.3010 gn=0.93 lr=8.91e-05 tps=18298 mem=5.7G lam=0.2966192088400324 Gm=0.331117530586198 +[full s2] step 10260/12000 loss=4.3111 gn=0.94 lr=8.85e-05 tps=18298 mem=5.7G lam=0.2967294656361143 Gm=0.3397163654056688 +[full s2] step 10280/12000 loss=4.4568 gn=0.96 lr=8.78e-05 tps=18298 mem=5.7G lam=0.29682743549346924 Gm=0.3370238135879238 +[full s2] step 10300/12000 loss=4.2900 gn=1.01 lr=8.72e-05 tps=18298 mem=5.7G lam=0.296941928875943 Gm=0.33729294082149863 +[full s2] step 10320/12000 loss=4.4875 gn=0.98 lr=8.66e-05 tps=18299 mem=5.7G lam=0.29703755987187225 Gm=0.34162324977417785 +[full s2] step 10340/12000 loss=4.2981 gn=0.94 lr=8.59e-05 tps=18300 mem=5.7G lam=0.29712083221723634 Gm=0.33505570151222247 +[full s2] step 10360/12000 loss=4.3197 gn=0.97 lr=8.53e-05 tps=18300 mem=5.7G lam=0.2971818794806798 Gm=0.3349855304695666 +[full s2] step 10380/12000 loss=4.4258 gn=1.05 lr=8.47e-05 tps=18300 mem=5.7G lam=0.2972594431291024 Gm=0.33489597293858725 +[full s2] step 10400/12000 loss=4.4407 gn=0.98 lr=8.41e-05 tps=18300 mem=5.7G lam=0.2973357926433285 Gm=0.34063150531922776 +[full s2] step 10420/12000 loss=4.2587 gn=0.97 lr=8.35e-05 tps=18301 mem=5.7G lam=0.2974061307807763 Gm=0.33310824042807025 +[full s2] step 10440/12000 loss=4.2100 gn=1.08 lr=8.30e-05 tps=18301 mem=5.7G lam=0.29749381138632697 Gm=0.33501911008109647 +[full s2] step 10460/12000 loss=4.3595 gn=1.02 lr=8.24e-05 tps=18301 mem=5.7G lam=0.29757903578380746 Gm=0.3378456477851917 +[full s2] step 10480/12000 loss=4.3731 gn=0.96 lr=8.18e-05 tps=18301 mem=5.7G lam=0.29769213559726876 Gm=0.3355237925425172 +[full s2] step 10500/12000 loss=4.2692 gn=0.96 lr=8.12e-05 tps=18301 mem=5.7G lam=0.2977768012012045 Gm=0.3309869532628606 +[full s2] step 10520/12000 loss=4.2185 gn=0.96 lr=8.07e-05 tps=18302 mem=5.7G lam=0.29787065119793016 Gm=0.33339215551192564 +[full s2] step 10540/12000 loss=4.0970 gn=0.98 lr=8.01e-05 tps=18302 mem=5.7G lam=0.2979574119672179 Gm=0.330084431450814 +[full s2] step 10560/12000 loss=4.3473 gn=0.99 lr=7.96e-05 tps=18302 mem=5.7G lam=0.29804212382684153 Gm=0.3382441345602274 +[full s2] step 10580/12000 loss=4.4054 gn=1.02 lr=7.91e-05 tps=18302 mem=5.7G lam=0.29812996586163837 Gm=0.3381600435823202 +[full s2] step 10600/12000 loss=4.4437 gn=1.00 lr=7.85e-05 tps=18302 mem=5.7G lam=0.2982260699694355 Gm=0.33483957685530186 +[full s2] step 10620/12000 loss=4.3811 gn=1.00 lr=7.80e-05 tps=18302 mem=5.7G lam=0.29827895543227595 Gm=0.3335218519593279 +[full s2] step 10640/12000 loss=4.3628 gn=0.93 lr=7.75e-05 tps=18303 mem=5.7G lam=0.29834595633049804 Gm=0.3355657681822777 +[full s2] step 10660/12000 loss=4.3397 gn=0.99 lr=7.70e-05 tps=18303 mem=5.7G lam=0.29840943186233443 Gm=0.33601428552841145 +[full s2] step 10680/12000 loss=4.4184 gn=0.94 lr=7.65e-05 tps=18303 mem=5.7G lam=0.2984905168414116 Gm=0.3320116763934493 +[full s2] step 10700/12000 loss=4.4439 gn=0.96 lr=7.60e-05 tps=18303 mem=5.7G lam=0.2985793311769764 Gm=0.33309401882191497 +[full s2] step 10720/12000 loss=4.3144 gn=0.94 lr=7.55e-05 tps=18303 mem=5.7G lam=0.29866899953534204 Gm=0.3293830219190568 +[full s2] step 10740/12000 loss=4.1968 gn=0.92 lr=7.50e-05 tps=18304 mem=5.7G lam=0.29875212348997593 Gm=0.33091396605595946 +[full s2] step 10760/12000 loss=4.3248 gn=0.99 lr=7.46e-05 tps=18303 mem=5.7G lam=0.29883819352835417 Gm=0.3360634380951524 +[full s2] step 10780/12000 loss=4.4318 gn=0.98 lr=7.41e-05 tps=18304 mem=5.7G lam=0.2989079315836231 Gm=0.338360452093184 +[full s2] step 10800/12000 loss=4.3842 gn=1.17 lr=7.37e-05 tps=18304 mem=5.7G lam=0.2989699486643076 Gm=0.33223221303584677 +[full s2] step 10820/12000 loss=4.3256 gn=1.03 lr=7.32e-05 tps=18304 mem=5.7G lam=0.2990274153028925 Gm=0.3372191662589709 +[full s2] step 10840/12000 loss=4.3152 gn=1.00 lr=7.28e-05 tps=18304 mem=5.7G lam=0.29912532183031243 Gm=0.3354375382574896 +[full s2] step 10860/12000 loss=4.3049 gn=0.99 lr=7.23e-05 tps=18304 mem=5.7G lam=0.29917903058230877 Gm=0.3341499847980837 +[full s2] step 10880/12000 loss=4.4307 gn=1.02 lr=7.19e-05 tps=18303 mem=5.7G lam=0.29924711895485717 Gm=0.3305690231888245 +[full s2] step 10900/12000 loss=4.4936 gn=0.98 lr=7.15e-05 tps=18303 mem=5.7G lam=0.2993392863621314 Gm=0.33885904937051237 +[full s2] step 10920/12000 loss=4.5191 gn=1.15 lr=7.11e-05 tps=18304 mem=5.7G lam=0.2994183097034693 Gm=0.3364360462874174 +[full s2] step 10940/12000 loss=4.5151 gn=0.97 lr=7.07e-05 tps=18304 mem=5.7G lam=0.29951002666105825 Gm=0.33465244015678763 +[full s2] step 10960/12000 loss=4.2980 gn=0.94 lr=7.03e-05 tps=18304 mem=5.7G lam=0.29959105141460896 Gm=0.3320754912371437 +[full s2] step 10980/12000 loss=4.1673 gn=0.96 lr=6.99e-05 tps=18304 mem=5.7G lam=0.2996717893208067 Gm=0.3328418214805424 +[full s2] step 11000/12000 loss=4.3976 gn=0.99 lr=6.95e-05 tps=18304 mem=5.7G lam=0.29974689210454625 Gm=0.33909621454464894 +[full s2] EVAL step 11000: val_loss=4.4008 val_ppl=81.51 +[full s2] step 11020/12000 loss=4.3394 gn=1.06 lr=6.91e-05 tps=18298 mem=5.7G lam=0.2998280708367626 Gm=0.33585909722993773 +[full s2] step 11040/12000 loss=4.2727 gn=1.02 lr=6.88e-05 tps=18298 mem=5.7G lam=0.2999063804745674 Gm=0.3344186300722261 +[full s2] step 11060/12000 loss=4.3882 gn=1.00 lr=6.84e-05 tps=18298 mem=5.7G lam=0.29997246836622554 Gm=0.3404817064292729 +[full s2] step 11080/12000 loss=4.2957 gn=1.11 lr=6.81e-05 tps=18298 mem=5.7G lam=0.3000399845962723 Gm=0.33245221005442244 +[full s2] step 11100/12000 loss=4.4460 gn=0.94 lr=6.77e-05 tps=18298 mem=5.7G lam=0.3001103528464834 Gm=0.3366591820959002 +[full s2] step 11120/12000 loss=4.1792 gn=0.95 lr=6.74e-05 tps=18299 mem=5.7G lam=0.3001735821987192 Gm=0.3322420703091969 +[full s2] step 11140/12000 loss=4.3246 gn=0.94 lr=6.70e-05 tps=18299 mem=5.7G lam=0.30025535232077044 Gm=0.3340846349019557 +[full s2] step 11160/12000 loss=4.3590 gn=0.98 lr=6.67e-05 tps=18299 mem=5.7G lam=0.3003443709264199 Gm=0.3310563878621906 +[full s2] step 11180/12000 loss=4.3413 gn=1.01 lr=6.64e-05 tps=18299 mem=5.7G lam=0.3004110446199775 Gm=0.3342722758340339 +[full s2] step 11200/12000 loss=4.1781 gn=0.96 lr=6.61e-05 tps=18299 mem=5.7G lam=0.30048211570829153 Gm=0.3339957483112812 +[full s2] step 11220/12000 loss=4.1240 gn=0.97 lr=6.58e-05 tps=18299 mem=5.7G lam=0.3005703358600537 Gm=0.3322759289294481 +[full s2] step 11240/12000 loss=4.3414 gn=1.05 lr=6.55e-05 tps=18299 mem=5.7G lam=0.3006470849116643 Gm=0.3359257663444926 +[full s2] step 11260/12000 loss=4.3582 gn=1.04 lr=6.52e-05 tps=18299 mem=5.7G lam=0.3007193918650349 Gm=0.33518690429627895 +[full s2] step 11280/12000 loss=4.3194 gn=0.97 lr=6.49e-05 tps=18299 mem=5.7G lam=0.3007976400355498 Gm=0.3354551646237572 +[full s2] step 11300/12000 loss=4.3561 gn=1.05 lr=6.47e-05 tps=18299 mem=5.7G lam=0.3008703788121541 Gm=0.32774160988628864 +[full s2] step 11320/12000 loss=4.2173 gn=0.97 lr=6.44e-05 tps=18300 mem=5.7G lam=0.30093539350976545 Gm=0.3309194231405854 +[full s2] step 11340/12000 loss=4.3989 gn=1.04 lr=6.42e-05 tps=18300 mem=5.7G lam=0.3010226304953297 Gm=0.33545717131346464 +[full s2] step 11360/12000 loss=4.3132 gn=1.00 lr=6.39e-05 tps=18300 mem=5.7G lam=0.3010765478635828 Gm=0.3337508357750873 +[full s2] step 11380/12000 loss=4.3315 gn=0.99 lr=6.37e-05 tps=18300 mem=5.7G lam=0.30114170846839744 Gm=0.3364961484136681 +[full s2] step 11400/12000 loss=4.3581 gn=1.16 lr=6.34e-05 tps=18300 mem=5.7G lam=0.30120555125176907 Gm=0.3350443052283178 +[full s2] step 11420/12000 loss=4.3313 gn=1.01 lr=6.32e-05 tps=18300 mem=5.7G lam=0.3012923700734973 Gm=0.3355515001652141 +[full s2] step 11440/12000 loss=4.2027 gn=1.01 lr=6.30e-05 tps=18300 mem=5.7G lam=0.30137043446302414 Gm=0.3367900756808619 +[full s2] step 11460/12000 loss=4.2644 gn=1.01 lr=6.28e-05 tps=18300 mem=5.7G lam=0.3014207547530532 Gm=0.3333430814867218 +[full s2] step 11480/12000 loss=4.2372 gn=0.99 lr=6.26e-05 tps=18300 mem=5.7G lam=0.30149094946682453 Gm=0.3290424144361168 +[full s2] step 11500/12000 loss=4.3741 gn=1.05 lr=6.24e-05 tps=18300 mem=5.7G lam=0.3015477191656828 Gm=0.33636049134656787 +[full s2] step 11520/12000 loss=4.4323 gn=1.00 lr=6.22e-05 tps=18300 mem=5.7G lam=0.3016110733151436 Gm=0.33577623140687746 +[full s2] step 11540/12000 loss=4.3232 gn=1.07 lr=6.20e-05 tps=18300 mem=5.7G lam=0.3016840474059184 Gm=0.3353229062631726 +[full s2] step 11560/12000 loss=4.2707 gn=1.03 lr=6.19e-05 tps=18300 mem=5.7G lam=0.30174472214033204 Gm=0.33598918669546646 +[full s2] step 11580/12000 loss=4.4874 gn=1.19 lr=6.17e-05 tps=18301 mem=5.7G lam=0.30178547588487464 Gm=0.33549159734199446 +[full s2] step 11600/12000 loss=4.2109 gn=0.99 lr=6.15e-05 tps=18301 mem=5.7G lam=0.3018618741383155 Gm=0.33840199327096343 +[full s2] step 11620/12000 loss=4.3223 gn=1.03 lr=6.14e-05 tps=18301 mem=5.7G lam=0.30193774309009314 Gm=0.3322816089882205 +[full s2] step 11640/12000 loss=4.2298 gn=0.92 lr=6.12e-05 tps=18301 mem=5.7G lam=0.3020254758497079 Gm=0.33092236580948037 +[full s2] step 11660/12000 loss=4.2679 gn=1.02 lr=6.11e-05 tps=18301 mem=5.7G lam=0.3020991949985425 Gm=0.3331262009839217 +[full s2] step 11680/12000 loss=4.3035 gn=0.96 lr=6.10e-05 tps=18302 mem=5.7G lam=0.30217099376022816 Gm=0.33133028556282323 +[full s2] step 11700/12000 loss=4.3462 gn=1.04 lr=6.09e-05 tps=18302 mem=5.7G lam=0.30223641358315945 Gm=0.33266834588721395 +[full s2] step 11720/12000 loss=4.3007 gn=1.00 lr=6.07e-05 tps=18302 mem=5.7G lam=0.30230084465195733 Gm=0.332668022873501 +[full s2] step 11740/12000 loss=4.3439 gn=1.11 lr=6.06e-05 tps=18302 mem=5.7G lam=0.30235824082046747 Gm=0.33805657100553316 +[full s2] step 11760/12000 loss=4.2244 gn=1.00 lr=6.06e-05 tps=18302 mem=5.7G lam=0.30240761302411556 Gm=0.32713424678271014 +[full s2] step 11780/12000 loss=4.4177 gn=1.02 lr=6.05e-05 tps=18302 mem=5.7G lam=0.30246808255712193 Gm=0.3366701540847619 +[full s2] step 11800/12000 loss=4.3225 gn=0.97 lr=6.04e-05 tps=18303 mem=5.7G lam=0.30254902100811404 Gm=0.3355330795360108 +[full s2] step 11820/12000 loss=4.4871 gn=0.98 lr=6.03e-05 tps=18303 mem=5.7G lam=0.30261956869314116 Gm=0.3343801381221662 +[full s2] step 11840/12000 loss=4.2584 gn=1.00 lr=6.02e-05 tps=18303 mem=5.7G lam=0.3026861995458603 Gm=0.33055491792038083 +[full s2] step 11860/12000 loss=4.2239 gn=1.06 lr=6.02e-05 tps=18303 mem=5.7G lam=0.3027473653977116 Gm=0.3318434583488852 +[full s2] step 11880/12000 loss=4.3311 gn=1.14 lr=6.01e-05 tps=18303 mem=5.7G lam=0.30281743717690307 Gm=0.33413421533380944 +[full s2] step 11900/12000 loss=4.2967 gn=0.96 lr=6.01e-05 tps=18303 mem=5.7G lam=0.3028759406879544 Gm=0.3323155481678744 +[full s2] step 11920/12000 loss=4.2085 gn=1.08 lr=6.01e-05 tps=18303 mem=5.7G lam=0.3029349548742175 Gm=0.33602407574653625 +[full s2] step 11940/12000 loss=4.1956 gn=1.12 lr=6.00e-05 tps=18303 mem=5.7G lam=0.30299328671147424 Gm=0.3265834611374885 +[full s2] step 11960/12000 loss=4.2941 gn=1.03 lr=6.00e-05 tps=18304 mem=5.7G lam=0.30304825988908607 Gm=0.3326101692703863 +[full s2] step 11980/12000 loss=4.2694 gn=1.03 lr=6.00e-05 tps=18304 mem=5.7G lam=0.30312894998739165 Gm=0.33056403052372235 +[full s2] EVAL step 11999: val_loss=4.3977 val_ppl=81.26 +[init] variant=baseline seed=3 params=63.38M +[init] use_S=False use_G=False +[baseline s3] step 0/12000 loss=10.9114 gn=13.48 lr=3.00e-06 tps=10437 mem=4.8G lam=None Gm=None +[baseline s3] step 20/12000 loss=9.9122 gn=2.20 lr=6.30e-05 tps=23331 mem=5.4G lam=None Gm=None +[baseline s3] step 40/12000 loss=9.1606 gn=1.82 lr=1.23e-04 tps=24062 mem=5.4G lam=None Gm=None +[baseline s3] step 60/12000 loss=8.2196 gn=1.62 lr=1.83e-04 tps=24277 mem=5.4G lam=None Gm=None +[baseline s3] step 80/12000 loss=7.5543 gn=0.95 lr=2.43e-04 tps=24356 mem=5.4G lam=None Gm=None +[baseline s3] step 100/12000 loss=7.1170 gn=0.70 lr=3.03e-04 tps=24291 mem=5.4G lam=None Gm=None +[baseline s3] step 120/12000 loss=7.2036 gn=0.68 lr=3.63e-04 tps=24370 mem=5.4G lam=None Gm=None +[baseline s3] step 140/12000 loss=6.9144 gn=0.67 lr=4.23e-04 tps=24411 mem=5.4G lam=None Gm=None +[baseline s3] step 160/12000 loss=6.8611 gn=0.67 lr=4.83e-04 tps=24452 mem=5.4G lam=None Gm=None +[baseline s3] step 180/12000 loss=6.7627 gn=0.69 lr=5.43e-04 tps=24470 mem=5.4G lam=None Gm=None +[baseline s3] step 200/12000 loss=6.6926 gn=0.63 lr=6.00e-04 tps=24497 mem=5.4G lam=None Gm=None +[baseline s3] step 220/12000 loss=6.6015 gn=0.56 lr=6.00e-04 tps=24513 mem=5.4G lam=None Gm=None +[baseline s3] step 240/12000 loss=6.6267 gn=0.54 lr=6.00e-04 tps=24530 mem=5.4G lam=None Gm=None +[baseline s3] step 260/12000 loss=6.6433 gn=0.57 lr=6.00e-04 tps=24543 mem=5.4G lam=None Gm=None +[baseline s3] step 280/12000 loss=6.4615 gn=0.63 lr=6.00e-04 tps=24553 mem=5.4G lam=None Gm=None +[baseline s3] step 300/12000 loss=6.3438 gn=0.52 lr=6.00e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 320/12000 loss=6.4371 gn=0.46 lr=6.00e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 340/12000 loss=6.5003 gn=0.57 lr=6.00e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 360/12000 loss=6.2773 gn=0.58 lr=6.00e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 380/12000 loss=6.3127 gn=0.55 lr=6.00e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s3] step 400/12000 loss=6.3850 gn=0.53 lr=6.00e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s3] step 420/12000 loss=6.2952 gn=0.50 lr=6.00e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s3] step 440/12000 loss=6.2434 gn=0.61 lr=5.99e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s3] step 460/12000 loss=6.3590 gn=0.55 lr=5.99e-04 tps=24606 mem=5.4G lam=None Gm=None +[baseline s3] step 480/12000 loss=6.2150 gn=0.51 lr=5.99e-04 tps=24610 mem=5.4G lam=None Gm=None +[baseline s3] step 500/12000 loss=6.2556 gn=0.53 lr=5.99e-04 tps=24613 mem=5.4G lam=None Gm=None +[baseline s3] step 520/12000 loss=6.2658 gn=0.54 lr=5.99e-04 tps=24616 mem=5.4G lam=None Gm=None +[baseline s3] step 540/12000 loss=6.1726 gn=0.56 lr=5.99e-04 tps=24617 mem=5.4G lam=None Gm=None +[baseline s3] step 560/12000 loss=6.2168 gn=0.54 lr=5.99e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s3] step 580/12000 loss=6.1943 gn=0.53 lr=5.99e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s3] step 600/12000 loss=6.0998 gn=0.48 lr=5.98e-04 tps=24614 mem=5.4G lam=None Gm=None +[baseline s3] step 620/12000 loss=6.0107 gn=0.54 lr=5.98e-04 tps=24615 mem=5.4G lam=None Gm=None +[baseline s3] step 640/12000 loss=6.0965 gn=0.54 lr=5.98e-04 tps=24618 mem=5.4G lam=None Gm=None +[baseline s3] step 660/12000 loss=5.8990 gn=0.52 lr=5.98e-04 tps=24618 mem=5.4G lam=None Gm=None +[baseline s3] step 680/12000 loss=6.0254 gn=0.51 lr=5.98e-04 tps=24618 mem=5.4G lam=None Gm=None +[baseline s3] step 700/12000 loss=6.0725 gn=0.51 lr=5.98e-04 tps=24619 mem=5.4G lam=None Gm=None +[baseline s3] step 720/12000 loss=5.8644 gn=0.52 lr=5.97e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s3] step 740/12000 loss=6.0486 gn=0.55 lr=5.97e-04 tps=24622 mem=5.4G lam=None Gm=None +[baseline s3] step 760/12000 loss=5.9352 gn=0.56 lr=5.97e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s3] step 780/12000 loss=6.0132 gn=0.52 lr=5.97e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s3] step 800/12000 loss=5.9363 gn=0.51 lr=5.97e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s3] step 820/12000 loss=5.8584 gn=0.55 lr=5.96e-04 tps=24623 mem=5.4G lam=None Gm=None +[baseline s3] step 840/12000 loss=5.8853 gn=0.59 lr=5.96e-04 tps=24619 mem=5.4G lam=None Gm=None +[baseline s3] step 860/12000 loss=5.8607 gn=0.56 lr=5.96e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s3] step 880/12000 loss=5.9693 gn=0.52 lr=5.96e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s3] step 900/12000 loss=5.7708 gn=0.50 lr=5.95e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s3] step 920/12000 loss=5.7981 gn=0.56 lr=5.95e-04 tps=24620 mem=5.4G lam=None Gm=None +[baseline s3] step 940/12000 loss=5.6471 gn=0.51 lr=5.95e-04 tps=24621 mem=5.4G lam=None Gm=None +[baseline s3] step 960/12000 loss=5.8175 gn=0.59 lr=5.94e-04 tps=24623 mem=5.4G lam=None Gm=None +[baseline s3] step 980/12000 loss=5.7338 gn=0.49 lr=5.94e-04 tps=24624 mem=5.4G lam=None Gm=None +[baseline s3] step 1000/12000 loss=5.9067 gn=0.63 lr=5.94e-04 tps=24626 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 1000: val_loss=5.8590 val_ppl=350.37 +[baseline s3] step 1020/12000 loss=5.7402 gn=0.51 lr=5.94e-04 tps=24543 mem=5.4G lam=None Gm=None +[baseline s3] step 1040/12000 loss=5.6488 gn=0.51 lr=5.93e-04 tps=24546 mem=5.4G lam=None Gm=None +[baseline s3] step 1060/12000 loss=5.6252 gn=0.54 lr=5.93e-04 tps=24548 mem=5.4G lam=None Gm=None +[baseline s3] step 1080/12000 loss=5.6993 gn=0.53 lr=5.93e-04 tps=24550 mem=5.4G lam=None Gm=None +[baseline s3] step 1100/12000 loss=5.6645 gn=0.57 lr=5.92e-04 tps=24551 mem=5.4G lam=None Gm=None +[baseline s3] step 1120/12000 loss=5.7535 gn=0.53 lr=5.92e-04 tps=24553 mem=5.4G lam=None Gm=None +[baseline s3] step 1140/12000 loss=5.6475 gn=0.53 lr=5.92e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s3] step 1160/12000 loss=5.7055 gn=0.57 lr=5.91e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 1180/12000 loss=5.6323 gn=0.54 lr=5.91e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 1200/12000 loss=5.7008 gn=0.59 lr=5.90e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 1220/12000 loss=5.5084 gn=0.53 lr=5.90e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 1240/12000 loss=5.5478 gn=0.53 lr=5.90e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 1260/12000 loss=5.7204 gn=0.54 lr=5.89e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 1280/12000 loss=5.5685 gn=0.56 lr=5.89e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 1300/12000 loss=5.6690 gn=0.58 lr=5.89e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 1320/12000 loss=5.5577 gn=0.58 lr=5.88e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 1340/12000 loss=5.6259 gn=0.58 lr=5.88e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 1360/12000 loss=5.4661 gn=0.59 lr=5.87e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 1380/12000 loss=5.4320 gn=0.59 lr=5.87e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 1400/12000 loss=5.6185 gn=0.60 lr=5.86e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 1420/12000 loss=5.4887 gn=0.58 lr=5.86e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 1440/12000 loss=5.3776 gn=0.55 lr=5.85e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 1460/12000 loss=5.5481 gn=0.63 lr=5.85e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 1480/12000 loss=5.3899 gn=0.58 lr=5.84e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 1500/12000 loss=5.5063 gn=0.59 lr=5.84e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 1520/12000 loss=5.4973 gn=0.59 lr=5.83e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 1540/12000 loss=5.3640 gn=0.61 lr=5.83e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 1560/12000 loss=5.4229 gn=0.60 lr=5.82e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 1580/12000 loss=5.4993 gn=0.56 lr=5.82e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 1600/12000 loss=5.2761 gn=0.59 lr=5.81e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s3] step 1620/12000 loss=5.4775 gn=0.58 lr=5.81e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s3] step 1640/12000 loss=5.3555 gn=0.59 lr=5.80e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s3] step 1660/12000 loss=5.3775 gn=0.61 lr=5.80e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s3] step 1680/12000 loss=5.3862 gn=0.57 lr=5.79e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s3] step 1700/12000 loss=5.5440 gn=0.61 lr=5.79e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s3] step 1720/12000 loss=5.3912 gn=0.63 lr=5.78e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s3] step 1740/12000 loss=5.3404 gn=0.59 lr=5.78e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s3] step 1760/12000 loss=5.2744 gn=0.57 lr=5.77e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s3] step 1780/12000 loss=5.4244 gn=0.62 lr=5.76e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s3] step 1800/12000 loss=5.3310 gn=0.60 lr=5.76e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s3] step 1820/12000 loss=5.2838 gn=0.59 lr=5.75e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s3] step 1840/12000 loss=5.4461 gn=0.62 lr=5.75e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s3] step 1860/12000 loss=5.4029 gn=0.62 lr=5.74e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s3] step 1880/12000 loss=5.2906 gn=0.58 lr=5.73e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s3] step 1900/12000 loss=5.2187 gn=0.65 lr=5.73e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s3] step 1920/12000 loss=5.3155 gn=0.59 lr=5.72e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s3] step 1940/12000 loss=5.4311 gn=0.61 lr=5.72e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s3] step 1960/12000 loss=5.3726 gn=0.62 lr=5.71e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s3] step 1980/12000 loss=5.1066 gn=0.60 lr=5.70e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s3] step 2000/12000 loss=5.3560 gn=0.56 lr=5.70e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 2000: val_loss=5.3649 val_ppl=213.78 +[baseline s3] step 2020/12000 loss=5.3843 gn=0.61 lr=5.69e-04 tps=24551 mem=5.4G lam=None Gm=None +[baseline s3] step 2040/12000 loss=5.4138 gn=0.63 lr=5.68e-04 tps=24552 mem=5.4G lam=None Gm=None +[baseline s3] step 2060/12000 loss=5.2996 gn=0.61 lr=5.68e-04 tps=24552 mem=5.4G lam=None Gm=None +[baseline s3] step 2080/12000 loss=5.2104 gn=0.61 lr=5.67e-04 tps=24554 mem=5.4G lam=None Gm=None +[baseline s3] step 2100/12000 loss=5.3015 gn=0.60 lr=5.66e-04 tps=24554 mem=5.4G lam=None Gm=None +[baseline s3] step 2120/12000 loss=5.3284 gn=0.60 lr=5.65e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s3] step 2140/12000 loss=5.1465 gn=0.62 lr=5.65e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s3] step 2160/12000 loss=5.1244 gn=0.63 lr=5.64e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 2180/12000 loss=5.1736 gn=0.64 lr=5.63e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 2200/12000 loss=5.3564 gn=0.65 lr=5.63e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 2220/12000 loss=5.2752 gn=0.61 lr=5.62e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 2240/12000 loss=5.2666 gn=0.69 lr=5.61e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 2260/12000 loss=5.1377 gn=0.64 lr=5.60e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 2280/12000 loss=5.2310 gn=0.88 lr=5.60e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 2300/12000 loss=5.2883 gn=0.63 lr=5.59e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 2320/12000 loss=5.1914 gn=0.62 lr=5.58e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 2340/12000 loss=5.0065 gn=0.68 lr=5.57e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 2360/12000 loss=5.2104 gn=0.66 lr=5.57e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 2380/12000 loss=5.1504 gn=0.68 lr=5.56e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 2400/12000 loss=5.0992 gn=0.61 lr=5.55e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 2420/12000 loss=5.1270 gn=0.67 lr=5.54e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 2440/12000 loss=5.1507 gn=0.64 lr=5.53e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 2460/12000 loss=5.0989 gn=0.63 lr=5.53e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 2480/12000 loss=5.1131 gn=0.61 lr=5.52e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 2500/12000 loss=5.1221 gn=0.61 lr=5.51e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 2520/12000 loss=5.2352 gn=0.63 lr=5.50e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 2540/12000 loss=5.0895 gn=0.61 lr=5.49e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 2560/12000 loss=5.1998 gn=0.80 lr=5.48e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 2580/12000 loss=5.0493 gn=0.64 lr=5.48e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 2600/12000 loss=5.1181 gn=0.64 lr=5.47e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 2620/12000 loss=5.0237 gn=0.67 lr=5.46e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 2640/12000 loss=4.9731 gn=0.78 lr=5.45e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 2660/12000 loss=5.0928 gn=0.65 lr=5.44e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 2680/12000 loss=5.0756 gn=0.66 lr=5.43e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 2700/12000 loss=5.0614 gn=0.67 lr=5.42e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 2720/12000 loss=5.1274 gn=0.73 lr=5.41e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 2740/12000 loss=5.0989 gn=0.67 lr=5.41e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 2760/12000 loss=5.1185 gn=0.72 lr=5.40e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 2780/12000 loss=5.0470 gn=0.69 lr=5.39e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 2800/12000 loss=5.0093 gn=0.69 lr=5.38e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 2820/12000 loss=5.1816 gn=0.72 lr=5.37e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 2840/12000 loss=5.1085 gn=0.68 lr=5.36e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s3] step 2860/12000 loss=4.9669 gn=0.70 lr=5.35e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s3] step 2880/12000 loss=5.0195 gn=0.67 lr=5.34e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s3] step 2900/12000 loss=4.8728 gn=0.69 lr=5.33e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s3] step 2920/12000 loss=4.9963 gn=0.69 lr=5.32e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s3] step 2940/12000 loss=5.0402 gn=0.70 lr=5.31e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s3] step 2960/12000 loss=4.8702 gn=0.71 lr=5.30e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s3] step 2980/12000 loss=4.8033 gn=1.02 lr=5.29e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s3] step 3000/12000 loss=5.0822 gn=0.81 lr=5.28e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 3000: val_loss=5.0396 val_ppl=154.41 +[baseline s3] step 3020/12000 loss=5.1267 gn=0.72 lr=5.27e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 3040/12000 loss=4.8589 gn=0.66 lr=5.26e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 3060/12000 loss=4.9337 gn=0.75 lr=5.25e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 3080/12000 loss=4.8880 gn=0.69 lr=5.24e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 3100/12000 loss=5.0400 gn=0.70 lr=5.23e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 3120/12000 loss=4.7936 gn=0.71 lr=5.22e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 3140/12000 loss=4.9200 gn=0.68 lr=5.21e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 3160/12000 loss=4.8702 gn=0.73 lr=5.20e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 3180/12000 loss=4.8962 gn=0.75 lr=5.19e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 3200/12000 loss=4.8673 gn=0.71 lr=5.18e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 3220/12000 loss=4.9684 gn=0.77 lr=5.17e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 3240/12000 loss=4.9773 gn=0.73 lr=5.16e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 3260/12000 loss=5.0071 gn=0.69 lr=5.15e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 3280/12000 loss=4.8178 gn=0.66 lr=5.14e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 3300/12000 loss=5.0747 gn=0.67 lr=5.13e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 3320/12000 loss=4.9283 gn=0.69 lr=5.12e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 3340/12000 loss=4.9037 gn=0.75 lr=5.11e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 3360/12000 loss=4.9320 gn=0.71 lr=5.10e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 3380/12000 loss=5.0718 gn=0.80 lr=5.09e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 3400/12000 loss=4.9834 gn=0.75 lr=5.08e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 3420/12000 loss=4.9429 gn=0.76 lr=5.07e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 3440/12000 loss=4.7289 gn=0.68 lr=5.06e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 3460/12000 loss=4.9469 gn=0.70 lr=5.05e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 3480/12000 loss=4.9701 gn=0.75 lr=5.03e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 3500/12000 loss=5.0811 gn=0.72 lr=5.02e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 3520/12000 loss=4.8385 gn=0.71 lr=5.01e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 3540/12000 loss=4.8548 gn=0.68 lr=5.00e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 3560/12000 loss=4.9218 gn=0.78 lr=4.99e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3580/12000 loss=4.9307 gn=0.71 lr=4.98e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3600/12000 loss=4.9094 gn=0.74 lr=4.97e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3620/12000 loss=4.8991 gn=0.69 lr=4.96e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3640/12000 loss=4.8054 gn=0.72 lr=4.94e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 3660/12000 loss=5.0216 gn=0.74 lr=4.93e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3680/12000 loss=4.8469 gn=0.72 lr=4.92e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 3700/12000 loss=5.0132 gn=0.75 lr=4.91e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3720/12000 loss=4.9734 gn=0.78 lr=4.90e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 3740/12000 loss=4.8851 gn=0.75 lr=4.89e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3760/12000 loss=4.8189 gn=0.80 lr=4.88e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3780/12000 loss=4.7751 gn=0.76 lr=4.86e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 3800/12000 loss=4.9181 gn=0.76 lr=4.85e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 3820/12000 loss=5.0341 gn=0.78 lr=4.84e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 3840/12000 loss=4.9222 gn=0.78 lr=4.83e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 3860/12000 loss=4.8868 gn=0.71 lr=4.82e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 3880/12000 loss=4.9190 gn=0.73 lr=4.80e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 3900/12000 loss=4.9013 gn=0.74 lr=4.79e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 3920/12000 loss=4.8880 gn=0.71 lr=4.78e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 3940/12000 loss=4.7801 gn=0.80 lr=4.77e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 3960/12000 loss=4.7614 gn=0.81 lr=4.76e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 3980/12000 loss=4.8489 gn=0.73 lr=4.74e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 4000/12000 loss=4.7820 gn=0.76 lr=4.73e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 4000: val_loss=4.9010 val_ppl=134.43 +[baseline s3] step 4020/12000 loss=4.7293 gn=0.76 lr=4.72e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 4040/12000 loss=5.1089 gn=0.82 lr=4.71e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 4060/12000 loss=4.7707 gn=0.79 lr=4.70e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 4080/12000 loss=4.9279 gn=0.84 lr=4.68e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 4100/12000 loss=4.8439 gn=0.77 lr=4.67e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 4120/12000 loss=4.8110 gn=0.76 lr=4.66e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 4140/12000 loss=4.8312 gn=0.71 lr=4.65e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 4160/12000 loss=4.7723 gn=0.84 lr=4.63e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 4180/12000 loss=4.7573 gn=0.73 lr=4.62e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 4200/12000 loss=4.8557 gn=0.83 lr=4.61e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 4220/12000 loss=4.6853 gn=0.78 lr=4.60e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 4240/12000 loss=4.8711 gn=0.82 lr=4.58e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 4260/12000 loss=4.8910 gn=0.76 lr=4.57e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 4280/12000 loss=4.7954 gn=0.75 lr=4.56e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 4300/12000 loss=4.7202 gn=0.73 lr=4.54e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 4320/12000 loss=4.7721 gn=0.77 lr=4.53e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 4340/12000 loss=4.8774 gn=0.80 lr=4.52e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 4360/12000 loss=4.9196 gn=0.79 lr=4.51e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 4380/12000 loss=4.7934 gn=0.77 lr=4.49e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 4400/12000 loss=4.7009 gn=0.76 lr=4.48e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 4420/12000 loss=5.0152 gn=0.78 lr=4.47e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 4440/12000 loss=4.9138 gn=0.80 lr=4.45e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 4460/12000 loss=4.7862 gn=0.76 lr=4.44e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 4480/12000 loss=4.9836 gn=0.79 lr=4.43e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 4500/12000 loss=4.9216 gn=0.84 lr=4.42e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 4520/12000 loss=4.7776 gn=0.88 lr=4.40e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 4540/12000 loss=4.8168 gn=0.80 lr=4.39e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 4560/12000 loss=4.9579 gn=0.90 lr=4.38e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 4580/12000 loss=4.8217 gn=0.76 lr=4.36e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 4600/12000 loss=4.8176 gn=0.75 lr=4.35e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 4620/12000 loss=4.7610 gn=0.84 lr=4.34e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 4640/12000 loss=4.7475 gn=0.76 lr=4.32e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 4660/12000 loss=4.9473 gn=0.78 lr=4.31e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 4680/12000 loss=4.6323 gn=0.78 lr=4.30e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 4700/12000 loss=4.6491 gn=0.78 lr=4.28e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 4720/12000 loss=4.9053 gn=0.81 lr=4.27e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 4740/12000 loss=4.6636 gn=0.82 lr=4.26e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 4760/12000 loss=4.8081 gn=0.76 lr=4.24e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 4780/12000 loss=4.7624 gn=0.78 lr=4.23e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 4800/12000 loss=4.7756 gn=0.79 lr=4.22e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 4820/12000 loss=4.6294 gn=0.79 lr=4.20e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 4840/12000 loss=4.7430 gn=0.75 lr=4.19e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 4860/12000 loss=4.7752 gn=0.80 lr=4.18e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 4880/12000 loss=4.7837 gn=0.81 lr=4.16e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 4900/12000 loss=4.5858 gn=0.80 lr=4.15e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 4920/12000 loss=4.8026 gn=0.80 lr=4.13e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 4940/12000 loss=4.6733 gn=0.78 lr=4.12e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 4960/12000 loss=4.7417 gn=0.82 lr=4.11e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 4980/12000 loss=4.7020 gn=0.76 lr=4.09e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 5000/12000 loss=4.5902 gn=0.87 lr=4.08e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 5000: val_loss=4.7875 val_ppl=120.00 +[baseline s3] step 5020/12000 loss=4.7631 gn=0.81 lr=4.07e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 5040/12000 loss=4.6250 gn=0.80 lr=4.05e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 5060/12000 loss=4.6736 gn=0.93 lr=4.04e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 5080/12000 loss=4.7325 gn=0.82 lr=4.02e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 5100/12000 loss=4.6324 gn=0.87 lr=4.01e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 5120/12000 loss=4.7487 gn=0.75 lr=4.00e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 5140/12000 loss=4.6706 gn=0.86 lr=3.98e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 5160/12000 loss=4.7846 gn=0.78 lr=3.97e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 5180/12000 loss=4.5184 gn=0.81 lr=3.95e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 5200/12000 loss=4.5519 gn=0.84 lr=3.94e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 5220/12000 loss=4.7531 gn=0.79 lr=3.93e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 5240/12000 loss=4.5709 gn=0.84 lr=3.91e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 5260/12000 loss=4.7150 gn=0.83 lr=3.90e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 5280/12000 loss=4.5523 gn=0.81 lr=3.88e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 5300/12000 loss=4.7523 gn=0.88 lr=3.87e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5320/12000 loss=4.6058 gn=0.80 lr=3.86e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5340/12000 loss=4.5838 gn=0.79 lr=3.84e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5360/12000 loss=4.6263 gn=0.83 lr=3.83e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5380/12000 loss=4.7453 gn=0.82 lr=3.81e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5400/12000 loss=4.6762 gn=0.81 lr=3.80e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5420/12000 loss=4.8283 gn=0.85 lr=3.79e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 5440/12000 loss=4.5465 gn=0.82 lr=3.77e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 5460/12000 loss=4.5995 gn=0.79 lr=3.76e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 5480/12000 loss=4.5882 gn=0.79 lr=3.74e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 5500/12000 loss=4.7905 gn=0.95 lr=3.73e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 5520/12000 loss=4.8025 gn=0.88 lr=3.72e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5540/12000 loss=4.6884 gn=0.93 lr=3.70e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 5560/12000 loss=4.7111 gn=0.99 lr=3.69e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5580/12000 loss=4.5460 gn=0.84 lr=3.67e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 5600/12000 loss=4.7336 gn=0.93 lr=3.66e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 5620/12000 loss=4.6212 gn=0.88 lr=3.64e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 5640/12000 loss=4.6017 gn=0.89 lr=3.63e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 5660/12000 loss=4.6566 gn=0.84 lr=3.62e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 5680/12000 loss=4.7616 gn=0.84 lr=3.60e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 5700/12000 loss=4.6430 gn=0.85 lr=3.59e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 5720/12000 loss=4.6317 gn=0.82 lr=3.57e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 5740/12000 loss=4.6519 gn=0.89 lr=3.56e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 5760/12000 loss=4.5748 gn=0.79 lr=3.54e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 5780/12000 loss=4.6665 gn=0.85 lr=3.53e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 5800/12000 loss=4.6574 gn=0.84 lr=3.52e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 5820/12000 loss=4.4838 gn=0.87 lr=3.50e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 5840/12000 loss=4.5635 gn=0.84 lr=3.49e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 5860/12000 loss=4.5039 gn=0.83 lr=3.47e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 5880/12000 loss=4.4945 gn=0.85 lr=3.46e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 5900/12000 loss=4.5517 gn=0.90 lr=3.44e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 5920/12000 loss=4.5498 gn=0.83 lr=3.43e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 5940/12000 loss=4.6995 gn=0.83 lr=3.41e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 5960/12000 loss=4.6299 gn=0.82 lr=3.40e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 5980/12000 loss=4.4736 gn=0.88 lr=3.39e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 6000/12000 loss=4.6977 gn=0.84 lr=3.37e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 6000: val_loss=4.7332 val_ppl=113.66 +[baseline s3] step 6020/12000 loss=4.7908 gn=0.86 lr=3.36e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s3] step 6040/12000 loss=4.5770 gn=0.82 lr=3.34e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s3] step 6060/12000 loss=4.5042 gn=0.89 lr=3.33e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s3] step 6080/12000 loss=4.5372 gn=0.79 lr=3.31e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 6100/12000 loss=4.6748 gn=0.84 lr=3.30e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s3] step 6120/12000 loss=4.6991 gn=0.90 lr=3.29e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 6140/12000 loss=4.6526 gn=0.85 lr=3.27e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 6160/12000 loss=4.5346 gn=0.85 lr=3.26e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s3] step 6180/12000 loss=4.6078 gn=0.88 lr=3.24e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 6200/12000 loss=4.5772 gn=1.06 lr=3.23e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 6220/12000 loss=4.5968 gn=0.88 lr=3.21e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s3] step 6240/12000 loss=4.6883 gn=0.81 lr=3.20e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 6260/12000 loss=4.5764 gn=0.83 lr=3.19e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s3] step 6280/12000 loss=4.5607 gn=0.87 lr=3.17e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 6300/12000 loss=4.6819 gn=0.81 lr=3.16e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s3] step 6320/12000 loss=4.5276 gn=0.88 lr=3.14e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 6340/12000 loss=4.6492 gn=0.91 lr=3.13e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 6360/12000 loss=4.7759 gn=0.95 lr=3.11e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 6380/12000 loss=4.3180 gn=0.87 lr=3.10e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 6400/12000 loss=4.5449 gn=0.81 lr=3.08e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 6420/12000 loss=4.4882 gn=0.88 lr=3.07e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 6440/12000 loss=4.6554 gn=0.84 lr=3.06e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 6460/12000 loss=4.5274 gn=0.90 lr=3.04e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 6480/12000 loss=4.5642 gn=0.98 lr=3.03e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 6500/12000 loss=4.5089 gn=0.82 lr=3.01e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 6520/12000 loss=4.6204 gn=0.89 lr=3.00e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 6540/12000 loss=4.5336 gn=0.85 lr=2.98e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 6560/12000 loss=4.6784 gn=0.85 lr=2.97e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 6580/12000 loss=4.6149 gn=0.90 lr=2.96e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 6600/12000 loss=4.5153 gn=0.88 lr=2.94e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 6620/12000 loss=4.5475 gn=0.98 lr=2.93e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 6640/12000 loss=4.4153 gn=0.92 lr=2.91e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 6660/12000 loss=4.4639 gn=0.83 lr=2.90e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 6680/12000 loss=4.4875 gn=0.91 lr=2.88e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 6700/12000 loss=4.7967 gn=0.96 lr=2.87e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 6720/12000 loss=4.4422 gn=0.88 lr=2.86e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 6740/12000 loss=4.4884 gn=0.88 lr=2.84e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 6760/12000 loss=4.5905 gn=0.95 lr=2.83e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 6780/12000 loss=4.5489 gn=0.82 lr=2.81e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 6800/12000 loss=4.7064 gn=0.94 lr=2.80e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 6820/12000 loss=4.5966 gn=0.92 lr=2.79e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 6840/12000 loss=4.5831 gn=0.95 lr=2.77e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 6860/12000 loss=4.5805 gn=0.92 lr=2.76e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 6880/12000 loss=4.5784 gn=0.90 lr=2.74e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 6900/12000 loss=4.5668 gn=0.89 lr=2.73e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 6920/12000 loss=4.4952 gn=0.93 lr=2.72e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 6940/12000 loss=4.3886 gn=0.82 lr=2.70e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 6960/12000 loss=4.4527 gn=0.88 lr=2.69e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 6980/12000 loss=4.6287 gn=0.98 lr=2.67e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 7000/12000 loss=4.4164 gn=0.83 lr=2.66e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 7000: val_loss=4.6259 val_ppl=102.10 +[baseline s3] step 7020/12000 loss=4.5065 gn=0.84 lr=2.65e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s3] step 7040/12000 loss=4.4877 gn=0.92 lr=2.63e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 7060/12000 loss=4.5556 gn=0.94 lr=2.62e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 7080/12000 loss=4.4955 gn=0.97 lr=2.60e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 7100/12000 loss=4.4911 gn=0.87 lr=2.59e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s3] step 7120/12000 loss=4.6229 gn=0.92 lr=2.58e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 7140/12000 loss=4.5255 gn=0.96 lr=2.56e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 7160/12000 loss=4.6908 gn=0.92 lr=2.55e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 7180/12000 loss=4.5083 gn=0.86 lr=2.53e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 7200/12000 loss=4.4249 gn=0.87 lr=2.52e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s3] step 7220/12000 loss=4.4560 gn=0.90 lr=2.51e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 7240/12000 loss=4.5891 gn=0.86 lr=2.49e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 7260/12000 loss=4.4211 gn=1.01 lr=2.48e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s3] step 7280/12000 loss=4.3915 gn=0.86 lr=2.47e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 7300/12000 loss=4.6424 gn=0.93 lr=2.45e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s3] step 7320/12000 loss=4.6485 gn=0.94 lr=2.44e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 7340/12000 loss=4.4469 gn=0.89 lr=2.42e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 7360/12000 loss=4.4262 gn=0.90 lr=2.41e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 7380/12000 loss=4.4746 gn=0.90 lr=2.40e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 7400/12000 loss=4.6049 gn=0.90 lr=2.38e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 7420/12000 loss=4.4650 gn=0.90 lr=2.37e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 7440/12000 loss=4.5241 gn=0.94 lr=2.36e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 7460/12000 loss=4.5992 gn=0.92 lr=2.34e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 7480/12000 loss=4.5267 gn=0.93 lr=2.33e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 7500/12000 loss=4.5916 gn=0.90 lr=2.32e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 7520/12000 loss=4.4055 gn=0.89 lr=2.30e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 7540/12000 loss=4.5050 gn=0.92 lr=2.29e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 7560/12000 loss=4.5384 gn=0.87 lr=2.28e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 7580/12000 loss=4.4672 gn=0.88 lr=2.26e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 7600/12000 loss=4.4122 gn=0.98 lr=2.25e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 7620/12000 loss=4.3242 gn=0.96 lr=2.24e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 7640/12000 loss=4.5264 gn=0.92 lr=2.22e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 7660/12000 loss=4.4276 gn=0.98 lr=2.21e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 7680/12000 loss=4.5293 gn=0.93 lr=2.20e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 7700/12000 loss=4.4680 gn=0.93 lr=2.18e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 7720/12000 loss=4.3635 gn=0.94 lr=2.17e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 7740/12000 loss=4.3533 gn=0.92 lr=2.16e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 7760/12000 loss=4.4386 gn=0.95 lr=2.15e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 7780/12000 loss=4.4696 gn=0.88 lr=2.13e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 7800/12000 loss=4.4409 gn=0.98 lr=2.12e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 7820/12000 loss=4.5578 gn=0.92 lr=2.11e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 7840/12000 loss=4.4919 gn=0.95 lr=2.09e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 7860/12000 loss=4.2528 gn=0.90 lr=2.08e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 7880/12000 loss=4.5519 gn=1.02 lr=2.07e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 7900/12000 loss=4.5535 gn=1.03 lr=2.06e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 7920/12000 loss=4.3406 gn=0.96 lr=2.04e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 7940/12000 loss=4.4964 gn=0.94 lr=2.03e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 7960/12000 loss=4.5487 gn=0.94 lr=2.02e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 7980/12000 loss=4.4281 gn=0.92 lr=2.00e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8000/12000 loss=4.4898 gn=0.90 lr=1.99e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 8000: val_loss=4.5128 val_ppl=91.17 +[baseline s3] step 8020/12000 loss=4.5364 gn=0.97 lr=1.98e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s3] step 8040/12000 loss=4.5537 gn=0.96 lr=1.97e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 8060/12000 loss=4.5083 gn=0.95 lr=1.95e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 8080/12000 loss=4.4166 gn=1.00 lr=1.94e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 8100/12000 loss=4.3151 gn=0.95 lr=1.93e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 8120/12000 loss=4.7354 gn=1.02 lr=1.92e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 8140/12000 loss=4.4248 gn=0.96 lr=1.90e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 8160/12000 loss=4.4084 gn=0.92 lr=1.89e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 8180/12000 loss=4.5053 gn=1.05 lr=1.88e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 8200/12000 loss=4.4627 gn=0.92 lr=1.87e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 8220/12000 loss=4.4797 gn=1.02 lr=1.86e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 8240/12000 loss=4.6057 gn=0.99 lr=1.84e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 8260/12000 loss=4.4386 gn=0.91 lr=1.83e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 8280/12000 loss=4.3924 gn=0.93 lr=1.82e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 8300/12000 loss=4.4207 gn=0.97 lr=1.81e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 8320/12000 loss=4.3962 gn=1.00 lr=1.80e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 8340/12000 loss=4.4828 gn=1.00 lr=1.78e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 8360/12000 loss=4.3428 gn=1.00 lr=1.77e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 8380/12000 loss=4.4244 gn=0.96 lr=1.76e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 8400/12000 loss=4.4984 gn=1.01 lr=1.75e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 8420/12000 loss=4.4784 gn=1.01 lr=1.74e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 8440/12000 loss=4.4338 gn=0.95 lr=1.72e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 8460/12000 loss=4.5092 gn=1.01 lr=1.71e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 8480/12000 loss=4.4629 gn=1.10 lr=1.70e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 8500/12000 loss=4.4299 gn=1.04 lr=1.69e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 8520/12000 loss=4.4377 gn=1.02 lr=1.68e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 8540/12000 loss=4.4238 gn=0.91 lr=1.67e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 8560/12000 loss=4.3979 gn=1.06 lr=1.66e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 8580/12000 loss=4.5119 gn=1.06 lr=1.64e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 8600/12000 loss=4.3445 gn=0.97 lr=1.63e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 8620/12000 loss=4.4495 gn=0.99 lr=1.62e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 8640/12000 loss=4.4625 gn=0.95 lr=1.61e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 8660/12000 loss=4.4796 gn=0.96 lr=1.60e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8680/12000 loss=4.4901 gn=0.94 lr=1.59e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8700/12000 loss=4.4437 gn=0.96 lr=1.58e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8720/12000 loss=4.4620 gn=1.00 lr=1.57e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8740/12000 loss=4.3729 gn=0.98 lr=1.55e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8760/12000 loss=4.4548 gn=0.98 lr=1.54e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8780/12000 loss=4.2941 gn=0.96 lr=1.53e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 8800/12000 loss=4.4787 gn=1.03 lr=1.52e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8820/12000 loss=4.5140 gn=0.97 lr=1.51e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8840/12000 loss=4.4527 gn=0.98 lr=1.50e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8860/12000 loss=4.3625 gn=0.92 lr=1.49e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8880/12000 loss=4.5655 gn=1.00 lr=1.48e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s3] step 8900/12000 loss=4.4335 gn=0.96 lr=1.47e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 8920/12000 loss=4.4362 gn=1.04 lr=1.46e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 8940/12000 loss=4.2879 gn=1.00 lr=1.45e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 8960/12000 loss=4.4540 gn=0.95 lr=1.44e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 8980/12000 loss=4.4624 gn=1.12 lr=1.43e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] step 9000/12000 loss=4.5518 gn=0.98 lr=1.42e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 9000: val_loss=4.4801 val_ppl=88.25 +[baseline s3] step 9020/12000 loss=4.3560 gn=1.04 lr=1.41e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 9040/12000 loss=4.4644 gn=0.98 lr=1.40e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 9060/12000 loss=4.2369 gn=1.00 lr=1.39e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 9080/12000 loss=4.4199 gn=1.01 lr=1.38e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 9100/12000 loss=4.5150 gn=1.02 lr=1.37e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9120/12000 loss=4.4866 gn=0.93 lr=1.36e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9140/12000 loss=4.2669 gn=0.98 lr=1.35e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9160/12000 loss=4.5629 gn=1.20 lr=1.34e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9180/12000 loss=4.3470 gn=0.97 lr=1.33e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9200/12000 loss=4.4363 gn=0.98 lr=1.32e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9220/12000 loss=4.4081 gn=1.07 lr=1.31e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9240/12000 loss=4.4238 gn=1.09 lr=1.30e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9260/12000 loss=4.4348 gn=1.05 lr=1.29e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9280/12000 loss=4.4311 gn=1.00 lr=1.28e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9300/12000 loss=4.3185 gn=1.20 lr=1.27e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 9320/12000 loss=4.5431 gn=1.09 lr=1.26e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9340/12000 loss=4.5564 gn=1.08 lr=1.25e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9360/12000 loss=4.3772 gn=1.09 lr=1.24e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9380/12000 loss=4.4444 gn=1.01 lr=1.23e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9400/12000 loss=4.4187 gn=1.09 lr=1.22e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9420/12000 loss=4.4037 gn=1.10 lr=1.21e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9440/12000 loss=4.2833 gn=1.01 lr=1.20e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9460/12000 loss=4.5062 gn=1.06 lr=1.19e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 9480/12000 loss=4.5611 gn=1.16 lr=1.19e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9500/12000 loss=4.3066 gn=1.02 lr=1.18e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9520/12000 loss=4.4245 gn=1.07 lr=1.17e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9540/12000 loss=4.3790 gn=1.00 lr=1.16e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9560/12000 loss=4.3185 gn=0.95 lr=1.15e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9580/12000 loss=4.4025 gn=1.07 lr=1.14e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9600/12000 loss=4.6061 gn=1.07 lr=1.13e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9620/12000 loss=4.4327 gn=1.10 lr=1.12e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 9640/12000 loss=4.2685 gn=1.08 lr=1.12e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 9660/12000 loss=4.5086 gn=1.03 lr=1.11e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 9680/12000 loss=4.4223 gn=1.02 lr=1.10e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 9700/12000 loss=4.2798 gn=1.00 lr=1.09e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 9720/12000 loss=4.3475 gn=0.98 lr=1.08e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 9740/12000 loss=4.3361 gn=1.08 lr=1.07e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 9760/12000 loss=4.4092 gn=1.03 lr=1.07e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 9780/12000 loss=4.3316 gn=0.99 lr=1.06e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 9800/12000 loss=4.3471 gn=1.12 lr=1.05e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 9820/12000 loss=4.3214 gn=1.08 lr=1.04e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 9840/12000 loss=4.3413 gn=0.97 lr=1.03e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 9860/12000 loss=4.3340 gn=1.04 lr=1.03e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 9880/12000 loss=4.2665 gn=1.02 lr=1.02e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 9900/12000 loss=4.3551 gn=1.05 lr=1.01e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 9920/12000 loss=4.3569 gn=1.02 lr=1.00e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 9940/12000 loss=4.3230 gn=1.16 lr=9.96e-05 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 9960/12000 loss=4.5070 gn=1.22 lr=9.89e-05 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 9980/12000 loss=4.2698 gn=1.10 lr=9.81e-05 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] step 10000/12000 loss=4.4129 gn=1.05 lr=9.74e-05 tps=24577 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 10000: val_loss=4.4777 val_ppl=88.03 +[baseline s3] step 10020/12000 loss=4.2730 gn=1.15 lr=9.67e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 10040/12000 loss=4.3734 gn=1.07 lr=9.59e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 10060/12000 loss=4.2654 gn=1.02 lr=9.52e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 10080/12000 loss=4.2103 gn=0.97 lr=9.45e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10100/12000 loss=4.5193 gn=1.10 lr=9.38e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10120/12000 loss=4.4165 gn=1.03 lr=9.31e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10140/12000 loss=4.2914 gn=1.03 lr=9.24e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10160/12000 loss=4.4215 gn=1.07 lr=9.18e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10180/12000 loss=4.5175 gn=1.07 lr=9.11e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 10200/12000 loss=4.5061 gn=1.05 lr=9.04e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10220/12000 loss=4.4143 gn=1.11 lr=8.98e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10240/12000 loss=4.2719 gn=1.07 lr=8.91e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10260/12000 loss=4.4186 gn=1.15 lr=8.85e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10280/12000 loss=4.4341 gn=1.16 lr=8.78e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10300/12000 loss=4.3148 gn=1.11 lr=8.72e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10320/12000 loss=4.4542 gn=1.02 lr=8.66e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 10340/12000 loss=4.2675 gn=1.04 lr=8.59e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 10360/12000 loss=4.3070 gn=1.02 lr=8.53e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 10380/12000 loss=4.4473 gn=1.04 lr=8.47e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 10400/12000 loss=4.4064 gn=1.07 lr=8.41e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 10420/12000 loss=4.2504 gn=0.98 lr=8.35e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 10440/12000 loss=4.2585 gn=1.07 lr=8.30e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10460/12000 loss=4.4643 gn=1.29 lr=8.24e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10480/12000 loss=4.5828 gn=1.12 lr=8.18e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10500/12000 loss=4.4147 gn=1.19 lr=8.12e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10520/12000 loss=4.3162 gn=1.09 lr=8.07e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10540/12000 loss=4.2827 gn=1.05 lr=8.01e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 10560/12000 loss=4.2460 gn=1.11 lr=7.96e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10580/12000 loss=4.4442 gn=1.03 lr=7.91e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10600/12000 loss=4.2598 gn=1.03 lr=7.85e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10620/12000 loss=4.4145 gn=1.07 lr=7.80e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10640/12000 loss=4.2702 gn=1.09 lr=7.75e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10660/12000 loss=4.3425 gn=1.17 lr=7.70e-05 tps=24574 mem=5.4G lam=None Gm=None +[baseline s3] step 10680/12000 loss=4.4145 gn=1.18 lr=7.65e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10700/12000 loss=4.4238 gn=1.08 lr=7.60e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10720/12000 loss=4.3224 gn=1.01 lr=7.55e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10740/12000 loss=4.2670 gn=1.04 lr=7.50e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10760/12000 loss=4.3646 gn=1.09 lr=7.46e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10780/12000 loss=4.3552 gn=1.13 lr=7.41e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10800/12000 loss=4.3919 gn=1.10 lr=7.37e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10820/12000 loss=4.3324 gn=1.17 lr=7.32e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10840/12000 loss=4.4895 gn=1.20 lr=7.28e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10860/12000 loss=4.4334 gn=1.20 lr=7.23e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10880/12000 loss=4.4374 gn=1.06 lr=7.19e-05 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] step 10900/12000 loss=4.2947 gn=1.10 lr=7.15e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10920/12000 loss=4.1761 gn=1.09 lr=7.11e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10940/12000 loss=4.4421 gn=1.05 lr=7.07e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10960/12000 loss=4.3651 gn=1.14 lr=7.03e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 10980/12000 loss=4.3494 gn=1.28 lr=6.99e-05 tps=24575 mem=5.4G lam=None Gm=None +[baseline s3] step 11000/12000 loss=4.3403 gn=1.18 lr=6.95e-05 tps=24576 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 11000: val_loss=4.3897 val_ppl=80.61 +[baseline s3] step 11020/12000 loss=4.2964 gn=1.07 lr=6.91e-05 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 11040/12000 loss=4.3059 gn=1.04 lr=6.88e-05 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 11060/12000 loss=4.4136 gn=1.07 lr=6.84e-05 tps=24568 mem=5.4G lam=None Gm=None +[baseline s3] step 11080/12000 loss=4.1593 gn=1.13 lr=6.81e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11100/12000 loss=4.3929 gn=1.31 lr=6.77e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11120/12000 loss=4.3232 gn=1.01 lr=6.74e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11140/12000 loss=4.3487 gn=1.10 lr=6.70e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11160/12000 loss=4.3645 gn=1.02 lr=6.67e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11180/12000 loss=4.3178 gn=1.13 lr=6.64e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11200/12000 loss=4.2417 gn=1.11 lr=6.61e-05 tps=24569 mem=5.4G lam=None Gm=None +[baseline s3] step 11220/12000 loss=4.2896 gn=1.12 lr=6.58e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11240/12000 loss=4.3467 gn=1.10 lr=6.55e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11260/12000 loss=4.3634 gn=1.26 lr=6.52e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11280/12000 loss=4.2993 gn=1.09 lr=6.49e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11300/12000 loss=4.2978 gn=1.13 lr=6.47e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11320/12000 loss=4.3196 gn=1.13 lr=6.44e-05 tps=24570 mem=5.4G lam=None Gm=None +[baseline s3] step 11340/12000 loss=4.3945 gn=1.20 lr=6.42e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11360/12000 loss=4.4946 gn=1.13 lr=6.39e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11380/12000 loss=4.3189 gn=1.15 lr=6.37e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11400/12000 loss=4.4036 gn=1.13 lr=6.34e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11420/12000 loss=4.3125 gn=1.06 lr=6.32e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11440/12000 loss=4.3456 gn=1.06 lr=6.30e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11460/12000 loss=4.3805 gn=1.16 lr=6.28e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11480/12000 loss=4.3730 gn=1.10 lr=6.26e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11500/12000 loss=4.2762 gn=1.16 lr=6.24e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11520/12000 loss=4.1864 gn=1.21 lr=6.22e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11540/12000 loss=4.4457 gn=1.10 lr=6.20e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11560/12000 loss=4.3353 gn=1.17 lr=6.19e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11580/12000 loss=4.2062 gn=1.08 lr=6.17e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11600/12000 loss=4.3415 gn=1.05 lr=6.15e-05 tps=24571 mem=5.4G lam=None Gm=None +[baseline s3] step 11620/12000 loss=4.2916 gn=1.22 lr=6.14e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11640/12000 loss=4.3572 gn=1.06 lr=6.12e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11660/12000 loss=4.4375 gn=1.25 lr=6.11e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11680/12000 loss=4.2270 gn=1.15 lr=6.10e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11700/12000 loss=4.3745 gn=1.05 lr=6.09e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11720/12000 loss=4.2871 gn=1.18 lr=6.07e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11740/12000 loss=4.3310 gn=1.22 lr=6.06e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11760/12000 loss=4.3345 gn=1.10 lr=6.06e-05 tps=24572 mem=5.4G lam=None Gm=None +[baseline s3] step 11780/12000 loss=4.3628 gn=1.08 lr=6.05e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11800/12000 loss=4.3317 gn=1.09 lr=6.04e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11820/12000 loss=4.3704 gn=1.19 lr=6.03e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11840/12000 loss=4.2606 gn=1.25 lr=6.02e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11860/12000 loss=4.2839 gn=1.08 lr=6.02e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11880/12000 loss=4.4770 gn=1.19 lr=6.01e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11900/12000 loss=4.2925 gn=1.20 lr=6.01e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11920/12000 loss=4.2869 gn=1.15 lr=6.01e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11940/12000 loss=4.4190 gn=1.34 lr=6.00e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11960/12000 loss=4.2699 gn=1.10 lr=6.00e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] step 11980/12000 loss=4.3054 gn=1.17 lr=6.00e-05 tps=24573 mem=5.4G lam=None Gm=None +[baseline s3] EVAL step 11999: val_loss=4.3744 val_ppl=79.39 +[init] variant=s_only seed=3 params=63.97M +[init] use_S=True use_G=False +[s_only s3] step 0/12000 loss=10.8947 gn=14.87 lr=3.00e-06 tps=8439 mem=5.0G lam=0.12692801654338837 Gm=None +[s_only s3] step 20/12000 loss=9.8787 gn=1.97 lr=6.30e-05 tps=19656 mem=5.5G lam=0.12694220369060835 Gm=None +[s_only s3] step 40/12000 loss=9.1230 gn=1.77 lr=1.23e-04 tps=20283 mem=5.5G lam=0.126886830975612 Gm=None +[s_only s3] step 60/12000 loss=8.1696 gn=1.58 lr=1.83e-04 tps=20614 mem=5.5G lam=0.12679918544987837 Gm=None +[s_only s3] step 80/12000 loss=7.5320 gn=0.94 lr=2.43e-04 tps=20750 mem=5.5G lam=0.12672995527585348 Gm=None +[s_only s3] step 100/12000 loss=7.1228 gn=0.64 lr=3.03e-04 tps=20829 mem=5.5G lam=0.1266972292214632 Gm=None +[s_only s3] step 120/12000 loss=7.1949 gn=0.68 lr=3.63e-04 tps=20897 mem=5.5G lam=0.12658714999755225 Gm=None +[s_only s3] step 140/12000 loss=6.9264 gn=0.77 lr=4.23e-04 tps=20937 mem=5.5G lam=0.12656180560588837 Gm=None +[s_only s3] step 160/12000 loss=6.8878 gn=1.04 lr=4.83e-04 tps=20966 mem=5.5G lam=0.12656703094641367 Gm=None +[s_only s3] step 180/12000 loss=6.7629 gn=0.78 lr=5.43e-04 tps=20986 mem=5.5G lam=0.12656066628793874 Gm=None +[s_only s3] step 200/12000 loss=6.6858 gn=0.64 lr=6.00e-04 tps=21014 mem=5.5G lam=0.12658241080741087 Gm=None +[s_only s3] step 220/12000 loss=6.5929 gn=0.55 lr=6.00e-04 tps=21023 mem=5.5G lam=0.1266044502456983 Gm=None +[s_only s3] step 240/12000 loss=6.6295 gn=0.52 lr=6.00e-04 tps=21036 mem=5.5G lam=0.1266349603732427 Gm=None +[s_only s3] step 260/12000 loss=6.6218 gn=0.48 lr=6.00e-04 tps=21043 mem=5.5G lam=0.12663273823757967 Gm=None +[s_only s3] step 280/12000 loss=6.4487 gn=0.62 lr=6.00e-04 tps=21055 mem=5.5G lam=0.1266415569310387 Gm=None +[s_only s3] step 300/12000 loss=6.3327 gn=0.51 lr=6.00e-04 tps=21053 mem=5.5G lam=0.12665006828804812 Gm=None +[s_only s3] step 320/12000 loss=6.4291 gn=0.56 lr=6.00e-04 tps=21047 mem=5.5G lam=0.12663793377578259 Gm=None +[s_only s3] step 340/12000 loss=6.4897 gn=0.61 lr=6.00e-04 tps=21024 mem=5.5G lam=0.1266579208895564 Gm=None +[s_only s3] step 360/12000 loss=6.2732 gn=0.56 lr=6.00e-04 tps=21002 mem=5.5G lam=0.1267132411400477 Gm=None +[s_only s3] step 380/12000 loss=6.3135 gn=0.57 lr=6.00e-04 tps=20973 mem=5.5G lam=0.12670761936654648 Gm=None +[s_only s3] step 400/12000 loss=6.3679 gn=0.59 lr=6.00e-04 tps=20953 mem=5.5G lam=0.12676518689841032 Gm=None +[s_only s3] step 420/12000 loss=6.2888 gn=0.47 lr=6.00e-04 tps=20937 mem=5.5G lam=0.1267999398211638 Gm=None +[s_only s3] step 440/12000 loss=6.2291 gn=0.56 lr=5.99e-04 tps=20876 mem=5.5G lam=0.12688300355027118 Gm=None +[s_only s3] step 460/12000 loss=6.3626 gn=0.55 lr=5.99e-04 tps=20840 mem=5.5G lam=0.12695795483887196 Gm=None +[s_only s3] step 480/12000 loss=6.2115 gn=0.51 lr=5.99e-04 tps=20813 mem=5.5G lam=0.12707082461565733 Gm=None +[s_only s3] step 500/12000 loss=6.2465 gn=0.52 lr=5.99e-04 tps=20792 mem=5.5G lam=0.12718711979687214 Gm=None +[s_only s3] step 520/12000 loss=6.2745 gn=0.58 lr=5.99e-04 tps=20763 mem=5.5G lam=0.12737652597328028 Gm=None +[s_only s3] step 540/12000 loss=6.1698 gn=0.52 lr=5.99e-04 tps=20745 mem=5.5G lam=0.12749819147090116 Gm=None +[s_only s3] step 560/12000 loss=6.2090 gn=0.57 lr=5.99e-04 tps=20719 mem=5.5G lam=0.12764237727969885 Gm=None +[s_only s3] step 580/12000 loss=6.2011 gn=0.55 lr=5.99e-04 tps=20709 mem=5.5G lam=0.127825283134977 Gm=None +[s_only s3] step 600/12000 loss=6.1010 gn=0.48 lr=5.98e-04 tps=20704 mem=5.5G lam=0.12795953080058098 Gm=None +[s_only s3] step 620/12000 loss=6.0148 gn=0.52 lr=5.98e-04 tps=20707 mem=5.5G lam=0.12812368975331387 Gm=None +[s_only s3] step 640/12000 loss=6.1043 gn=0.54 lr=5.98e-04 tps=20717 mem=5.5G lam=0.12832175133128962 Gm=None +[s_only s3] step 660/12000 loss=5.9025 gn=0.54 lr=5.98e-04 tps=20734 mem=5.5G lam=0.12851136705527702 Gm=None +[s_only s3] step 680/12000 loss=6.0172 gn=0.50 lr=5.98e-04 tps=20746 mem=5.5G lam=0.12868034715453783 Gm=None +[s_only s3] step 700/12000 loss=6.0751 gn=0.56 lr=5.98e-04 tps=20760 mem=5.5G lam=0.12891405262053013 Gm=None +[s_only s3] step 720/12000 loss=5.8826 gn=0.54 lr=5.97e-04 tps=20770 mem=5.5G lam=0.12920820309470096 Gm=None +[s_only s3] step 740/12000 loss=6.0464 gn=0.55 lr=5.97e-04 tps=20773 mem=5.5G lam=0.12944251050551733 Gm=None +[s_only s3] step 760/12000 loss=5.9405 gn=0.53 lr=5.97e-04 tps=20780 mem=5.5G lam=0.12973964431633553 Gm=None +[s_only s3] step 780/12000 loss=6.0210 gn=0.59 lr=5.97e-04 tps=20791 mem=5.5G lam=0.13004410670449337 Gm=None +[s_only s3] step 800/12000 loss=5.9413 gn=0.51 lr=5.97e-04 tps=20800 mem=5.5G lam=0.13040621299296618 Gm=None +[s_only s3] step 820/12000 loss=5.8643 gn=0.54 lr=5.96e-04 tps=20805 mem=5.5G lam=0.1307692745079597 Gm=None +[s_only s3] step 840/12000 loss=5.8955 gn=0.58 lr=5.96e-04 tps=20816 mem=5.5G lam=0.13110253680497408 Gm=None +[s_only s3] step 860/12000 loss=5.8447 gn=0.51 lr=5.96e-04 tps=20824 mem=5.5G lam=0.13148421545823416 Gm=None +[s_only s3] step 880/12000 loss=5.9974 gn=0.63 lr=5.96e-04 tps=20833 mem=5.5G lam=0.13183280701438585 Gm=None +[s_only s3] step 900/12000 loss=5.7843 gn=0.57 lr=5.95e-04 tps=20839 mem=5.5G lam=0.13218222682674727 Gm=None +[s_only s3] step 920/12000 loss=5.8090 gn=0.52 lr=5.95e-04 tps=20846 mem=5.5G lam=0.13257871506114802 Gm=None +[s_only s3] step 940/12000 loss=5.6490 gn=0.53 lr=5.95e-04 tps=20852 mem=5.5G lam=0.1329437584305803 Gm=None +[s_only s3] step 960/12000 loss=5.8092 gn=0.52 lr=5.94e-04 tps=20859 mem=5.5G lam=0.13336021018524966 Gm=None +[s_only s3] step 980/12000 loss=5.7391 gn=0.52 lr=5.94e-04 tps=20866 mem=5.5G lam=0.133741723994414 Gm=None +[s_only s3] step 1000/12000 loss=5.9076 gn=0.58 lr=5.94e-04 tps=20873 mem=5.5G lam=0.13417456454286972 Gm=None +[s_only s3] EVAL step 1000: val_loss=5.8518 val_ppl=347.86 +[s_only s3] step 1020/12000 loss=5.7333 gn=0.52 lr=5.94e-04 tps=20805 mem=5.5G lam=0.13449585065245628 Gm=None +[s_only s3] step 1040/12000 loss=5.6524 gn=0.55 lr=5.93e-04 tps=20812 mem=5.5G lam=0.13490383668492237 Gm=None +[s_only s3] step 1060/12000 loss=5.6541 gn=0.61 lr=5.93e-04 tps=20816 mem=5.5G lam=0.13530297266940275 Gm=None +[s_only s3] step 1080/12000 loss=5.7044 gn=0.52 lr=5.93e-04 tps=20825 mem=5.5G lam=0.1357328516120712 Gm=None +[s_only s3] step 1100/12000 loss=5.6635 gn=0.56 lr=5.92e-04 tps=20829 mem=5.5G lam=0.13609287949899831 Gm=None +[s_only s3] step 1120/12000 loss=5.7738 gn=0.63 lr=5.92e-04 tps=20837 mem=5.5G lam=0.13648841002335152 Gm=None +[s_only s3] step 1140/12000 loss=5.6606 gn=0.57 lr=5.92e-04 tps=20839 mem=5.5G lam=0.1368508137141665 Gm=None +[s_only s3] step 1160/12000 loss=5.7048 gn=0.58 lr=5.91e-04 tps=20845 mem=5.5G lam=0.1373081263154745 Gm=None +[s_only s3] step 1180/12000 loss=5.6299 gn=0.54 lr=5.91e-04 tps=20849 mem=5.5G lam=0.13778687144319215 Gm=None +[s_only s3] step 1200/12000 loss=5.7025 gn=0.62 lr=5.90e-04 tps=20855 mem=5.5G lam=0.13828484155237675 Gm=None +[s_only s3] step 1220/12000 loss=5.5112 gn=0.55 lr=5.90e-04 tps=20861 mem=5.5G lam=0.13870205885420242 Gm=None +[s_only s3] step 1240/12000 loss=5.5611 gn=0.56 lr=5.90e-04 tps=20866 mem=5.5G lam=0.1391298739860455 Gm=None +[s_only s3] step 1260/12000 loss=5.7347 gn=0.56 lr=5.89e-04 tps=20868 mem=5.5G lam=0.13961359423895678 Gm=None +[s_only s3] step 1280/12000 loss=5.5690 gn=0.58 lr=5.89e-04 tps=20871 mem=5.5G lam=0.1400772612541914 Gm=None +[s_only s3] step 1300/12000 loss=5.6611 gn=0.53 lr=5.89e-04 tps=20874 mem=5.5G lam=0.14053657744079828 Gm=None +[s_only s3] step 1320/12000 loss=5.5688 gn=0.58 lr=5.88e-04 tps=20879 mem=5.5G lam=0.14098914340138435 Gm=None +[s_only s3] step 1340/12000 loss=5.6360 gn=0.58 lr=5.88e-04 tps=20882 mem=5.5G lam=0.14152920028815666 Gm=None +[s_only s3] step 1360/12000 loss=5.4790 gn=0.60 lr=5.87e-04 tps=20886 mem=5.5G lam=0.14196714044859013 Gm=None +[s_only s3] step 1380/12000 loss=5.4218 gn=0.55 lr=5.87e-04 tps=20891 mem=5.5G lam=0.14235272475828728 Gm=None +[s_only s3] step 1400/12000 loss=5.6239 gn=0.63 lr=5.86e-04 tps=20895 mem=5.5G lam=0.1428036680445075 Gm=None +[s_only s3] step 1420/12000 loss=5.4842 gn=0.62 lr=5.86e-04 tps=20898 mem=5.5G lam=0.14324054711808762 Gm=None +[s_only s3] step 1440/12000 loss=5.3740 gn=0.57 lr=5.85e-04 tps=20902 mem=5.5G lam=0.14371431898325682 Gm=None +[s_only s3] step 1460/12000 loss=5.5496 gn=0.61 lr=5.85e-04 tps=20906 mem=5.5G lam=0.14421408313016096 Gm=None +[s_only s3] step 1480/12000 loss=5.3994 gn=0.60 lr=5.84e-04 tps=20909 mem=5.5G lam=0.14469032486279806 Gm=None +[s_only s3] step 1500/12000 loss=5.5122 gn=0.58 lr=5.84e-04 tps=20913 mem=5.5G lam=0.14517768317212662 Gm=None +[s_only s3] step 1520/12000 loss=5.5056 gn=0.57 lr=5.83e-04 tps=20914 mem=5.5G lam=0.14564614246288934 Gm=None +[s_only s3] step 1540/12000 loss=5.3640 gn=0.59 lr=5.83e-04 tps=20916 mem=5.5G lam=0.1461265878751874 Gm=None +[s_only s3] step 1560/12000 loss=5.4231 gn=0.56 lr=5.82e-04 tps=20918 mem=5.5G lam=0.14658704648415247 Gm=None +[s_only s3] step 1580/12000 loss=5.4919 gn=0.57 lr=5.82e-04 tps=20923 mem=5.5G lam=0.14701714956512055 Gm=None +[s_only s3] step 1600/12000 loss=5.2736 gn=0.58 lr=5.81e-04 tps=20926 mem=5.5G lam=0.14750135131180286 Gm=None +[s_only s3] step 1620/12000 loss=5.4942 gn=0.59 lr=5.81e-04 tps=20930 mem=5.5G lam=0.1479597290356954 Gm=None +[s_only s3] step 1640/12000 loss=5.3593 gn=0.59 lr=5.80e-04 tps=20934 mem=5.5G lam=0.14843208498011032 Gm=None +[s_only s3] step 1660/12000 loss=5.3763 gn=0.61 lr=5.80e-04 tps=20938 mem=5.5G lam=0.14888587904473147 Gm=None +[s_only s3] step 1680/12000 loss=5.3820 gn=0.56 lr=5.79e-04 tps=20941 mem=5.5G lam=0.14940758794546127 Gm=None +[s_only s3] step 1700/12000 loss=5.5410 gn=0.60 lr=5.79e-04 tps=20945 mem=5.5G lam=0.14982634441306195 Gm=None +[s_only s3] step 1720/12000 loss=5.3974 gn=0.60 lr=5.78e-04 tps=20946 mem=5.5G lam=0.1503677119811376 Gm=None +[s_only s3] step 1740/12000 loss=5.3413 gn=0.65 lr=5.78e-04 tps=20949 mem=5.5G lam=0.1508328371370832 Gm=None +[s_only s3] step 1760/12000 loss=5.2739 gn=0.58 lr=5.77e-04 tps=20951 mem=5.5G lam=0.1513590533286333 Gm=None +[s_only s3] step 1780/12000 loss=5.4289 gn=0.62 lr=5.76e-04 tps=20953 mem=5.5G lam=0.15195063843081394 Gm=None +[s_only s3] step 1800/12000 loss=5.3411 gn=0.60 lr=5.76e-04 tps=20955 mem=5.5G lam=0.1524457804237803 Gm=None +[s_only s3] step 1820/12000 loss=5.2930 gn=0.61 lr=5.75e-04 tps=20958 mem=5.5G lam=0.15293488713602224 Gm=None +[s_only s3] step 1840/12000 loss=5.4547 gn=0.63 lr=5.75e-04 tps=20960 mem=5.5G lam=0.15345919442673525 Gm=None +[s_only s3] step 1860/12000 loss=5.4131 gn=0.63 lr=5.74e-04 tps=20963 mem=5.5G lam=0.15391629996399084 Gm=None +[s_only s3] step 1880/12000 loss=5.3073 gn=0.60 lr=5.73e-04 tps=20965 mem=5.5G lam=0.15444200237592062 Gm=None +[s_only s3] step 1900/12000 loss=5.2185 gn=0.60 lr=5.73e-04 tps=20967 mem=5.5G lam=0.15496338810771704 Gm=None +[s_only s3] step 1920/12000 loss=5.3068 gn=0.57 lr=5.72e-04 tps=20966 mem=5.5G lam=0.15550452874352536 Gm=None +[s_only s3] step 1940/12000 loss=5.4119 gn=0.60 lr=5.72e-04 tps=20969 mem=5.5G lam=0.15604656096547842 Gm=None +[s_only s3] step 1960/12000 loss=5.3566 gn=0.58 lr=5.71e-04 tps=20970 mem=5.5G lam=0.15654522522042194 Gm=None +[s_only s3] step 1980/12000 loss=5.1046 gn=0.56 lr=5.70e-04 tps=20973 mem=5.5G lam=0.1571055787305037 Gm=None +[s_only s3] step 2000/12000 loss=5.3473 gn=0.56 lr=5.70e-04 tps=20975 mem=5.5G lam=0.157640913190941 Gm=None +[s_only s3] EVAL step 2000: val_loss=5.3642 val_ppl=213.63 +[s_only s3] step 2020/12000 loss=5.3967 gn=0.61 lr=5.69e-04 tps=20938 mem=5.5G lam=0.15812867445250353 Gm=None +[s_only s3] step 2040/12000 loss=5.4042 gn=0.60 lr=5.68e-04 tps=20941 mem=5.5G lam=0.15867000973472992 Gm=None +[s_only s3] step 2060/12000 loss=5.3086 gn=0.63 lr=5.68e-04 tps=20944 mem=5.5G lam=0.1591886511693398 Gm=None +[s_only s3] step 2080/12000 loss=5.2221 gn=0.58 lr=5.67e-04 tps=20946 mem=5.5G lam=0.15965720762809119 Gm=None +[s_only s3] step 2100/12000 loss=5.2930 gn=0.58 lr=5.66e-04 tps=20948 mem=5.5G lam=0.16022760265817246 Gm=None +[s_only s3] step 2120/12000 loss=5.3471 gn=0.62 lr=5.65e-04 tps=20951 mem=5.5G lam=0.16081504120181003 Gm=None +[s_only s3] step 2140/12000 loss=5.1443 gn=0.60 lr=5.65e-04 tps=20951 mem=5.5G lam=0.16131479510416588 Gm=None +[s_only s3] step 2160/12000 loss=5.1243 gn=0.62 lr=5.64e-04 tps=20953 mem=5.5G lam=0.1618762823442618 Gm=None +[s_only s3] step 2180/12000 loss=5.1559 gn=0.63 lr=5.63e-04 tps=20955 mem=5.5G lam=0.16239843952159086 Gm=None +[s_only s3] step 2200/12000 loss=5.3548 gn=0.62 lr=5.63e-04 tps=20958 mem=5.5G lam=0.16292800102382898 Gm=None +[s_only s3] step 2220/12000 loss=5.2824 gn=0.63 lr=5.62e-04 tps=20960 mem=5.5G lam=0.1634970015535752 Gm=None +[s_only s3] step 2240/12000 loss=5.2789 gn=0.70 lr=5.61e-04 tps=20963 mem=5.5G lam=0.1640439551944534 Gm=None +[s_only s3] step 2260/12000 loss=5.1492 gn=0.66 lr=5.60e-04 tps=20964 mem=5.5G lam=0.16461799448976913 Gm=None +[s_only s3] step 2280/12000 loss=5.2255 gn=1.05 lr=5.60e-04 tps=20967 mem=5.5G lam=0.1651757045959433 Gm=None +[s_only s3] step 2300/12000 loss=5.2786 gn=0.64 lr=5.59e-04 tps=20969 mem=5.5G lam=0.16579562425613403 Gm=None +[s_only s3] step 2320/12000 loss=5.2192 gn=0.68 lr=5.58e-04 tps=20971 mem=5.5G lam=0.16640225580583015 Gm=None +[s_only s3] step 2340/12000 loss=5.0022 gn=0.66 lr=5.57e-04 tps=20971 mem=5.5G lam=0.1668911144758264 Gm=None +[s_only s3] step 2360/12000 loss=5.2226 gn=0.61 lr=5.57e-04 tps=20973 mem=5.5G lam=0.16745880153030157 Gm=None +[s_only s3] step 2380/12000 loss=5.1478 gn=0.69 lr=5.56e-04 tps=20975 mem=5.5G lam=0.1680245685080687 Gm=None +[s_only s3] step 2400/12000 loss=5.1173 gn=0.63 lr=5.55e-04 tps=20977 mem=5.5G lam=0.16856745351105928 Gm=None +[s_only s3] step 2420/12000 loss=5.1471 gn=0.72 lr=5.54e-04 tps=20979 mem=5.5G lam=0.16916084848344326 Gm=None +[s_only s3] step 2440/12000 loss=5.1603 gn=0.61 lr=5.53e-04 tps=20980 mem=5.5G lam=0.16971150071670613 Gm=None +[s_only s3] step 2460/12000 loss=5.1129 gn=0.62 lr=5.53e-04 tps=20981 mem=5.5G lam=0.1702629942446947 Gm=None +[s_only s3] step 2480/12000 loss=5.1421 gn=0.65 lr=5.52e-04 tps=20982 mem=5.5G lam=0.17080736408631006 Gm=None +[s_only s3] step 2500/12000 loss=5.1330 gn=0.62 lr=5.51e-04 tps=20985 mem=5.5G lam=0.17129923589527607 Gm=None +[s_only s3] step 2520/12000 loss=5.2338 gn=0.59 lr=5.50e-04 tps=20984 mem=5.5G lam=0.17186312874158224 Gm=None +[s_only s3] step 2540/12000 loss=5.0934 gn=0.65 lr=5.49e-04 tps=20987 mem=5.5G lam=0.17238754530747732 Gm=None +[s_only s3] step 2560/12000 loss=5.2068 gn=0.71 lr=5.48e-04 tps=20987 mem=5.5G lam=0.17288817061732212 Gm=None +[s_only s3] step 2580/12000 loss=5.0538 gn=0.65 lr=5.48e-04 tps=20987 mem=5.5G lam=0.17347303529580435 Gm=None +[s_only s3] step 2600/12000 loss=5.1140 gn=0.62 lr=5.47e-04 tps=20988 mem=5.5G lam=0.17405547574162483 Gm=None +[s_only s3] step 2620/12000 loss=5.0385 gn=0.63 lr=5.46e-04 tps=20989 mem=5.5G lam=0.174584886059165 Gm=None +[s_only s3] step 2640/12000 loss=5.0004 gn=0.75 lr=5.45e-04 tps=20991 mem=5.5G lam=0.17514473665505648 Gm=None +[s_only s3] step 2660/12000 loss=5.0964 gn=0.64 lr=5.44e-04 tps=20992 mem=5.5G lam=0.17571115183333555 Gm=None +[s_only s3] step 2680/12000 loss=5.0865 gn=0.65 lr=5.43e-04 tps=20993 mem=5.5G lam=0.17630939868589243 Gm=None +[s_only s3] step 2700/12000 loss=5.0500 gn=0.64 lr=5.42e-04 tps=20993 mem=5.5G lam=0.17686768559118113 Gm=None +[s_only s3] step 2720/12000 loss=5.1562 gn=0.78 lr=5.41e-04 tps=20995 mem=5.5G lam=0.17745353250453869 Gm=None +[s_only s3] step 2740/12000 loss=5.1139 gn=0.67 lr=5.41e-04 tps=20996 mem=5.5G lam=0.17796417605131865 Gm=None +[s_only s3] step 2760/12000 loss=5.1309 gn=0.71 lr=5.40e-04 tps=20997 mem=5.5G lam=0.1784825442979733 Gm=None +[s_only s3] step 2780/12000 loss=5.0624 gn=0.68 lr=5.39e-04 tps=20998 mem=5.5G lam=0.17888430102417865 Gm=None +[s_only s3] step 2800/12000 loss=5.0144 gn=0.71 lr=5.38e-04 tps=20999 mem=5.5G lam=0.17934417507300773 Gm=None +[s_only s3] step 2820/12000 loss=5.1634 gn=0.76 lr=5.37e-04 tps=21002 mem=5.5G lam=0.1799729916577538 Gm=None +[s_only s3] step 2840/12000 loss=5.1202 gn=0.67 lr=5.36e-04 tps=21003 mem=5.5G lam=0.18065636170407137 Gm=None +[s_only s3] step 2860/12000 loss=4.9527 gn=0.72 lr=5.35e-04 tps=21004 mem=5.5G lam=0.18117472529411316 Gm=None +[s_only s3] step 2880/12000 loss=5.0321 gn=0.68 lr=5.34e-04 tps=21006 mem=5.5G lam=0.18174382330228886 Gm=None +[s_only s3] step 2900/12000 loss=4.8904 gn=0.66 lr=5.33e-04 tps=21007 mem=5.5G lam=0.1822758965815107 Gm=None +[s_only s3] step 2920/12000 loss=4.9885 gn=0.67 lr=5.32e-04 tps=21008 mem=5.5G lam=0.1827599204455813 Gm=None +[s_only s3] step 2940/12000 loss=5.0275 gn=0.73 lr=5.31e-04 tps=21010 mem=5.5G lam=0.18328631669282913 Gm=None +[s_only s3] step 2960/12000 loss=4.8790 gn=0.69 lr=5.30e-04 tps=21011 mem=5.5G lam=0.18391878250986338 Gm=None +[s_only s3] step 2980/12000 loss=4.7845 gn=0.89 lr=5.29e-04 tps=21012 mem=5.5G lam=0.18440715440859398 Gm=None +[s_only s3] step 3000/12000 loss=5.0700 gn=0.74 lr=5.28e-04 tps=21013 mem=5.5G lam=0.18498189436892667 Gm=None +[s_only s3] EVAL step 3000: val_loss=5.0432 val_ppl=154.97 +[s_only s3] step 3020/12000 loss=5.1426 gn=0.75 lr=5.27e-04 tps=20990 mem=5.5G lam=0.1855556114266316 Gm=None +[s_only s3] step 3040/12000 loss=4.8466 gn=0.67 lr=5.26e-04 tps=20990 mem=5.5G lam=0.18607309367507696 Gm=None +[s_only s3] step 3060/12000 loss=4.9375 gn=0.69 lr=5.25e-04 tps=20990 mem=5.5G lam=0.18656674989809593 Gm=None +[s_only s3] step 3080/12000 loss=4.8801 gn=0.71 lr=5.24e-04 tps=20992 mem=5.5G lam=0.18712459163119396 Gm=None +[s_only s3] step 3100/12000 loss=5.0334 gn=0.79 lr=5.23e-04 tps=20993 mem=5.5G lam=0.18768051204582056 Gm=None +[s_only s3] step 3120/12000 loss=4.7990 gn=0.74 lr=5.22e-04 tps=20994 mem=5.5G lam=0.18817588003973165 Gm=None +[s_only s3] step 3140/12000 loss=4.9271 gn=0.68 lr=5.21e-04 tps=20995 mem=5.5G lam=0.1887569141884645 Gm=None +[s_only s3] step 3160/12000 loss=4.8710 gn=0.75 lr=5.20e-04 tps=20997 mem=5.5G lam=0.18937193602323532 Gm=None +[s_only s3] step 3180/12000 loss=4.9012 gn=0.72 lr=5.19e-04 tps=20998 mem=5.5G lam=0.1899965868021051 Gm=None +[s_only s3] step 3200/12000 loss=4.8792 gn=0.71 lr=5.18e-04 tps=20999 mem=5.5G lam=0.19056051845351854 Gm=None +[s_only s3] step 3220/12000 loss=4.9825 gn=0.83 lr=5.17e-04 tps=21000 mem=5.5G lam=0.19111564382910728 Gm=None +[s_only s3] step 3240/12000 loss=4.9550 gn=0.69 lr=5.16e-04 tps=21001 mem=5.5G lam=0.1916476171463728 Gm=None +[s_only s3] step 3260/12000 loss=5.0246 gn=0.72 lr=5.15e-04 tps=21001 mem=5.5G lam=0.19216732742885748 Gm=None +[s_only s3] step 3280/12000 loss=4.8351 gn=0.68 lr=5.14e-04 tps=21002 mem=5.5G lam=0.19270867357651392 Gm=None +[s_only s3] step 3300/12000 loss=5.0925 gn=0.70 lr=5.13e-04 tps=21004 mem=5.5G lam=0.193344515748322 Gm=None +[s_only s3] step 3320/12000 loss=4.9303 gn=0.69 lr=5.12e-04 tps=21004 mem=5.5G lam=0.19388321538766226 Gm=None +[s_only s3] step 3340/12000 loss=4.8977 gn=0.68 lr=5.11e-04 tps=21005 mem=5.5G lam=0.19441270641982555 Gm=None +[s_only s3] step 3360/12000 loss=4.9486 gn=0.90 lr=5.10e-04 tps=21007 mem=5.5G lam=0.19506384525448084 Gm=None +[s_only s3] step 3380/12000 loss=5.0753 gn=0.75 lr=5.09e-04 tps=21007 mem=5.5G lam=0.19559775634358326 Gm=None +[s_only s3] step 3400/12000 loss=5.0080 gn=0.86 lr=5.08e-04 tps=21006 mem=5.5G lam=0.19604836435367665 Gm=None +[s_only s3] step 3420/12000 loss=4.9251 gn=0.72 lr=5.07e-04 tps=21006 mem=5.5G lam=0.19656820936749378 Gm=None +[s_only s3] step 3440/12000 loss=4.7397 gn=0.68 lr=5.06e-04 tps=21008 mem=5.5G lam=0.19712105486541986 Gm=None +[s_only s3] step 3460/12000 loss=4.9556 gn=0.74 lr=5.05e-04 tps=21009 mem=5.5G lam=0.19768475896368423 Gm=None +[s_only s3] step 3480/12000 loss=4.9662 gn=0.75 lr=5.03e-04 tps=21010 mem=5.5G lam=0.19824143840620914 Gm=None +[s_only s3] step 3500/12000 loss=5.0849 gn=0.74 lr=5.02e-04 tps=21011 mem=5.5G lam=0.19879716355353594 Gm=None +[s_only s3] step 3520/12000 loss=4.8439 gn=0.78 lr=5.01e-04 tps=21013 mem=5.5G lam=0.1993343730767568 Gm=None +[s_only s3] step 3540/12000 loss=4.8639 gn=0.74 lr=5.00e-04 tps=21014 mem=5.5G lam=0.19994602942218384 Gm=None +[s_only s3] step 3560/12000 loss=4.9125 gn=0.76 lr=4.99e-04 tps=21016 mem=5.5G lam=0.2004567307109634 Gm=None +[s_only s3] step 3580/12000 loss=4.9264 gn=0.71 lr=4.98e-04 tps=21017 mem=5.5G lam=0.2010104317838947 Gm=None +[s_only s3] step 3600/12000 loss=4.9223 gn=0.75 lr=4.97e-04 tps=21018 mem=5.5G lam=0.20156620411823192 Gm=None +[s_only s3] step 3620/12000 loss=4.9067 gn=0.74 lr=4.96e-04 tps=21020 mem=5.5G lam=0.20216343582918248 Gm=None +[s_only s3] step 3640/12000 loss=4.8079 gn=0.69 lr=4.94e-04 tps=21021 mem=5.5G lam=0.20265825962026915 Gm=None +[s_only s3] step 3660/12000 loss=5.0376 gn=0.78 lr=4.93e-04 tps=21015 mem=5.5G lam=0.20320954297979674 Gm=None +[s_only s3] step 3680/12000 loss=4.8581 gn=0.76 lr=4.92e-04 tps=21016 mem=5.5G lam=0.20378255347410837 Gm=None +[s_only s3] step 3700/12000 loss=4.9982 gn=0.70 lr=4.91e-04 tps=21017 mem=5.5G lam=0.2043424757818381 Gm=None +[s_only s3] step 3720/12000 loss=4.9720 gn=0.84 lr=4.90e-04 tps=21017 mem=5.5G lam=0.2048528160278996 Gm=None +[s_only s3] step 3740/12000 loss=4.8881 gn=0.84 lr=4.89e-04 tps=21019 mem=5.5G lam=0.2053167400881648 Gm=None +[s_only s3] step 3760/12000 loss=4.8170 gn=0.77 lr=4.88e-04 tps=21019 mem=5.5G lam=0.2059400804961721 Gm=None +[s_only s3] step 3780/12000 loss=4.7682 gn=0.77 lr=4.86e-04 tps=21020 mem=5.5G lam=0.20650208772470555 Gm=None +[s_only s3] step 3800/12000 loss=4.9124 gn=0.76 lr=4.85e-04 tps=21021 mem=5.5G lam=0.20702394905189672 Gm=None +[s_only s3] step 3820/12000 loss=5.0250 gn=0.80 lr=4.84e-04 tps=21023 mem=5.5G lam=0.20754016314943632 Gm=None +[s_only s3] step 3840/12000 loss=4.9183 gn=0.78 lr=4.83e-04 tps=21024 mem=5.5G lam=0.20805131799230972 Gm=None +[s_only s3] step 3860/12000 loss=4.8745 gn=0.76 lr=4.82e-04 tps=21024 mem=5.5G lam=0.20862161523352066 Gm=None +[s_only s3] step 3880/12000 loss=4.9252 gn=0.75 lr=4.80e-04 tps=21025 mem=5.5G lam=0.20912469395746788 Gm=None +[s_only s3] step 3900/12000 loss=4.9183 gn=0.80 lr=4.79e-04 tps=21025 mem=5.5G lam=0.20965508837252855 Gm=None +[s_only s3] step 3920/12000 loss=4.8815 gn=0.69 lr=4.78e-04 tps=21026 mem=5.5G lam=0.21024226179967323 Gm=None +[s_only s3] step 3940/12000 loss=4.7559 gn=0.81 lr=4.77e-04 tps=21027 mem=5.5G lam=0.21078395719329515 Gm=None +[s_only s3] step 3960/12000 loss=4.7669 gn=0.82 lr=4.76e-04 tps=21028 mem=5.5G lam=0.21136238146573305 Gm=None +[s_only s3] step 3980/12000 loss=4.8314 gn=0.77 lr=4.74e-04 tps=21030 mem=5.5G lam=0.2118435949087143 Gm=None +[s_only s3] step 4000/12000 loss=4.7736 gn=0.76 lr=4.73e-04 tps=21031 mem=5.5G lam=0.21241079146663347 Gm=None +[s_only s3] EVAL step 4000: val_loss=4.8969 val_ppl=133.88 +[s_only s3] step 4020/12000 loss=4.7406 gn=0.80 lr=4.72e-04 tps=21013 mem=5.5G lam=0.21301633026450872 Gm=None +[s_only s3] step 4040/12000 loss=5.0994 gn=0.83 lr=4.71e-04 tps=21013 mem=5.5G lam=0.21350487073262533 Gm=None +[s_only s3] step 4060/12000 loss=4.7545 gn=0.77 lr=4.70e-04 tps=21014 mem=5.5G lam=0.2140956592435638 Gm=None +[s_only s3] step 4080/12000 loss=4.9120 gn=0.83 lr=4.68e-04 tps=21015 mem=5.5G lam=0.2146022000039617 Gm=None +[s_only s3] step 4100/12000 loss=4.8296 gn=0.75 lr=4.67e-04 tps=21016 mem=5.5G lam=0.21505094785243273 Gm=None +[s_only s3] step 4120/12000 loss=4.8010 gn=0.77 lr=4.66e-04 tps=21016 mem=5.5G lam=0.2155224603290359 Gm=None +[s_only s3] step 4140/12000 loss=4.8341 gn=0.73 lr=4.65e-04 tps=21017 mem=5.5G lam=0.2160450452938676 Gm=None +[s_only s3] step 4160/12000 loss=4.7734 gn=0.79 lr=4.63e-04 tps=21018 mem=5.5G lam=0.21655108965933323 Gm=None +[s_only s3] step 4180/12000 loss=4.7292 gn=0.71 lr=4.62e-04 tps=21018 mem=5.5G lam=0.21710350923240185 Gm=None +[s_only s3] step 4200/12000 loss=4.8384 gn=0.83 lr=4.61e-04 tps=21019 mem=5.5G lam=0.21774428865561882 Gm=None +[s_only s3] step 4220/12000 loss=4.6862 gn=0.75 lr=4.60e-04 tps=21019 mem=5.5G lam=0.21827208964775005 Gm=None +[s_only s3] step 4240/12000 loss=4.8560 gn=0.77 lr=4.58e-04 tps=21020 mem=5.5G lam=0.2188006949921449 Gm=None +[s_only s3] step 4260/12000 loss=4.8969 gn=0.79 lr=4.57e-04 tps=21021 mem=5.5G lam=0.2194276244069139 Gm=None +[s_only s3] step 4280/12000 loss=4.8045 gn=0.79 lr=4.56e-04 tps=21021 mem=5.5G lam=0.21989780416091284 Gm=None +[s_only s3] step 4300/12000 loss=4.7267 gn=0.74 lr=4.54e-04 tps=21023 mem=5.5G lam=0.22049069652954736 Gm=None +[s_only s3] step 4320/12000 loss=4.7667 gn=0.88 lr=4.53e-04 tps=21023 mem=5.5G lam=0.22094896590958038 Gm=None +[s_only s3] step 4340/12000 loss=4.8827 gn=0.82 lr=4.52e-04 tps=21024 mem=5.5G lam=0.22141566158582768 Gm=None +[s_only s3] step 4360/12000 loss=4.9000 gn=0.77 lr=4.51e-04 tps=21024 mem=5.5G lam=0.22185522566239038 Gm=None +[s_only s3] step 4380/12000 loss=4.7774 gn=0.76 lr=4.49e-04 tps=21024 mem=5.5G lam=0.22237294539809227 Gm=None +[s_only s3] step 4400/12000 loss=4.6927 gn=0.82 lr=4.48e-04 tps=21025 mem=5.5G lam=0.22294518227378526 Gm=None +[s_only s3] step 4420/12000 loss=4.9900 gn=0.78 lr=4.47e-04 tps=21026 mem=5.5G lam=0.22349098231643438 Gm=None +[s_only s3] step 4440/12000 loss=4.9184 gn=0.88 lr=4.45e-04 tps=21026 mem=5.5G lam=0.22402157603452602 Gm=None +[s_only s3] step 4460/12000 loss=4.8016 gn=0.80 lr=4.44e-04 tps=21027 mem=5.5G lam=0.22449471708387136 Gm=None +[s_only s3] step 4480/12000 loss=4.9758 gn=0.77 lr=4.43e-04 tps=21028 mem=5.5G lam=0.22501828521490097 Gm=None +[s_only s3] step 4500/12000 loss=4.9120 gn=0.87 lr=4.42e-04 tps=21028 mem=5.5G lam=0.22554088259736696 Gm=None +[s_only s3] step 4520/12000 loss=4.7595 gn=0.83 lr=4.40e-04 tps=21029 mem=5.5G lam=0.2259930499518911 Gm=None +[s_only s3] step 4540/12000 loss=4.7931 gn=0.77 lr=4.39e-04 tps=21030 mem=5.5G lam=0.22653298390408358 Gm=None +[s_only s3] step 4560/12000 loss=4.9512 gn=0.82 lr=4.38e-04 tps=21030 mem=5.5G lam=0.22706285491585732 Gm=None +[s_only s3] step 4580/12000 loss=4.8110 gn=0.82 lr=4.36e-04 tps=21030 mem=5.5G lam=0.22760037519037724 Gm=None +[s_only s3] step 4600/12000 loss=4.7991 gn=0.78 lr=4.35e-04 tps=21031 mem=5.5G lam=0.22808931364367405 Gm=None +[s_only s3] step 4620/12000 loss=4.7683 gn=0.86 lr=4.34e-04 tps=21032 mem=5.5G lam=0.2284925344089667 Gm=None +[s_only s3] step 4640/12000 loss=4.7365 gn=0.75 lr=4.32e-04 tps=21032 mem=5.5G lam=0.22888869885355234 Gm=None +[s_only s3] step 4660/12000 loss=4.9663 gn=0.87 lr=4.31e-04 tps=21033 mem=5.5G lam=0.22943979346503815 Gm=None +[s_only s3] step 4680/12000 loss=4.6270 gn=0.81 lr=4.30e-04 tps=21034 mem=5.5G lam=0.23000638404240212 Gm=None +[s_only s3] step 4700/12000 loss=4.6364 gn=0.82 lr=4.28e-04 tps=21034 mem=5.5G lam=0.2304636680831512 Gm=None +[s_only s3] step 4720/12000 loss=4.8949 gn=0.87 lr=4.27e-04 tps=21034 mem=5.5G lam=0.23095131572335958 Gm=None +[s_only s3] step 4740/12000 loss=4.6821 gn=1.05 lr=4.26e-04 tps=21035 mem=5.5G lam=0.23144968940565983 Gm=None +[s_only s3] step 4760/12000 loss=4.8174 gn=0.78 lr=4.24e-04 tps=21036 mem=5.5G lam=0.23195957640806833 Gm=None +[s_only s3] step 4780/12000 loss=4.7545 gn=0.84 lr=4.23e-04 tps=21037 mem=5.5G lam=0.23244882809619108 Gm=None +[s_only s3] step 4800/12000 loss=4.7621 gn=0.75 lr=4.22e-04 tps=21038 mem=5.5G lam=0.2329943006237348 Gm=None +[s_only s3] step 4820/12000 loss=4.6186 gn=0.84 lr=4.20e-04 tps=21038 mem=5.5G lam=0.23355562736590704 Gm=None +[s_only s3] step 4840/12000 loss=4.7317 gn=0.81 lr=4.19e-04 tps=21039 mem=5.5G lam=0.23402741147826114 Gm=None +[s_only s3] step 4860/12000 loss=4.7833 gn=0.86 lr=4.18e-04 tps=21039 mem=5.5G lam=0.23454075337698063 Gm=None +[s_only s3] step 4880/12000 loss=4.8047 gn=0.81 lr=4.16e-04 tps=21040 mem=5.5G lam=0.23501304009308419 Gm=None +[s_only s3] step 4900/12000 loss=4.5813 gn=0.79 lr=4.15e-04 tps=21041 mem=5.5G lam=0.2354985506584247 Gm=None +[s_only s3] step 4920/12000 loss=4.8040 gn=0.84 lr=4.13e-04 tps=21041 mem=5.5G lam=0.2359691954528292 Gm=None +[s_only s3] step 4940/12000 loss=4.6653 gn=0.80 lr=4.12e-04 tps=21042 mem=5.5G lam=0.23641565317908922 Gm=None +[s_only s3] step 4960/12000 loss=4.7336 gn=0.80 lr=4.11e-04 tps=21041 mem=5.5G lam=0.23693146090954542 Gm=None +[s_only s3] step 4980/12000 loss=4.6896 gn=0.80 lr=4.09e-04 tps=21042 mem=5.5G lam=0.2375046011681358 Gm=None +[s_only s3] step 5000/12000 loss=4.5805 gn=0.84 lr=4.08e-04 tps=21042 mem=5.5G lam=0.2379760518670082 Gm=None +[s_only s3] EVAL step 5000: val_loss=4.7766 val_ppl=118.70 +[s_only s3] step 5020/12000 loss=4.7582 gn=0.79 lr=4.07e-04 tps=21028 mem=5.5G lam=0.23840442020446062 Gm=None +[s_only s3] step 5040/12000 loss=4.6286 gn=0.77 lr=4.05e-04 tps=21027 mem=5.5G lam=0.23876803802947202 Gm=None +[s_only s3] step 5060/12000 loss=4.6618 gn=0.92 lr=4.04e-04 tps=21027 mem=5.5G lam=0.23923390079289675 Gm=None +[s_only s3] step 5080/12000 loss=4.7181 gn=0.86 lr=4.02e-04 tps=21028 mem=5.5G lam=0.23975531694789728 Gm=None +[s_only s3] step 5100/12000 loss=4.6143 gn=0.85 lr=4.01e-04 tps=21029 mem=5.5G lam=0.2402453419441978 Gm=None +[s_only s3] step 5120/12000 loss=4.7528 gn=0.79 lr=4.00e-04 tps=21029 mem=5.5G lam=0.24069552713384232 Gm=None +[s_only s3] step 5140/12000 loss=4.6667 gn=0.78 lr=3.98e-04 tps=21030 mem=5.5G lam=0.24114000828315815 Gm=None +[s_only s3] step 5160/12000 loss=4.7748 gn=0.77 lr=3.97e-04 tps=21030 mem=5.5G lam=0.24160239907602468 Gm=None +[s_only s3] step 5180/12000 loss=4.5057 gn=0.79 lr=3.95e-04 tps=21030 mem=5.5G lam=0.2420852491632104 Gm=None +[s_only s3] step 5200/12000 loss=4.5419 gn=0.87 lr=3.94e-04 tps=21029 mem=5.5G lam=0.24252531367043653 Gm=None +[s_only s3] step 5220/12000 loss=4.7423 gn=0.82 lr=3.93e-04 tps=21030 mem=5.5G lam=0.2429589688157042 Gm=None +[s_only s3] step 5240/12000 loss=4.5570 gn=0.80 lr=3.91e-04 tps=21030 mem=5.5G lam=0.24342031901081404 Gm=None +[s_only s3] step 5260/12000 loss=4.7172 gn=0.85 lr=3.90e-04 tps=21031 mem=5.5G lam=0.2438338895638784 Gm=None +[s_only s3] step 5280/12000 loss=4.5545 gn=0.78 lr=3.88e-04 tps=21031 mem=5.5G lam=0.2442765754337112 Gm=None +[s_only s3] step 5300/12000 loss=4.7641 gn=0.94 lr=3.87e-04 tps=21032 mem=5.5G lam=0.24483568531771502 Gm=None +[s_only s3] step 5320/12000 loss=4.5930 gn=0.85 lr=3.86e-04 tps=21032 mem=5.5G lam=0.2453006493548552 Gm=None +[s_only s3] step 5340/12000 loss=4.5797 gn=0.83 lr=3.84e-04 tps=21032 mem=5.5G lam=0.24576422230650982 Gm=None +[s_only s3] step 5360/12000 loss=4.6005 gn=0.86 lr=3.83e-04 tps=21033 mem=5.5G lam=0.24628845850626627 Gm=None +[s_only s3] step 5380/12000 loss=4.7554 gn=0.92 lr=3.81e-04 tps=21033 mem=5.5G lam=0.24681076686829329 Gm=None +[s_only s3] step 5400/12000 loss=4.6640 gn=0.81 lr=3.80e-04 tps=21034 mem=5.5G lam=0.24731259668866792 Gm=None +[s_only s3] step 5420/12000 loss=4.8133 gn=0.89 lr=3.79e-04 tps=21035 mem=5.5G lam=0.24777440199007592 Gm=None +[s_only s3] step 5440/12000 loss=4.5426 gn=0.87 lr=3.77e-04 tps=21035 mem=5.5G lam=0.24821137295415005 Gm=None +[s_only s3] step 5460/12000 loss=4.5994 gn=0.82 lr=3.76e-04 tps=21036 mem=5.5G lam=0.24866293898473182 Gm=None +[s_only s3] step 5480/12000 loss=4.5816 gn=0.82 lr=3.74e-04 tps=21036 mem=5.5G lam=0.2491338743517796 Gm=None +[s_only s3] step 5500/12000 loss=4.7776 gn=0.87 lr=3.73e-04 tps=21036 mem=5.5G lam=0.24959931491563717 Gm=None +[s_only s3] step 5520/12000 loss=4.7901 gn=0.86 lr=3.72e-04 tps=21037 mem=5.5G lam=0.250055651490887 Gm=None +[s_only s3] step 5540/12000 loss=4.6810 gn=0.88 lr=3.70e-04 tps=21037 mem=5.5G lam=0.2504699506486456 Gm=None +[s_only s3] step 5560/12000 loss=4.7170 gn=1.02 lr=3.69e-04 tps=21038 mem=5.5G lam=0.25087631524850923 Gm=None +[s_only s3] step 5580/12000 loss=4.5142 gn=0.82 lr=3.67e-04 tps=21038 mem=5.5G lam=0.25127558192859095 Gm=None +[s_only s3] step 5600/12000 loss=4.7225 gn=0.90 lr=3.66e-04 tps=21038 mem=5.5G lam=0.25180986647804576 Gm=None +[s_only s3] step 5620/12000 loss=4.6070 gn=0.89 lr=3.64e-04 tps=21039 mem=5.5G lam=0.2523468093325694 Gm=None +[s_only s3] step 5640/12000 loss=4.6056 gn=0.93 lr=3.63e-04 tps=21039 mem=5.5G lam=0.25270708308865625 Gm=None +[s_only s3] step 5660/12000 loss=4.6534 gn=0.90 lr=3.62e-04 tps=21039 mem=5.5G lam=0.2530415067449212 Gm=None +[s_only s3] step 5680/12000 loss=4.7474 gn=0.89 lr=3.60e-04 tps=21040 mem=5.5G lam=0.2534307176247239 Gm=None +[s_only s3] step 5700/12000 loss=4.6491 gn=0.95 lr=3.59e-04 tps=21040 mem=5.5G lam=0.2538331837082903 Gm=None +[s_only s3] step 5720/12000 loss=4.6087 gn=0.77 lr=3.57e-04 tps=21041 mem=5.5G lam=0.254263852412502 Gm=None +[s_only s3] step 5740/12000 loss=4.6446 gn=0.88 lr=3.56e-04 tps=21041 mem=5.5G lam=0.25476418311397236 Gm=None +[s_only s3] step 5760/12000 loss=4.5598 gn=0.83 lr=3.54e-04 tps=21042 mem=5.5G lam=0.25514222991963226 Gm=None +[s_only s3] step 5780/12000 loss=4.6624 gn=0.90 lr=3.53e-04 tps=21042 mem=5.5G lam=0.2555828085169196 Gm=None +[s_only s3] step 5800/12000 loss=4.6437 gn=0.85 lr=3.52e-04 tps=21043 mem=5.5G lam=0.2559935025249918 Gm=None +[s_only s3] step 5820/12000 loss=4.4680 gn=0.93 lr=3.50e-04 tps=21043 mem=5.5G lam=0.25641677249222994 Gm=None +[s_only s3] step 5840/12000 loss=4.5638 gn=0.85 lr=3.49e-04 tps=21042 mem=5.5G lam=0.25685216176013154 Gm=None +[s_only s3] step 5860/12000 loss=4.5000 gn=0.83 lr=3.47e-04 tps=21043 mem=5.5G lam=0.2572278743609786 Gm=None +[s_only s3] step 5880/12000 loss=4.4920 gn=0.88 lr=3.46e-04 tps=21042 mem=5.5G lam=0.2576815774664283 Gm=None +[s_only s3] step 5900/12000 loss=4.5395 gn=0.85 lr=3.44e-04 tps=21041 mem=5.5G lam=0.25807982807358104 Gm=None +[s_only s3] step 5920/12000 loss=4.5329 gn=0.83 lr=3.43e-04 tps=21042 mem=5.5G lam=0.2584884262954195 Gm=None +[s_only s3] step 5940/12000 loss=4.6880 gn=0.89 lr=3.41e-04 tps=21042 mem=5.5G lam=0.2589837433770299 Gm=None +[s_only s3] step 5960/12000 loss=4.6257 gn=0.84 lr=3.40e-04 tps=21043 mem=5.5G lam=0.2593919973199566 Gm=None +[s_only s3] step 5980/12000 loss=4.4627 gn=0.85 lr=3.39e-04 tps=21043 mem=5.5G lam=0.25977535483737785 Gm=None +[s_only s3] step 6000/12000 loss=4.6889 gn=0.88 lr=3.37e-04 tps=21044 mem=5.5G lam=0.26010638816903037 Gm=None +[s_only s3] EVAL step 6000: val_loss=4.7251 val_ppl=112.74 +[s_only s3] step 6020/12000 loss=4.7739 gn=0.84 lr=3.36e-04 tps=21032 mem=5.5G lam=0.2605509363735716 Gm=None +[s_only s3] step 6040/12000 loss=4.5480 gn=0.86 lr=3.34e-04 tps=21033 mem=5.5G lam=0.26097840815782547 Gm=None +[s_only s3] step 6060/12000 loss=4.4970 gn=0.89 lr=3.33e-04 tps=21032 mem=5.5G lam=0.2614562710126241 Gm=None +[s_only s3] step 6080/12000 loss=4.5309 gn=0.82 lr=3.31e-04 tps=21033 mem=5.5G lam=0.2618637339522441 Gm=None +[s_only s3] step 6100/12000 loss=4.6644 gn=0.87 lr=3.30e-04 tps=21032 mem=5.5G lam=0.2621789959569772 Gm=None +[s_only s3] step 6120/12000 loss=4.6859 gn=0.89 lr=3.29e-04 tps=21033 mem=5.5G lam=0.26250711486985284 Gm=None +[s_only s3] step 6140/12000 loss=4.6607 gn=0.90 lr=3.27e-04 tps=21033 mem=5.5G lam=0.2628472674017151 Gm=None +[s_only s3] step 6160/12000 loss=4.5251 gn=0.94 lr=3.26e-04 tps=21033 mem=5.5G lam=0.26328982319682837 Gm=None +[s_only s3] step 6180/12000 loss=4.6131 gn=0.99 lr=3.24e-04 tps=21033 mem=5.5G lam=0.2636699276044965 Gm=None +[s_only s3] step 6200/12000 loss=4.5562 gn=1.10 lr=3.23e-04 tps=21033 mem=5.5G lam=0.2640429263313611 Gm=None +[s_only s3] step 6220/12000 loss=4.5884 gn=0.92 lr=3.21e-04 tps=21034 mem=5.5G lam=0.26443176344037056 Gm=None +[s_only s3] step 6240/12000 loss=4.6762 gn=0.91 lr=3.20e-04 tps=21034 mem=5.5G lam=0.26479389363278943 Gm=None +[s_only s3] step 6260/12000 loss=4.5564 gn=0.86 lr=3.19e-04 tps=21034 mem=5.5G lam=0.2651548186937968 Gm=None +[s_only s3] step 6280/12000 loss=4.5468 gn=0.89 lr=3.17e-04 tps=21035 mem=5.5G lam=0.2655941757063071 Gm=None +[s_only s3] step 6300/12000 loss=4.6803 gn=0.83 lr=3.16e-04 tps=21036 mem=5.5G lam=0.2659634094064434 Gm=None +[s_only s3] step 6320/12000 loss=4.5118 gn=0.84 lr=3.14e-04 tps=21037 mem=5.5G lam=0.26632695893446606 Gm=None +[s_only s3] step 6340/12000 loss=4.6229 gn=0.93 lr=3.13e-04 tps=21037 mem=5.5G lam=0.2666599468017618 Gm=None +[s_only s3] step 6360/12000 loss=4.7461 gn=0.96 lr=3.11e-04 tps=21037 mem=5.5G lam=0.2670133359109362 Gm=None +[s_only s3] step 6380/12000 loss=4.3177 gn=0.85 lr=3.10e-04 tps=21037 mem=5.5G lam=0.26738848350942135 Gm=None +[s_only s3] step 6400/12000 loss=4.5271 gn=0.85 lr=3.08e-04 tps=21037 mem=5.5G lam=0.2677568967143695 Gm=None +[s_only s3] step 6420/12000 loss=4.4796 gn=0.89 lr=3.07e-04 tps=21037 mem=5.5G lam=0.2680551962306102 Gm=None +[s_only s3] step 6440/12000 loss=4.6515 gn=0.85 lr=3.06e-04 tps=21037 mem=5.5G lam=0.268474033412834 Gm=None +[s_only s3] step 6460/12000 loss=4.5176 gn=1.00 lr=3.04e-04 tps=21037 mem=5.5G lam=0.26887696143239737 Gm=None +[s_only s3] step 6480/12000 loss=4.5228 gn=0.93 lr=3.03e-04 tps=21035 mem=5.5G lam=0.26924001953254145 Gm=None +[s_only s3] step 6500/12000 loss=4.4865 gn=0.86 lr=3.01e-04 tps=21034 mem=5.5G lam=0.2696080633128683 Gm=None +[s_only s3] step 6520/12000 loss=4.6060 gn=0.88 lr=3.00e-04 tps=21034 mem=5.5G lam=0.2699444650982817 Gm=None +[s_only s3] step 6540/12000 loss=4.5248 gn=0.89 lr=2.98e-04 tps=21034 mem=5.5G lam=0.2702836294968923 Gm=None +[s_only s3] step 6560/12000 loss=4.6737 gn=0.88 lr=2.97e-04 tps=21034 mem=5.5G lam=0.2706401239459713 Gm=None +[s_only s3] step 6580/12000 loss=4.6098 gn=0.90 lr=2.96e-04 tps=21034 mem=5.5G lam=0.27100987049440545 Gm=None +[s_only s3] step 6600/12000 loss=4.4974 gn=0.87 lr=2.94e-04 tps=21034 mem=5.5G lam=0.27128772468616563 Gm=None +[s_only s3] step 6620/12000 loss=4.5213 gn=0.99 lr=2.93e-04 tps=21034 mem=5.5G lam=0.2716782350714008 Gm=None +[s_only s3] step 6640/12000 loss=4.4023 gn=0.97 lr=2.91e-04 tps=21034 mem=5.5G lam=0.27207538361350697 Gm=None +[s_only s3] step 6660/12000 loss=4.4613 gn=0.90 lr=2.90e-04 tps=21033 mem=5.5G lam=0.27247627017398673 Gm=None +[s_only s3] step 6680/12000 loss=4.4766 gn=0.93 lr=2.88e-04 tps=21033 mem=5.5G lam=0.2727378197014332 Gm=None +[s_only s3] step 6700/12000 loss=4.7808 gn=0.99 lr=2.87e-04 tps=21033 mem=5.5G lam=0.2730424103016655 Gm=None +[s_only s3] step 6720/12000 loss=4.4399 gn=0.92 lr=2.86e-04 tps=21033 mem=5.5G lam=0.27336607687175274 Gm=None +[s_only s3] step 6740/12000 loss=4.4891 gn=0.92 lr=2.84e-04 tps=21032 mem=5.5G lam=0.27376812044531107 Gm=None +[s_only s3] step 6760/12000 loss=4.5735 gn=0.94 lr=2.83e-04 tps=21033 mem=5.5G lam=0.27410754933953285 Gm=None +[s_only s3] step 6780/12000 loss=4.5335 gn=0.90 lr=2.81e-04 tps=21033 mem=5.5G lam=0.2744017243385315 Gm=None +[s_only s3] step 6800/12000 loss=4.6730 gn=0.95 lr=2.80e-04 tps=21032 mem=5.5G lam=0.27474741513530415 Gm=None +[s_only s3] step 6820/12000 loss=4.5842 gn=0.97 lr=2.79e-04 tps=21033 mem=5.5G lam=0.2750964475174745 Gm=None +[s_only s3] step 6840/12000 loss=4.5629 gn=1.07 lr=2.77e-04 tps=21033 mem=5.5G lam=0.2754027871415019 Gm=None +[s_only s3] step 6860/12000 loss=4.5719 gn=0.97 lr=2.76e-04 tps=21033 mem=5.5G lam=0.2757884655147791 Gm=None +[s_only s3] step 6880/12000 loss=4.5679 gn=0.99 lr=2.74e-04 tps=21032 mem=5.5G lam=0.27608213294297457 Gm=None +[s_only s3] step 6900/12000 loss=4.5623 gn=0.90 lr=2.73e-04 tps=21032 mem=5.5G lam=0.2763896118849516 Gm=None +[s_only s3] step 6920/12000 loss=4.4985 gn=0.95 lr=2.72e-04 tps=21032 mem=5.5G lam=0.2767505416025718 Gm=None +[s_only s3] step 6940/12000 loss=4.3811 gn=0.90 lr=2.70e-04 tps=21032 mem=5.5G lam=0.2771001846219103 Gm=None +[s_only s3] step 6960/12000 loss=4.4240 gn=0.99 lr=2.69e-04 tps=21032 mem=5.5G lam=0.2774897202228506 Gm=None +[s_only s3] step 6980/12000 loss=4.6301 gn=0.99 lr=2.67e-04 tps=21032 mem=5.5G lam=0.2777792022873958 Gm=None +[s_only s3] step 7000/12000 loss=4.3967 gn=0.86 lr=2.66e-04 tps=21032 mem=5.5G lam=0.2780758024503787 Gm=None +[s_only s3] EVAL step 7000: val_loss=4.6174 val_ppl=101.23 +[s_only s3] step 7020/12000 loss=4.5066 gn=0.94 lr=2.65e-04 tps=21021 mem=5.5G lam=0.2783716522778074 Gm=None +[s_only s3] step 7040/12000 loss=4.4858 gn=0.96 lr=2.63e-04 tps=21021 mem=5.5G lam=0.2786837083597978 Gm=None +[s_only s3] step 7060/12000 loss=4.5383 gn=1.01 lr=2.62e-04 tps=21021 mem=5.5G lam=0.2789633385837078 Gm=None +[s_only s3] step 7080/12000 loss=4.4879 gn=0.97 lr=2.60e-04 tps=21021 mem=5.5G lam=0.2792740287259221 Gm=None +[s_only s3] step 7100/12000 loss=4.4714 gn=0.88 lr=2.59e-04 tps=21021 mem=5.5G lam=0.27957747876644135 Gm=None +[s_only s3] step 7120/12000 loss=4.6199 gn=0.96 lr=2.58e-04 tps=21021 mem=5.5G lam=0.27987515553832054 Gm=None +[s_only s3] step 7140/12000 loss=4.5167 gn=1.03 lr=2.56e-04 tps=21021 mem=5.5G lam=0.28020206031699973 Gm=None +[s_only s3] step 7160/12000 loss=4.6837 gn=1.02 lr=2.55e-04 tps=21021 mem=5.5G lam=0.2805076325312257 Gm=None +[s_only s3] step 7180/12000 loss=4.5027 gn=0.95 lr=2.53e-04 tps=21021 mem=5.5G lam=0.28077773936092854 Gm=None +[s_only s3] step 7200/12000 loss=4.4121 gn=0.90 lr=2.52e-04 tps=21022 mem=5.5G lam=0.2811049235363801 Gm=None +[s_only s3] step 7220/12000 loss=4.4147 gn=0.97 lr=2.51e-04 tps=21022 mem=5.5G lam=0.28141277842223644 Gm=None +[s_only s3] step 7240/12000 loss=4.5761 gn=0.91 lr=2.49e-04 tps=21022 mem=5.5G lam=0.2817480228841305 Gm=None +[s_only s3] step 7260/12000 loss=4.3998 gn=0.97 lr=2.48e-04 tps=21023 mem=5.5G lam=0.28202537385125953 Gm=None +[s_only s3] step 7280/12000 loss=4.3861 gn=0.96 lr=2.47e-04 tps=21023 mem=5.5G lam=0.28232967139532167 Gm=None +[s_only s3] step 7300/12000 loss=4.6273 gn=0.91 lr=2.45e-04 tps=21023 mem=5.5G lam=0.2825979304810365 Gm=None +[s_only s3] step 7320/12000 loss=4.6243 gn=0.98 lr=2.44e-04 tps=21023 mem=5.5G lam=0.2828828416143854 Gm=None +[s_only s3] step 7340/12000 loss=4.4346 gn=0.92 lr=2.42e-04 tps=21023 mem=5.5G lam=0.2832132950425148 Gm=None +[s_only s3] step 7360/12000 loss=4.4038 gn=0.94 lr=2.41e-04 tps=21024 mem=5.5G lam=0.28358427466203767 Gm=None +[s_only s3] step 7380/12000 loss=4.4549 gn=0.93 lr=2.40e-04 tps=21024 mem=5.5G lam=0.2839121436700225 Gm=None +[s_only s3] step 7400/12000 loss=4.5798 gn=0.93 lr=2.38e-04 tps=21024 mem=5.5G lam=0.2841806523501873 Gm=None +[s_only s3] step 7420/12000 loss=4.4471 gn=0.93 lr=2.37e-04 tps=21024 mem=5.5G lam=0.2844470785930753 Gm=None +[s_only s3] step 7440/12000 loss=4.5167 gn=0.96 lr=2.36e-04 tps=21024 mem=5.5G lam=0.28473740505675477 Gm=None +[s_only s3] step 7460/12000 loss=4.5891 gn=0.94 lr=2.34e-04 tps=21024 mem=5.5G lam=0.285005207794408 Gm=None +[s_only s3] step 7480/12000 loss=4.5045 gn=1.02 lr=2.33e-04 tps=21024 mem=5.5G lam=0.28527701180428267 Gm=None +[s_only s3] step 7500/12000 loss=4.5894 gn=1.01 lr=2.32e-04 tps=21024 mem=5.5G lam=0.2855298410480221 Gm=None +[s_only s3] step 7520/12000 loss=4.4027 gn=0.92 lr=2.30e-04 tps=21024 mem=5.5G lam=0.28579395450651646 Gm=None +[s_only s3] step 7540/12000 loss=4.4906 gn=0.95 lr=2.29e-04 tps=21024 mem=5.5G lam=0.28603280025223893 Gm=None +[s_only s3] step 7560/12000 loss=4.5267 gn=0.89 lr=2.28e-04 tps=21024 mem=5.5G lam=0.28629320207983255 Gm=None +[s_only s3] step 7580/12000 loss=4.4609 gn=1.04 lr=2.26e-04 tps=21024 mem=5.5G lam=0.28661202763517696 Gm=None +[s_only s3] step 7600/12000 loss=4.4045 gn=0.99 lr=2.25e-04 tps=21024 mem=5.5G lam=0.2869257063915332 Gm=None +[s_only s3] step 7620/12000 loss=4.2916 gn=0.97 lr=2.24e-04 tps=21023 mem=5.5G lam=0.2871959653372566 Gm=None +[s_only s3] step 7640/12000 loss=4.5028 gn=0.95 lr=2.22e-04 tps=21024 mem=5.5G lam=0.2874716476847728 Gm=None +[s_only s3] step 7660/12000 loss=4.4310 gn=1.23 lr=2.21e-04 tps=21023 mem=5.5G lam=0.2877583159133792 Gm=None +[s_only s3] step 7680/12000 loss=4.5166 gn=1.01 lr=2.20e-04 tps=21023 mem=5.5G lam=0.2880103134860595 Gm=None +[s_only s3] step 7700/12000 loss=4.4572 gn=0.96 lr=2.18e-04 tps=21023 mem=5.5G lam=0.28828644069532555 Gm=None +[s_only s3] step 7720/12000 loss=4.3480 gn=1.07 lr=2.17e-04 tps=21023 mem=5.5G lam=0.2885421725610892 Gm=None +[s_only s3] step 7740/12000 loss=4.3316 gn=0.96 lr=2.16e-04 tps=21023 mem=5.5G lam=0.2887974229330818 Gm=None +[s_only s3] step 7760/12000 loss=4.4352 gn=1.04 lr=2.15e-04 tps=21023 mem=5.5G lam=0.28910736832767725 Gm=None +[s_only s3] step 7780/12000 loss=4.4502 gn=0.96 lr=2.13e-04 tps=21023 mem=5.5G lam=0.28933737830569345 Gm=None +[s_only s3] step 7800/12000 loss=4.4493 gn=1.04 lr=2.12e-04 tps=21024 mem=5.5G lam=0.2895875150958697 Gm=None +[s_only s3] step 7820/12000 loss=4.5422 gn=0.95 lr=2.11e-04 tps=21024 mem=5.5G lam=0.28985273869087297 Gm=None +[s_only s3] step 7840/12000 loss=4.4858 gn=1.00 lr=2.09e-04 tps=21024 mem=5.5G lam=0.29011591865370673 Gm=None +[s_only s3] step 7860/12000 loss=4.2389 gn=0.89 lr=2.08e-04 tps=21025 mem=5.5G lam=0.290333600093921 Gm=None +[s_only s3] step 7880/12000 loss=4.5332 gn=1.00 lr=2.07e-04 tps=21025 mem=5.5G lam=0.29056544322520494 Gm=None +[s_only s3] step 7900/12000 loss=4.5333 gn=1.10 lr=2.06e-04 tps=21025 mem=5.5G lam=0.29079249625404674 Gm=None +[s_only s3] step 7920/12000 loss=4.2975 gn=0.91 lr=2.04e-04 tps=21025 mem=5.5G lam=0.29100404276202124 Gm=None +[s_only s3] step 7940/12000 loss=4.4819 gn=0.97 lr=2.03e-04 tps=21025 mem=5.5G lam=0.2912351780881484 Gm=None +[s_only s3] step 7960/12000 loss=4.5178 gn=0.97 lr=2.02e-04 tps=21025 mem=5.5G lam=0.29150691100706655 Gm=None +[s_only s3] step 7980/12000 loss=4.4013 gn=0.97 lr=2.00e-04 tps=21025 mem=5.5G lam=0.2917752868185441 Gm=None +[s_only s3] step 8000/12000 loss=4.4691 gn=0.93 lr=1.99e-04 tps=21026 mem=5.5G lam=0.2919822692250212 Gm=None +[s_only s3] EVAL step 8000: val_loss=4.5004 val_ppl=90.06 +[s_only s3] step 8020/12000 loss=4.5189 gn=1.02 lr=1.98e-04 tps=21017 mem=5.5G lam=0.29226516280323267 Gm=None +[s_only s3] step 8040/12000 loss=4.5289 gn=0.99 lr=1.97e-04 tps=21016 mem=5.5G lam=0.2925489963963628 Gm=None +[s_only s3] step 8060/12000 loss=4.4815 gn=0.97 lr=1.95e-04 tps=21016 mem=5.5G lam=0.29277190876503784 Gm=None +[s_only s3] step 8080/12000 loss=4.4111 gn=1.00 lr=1.94e-04 tps=21015 mem=5.5G lam=0.29306782130151987 Gm=None +[s_only s3] step 8100/12000 loss=4.2946 gn=0.93 lr=1.93e-04 tps=21016 mem=5.5G lam=0.29327696810166043 Gm=None +[s_only s3] step 8120/12000 loss=4.7126 gn=1.04 lr=1.92e-04 tps=21015 mem=5.5G lam=0.29354423998544615 Gm=None +[s_only s3] step 8140/12000 loss=4.4179 gn=1.19 lr=1.90e-04 tps=21016 mem=5.5G lam=0.29382862616330385 Gm=None +[s_only s3] step 8160/12000 loss=4.3923 gn=0.95 lr=1.89e-04 tps=21016 mem=5.5G lam=0.2940215989947319 Gm=None +[s_only s3] step 8180/12000 loss=4.4967 gn=0.99 lr=1.88e-04 tps=21016 mem=5.5G lam=0.2941976059228182 Gm=None +[s_only s3] step 8200/12000 loss=4.4403 gn=1.00 lr=1.87e-04 tps=21016 mem=5.5G lam=0.2943640798330307 Gm=None +[s_only s3] step 8220/12000 loss=4.4606 gn=1.03 lr=1.86e-04 tps=21015 mem=5.5G lam=0.2945721506451567 Gm=None +[s_only s3] step 8240/12000 loss=4.5972 gn=1.05 lr=1.84e-04 tps=21016 mem=5.5G lam=0.2948270433892806 Gm=None +[s_only s3] step 8260/12000 loss=4.4252 gn=1.00 lr=1.83e-04 tps=21016 mem=5.5G lam=0.2950559767584006 Gm=None +[s_only s3] step 8280/12000 loss=4.3734 gn=0.97 lr=1.82e-04 tps=21014 mem=5.5G lam=0.2953093309576313 Gm=None +[s_only s3] step 8300/12000 loss=4.4128 gn=1.34 lr=1.81e-04 tps=21014 mem=5.5G lam=0.29555914519975585 Gm=None +[s_only s3] step 8320/12000 loss=4.3889 gn=1.14 lr=1.80e-04 tps=21014 mem=5.5G lam=0.29578693583607674 Gm=None +[s_only s3] step 8340/12000 loss=4.4575 gn=1.06 lr=1.78e-04 tps=21014 mem=5.5G lam=0.29600506493200857 Gm=None +[s_only s3] step 8360/12000 loss=4.3195 gn=1.02 lr=1.77e-04 tps=21014 mem=5.5G lam=0.2962373737245798 Gm=None +[s_only s3] step 8380/12000 loss=4.4201 gn=1.07 lr=1.76e-04 tps=21014 mem=5.5G lam=0.2964725581308206 Gm=None +[s_only s3] step 8400/12000 loss=4.4982 gn=1.06 lr=1.75e-04 tps=21014 mem=5.5G lam=0.29670541019489366 Gm=None +[s_only s3] step 8420/12000 loss=4.4779 gn=1.07 lr=1.74e-04 tps=21014 mem=5.5G lam=0.29693524818867445 Gm=None +[s_only s3] step 8440/12000 loss=4.4159 gn=1.02 lr=1.72e-04 tps=21014 mem=5.5G lam=0.29714473709464073 Gm=None +[s_only s3] step 8460/12000 loss=4.5146 gn=1.03 lr=1.71e-04 tps=21014 mem=5.5G lam=0.2973694521933794 Gm=None +[s_only s3] step 8480/12000 loss=4.4578 gn=1.06 lr=1.70e-04 tps=21014 mem=5.5G lam=0.2975805504247546 Gm=None +[s_only s3] step 8500/12000 loss=4.4154 gn=1.06 lr=1.69e-04 tps=21014 mem=5.5G lam=0.29778565373271704 Gm=None +[s_only s3] step 8520/12000 loss=4.4236 gn=1.06 lr=1.68e-04 tps=21014 mem=5.5G lam=0.29796665472288925 Gm=None +[s_only s3] step 8540/12000 loss=4.4158 gn=0.98 lr=1.67e-04 tps=21015 mem=5.5G lam=0.2981470062707861 Gm=None +[s_only s3] step 8560/12000 loss=4.3839 gn=1.07 lr=1.66e-04 tps=21015 mem=5.5G lam=0.2983068637549877 Gm=None +[s_only s3] step 8580/12000 loss=4.4962 gn=1.08 lr=1.64e-04 tps=21015 mem=5.5G lam=0.2984922140215834 Gm=None +[s_only s3] step 8600/12000 loss=4.3184 gn=1.02 lr=1.63e-04 tps=21015 mem=5.5G lam=0.29867590374002856 Gm=None +[s_only s3] step 8620/12000 loss=4.4251 gn=1.06 lr=1.62e-04 tps=21015 mem=5.5G lam=0.2988306765134136 Gm=None +[s_only s3] step 8640/12000 loss=4.4361 gn=1.04 lr=1.61e-04 tps=21015 mem=5.5G lam=0.2990329597766201 Gm=None +[s_only s3] step 8660/12000 loss=4.4480 gn=1.10 lr=1.60e-04 tps=21015 mem=5.5G lam=0.2992394467194875 Gm=None +[s_only s3] step 8680/12000 loss=4.4799 gn=1.05 lr=1.59e-04 tps=21016 mem=5.5G lam=0.29942688532173634 Gm=None +[s_only s3] step 8700/12000 loss=4.4469 gn=1.12 lr=1.58e-04 tps=21016 mem=5.5G lam=0.2996521483485897 Gm=None +[s_only s3] step 8720/12000 loss=4.4532 gn=0.99 lr=1.57e-04 tps=21016 mem=5.5G lam=0.299875491609176 Gm=None +[s_only s3] step 8740/12000 loss=4.3490 gn=1.06 lr=1.55e-04 tps=21016 mem=5.5G lam=0.30011247967680293 Gm=None +[s_only s3] step 8760/12000 loss=4.4401 gn=1.08 lr=1.54e-04 tps=21016 mem=5.5G lam=0.30033106977740925 Gm=None +[s_only s3] step 8780/12000 loss=4.2976 gn=1.00 lr=1.53e-04 tps=21016 mem=5.5G lam=0.3005097374940912 Gm=None +[s_only s3] step 8800/12000 loss=4.4678 gn=1.06 lr=1.52e-04 tps=21016 mem=5.5G lam=0.30064757478733856 Gm=None +[s_only s3] step 8820/12000 loss=4.4908 gn=1.25 lr=1.51e-04 tps=21016 mem=5.5G lam=0.3008455897991856 Gm=None +[s_only s3] step 8840/12000 loss=4.4223 gn=1.03 lr=1.50e-04 tps=21016 mem=5.5G lam=0.30098267178982496 Gm=None +[s_only s3] step 8860/12000 loss=4.3571 gn=1.04 lr=1.49e-04 tps=21017 mem=5.5G lam=0.301154020242393 Gm=None +[s_only s3] step 8880/12000 loss=4.5608 gn=1.09 lr=1.48e-04 tps=21017 mem=5.5G lam=0.30128964968025684 Gm=None +[s_only s3] step 8900/12000 loss=4.4040 gn=1.07 lr=1.47e-04 tps=21017 mem=5.5G lam=0.3014502314229806 Gm=None +[s_only s3] step 8920/12000 loss=4.4234 gn=1.11 lr=1.46e-04 tps=21017 mem=5.5G lam=0.30163124793519575 Gm=None +[s_only s3] step 8940/12000 loss=4.2797 gn=1.06 lr=1.45e-04 tps=21017 mem=5.5G lam=0.3018195467690627 Gm=None +[s_only s3] step 8960/12000 loss=4.4504 gn=1.06 lr=1.44e-04 tps=21017 mem=5.5G lam=0.3020295671497782 Gm=None +[s_only s3] step 8980/12000 loss=4.4419 gn=1.11 lr=1.43e-04 tps=21018 mem=5.5G lam=0.30221302186449367 Gm=None +[s_only s3] step 9000/12000 loss=4.5359 gn=1.01 lr=1.42e-04 tps=21018 mem=5.5G lam=0.3023571229229371 Gm=None +[s_only s3] EVAL step 9000: val_loss=4.4729 val_ppl=87.61 +[s_only s3] step 9020/12000 loss=4.3591 gn=0.99 lr=1.41e-04 tps=21010 mem=5.5G lam=0.30248314794152975 Gm=None +[s_only s3] step 9040/12000 loss=4.4516 gn=1.09 lr=1.40e-04 tps=21010 mem=5.5G lam=0.30262507932881516 Gm=None +[s_only s3] step 9060/12000 loss=4.2346 gn=0.99 lr=1.39e-04 tps=21010 mem=5.5G lam=0.3027652424449722 Gm=None +[s_only s3] step 9080/12000 loss=4.4063 gn=1.02 lr=1.38e-04 tps=21010 mem=5.5G lam=0.3029285604134202 Gm=None +[s_only s3] step 9100/12000 loss=4.5120 gn=1.14 lr=1.37e-04 tps=21010 mem=5.5G lam=0.30311941479643184 Gm=None +[s_only s3] step 9120/12000 loss=4.4666 gn=1.06 lr=1.36e-04 tps=21010 mem=5.5G lam=0.30330307222902775 Gm=None +[s_only s3] step 9140/12000 loss=4.2469 gn=1.04 lr=1.35e-04 tps=21010 mem=5.5G lam=0.3034737982476751 Gm=None +[s_only s3] step 9160/12000 loss=4.5399 gn=1.24 lr=1.34e-04 tps=21010 mem=5.5G lam=0.3036585608497262 Gm=None +[s_only s3] step 9180/12000 loss=4.3249 gn=1.04 lr=1.33e-04 tps=21010 mem=5.5G lam=0.30384047018984955 Gm=None +[s_only s3] step 9200/12000 loss=4.4384 gn=1.09 lr=1.32e-04 tps=21010 mem=5.5G lam=0.3040421061838667 Gm=None +[s_only s3] step 9220/12000 loss=4.3833 gn=1.15 lr=1.31e-04 tps=21010 mem=5.5G lam=0.3042104837174217 Gm=None +[s_only s3] step 9240/12000 loss=4.4236 gn=1.11 lr=1.30e-04 tps=21010 mem=5.5G lam=0.3043612226222952 Gm=None +[s_only s3] step 9260/12000 loss=4.4429 gn=1.08 lr=1.29e-04 tps=21010 mem=5.5G lam=0.304512751288712 Gm=None +[s_only s3] step 9280/12000 loss=4.4052 gn=1.00 lr=1.28e-04 tps=21010 mem=5.5G lam=0.3046166331817706 Gm=None +[s_only s3] step 9300/12000 loss=4.3112 gn=1.20 lr=1.27e-04 tps=21010 mem=5.5G lam=0.30475779840101797 Gm=None +[s_only s3] step 9320/12000 loss=4.5350 gn=1.09 lr=1.26e-04 tps=21011 mem=5.5G lam=0.3049195023874442 Gm=None +[s_only s3] step 9340/12000 loss=4.5325 gn=1.07 lr=1.25e-04 tps=21011 mem=5.5G lam=0.30508960566173 Gm=None +[s_only s3] step 9360/12000 loss=4.3559 gn=1.07 lr=1.24e-04 tps=21011 mem=5.5G lam=0.3052930148939292 Gm=None +[s_only s3] step 9380/12000 loss=4.4366 gn=1.06 lr=1.23e-04 tps=21012 mem=5.5G lam=0.30548496078699827 Gm=None +[s_only s3] step 9400/12000 loss=4.4015 gn=1.13 lr=1.22e-04 tps=21012 mem=5.5G lam=0.3056222964078188 Gm=None +[s_only s3] step 9420/12000 loss=4.3962 gn=1.12 lr=1.21e-04 tps=21012 mem=5.5G lam=0.30575343190381926 Gm=None +[s_only s3] step 9440/12000 loss=4.2597 gn=1.02 lr=1.20e-04 tps=21012 mem=5.5G lam=0.3058700455973546 Gm=None +[s_only s3] step 9460/12000 loss=4.4924 gn=1.03 lr=1.19e-04 tps=21013 mem=5.5G lam=0.3059997158125043 Gm=None +[s_only s3] step 9480/12000 loss=4.5495 gn=1.15 lr=1.19e-04 tps=21013 mem=5.5G lam=0.3061444725220402 Gm=None +[s_only s3] step 9500/12000 loss=4.2895 gn=0.96 lr=1.18e-04 tps=21013 mem=5.5G lam=0.306277137560149 Gm=None +[s_only s3] step 9520/12000 loss=4.4137 gn=1.24 lr=1.17e-04 tps=21013 mem=5.5G lam=0.3063956005498767 Gm=None +[s_only s3] step 9540/12000 loss=4.3573 gn=1.10 lr=1.16e-04 tps=21013 mem=5.5G lam=0.3064803583547473 Gm=None +[s_only s3] step 9560/12000 loss=4.2922 gn=1.01 lr=1.15e-04 tps=21013 mem=5.5G lam=0.30660199870665866 Gm=None +[s_only s3] step 9580/12000 loss=4.3941 gn=1.10 lr=1.14e-04 tps=21014 mem=5.5G lam=0.3067548271889488 Gm=None +[s_only s3] step 9600/12000 loss=4.5913 gn=1.06 lr=1.13e-04 tps=21014 mem=5.5G lam=0.306904342956841 Gm=None +[s_only s3] step 9620/12000 loss=4.4248 gn=1.08 lr=1.12e-04 tps=21014 mem=5.5G lam=0.307048549875617 Gm=None +[s_only s3] step 9640/12000 loss=4.2409 gn=1.10 lr=1.12e-04 tps=21014 mem=5.5G lam=0.3071735060463349 Gm=None +[s_only s3] step 9660/12000 loss=4.4957 gn=1.08 lr=1.11e-04 tps=21014 mem=5.5G lam=0.3072992271433274 Gm=None +[s_only s3] step 9680/12000 loss=4.4125 gn=1.21 lr=1.10e-04 tps=21015 mem=5.5G lam=0.3074504438166817 Gm=None +[s_only s3] step 9700/12000 loss=4.2538 gn=0.98 lr=1.09e-04 tps=21015 mem=5.5G lam=0.3075818444291751 Gm=None +[s_only s3] step 9720/12000 loss=4.3475 gn=1.06 lr=1.08e-04 tps=21015 mem=5.5G lam=0.30769234523177147 Gm=None +[s_only s3] step 9740/12000 loss=4.3267 gn=1.08 lr=1.07e-04 tps=21015 mem=5.5G lam=0.307798162723581 Gm=None +[s_only s3] step 9760/12000 loss=4.3960 gn=1.10 lr=1.07e-04 tps=21015 mem=5.5G lam=0.3079418533792098 Gm=None +[s_only s3] step 9780/12000 loss=4.3221 gn=0.99 lr=1.06e-04 tps=21014 mem=5.5G lam=0.30808891107638675 Gm=None +[s_only s3] step 9800/12000 loss=4.3389 gn=1.16 lr=1.05e-04 tps=21014 mem=5.5G lam=0.30822421175738174 Gm=None +[s_only s3] step 9820/12000 loss=4.3096 gn=1.02 lr=1.04e-04 tps=21014 mem=5.5G lam=0.30835101163635653 Gm=None +[s_only s3] step 9840/12000 loss=4.3378 gn=1.01 lr=1.03e-04 tps=21014 mem=5.5G lam=0.3084780365849535 Gm=None +[s_only s3] step 9860/12000 loss=4.3244 gn=1.12 lr=1.03e-04 tps=21013 mem=5.5G lam=0.30861766388018924 Gm=None +[s_only s3] step 9880/12000 loss=4.2499 gn=1.30 lr=1.02e-04 tps=21013 mem=5.5G lam=0.3087437491243084 Gm=None +[s_only s3] step 9900/12000 loss=4.3385 gn=1.01 lr=1.01e-04 tps=21014 mem=5.5G lam=0.3088924406717221 Gm=None +[s_only s3] step 9920/12000 loss=4.3347 gn=1.31 lr=1.00e-04 tps=21014 mem=5.5G lam=0.3090210898468892 Gm=None +[s_only s3] step 9940/12000 loss=4.3081 gn=1.20 lr=9.96e-05 tps=21014 mem=5.5G lam=0.30910401574025553 Gm=None +[s_only s3] step 9960/12000 loss=4.4791 gn=1.27 lr=9.89e-05 tps=21015 mem=5.5G lam=0.3092257206638654 Gm=None +[s_only s3] step 9980/12000 loss=4.2606 gn=1.03 lr=9.81e-05 tps=21015 mem=5.5G lam=0.3093541075165073 Gm=None +[s_only s3] step 10000/12000 loss=4.4013 gn=1.17 lr=9.74e-05 tps=21015 mem=5.5G lam=0.30948216157654923 Gm=None +[s_only s3] EVAL step 10000: val_loss=4.4604 val_ppl=86.52 +[s_only s3] step 10020/12000 loss=4.2483 gn=1.09 lr=9.67e-05 tps=21008 mem=5.5G lam=0.30958569298187893 Gm=None +[s_only s3] step 10040/12000 loss=4.3691 gn=1.27 lr=9.59e-05 tps=21008 mem=5.5G lam=0.3096873393903176 Gm=None +[s_only s3] step 10060/12000 loss=4.2522 gn=1.16 lr=9.52e-05 tps=21009 mem=5.5G lam=0.30981788008163375 Gm=None +[s_only s3] step 10080/12000 loss=4.1909 gn=1.00 lr=9.45e-05 tps=21008 mem=5.5G lam=0.3099231819311778 Gm=None +[s_only s3] step 10100/12000 loss=4.4941 gn=1.12 lr=9.38e-05 tps=21008 mem=5.5G lam=0.3100415092582504 Gm=None +[s_only s3] step 10120/12000 loss=4.3979 gn=1.14 lr=9.31e-05 tps=21008 mem=5.5G lam=0.3101574620231986 Gm=None +[s_only s3] step 10140/12000 loss=4.2890 gn=1.09 lr=9.24e-05 tps=21009 mem=5.5G lam=0.3102905598158638 Gm=None +[s_only s3] step 10160/12000 loss=4.4189 gn=1.16 lr=9.18e-05 tps=21009 mem=5.5G lam=0.3103799354285002 Gm=None +[s_only s3] step 10180/12000 loss=4.5187 gn=1.15 lr=9.11e-05 tps=21009 mem=5.5G lam=0.31047465559095144 Gm=None +[s_only s3] step 10200/12000 loss=4.4874 gn=1.17 lr=9.04e-05 tps=21010 mem=5.5G lam=0.3105418436850111 Gm=None +[s_only s3] step 10220/12000 loss=4.3898 gn=1.19 lr=8.98e-05 tps=21010 mem=5.5G lam=0.3106403707837065 Gm=None +[s_only s3] step 10240/12000 loss=4.2601 gn=1.19 lr=8.91e-05 tps=21010 mem=5.5G lam=0.31079629343003035 Gm=None +[s_only s3] step 10260/12000 loss=4.3851 gn=1.16 lr=8.85e-05 tps=21010 mem=5.5G lam=0.3109354094291727 Gm=None +[s_only s3] step 10280/12000 loss=4.4095 gn=1.33 lr=8.78e-05 tps=21010 mem=5.5G lam=0.3110424932092428 Gm=None +[s_only s3] step 10300/12000 loss=4.3013 gn=1.26 lr=8.72e-05 tps=21011 mem=5.5G lam=0.3111277852828304 Gm=None +[s_only s3] step 10320/12000 loss=4.4309 gn=1.07 lr=8.66e-05 tps=21011 mem=5.5G lam=0.3112040860578418 Gm=None +[s_only s3] step 10340/12000 loss=4.2580 gn=1.02 lr=8.59e-05 tps=21010 mem=5.5G lam=0.3113103797659278 Gm=None +[s_only s3] step 10360/12000 loss=4.2900 gn=1.08 lr=8.53e-05 tps=21010 mem=5.5G lam=0.31141173746436834 Gm=None +[s_only s3] step 10380/12000 loss=4.4450 gn=1.11 lr=8.47e-05 tps=21010 mem=5.5G lam=0.3114861762151122 Gm=None +[s_only s3] step 10400/12000 loss=4.3927 gn=1.10 lr=8.41e-05 tps=21010 mem=5.5G lam=0.31158564829578 Gm=None +[s_only s3] step 10420/12000 loss=4.2372 gn=1.17 lr=8.35e-05 tps=21010 mem=5.5G lam=0.3116895779967308 Gm=None +[s_only s3] step 10440/12000 loss=4.2406 gn=1.14 lr=8.30e-05 tps=21010 mem=5.5G lam=0.3118029907345772 Gm=None +[s_only s3] step 10460/12000 loss=4.4502 gn=1.22 lr=8.24e-05 tps=21010 mem=5.5G lam=0.3119182111695409 Gm=None +[s_only s3] step 10480/12000 loss=4.5603 gn=1.15 lr=8.18e-05 tps=21010 mem=5.5G lam=0.31201620027422905 Gm=None +[s_only s3] step 10500/12000 loss=4.3893 gn=1.18 lr=8.12e-05 tps=21010 mem=5.5G lam=0.3121182657778263 Gm=None +[s_only s3] step 10520/12000 loss=4.3030 gn=1.03 lr=8.07e-05 tps=21010 mem=5.5G lam=0.3122436326617996 Gm=None +[s_only s3] step 10540/12000 loss=4.2823 gn=1.08 lr=8.01e-05 tps=21009 mem=5.5G lam=0.312337056423227 Gm=None +[s_only s3] step 10560/12000 loss=4.2243 gn=1.14 lr=7.96e-05 tps=21009 mem=5.5G lam=0.3124554868166645 Gm=None +[s_only s3] step 10580/12000 loss=4.4395 gn=1.11 lr=7.91e-05 tps=21009 mem=5.5G lam=0.31256064772605896 Gm=None +[s_only s3] step 10600/12000 loss=4.2345 gn=1.09 lr=7.85e-05 tps=21009 mem=5.5G lam=0.3126483193288247 Gm=None +[s_only s3] step 10620/12000 loss=4.3956 gn=1.14 lr=7.80e-05 tps=21010 mem=5.5G lam=0.31275580171495676 Gm=None +[s_only s3] step 10640/12000 loss=4.2454 gn=1.08 lr=7.75e-05 tps=21010 mem=5.5G lam=0.3128508732964595 Gm=None +[s_only s3] step 10660/12000 loss=4.3225 gn=1.13 lr=7.70e-05 tps=21010 mem=5.5G lam=0.3129451849187414 Gm=None +[s_only s3] step 10680/12000 loss=4.3869 gn=1.18 lr=7.65e-05 tps=21010 mem=5.5G lam=0.3130538597082098 Gm=None +[s_only s3] step 10700/12000 loss=4.3870 gn=1.11 lr=7.60e-05 tps=21010 mem=5.5G lam=0.3131609366585811 Gm=None +[s_only s3] step 10720/12000 loss=4.3107 gn=1.12 lr=7.55e-05 tps=21010 mem=5.5G lam=0.31326413433998823 Gm=None +[s_only s3] step 10740/12000 loss=4.2542 gn=1.12 lr=7.50e-05 tps=21010 mem=5.5G lam=0.31335595312217873 Gm=None +[s_only s3] step 10760/12000 loss=4.3435 gn=1.23 lr=7.46e-05 tps=21010 mem=5.5G lam=0.31345368331919116 Gm=None +[s_only s3] step 10780/12000 loss=4.3417 gn=1.23 lr=7.41e-05 tps=21010 mem=5.5G lam=0.3135839095339179 Gm=None +[s_only s3] step 10800/12000 loss=4.3781 gn=1.35 lr=7.37e-05 tps=21010 mem=5.5G lam=0.3136873592933019 Gm=None +[s_only s3] step 10820/12000 loss=4.3091 gn=1.09 lr=7.32e-05 tps=21010 mem=5.5G lam=0.313779662673672 Gm=None +[s_only s3] step 10840/12000 loss=4.4702 gn=1.20 lr=7.28e-05 tps=21011 mem=5.5G lam=0.3138764553392927 Gm=None +[s_only s3] step 10860/12000 loss=4.4123 gn=1.13 lr=7.23e-05 tps=21011 mem=5.5G lam=0.31396608023593825 Gm=None +[s_only s3] step 10880/12000 loss=4.4039 gn=1.22 lr=7.19e-05 tps=21012 mem=5.5G lam=0.3140347022563219 Gm=None +[s_only s3] step 10900/12000 loss=4.2792 gn=1.19 lr=7.15e-05 tps=21011 mem=5.5G lam=0.31412109918892384 Gm=None +[s_only s3] step 10920/12000 loss=4.1602 gn=1.17 lr=7.11e-05 tps=21012 mem=5.5G lam=0.314202181994915 Gm=None +[s_only s3] step 10940/12000 loss=4.4321 gn=1.29 lr=7.07e-05 tps=21012 mem=5.5G lam=0.3143148592983683 Gm=None +[s_only s3] step 10960/12000 loss=4.3552 gn=1.16 lr=7.03e-05 tps=21012 mem=5.5G lam=0.31440695965041715 Gm=None +[s_only s3] step 10980/12000 loss=4.3279 gn=1.21 lr=6.99e-05 tps=21012 mem=5.5G lam=0.3144907004510363 Gm=None +[s_only s3] step 11000/12000 loss=4.3213 gn=1.15 lr=6.95e-05 tps=21012 mem=5.5G lam=0.3145651655892531 Gm=None +[s_only s3] EVAL step 11000: val_loss=4.3764 val_ppl=79.55 +[s_only s3] step 11020/12000 loss=4.2788 gn=1.17 lr=6.91e-05 tps=21005 mem=5.5G lam=0.3146486474821965 Gm=None +[s_only s3] step 11040/12000 loss=4.2776 gn=1.03 lr=6.88e-05 tps=21005 mem=5.5G lam=0.31471580856790143 Gm=None +[s_only s3] step 11060/12000 loss=4.4084 gn=1.12 lr=6.84e-05 tps=21005 mem=5.5G lam=0.3147904959817727 Gm=None +[s_only s3] step 11080/12000 loss=4.1436 gn=1.09 lr=6.81e-05 tps=21005 mem=5.5G lam=0.3148776063074668 Gm=None +[s_only s3] step 11100/12000 loss=4.3753 gn=1.27 lr=6.77e-05 tps=21005 mem=5.5G lam=0.314977222122252 Gm=None +[s_only s3] step 11120/12000 loss=4.3086 gn=1.02 lr=6.74e-05 tps=21005 mem=5.5G lam=0.3150577774892251 Gm=None +[s_only s3] step 11140/12000 loss=4.3277 gn=1.14 lr=6.70e-05 tps=21005 mem=5.5G lam=0.31513983476907015 Gm=None +[s_only s3] step 11160/12000 loss=4.3656 gn=1.14 lr=6.67e-05 tps=21005 mem=5.5G lam=0.31522727260986966 Gm=None +[s_only s3] step 11180/12000 loss=4.3089 gn=1.18 lr=6.64e-05 tps=21005 mem=5.5G lam=0.31530104267100495 Gm=None +[s_only s3] step 11200/12000 loss=4.2151 gn=1.23 lr=6.61e-05 tps=21006 mem=5.5G lam=0.3153944682950775 Gm=None +[s_only s3] step 11220/12000 loss=4.2767 gn=1.16 lr=6.58e-05 tps=21005 mem=5.5G lam=0.31546003309388954 Gm=None +[s_only s3] step 11240/12000 loss=4.3402 gn=1.11 lr=6.55e-05 tps=21006 mem=5.5G lam=0.3155176906536023 Gm=None +[s_only s3] step 11260/12000 loss=4.3441 gn=1.12 lr=6.52e-05 tps=21006 mem=5.5G lam=0.3155877587075035 Gm=None +[s_only s3] step 11280/12000 loss=4.2864 gn=1.21 lr=6.49e-05 tps=21006 mem=5.5G lam=0.31567253079265356 Gm=None +[s_only s3] step 11300/12000 loss=4.2830 gn=1.28 lr=6.47e-05 tps=21006 mem=5.5G lam=0.31575398209194344 Gm=None +[s_only s3] step 11320/12000 loss=4.3127 gn=1.15 lr=6.44e-05 tps=21006 mem=5.5G lam=0.3158643962815404 Gm=None +[s_only s3] step 11340/12000 loss=4.3633 gn=1.14 lr=6.42e-05 tps=21006 mem=5.5G lam=0.3159371443713705 Gm=None +[s_only s3] step 11360/12000 loss=4.4805 gn=1.22 lr=6.39e-05 tps=21006 mem=5.5G lam=0.3159923854594429 Gm=None +[s_only s3] step 11380/12000 loss=4.3110 gn=1.21 lr=6.37e-05 tps=21006 mem=5.5G lam=0.3160481195275982 Gm=None +[s_only s3] step 11400/12000 loss=4.3882 gn=1.22 lr=6.34e-05 tps=21006 mem=5.5G lam=0.3161244820803404 Gm=None +[s_only s3] step 11420/12000 loss=4.2950 gn=1.16 lr=6.32e-05 tps=21006 mem=5.5G lam=0.31620865128934383 Gm=None +[s_only s3] step 11440/12000 loss=4.3372 gn=1.16 lr=6.30e-05 tps=21006 mem=5.5G lam=0.3162989563619097 Gm=None +[s_only s3] step 11460/12000 loss=4.3631 gn=1.14 lr=6.28e-05 tps=21006 mem=5.5G lam=0.3163876834635933 Gm=None +[s_only s3] step 11480/12000 loss=4.3624 gn=1.31 lr=6.26e-05 tps=21007 mem=5.5G lam=0.3164501537879308 Gm=None +[s_only s3] step 11500/12000 loss=4.2652 gn=1.22 lr=6.24e-05 tps=21007 mem=5.5G lam=0.3165199737995863 Gm=None +[s_only s3] step 11520/12000 loss=4.1811 gn=1.15 lr=6.22e-05 tps=21007 mem=5.5G lam=0.316603130971392 Gm=None +[s_only s3] step 11540/12000 loss=4.4265 gn=1.12 lr=6.20e-05 tps=21007 mem=5.5G lam=0.31666958673546713 Gm=None +[s_only s3] step 11560/12000 loss=4.3366 gn=1.37 lr=6.19e-05 tps=21007 mem=5.5G lam=0.3167252391576767 Gm=None +[s_only s3] step 11580/12000 loss=4.1998 gn=1.13 lr=6.17e-05 tps=21007 mem=5.5G lam=0.3167978146423896 Gm=None +[s_only s3] step 11600/12000 loss=4.3330 gn=1.16 lr=6.15e-05 tps=21007 mem=5.5G lam=0.31688616580019396 Gm=None +[s_only s3] step 11620/12000 loss=4.2741 gn=1.13 lr=6.14e-05 tps=21007 mem=5.5G lam=0.3169640752797325 Gm=None +[s_only s3] step 11640/12000 loss=4.3376 gn=1.18 lr=6.12e-05 tps=21007 mem=5.5G lam=0.3170490333189567 Gm=None +[s_only s3] step 11660/12000 loss=4.4343 gn=1.20 lr=6.11e-05 tps=21007 mem=5.5G lam=0.31712669506669044 Gm=None +[s_only s3] step 11680/12000 loss=4.2094 gn=1.34 lr=6.10e-05 tps=21008 mem=5.5G lam=0.31721649784594774 Gm=None +[s_only s3] step 11700/12000 loss=4.3557 gn=1.15 lr=6.09e-05 tps=21007 mem=5.5G lam=0.31729846944411594 Gm=None +[s_only s3] step 11720/12000 loss=4.2835 gn=1.30 lr=6.07e-05 tps=21007 mem=5.5G lam=0.3173710924262802 Gm=None +[s_only s3] step 11740/12000 loss=4.3317 gn=1.24 lr=6.06e-05 tps=21008 mem=5.5G lam=0.317448900702099 Gm=None +[s_only s3] step 11760/12000 loss=4.3070 gn=1.17 lr=6.06e-05 tps=21008 mem=5.5G lam=0.31751856425156194 Gm=None +[s_only s3] step 11780/12000 loss=4.3688 gn=1.25 lr=6.05e-05 tps=21008 mem=5.5G lam=0.31759327184408903 Gm=None +[s_only s3] step 11800/12000 loss=4.3124 gn=1.23 lr=6.04e-05 tps=21008 mem=5.5G lam=0.31768871036668617 Gm=None +[s_only s3] step 11820/12000 loss=4.3524 gn=1.30 lr=6.03e-05 tps=21008 mem=5.5G lam=0.31775357325871784 Gm=None +[s_only s3] step 11840/12000 loss=4.2449 gn=1.15 lr=6.02e-05 tps=21009 mem=5.5G lam=0.31781917562087375 Gm=None +[s_only s3] step 11860/12000 loss=4.2664 gn=1.16 lr=6.02e-05 tps=21009 mem=5.5G lam=0.3178585882609089 Gm=None +[s_only s3] step 11880/12000 loss=4.4635 gn=1.30 lr=6.01e-05 tps=21009 mem=5.5G lam=0.31791513971984386 Gm=None +[s_only s3] step 11900/12000 loss=4.2846 gn=1.32 lr=6.01e-05 tps=21009 mem=5.5G lam=0.3179887297252814 Gm=None +[s_only s3] step 11920/12000 loss=4.2624 gn=1.23 lr=6.01e-05 tps=21009 mem=5.5G lam=0.3180524256701271 Gm=None +[s_only s3] step 11940/12000 loss=4.4075 gn=1.33 lr=6.00e-05 tps=21009 mem=5.5G lam=0.3181242533028126 Gm=None +[s_only s3] step 11960/12000 loss=4.2365 gn=1.18 lr=6.00e-05 tps=21009 mem=5.5G lam=0.31818261711547774 Gm=None +[s_only s3] step 11980/12000 loss=4.2919 gn=1.25 lr=6.00e-05 tps=21010 mem=5.5G lam=0.3182733968521158 Gm=None +[s_only s3] EVAL step 11999: val_loss=4.3565 val_ppl=77.98 +[init] variant=g_only seed=3 params=63.39M +[init] use_S=False use_G=True +[g_only s3] step 0/12000 loss=10.8757 gn=9.68 lr=3.00e-06 tps=8741 mem=5.0G lam=None Gm=0.4998372395833333 +[g_only s3] step 20/12000 loss=9.8379 gn=1.83 lr=6.30e-05 tps=21289 mem=5.5G lam=None Gm=0.4657389322916667 +[g_only s3] step 40/12000 loss=9.1136 gn=1.75 lr=1.23e-04 tps=21997 mem=5.5G lam=None Gm=0.452880859375 +[g_only s3] step 60/12000 loss=8.1876 gn=1.58 lr=1.83e-04 tps=22323 mem=5.5G lam=None Gm=0.4429524739583333 +[g_only s3] step 80/12000 loss=7.5293 gn=0.79 lr=2.43e-04 tps=22488 mem=5.5G lam=None Gm=0.4395345052083333 +[g_only s3] step 100/12000 loss=7.1120 gn=0.76 lr=3.03e-04 tps=22600 mem=5.5G lam=None Gm=0.430908203125 +[g_only s3] step 120/12000 loss=7.1915 gn=0.75 lr=3.63e-04 tps=22664 mem=5.5G lam=None Gm=0.4300130208333333 +[g_only s3] step 140/12000 loss=6.8942 gn=0.61 lr=4.23e-04 tps=22713 mem=5.5G lam=None Gm=0.421875 +[g_only s3] step 160/12000 loss=6.8625 gn=0.72 lr=4.83e-04 tps=22745 mem=5.5G lam=None Gm=0.4200846354166667 +[g_only s3] step 180/12000 loss=6.7406 gn=0.51 lr=5.43e-04 tps=22764 mem=5.5G lam=None Gm=0.4044596354166667 +[g_only s3] step 200/12000 loss=6.6746 gn=0.59 lr=6.00e-04 tps=22782 mem=5.5G lam=None Gm=0.42236328125 +[g_only s3] step 220/12000 loss=6.5995 gn=0.53 lr=6.00e-04 tps=22792 mem=5.5G lam=None Gm=0.3978678385416667 +[g_only s3] step 240/12000 loss=6.6137 gn=0.48 lr=6.00e-04 tps=22803 mem=5.5G lam=None Gm=0.3998209635416667 +[g_only s3] step 260/12000 loss=6.6120 gn=0.48 lr=6.00e-04 tps=22813 mem=5.5G lam=None Gm=0.40771484375 +[g_only s3] step 280/12000 loss=6.4401 gn=0.54 lr=6.00e-04 tps=22817 mem=5.5G lam=None Gm=0.4283854166666667 +[g_only s3] step 300/12000 loss=6.3312 gn=0.49 lr=6.00e-04 tps=22824 mem=5.5G lam=None Gm=0.4000651041666667 +[g_only s3] step 320/12000 loss=6.4253 gn=0.48 lr=6.00e-04 tps=22816 mem=5.5G lam=None Gm=0.3999837239583333 +[g_only s3] step 340/12000 loss=6.4758 gn=0.50 lr=6.00e-04 tps=22823 mem=5.5G lam=None Gm=0.4183756510416667 +[g_only s3] step 360/12000 loss=6.2547 gn=0.46 lr=6.00e-04 tps=22831 mem=5.5G lam=None Gm=0.3988444010416667 +[g_only s3] step 380/12000 loss=6.3012 gn=0.43 lr=6.00e-04 tps=22835 mem=5.5G lam=None Gm=0.4153645833333333 +[g_only s3] step 400/12000 loss=6.3643 gn=0.50 lr=6.00e-04 tps=22841 mem=5.5G lam=None Gm=0.4493001302083333 +[g_only s3] step 420/12000 loss=6.2832 gn=0.43 lr=6.00e-04 tps=22844 mem=5.5G lam=None Gm=0.4436848958333333 +[g_only s3] step 440/12000 loss=6.2207 gn=0.51 lr=5.99e-04 tps=22844 mem=5.5G lam=None Gm=0.4371744791666667 +[g_only s3] step 460/12000 loss=6.3509 gn=0.50 lr=5.99e-04 tps=22851 mem=5.5G lam=None Gm=0.4539388020833333 +[g_only s3] step 480/12000 loss=6.2145 gn=0.49 lr=5.99e-04 tps=22856 mem=5.5G lam=None Gm=0.4609375 +[g_only s3] step 500/12000 loss=6.2405 gn=0.49 lr=5.99e-04 tps=22857 mem=5.5G lam=None Gm=0.489501953125 +[g_only s3] step 520/12000 loss=6.2552 gn=0.52 lr=5.99e-04 tps=22858 mem=5.5G lam=None Gm=0.5037434895833334 +[g_only s3] step 540/12000 loss=6.1605 gn=0.47 lr=5.99e-04 tps=22861 mem=5.5G lam=None Gm=0.5077311197916666 +[g_only s3] step 560/12000 loss=6.2051 gn=0.55 lr=5.99e-04 tps=22863 mem=5.5G lam=None Gm=0.5309244791666666 +[g_only s3] step 580/12000 loss=6.1924 gn=0.46 lr=5.99e-04 tps=22866 mem=5.5G lam=None Gm=0.5105794270833334 +[g_only s3] step 600/12000 loss=6.0824 gn=0.43 lr=5.98e-04 tps=22866 mem=5.5G lam=None Gm=0.53466796875 +[g_only s3] step 620/12000 loss=5.9951 gn=0.49 lr=5.98e-04 tps=22862 mem=5.5G lam=None Gm=0.5511067708333334 +[g_only s3] step 640/12000 loss=6.0967 gn=0.49 lr=5.98e-04 tps=22862 mem=5.5G lam=None Gm=0.5537109375 +[g_only s3] step 660/12000 loss=5.8861 gn=0.47 lr=5.98e-04 tps=22861 mem=5.5G lam=None Gm=0.5729166666666666 +[g_only s3] step 680/12000 loss=6.0085 gn=0.47 lr=5.98e-04 tps=22861 mem=5.5G lam=None Gm=0.5673828125 +[g_only s3] step 700/12000 loss=6.0484 gn=0.49 lr=5.98e-04 tps=22865 mem=5.5G lam=None Gm=0.5575358072916666 +[g_only s3] step 720/12000 loss=5.8607 gn=0.46 lr=5.97e-04 tps=22862 mem=5.5G lam=None Gm=0.571044921875 +[g_only s3] step 740/12000 loss=6.0259 gn=0.47 lr=5.97e-04 tps=22865 mem=5.5G lam=None Gm=0.5669759114583334 +[g_only s3] step 760/12000 loss=5.9187 gn=0.47 lr=5.97e-04 tps=22866 mem=5.5G lam=None Gm=0.5955403645833334 +[g_only s3] step 780/12000 loss=5.9931 gn=0.54 lr=5.97e-04 tps=22867 mem=5.5G lam=None Gm=0.6043294270833334 +[g_only s3] step 800/12000 loss=5.9364 gn=0.46 lr=5.97e-04 tps=22870 mem=5.5G lam=None Gm=0.6045735677083334 +[g_only s3] step 820/12000 loss=5.8470 gn=0.50 lr=5.96e-04 tps=22872 mem=5.5G lam=None Gm=0.5884602864583334 +[g_only s3] step 840/12000 loss=5.8712 gn=0.51 lr=5.96e-04 tps=22873 mem=5.5G lam=None Gm=0.6038411458333334 +[g_only s3] step 860/12000 loss=5.8391 gn=0.49 lr=5.96e-04 tps=22873 mem=5.5G lam=None Gm=0.6110026041666666 +[g_only s3] step 880/12000 loss=5.9674 gn=0.52 lr=5.96e-04 tps=22873 mem=5.5G lam=None Gm=0.6027018229166666 +[g_only s3] step 900/12000 loss=5.7725 gn=0.51 lr=5.95e-04 tps=22874 mem=5.5G lam=None Gm=0.596435546875 +[g_only s3] step 920/12000 loss=5.8033 gn=0.51 lr=5.95e-04 tps=22876 mem=5.5G lam=None Gm=0.6217447916666666 +[g_only s3] step 940/12000 loss=5.6408 gn=0.50 lr=5.95e-04 tps=22846 mem=5.5G lam=None Gm=0.5756022135416666 +[g_only s3] step 960/12000 loss=5.8059 gn=0.50 lr=5.94e-04 tps=22847 mem=5.5G lam=None Gm=0.5938313802083334 +[g_only s3] step 980/12000 loss=5.7180 gn=0.46 lr=5.94e-04 tps=22847 mem=5.5G lam=None Gm=0.579345703125 +[g_only s3] step 1000/12000 loss=5.8888 gn=0.54 lr=5.94e-04 tps=22849 mem=5.5G lam=None Gm=0.5968831380208334 +[g_only s3] EVAL step 1000: val_loss=5.8372 val_ppl=342.83 +[g_only s3] step 1020/12000 loss=5.7369 gn=0.49 lr=5.94e-04 tps=22771 mem=5.5G lam=None Gm=0.5802001953125 +[g_only s3] step 1040/12000 loss=5.6482 gn=0.54 lr=5.93e-04 tps=22771 mem=5.5G lam=None Gm=0.60302734375 +[g_only s3] step 1060/12000 loss=5.6283 gn=0.52 lr=5.93e-04 tps=22774 mem=5.5G lam=None Gm=0.5662434895833334 +[g_only s3] step 1080/12000 loss=5.6945 gn=0.47 lr=5.93e-04 tps=22777 mem=5.5G lam=None Gm=0.5757242838541666 +[g_only s3] step 1100/12000 loss=5.6585 gn=0.57 lr=5.92e-04 tps=22778 mem=5.5G lam=None Gm=0.569091796875 +[g_only s3] step 1120/12000 loss=5.7428 gn=0.52 lr=5.92e-04 tps=22781 mem=5.5G lam=None Gm=0.5704752604166666 +[g_only s3] step 1140/12000 loss=5.6282 gn=0.50 lr=5.92e-04 tps=22783 mem=5.5G lam=None Gm=0.5753580729166666 +[g_only s3] step 1160/12000 loss=5.6916 gn=0.52 lr=5.91e-04 tps=22785 mem=5.5G lam=None Gm=0.5681966145833334 +[g_only s3] step 1180/12000 loss=5.6202 gn=0.53 lr=5.91e-04 tps=22788 mem=5.5G lam=None Gm=0.5658365885416666 +[g_only s3] step 1200/12000 loss=5.6872 gn=0.56 lr=5.90e-04 tps=22791 mem=5.5G lam=None Gm=0.5716145833333334 +[g_only s3] step 1220/12000 loss=5.4847 gn=0.51 lr=5.90e-04 tps=22794 mem=5.5G lam=None Gm=0.5725911458333334 +[g_only s3] step 1240/12000 loss=5.5462 gn=0.53 lr=5.90e-04 tps=22796 mem=5.5G lam=None Gm=0.5689697265625 +[g_only s3] step 1260/12000 loss=5.6990 gn=0.51 lr=5.89e-04 tps=22798 mem=5.5G lam=None Gm=0.5628662109375 +[g_only s3] step 1280/12000 loss=5.5625 gn=0.58 lr=5.89e-04 tps=22800 mem=5.5G lam=None Gm=0.5513509114583334 +[g_only s3] step 1300/12000 loss=5.6547 gn=0.51 lr=5.89e-04 tps=22801 mem=5.5G lam=None Gm=0.5518798828125 +[g_only s3] step 1320/12000 loss=5.5693 gn=0.59 lr=5.88e-04 tps=22803 mem=5.5G lam=None Gm=0.5572102864583334 +[g_only s3] step 1340/12000 loss=5.6097 gn=0.51 lr=5.88e-04 tps=22804 mem=5.5G lam=None Gm=0.55322265625 +[g_only s3] step 1360/12000 loss=5.4639 gn=0.53 lr=5.87e-04 tps=22804 mem=5.5G lam=None Gm=0.5423583984375 +[g_only s3] step 1380/12000 loss=5.4138 gn=0.51 lr=5.87e-04 tps=22806 mem=5.5G lam=None Gm=0.5546875 +[g_only s3] step 1400/12000 loss=5.6081 gn=0.56 lr=5.86e-04 tps=22807 mem=5.5G lam=None Gm=0.5486246744791666 +[g_only s3] step 1420/12000 loss=5.4698 gn=0.58 lr=5.86e-04 tps=22809 mem=5.5G lam=None Gm=0.5527750651041666 +[g_only s3] step 1440/12000 loss=5.3568 gn=0.50 lr=5.85e-04 tps=22811 mem=5.5G lam=None Gm=0.5410970052083334 +[g_only s3] step 1460/12000 loss=5.5341 gn=0.57 lr=5.85e-04 tps=22812 mem=5.5G lam=None Gm=0.5361328125 +[g_only s3] step 1480/12000 loss=5.3796 gn=0.56 lr=5.84e-04 tps=22813 mem=5.5G lam=None Gm=0.53466796875 +[g_only s3] step 1500/12000 loss=5.5036 gn=0.58 lr=5.84e-04 tps=22815 mem=5.5G lam=None Gm=0.5312296549479166 +[g_only s3] step 1520/12000 loss=5.4808 gn=0.55 lr=5.83e-04 tps=22815 mem=5.5G lam=None Gm=0.5228678385416666 +[g_only s3] step 1540/12000 loss=5.3413 gn=0.50 lr=5.83e-04 tps=22816 mem=5.5G lam=None Gm=0.5287068684895834 +[g_only s3] step 1560/12000 loss=5.4081 gn=0.52 lr=5.82e-04 tps=22815 mem=5.5G lam=None Gm=0.5293375651041666 +[g_only s3] step 1580/12000 loss=5.4833 gn=0.52 lr=5.82e-04 tps=22818 mem=5.5G lam=None Gm=0.5349934895833334 +[g_only s3] step 1600/12000 loss=5.2594 gn=0.52 lr=5.81e-04 tps=22819 mem=5.5G lam=None Gm=0.5268351236979166 +[g_only s3] step 1620/12000 loss=5.4752 gn=0.56 lr=5.81e-04 tps=22821 mem=5.5G lam=None Gm=0.5283406575520834 +[g_only s3] step 1640/12000 loss=5.3453 gn=0.55 lr=5.80e-04 tps=22823 mem=5.5G lam=None Gm=0.5275675455729166 +[g_only s3] step 1660/12000 loss=5.3492 gn=0.56 lr=5.80e-04 tps=22824 mem=5.5G lam=None Gm=0.5144246419270834 +[g_only s3] step 1680/12000 loss=5.3741 gn=0.58 lr=5.79e-04 tps=22826 mem=5.5G lam=None Gm=0.5149739583333334 +[g_only s3] step 1700/12000 loss=5.5275 gn=0.56 lr=5.79e-04 tps=22826 mem=5.5G lam=None Gm=0.5272013346354166 +[g_only s3] step 1720/12000 loss=5.3904 gn=0.55 lr=5.78e-04 tps=22827 mem=5.5G lam=None Gm=0.5162353515625 +[g_only s3] step 1740/12000 loss=5.3222 gn=0.54 lr=5.78e-04 tps=22828 mem=5.5G lam=None Gm=0.5182698567708334 +[g_only s3] step 1760/12000 loss=5.2653 gn=0.56 lr=5.77e-04 tps=22827 mem=5.5G lam=None Gm=0.5232950846354166 +[g_only s3] step 1780/12000 loss=5.4138 gn=0.56 lr=5.76e-04 tps=22828 mem=5.5G lam=None Gm=0.5093790690104166 +[g_only s3] step 1800/12000 loss=5.3249 gn=0.58 lr=5.76e-04 tps=22829 mem=5.5G lam=None Gm=0.52410888671875 +[g_only s3] step 1820/12000 loss=5.2641 gn=0.55 lr=5.75e-04 tps=22830 mem=5.5G lam=None Gm=0.51654052734375 +[g_only s3] step 1840/12000 loss=5.4317 gn=0.60 lr=5.75e-04 tps=22830 mem=5.5G lam=None Gm=0.5089925130208334 +[g_only s3] step 1860/12000 loss=5.3999 gn=0.58 lr=5.74e-04 tps=22831 mem=5.5G lam=None Gm=0.5112711588541666 +[g_only s3] step 1880/12000 loss=5.2872 gn=0.55 lr=5.73e-04 tps=22832 mem=5.5G lam=None Gm=0.5074259440104166 +[g_only s3] step 1900/12000 loss=5.1876 gn=0.55 lr=5.73e-04 tps=22833 mem=5.5G lam=None Gm=0.5054931640625 +[g_only s3] step 1920/12000 loss=5.3007 gn=0.55 lr=5.72e-04 tps=22834 mem=5.5G lam=None Gm=0.4974365234375 +[g_only s3] step 1940/12000 loss=5.4023 gn=0.54 lr=5.72e-04 tps=22835 mem=5.5G lam=None Gm=0.4929606119791667 +[g_only s3] step 1960/12000 loss=5.3542 gn=0.57 lr=5.71e-04 tps=22836 mem=5.5G lam=None Gm=0.4899698893229167 +[g_only s3] step 1980/12000 loss=5.1029 gn=0.56 lr=5.70e-04 tps=22837 mem=5.5G lam=None Gm=0.4876505533854167 +[g_only s3] step 2000/12000 loss=5.3456 gn=0.55 lr=5.70e-04 tps=22837 mem=5.5G lam=None Gm=0.4908854166666667 +[g_only s3] EVAL step 2000: val_loss=5.3507 val_ppl=210.75 +[g_only s3] step 2020/12000 loss=5.3575 gn=0.55 lr=5.69e-04 tps=22799 mem=5.5G lam=None Gm=0.4874267578125 +[g_only s3] step 2040/12000 loss=5.3892 gn=0.55 lr=5.68e-04 tps=22800 mem=5.5G lam=None Gm=0.4888102213541667 +[g_only s3] step 2060/12000 loss=5.2831 gn=0.61 lr=5.68e-04 tps=22801 mem=5.5G lam=None Gm=0.4858194986979167 +[g_only s3] step 2080/12000 loss=5.1932 gn=0.56 lr=5.67e-04 tps=22801 mem=5.5G lam=None Gm=0.4765625 +[g_only s3] step 2100/12000 loss=5.2693 gn=0.55 lr=5.66e-04 tps=22802 mem=5.5G lam=None Gm=0.4853922526041667 +[g_only s3] step 2120/12000 loss=5.3238 gn=0.58 lr=5.65e-04 tps=22805 mem=5.5G lam=None Gm=0.4911092122395833 +[g_only s3] step 2140/12000 loss=5.1176 gn=0.56 lr=5.65e-04 tps=22805 mem=5.5G lam=None Gm=0.4828084309895833 +[g_only s3] step 2160/12000 loss=5.0942 gn=0.58 lr=5.64e-04 tps=22806 mem=5.5G lam=None Gm=0.4670613606770833 +[g_only s3] step 2180/12000 loss=5.1370 gn=0.62 lr=5.63e-04 tps=22808 mem=5.5G lam=None Gm=0.48150634765625 +[g_only s3] step 2200/12000 loss=5.3319 gn=0.59 lr=5.63e-04 tps=22809 mem=5.5G lam=None Gm=0.4729817708333333 +[g_only s3] step 2220/12000 loss=5.2764 gn=0.60 lr=5.62e-04 tps=22808 mem=5.5G lam=None Gm=0.4620157877604167 +[g_only s3] step 2240/12000 loss=5.2580 gn=0.65 lr=5.61e-04 tps=22810 mem=5.5G lam=None Gm=0.4705403645833333 +[g_only s3] step 2260/12000 loss=5.1206 gn=0.64 lr=5.60e-04 tps=22810 mem=5.5G lam=None Gm=0.476348876953125 +[g_only s3] step 2280/12000 loss=5.2368 gn=0.79 lr=5.60e-04 tps=22811 mem=5.5G lam=None Gm=0.4753926595052083 +[g_only s3] step 2300/12000 loss=5.2434 gn=0.58 lr=5.59e-04 tps=22811 mem=5.5G lam=None Gm=0.4723103841145833 +[g_only s3] step 2320/12000 loss=5.1854 gn=0.60 lr=5.58e-04 tps=22813 mem=5.5G lam=None Gm=0.4684041341145833 +[g_only s3] step 2340/12000 loss=4.9663 gn=0.61 lr=5.57e-04 tps=22813 mem=5.5G lam=None Gm=0.4742533365885417 +[g_only s3] step 2360/12000 loss=5.2013 gn=0.61 lr=5.57e-04 tps=22812 mem=5.5G lam=None Gm=0.4731953938802083 +[g_only s3] step 2380/12000 loss=5.1285 gn=0.67 lr=5.56e-04 tps=22813 mem=5.5G lam=None Gm=0.4737548828125 +[g_only s3] step 2400/12000 loss=5.0776 gn=0.70 lr=5.55e-04 tps=22813 mem=5.5G lam=None Gm=0.4788411458333333 +[g_only s3] step 2420/12000 loss=5.1082 gn=0.68 lr=5.54e-04 tps=22814 mem=5.5G lam=None Gm=0.4533894856770833 +[g_only s3] step 2440/12000 loss=5.1303 gn=0.63 lr=5.53e-04 tps=22815 mem=5.5G lam=None Gm=0.4674072265625 +[g_only s3] step 2460/12000 loss=5.0783 gn=0.59 lr=5.53e-04 tps=22813 mem=5.5G lam=None Gm=0.4534912109375 +[g_only s3] step 2480/12000 loss=5.0938 gn=0.62 lr=5.52e-04 tps=22814 mem=5.5G lam=None Gm=0.4597981770833333 +[g_only s3] step 2500/12000 loss=5.1236 gn=0.63 lr=5.51e-04 tps=22815 mem=5.5G lam=None Gm=0.4583841959635417 +[g_only s3] step 2520/12000 loss=5.2023 gn=0.63 lr=5.50e-04 tps=22815 mem=5.5G lam=None Gm=0.4536234537760417 +[g_only s3] step 2540/12000 loss=5.0587 gn=0.60 lr=5.49e-04 tps=22816 mem=5.5G lam=None Gm=0.4603068033854167 +[g_only s3] step 2560/12000 loss=5.1693 gn=0.64 lr=5.48e-04 tps=22818 mem=5.5G lam=None Gm=0.466522216796875 +[g_only s3] step 2580/12000 loss=5.0341 gn=0.64 lr=5.48e-04 tps=22818 mem=5.5G lam=None Gm=0.4557596842447917 +[g_only s3] step 2600/12000 loss=5.0893 gn=0.66 lr=5.47e-04 tps=22819 mem=5.5G lam=None Gm=0.463836669921875 +[g_only s3] step 2620/12000 loss=5.0053 gn=0.64 lr=5.46e-04 tps=22820 mem=5.5G lam=None Gm=0.4479268391927083 +[g_only s3] step 2640/12000 loss=4.9326 gn=0.68 lr=5.45e-04 tps=22820 mem=5.5G lam=None Gm=0.4512736002604167 +[g_only s3] step 2660/12000 loss=5.0601 gn=0.60 lr=5.44e-04 tps=22821 mem=5.5G lam=None Gm=0.4517110188802083 +[g_only s3] step 2680/12000 loss=5.0370 gn=0.58 lr=5.43e-04 tps=22822 mem=5.5G lam=None Gm=0.4580586751302083 +[g_only s3] step 2700/12000 loss=5.0348 gn=0.62 lr=5.42e-04 tps=22822 mem=5.5G lam=None Gm=0.4495340983072917 +[g_only s3] step 2720/12000 loss=5.1068 gn=0.73 lr=5.41e-04 tps=22823 mem=5.5G lam=None Gm=0.4566243489583333 +[g_only s3] step 2740/12000 loss=5.0778 gn=0.62 lr=5.41e-04 tps=22824 mem=5.5G lam=None Gm=0.4525248209635417 +[g_only s3] step 2760/12000 loss=5.0945 gn=0.68 lr=5.40e-04 tps=22824 mem=5.5G lam=None Gm=0.451171875 +[g_only s3] step 2780/12000 loss=5.0141 gn=0.66 lr=5.39e-04 tps=22826 mem=5.5G lam=None Gm=0.4556172688802083 +[g_only s3] step 2800/12000 loss=4.9787 gn=0.60 lr=5.38e-04 tps=22827 mem=5.5G lam=None Gm=0.4503275553385417 +[g_only s3] step 2820/12000 loss=5.1224 gn=0.64 lr=5.37e-04 tps=22827 mem=5.5G lam=None Gm=0.447479248046875 +[g_only s3] step 2840/12000 loss=5.0936 gn=0.63 lr=5.36e-04 tps=22829 mem=5.5G lam=None Gm=0.4444783528645833 +[g_only s3] step 2860/12000 loss=4.9397 gn=0.67 lr=5.35e-04 tps=22829 mem=5.5G lam=None Gm=0.454376220703125 +[g_only s3] step 2880/12000 loss=5.0009 gn=0.64 lr=5.34e-04 tps=22829 mem=5.5G lam=None Gm=0.4485066731770833 +[g_only s3] step 2900/12000 loss=4.8393 gn=0.61 lr=5.33e-04 tps=22830 mem=5.5G lam=None Gm=0.4518534342447917 +[g_only s3] step 2920/12000 loss=4.9500 gn=0.64 lr=5.32e-04 tps=22831 mem=5.5G lam=None Gm=0.4477640787760417 +[g_only s3] step 2940/12000 loss=5.0014 gn=0.61 lr=5.31e-04 tps=22831 mem=5.5G lam=None Gm=0.44293212890625 +[g_only s3] step 2960/12000 loss=4.8489 gn=0.68 lr=5.30e-04 tps=22832 mem=5.5G lam=None Gm=0.452545166015625 +[g_only s3] step 2980/12000 loss=4.7656 gn=0.99 lr=5.29e-04 tps=22832 mem=5.5G lam=None Gm=0.4368489583333333 +[g_only s3] step 3000/12000 loss=5.0423 gn=0.71 lr=5.28e-04 tps=22832 mem=5.5G lam=None Gm=0.44708251953125 +[g_only s3] EVAL step 3000: val_loss=5.0072 val_ppl=149.48 +[g_only s3] step 3020/12000 loss=5.1063 gn=0.71 lr=5.27e-04 tps=22807 mem=5.5G lam=None Gm=0.4429728190104167 +[g_only s3] step 3040/12000 loss=4.8311 gn=0.63 lr=5.26e-04 tps=22807 mem=5.5G lam=None Gm=0.436279296875 +[g_only s3] step 3060/12000 loss=4.9092 gn=0.64 lr=5.25e-04 tps=22807 mem=5.5G lam=None Gm=0.4434000651041667 +[g_only s3] step 3080/12000 loss=4.8516 gn=0.66 lr=5.24e-04 tps=22808 mem=5.5G lam=None Gm=0.4447123209635417 +[g_only s3] step 3100/12000 loss=5.0112 gn=0.74 lr=5.23e-04 tps=22808 mem=5.5G lam=None Gm=0.4425862630208333 +[g_only s3] step 3120/12000 loss=4.7611 gn=0.72 lr=5.22e-04 tps=22809 mem=5.5G lam=None Gm=0.4352518717447917 +[g_only s3] step 3140/12000 loss=4.8983 gn=0.66 lr=5.21e-04 tps=22810 mem=5.5G lam=None Gm=0.445343017578125 +[g_only s3] step 3160/12000 loss=4.8327 gn=0.71 lr=5.20e-04 tps=22811 mem=5.5G lam=None Gm=0.4369913736979167 +[g_only s3] step 3180/12000 loss=4.8705 gn=0.69 lr=5.19e-04 tps=22812 mem=5.5G lam=None Gm=0.4369303385416667 +[g_only s3] step 3200/12000 loss=4.8495 gn=0.65 lr=5.18e-04 tps=22812 mem=5.5G lam=None Gm=0.4281005859375 +[g_only s3] step 3220/12000 loss=4.9444 gn=0.68 lr=5.17e-04 tps=22813 mem=5.5G lam=None Gm=0.4395039876302083 +[g_only s3] step 3240/12000 loss=4.9310 gn=0.66 lr=5.16e-04 tps=22813 mem=5.5G lam=None Gm=0.4372151692708333 +[g_only s3] step 3260/12000 loss=4.9875 gn=0.68 lr=5.15e-04 tps=22813 mem=5.5G lam=None Gm=0.438079833984375 +[g_only s3] step 3280/12000 loss=4.8005 gn=0.63 lr=5.14e-04 tps=22814 mem=5.5G lam=None Gm=0.4372660319010417 +[g_only s3] step 3300/12000 loss=5.0691 gn=0.68 lr=5.13e-04 tps=22813 mem=5.5G lam=None Gm=0.4263916015625 +[g_only s3] step 3320/12000 loss=4.9128 gn=0.68 lr=5.12e-04 tps=22813 mem=5.5G lam=None Gm=0.4330647786458333 +[g_only s3] step 3340/12000 loss=4.8751 gn=0.69 lr=5.11e-04 tps=22814 mem=5.5G lam=None Gm=0.4308980305989583 +[g_only s3] step 3360/12000 loss=4.9072 gn=0.65 lr=5.10e-04 tps=22815 mem=5.5G lam=None Gm=0.431793212890625 +[g_only s3] step 3380/12000 loss=5.0364 gn=0.70 lr=5.09e-04 tps=22816 mem=5.5G lam=None Gm=0.4320576985677083 +[g_only s3] step 3400/12000 loss=4.9772 gn=0.67 lr=5.08e-04 tps=22817 mem=5.5G lam=None Gm=0.4368184407552083 +[g_only s3] step 3420/12000 loss=4.9119 gn=0.69 lr=5.07e-04 tps=22817 mem=5.5G lam=None Gm=0.4283955891927083 +[g_only s3] step 3440/12000 loss=4.7125 gn=0.65 lr=5.06e-04 tps=22818 mem=5.5G lam=None Gm=0.4224650065104167 +[g_only s3] step 3460/12000 loss=4.9407 gn=0.68 lr=5.05e-04 tps=22818 mem=5.5G lam=None Gm=0.4336140950520833 +[g_only s3] step 3480/12000 loss=4.9475 gn=0.74 lr=5.03e-04 tps=22818 mem=5.5G lam=None Gm=0.4278564453125 +[g_only s3] step 3500/12000 loss=5.0538 gn=0.68 lr=5.02e-04 tps=22818 mem=5.5G lam=None Gm=0.4419148763020833 +[g_only s3] step 3520/12000 loss=4.8168 gn=0.68 lr=5.01e-04 tps=22819 mem=5.5G lam=None Gm=0.4314676920572917 +[g_only s3] step 3540/12000 loss=4.8347 gn=0.66 lr=5.00e-04 tps=22820 mem=5.5G lam=None Gm=0.418548583984375 +[g_only s3] step 3560/12000 loss=4.8868 gn=0.71 lr=4.99e-04 tps=22821 mem=5.5G lam=None Gm=0.422271728515625 +[g_only s3] step 3580/12000 loss=4.9078 gn=0.71 lr=4.98e-04 tps=22821 mem=5.5G lam=None Gm=0.4254964192708333 +[g_only s3] step 3600/12000 loss=4.8892 gn=0.69 lr=4.97e-04 tps=22820 mem=5.5G lam=None Gm=0.4209391276041667 +[g_only s3] step 3620/12000 loss=4.8935 gn=0.68 lr=4.96e-04 tps=22821 mem=5.5G lam=None Gm=0.422454833984375 +[g_only s3] step 3640/12000 loss=4.7890 gn=0.65 lr=4.94e-04 tps=22821 mem=5.5G lam=None Gm=0.42218017578125 +[g_only s3] step 3660/12000 loss=5.0048 gn=0.70 lr=4.93e-04 tps=22821 mem=5.5G lam=None Gm=0.42950439453125 +[g_only s3] step 3680/12000 loss=4.8322 gn=0.82 lr=4.92e-04 tps=22822 mem=5.5G lam=None Gm=0.422943115234375 +[g_only s3] step 3700/12000 loss=4.9836 gn=0.72 lr=4.91e-04 tps=22822 mem=5.5G lam=None Gm=0.427398681640625 +[g_only s3] step 3720/12000 loss=4.9458 gn=0.73 lr=4.90e-04 tps=22823 mem=5.5G lam=None Gm=0.4256490071614583 +[g_only s3] step 3740/12000 loss=4.8723 gn=0.74 lr=4.89e-04 tps=22823 mem=5.5G lam=None Gm=0.424468994140625 +[g_only s3] step 3760/12000 loss=4.8010 gn=0.70 lr=4.88e-04 tps=22824 mem=5.5G lam=None Gm=0.4154510498046875 +[g_only s3] step 3780/12000 loss=4.7639 gn=0.74 lr=4.86e-04 tps=22824 mem=5.5G lam=None Gm=0.4186299641927083 +[g_only s3] step 3800/12000 loss=4.8932 gn=0.69 lr=4.85e-04 tps=22824 mem=5.5G lam=None Gm=0.4244283040364583 +[g_only s3] step 3820/12000 loss=5.0038 gn=0.74 lr=4.84e-04 tps=22825 mem=5.5G lam=None Gm=0.4277852376302083 +[g_only s3] step 3840/12000 loss=4.9042 gn=0.71 lr=4.83e-04 tps=22825 mem=5.5G lam=None Gm=0.4226481119791667 +[g_only s3] step 3860/12000 loss=4.8623 gn=0.67 lr=4.82e-04 tps=22824 mem=5.5G lam=None Gm=0.413299560546875 +[g_only s3] step 3880/12000 loss=4.8815 gn=0.70 lr=4.80e-04 tps=22822 mem=5.5G lam=None Gm=0.4314982096354167 +[g_only s3] step 3900/12000 loss=4.9002 gn=0.69 lr=4.79e-04 tps=22821 mem=5.5G lam=None Gm=0.4247538248697917 +[g_only s3] step 3920/12000 loss=4.8627 gn=0.66 lr=4.78e-04 tps=22819 mem=5.5G lam=None Gm=0.4154103597005208 +[g_only s3] step 3940/12000 loss=4.7456 gn=0.71 lr=4.77e-04 tps=22817 mem=5.5G lam=None Gm=0.4094696044921875 +[g_only s3] step 3960/12000 loss=4.7379 gn=0.76 lr=4.76e-04 tps=22815 mem=5.5G lam=None Gm=0.4175059000651042 +[g_only s3] step 3980/12000 loss=4.8237 gn=0.70 lr=4.74e-04 tps=22813 mem=5.5G lam=None Gm=0.4167734781901042 +[g_only s3] step 4000/12000 loss=4.7505 gn=0.72 lr=4.73e-04 tps=22811 mem=5.5G lam=None Gm=0.423309326171875 +[g_only s3] EVAL step 4000: val_loss=4.8825 val_ppl=131.96 +[g_only s3] step 4020/12000 loss=4.7121 gn=0.70 lr=4.72e-04 tps=22789 mem=5.5G lam=None Gm=0.40753173828125 +[g_only s3] step 4040/12000 loss=5.0915 gn=0.72 lr=4.71e-04 tps=22788 mem=5.5G lam=None Gm=0.4153238932291667 +[g_only s3] step 4060/12000 loss=4.7406 gn=0.72 lr=4.70e-04 tps=22785 mem=5.5G lam=None Gm=0.4163818359375 +[g_only s3] step 4080/12000 loss=4.8865 gn=0.75 lr=4.68e-04 tps=22783 mem=5.5G lam=None Gm=0.417144775390625 +[g_only s3] step 4100/12000 loss=4.8184 gn=0.70 lr=4.67e-04 tps=22782 mem=5.5G lam=None Gm=0.4085489908854167 +[g_only s3] step 4120/12000 loss=4.7830 gn=0.72 lr=4.66e-04 tps=22780 mem=5.5G lam=None Gm=0.4178873697916667 +[g_only s3] step 4140/12000 loss=4.8094 gn=0.64 lr=4.65e-04 tps=22780 mem=5.5G lam=None Gm=0.41571044921875 +[g_only s3] step 4160/12000 loss=4.7576 gn=0.74 lr=4.63e-04 tps=22779 mem=5.5G lam=None Gm=0.410675048828125 +[g_only s3] step 4180/12000 loss=4.7271 gn=0.68 lr=4.62e-04 tps=22778 mem=5.5G lam=None Gm=0.4163411458333333 +[g_only s3] step 4200/12000 loss=4.8281 gn=0.69 lr=4.61e-04 tps=22776 mem=5.5G lam=None Gm=0.4108734130859375 +[g_only s3] step 4220/12000 loss=4.6619 gn=0.71 lr=4.60e-04 tps=22774 mem=5.5G lam=None Gm=0.4121449788411458 +[g_only s3] step 4240/12000 loss=4.8447 gn=0.72 lr=4.58e-04 tps=22772 mem=5.5G lam=None Gm=0.4267069498697917 +[g_only s3] step 4260/12000 loss=4.8800 gn=0.74 lr=4.57e-04 tps=22771 mem=5.5G lam=None Gm=0.410614013671875 +[g_only s3] step 4280/12000 loss=4.7956 gn=0.72 lr=4.56e-04 tps=22771 mem=5.5G lam=None Gm=0.4134063720703125 +[g_only s3] step 4300/12000 loss=4.7121 gn=0.66 lr=4.54e-04 tps=22769 mem=5.5G lam=None Gm=0.413177490234375 +[g_only s3] step 4320/12000 loss=4.7509 gn=0.72 lr=4.53e-04 tps=22769 mem=5.5G lam=None Gm=0.407379150390625 +[g_only s3] step 4340/12000 loss=4.8635 gn=0.74 lr=4.52e-04 tps=22769 mem=5.5G lam=None Gm=0.4117991129557292 +[g_only s3] step 4360/12000 loss=4.9057 gn=0.74 lr=4.51e-04 tps=22769 mem=5.5G lam=None Gm=0.4074147542317708 +[g_only s3] step 4380/12000 loss=4.7798 gn=0.73 lr=4.49e-04 tps=22769 mem=5.5G lam=None Gm=0.4029998779296875 +[g_only s3] step 4400/12000 loss=4.6842 gn=0.76 lr=4.48e-04 tps=22768 mem=5.5G lam=None Gm=0.4013519287109375 +[g_only s3] step 4420/12000 loss=5.0056 gn=0.73 lr=4.47e-04 tps=22768 mem=5.5G lam=None Gm=0.4063975016276042 +[g_only s3] step 4440/12000 loss=4.8836 gn=0.79 lr=4.45e-04 tps=22767 mem=5.5G lam=None Gm=0.405426025390625 +[g_only s3] step 4460/12000 loss=4.7749 gn=0.71 lr=4.44e-04 tps=22765 mem=5.5G lam=None Gm=0.4107157389322917 +[g_only s3] step 4480/12000 loss=4.9654 gn=0.71 lr=4.43e-04 tps=22765 mem=5.5G lam=None Gm=0.4049072265625 +[g_only s3] step 4500/12000 loss=4.9080 gn=0.77 lr=4.42e-04 tps=22765 mem=5.5G lam=None Gm=0.4048868815104167 +[g_only s3] step 4520/12000 loss=4.7434 gn=0.76 lr=4.40e-04 tps=22765 mem=5.5G lam=None Gm=0.4059295654296875 +[g_only s3] step 4540/12000 loss=4.7966 gn=0.72 lr=4.39e-04 tps=22765 mem=5.5G lam=None Gm=0.4032847086588542 +[g_only s3] step 4560/12000 loss=4.9417 gn=0.76 lr=4.38e-04 tps=22765 mem=5.5G lam=None Gm=0.4101206461588542 +[g_only s3] step 4580/12000 loss=4.8007 gn=0.72 lr=4.36e-04 tps=22765 mem=5.5G lam=None Gm=0.3996378580729167 +[g_only s3] step 4600/12000 loss=4.8053 gn=0.69 lr=4.35e-04 tps=22763 mem=5.5G lam=None Gm=0.4051259358723958 +[g_only s3] step 4620/12000 loss=4.7432 gn=0.82 lr=4.34e-04 tps=22763 mem=5.5G lam=None Gm=0.3959910074869792 +[g_only s3] step 4640/12000 loss=4.7236 gn=0.71 lr=4.32e-04 tps=22763 mem=5.5G lam=None Gm=0.4019775390625 +[g_only s3] step 4660/12000 loss=4.9392 gn=0.74 lr=4.31e-04 tps=22763 mem=5.5G lam=None Gm=0.4112650553385417 +[g_only s3] step 4680/12000 loss=4.5951 gn=0.72 lr=4.30e-04 tps=22763 mem=5.5G lam=None Gm=0.39990234375 +[g_only s3] step 4700/12000 loss=4.6301 gn=0.74 lr=4.28e-04 tps=22763 mem=5.5G lam=None Gm=0.3994547526041667 +[g_only s3] step 4720/12000 loss=4.8736 gn=0.72 lr=4.27e-04 tps=22763 mem=5.5G lam=None Gm=0.4100596110026042 +[g_only s3] step 4740/12000 loss=4.6395 gn=0.76 lr=4.26e-04 tps=22763 mem=5.5G lam=None Gm=0.40460205078125 +[g_only s3] step 4760/12000 loss=4.7990 gn=0.69 lr=4.24e-04 tps=22763 mem=5.5G lam=None Gm=0.4061330159505208 +[g_only s3] step 4780/12000 loss=4.7419 gn=0.78 lr=4.23e-04 tps=22763 mem=5.5G lam=None Gm=0.4083251953125 +[g_only s3] step 4800/12000 loss=4.7589 gn=0.75 lr=4.22e-04 tps=22763 mem=5.5G lam=None Gm=0.4020233154296875 +[g_only s3] step 4820/12000 loss=4.6094 gn=0.74 lr=4.20e-04 tps=22764 mem=5.5G lam=None Gm=0.4012705485026042 +[g_only s3] step 4840/12000 loss=4.7300 gn=0.71 lr=4.19e-04 tps=22764 mem=5.5G lam=None Gm=0.3994700113932292 +[g_only s3] step 4860/12000 loss=4.7668 gn=0.74 lr=4.18e-04 tps=22763 mem=5.5G lam=None Gm=0.4029388427734375 +[g_only s3] step 4880/12000 loss=4.7863 gn=0.76 lr=4.16e-04 tps=22763 mem=5.5G lam=None Gm=0.4005940755208333 +[g_only s3] step 4900/12000 loss=4.5717 gn=0.76 lr=4.15e-04 tps=22763 mem=5.5G lam=None Gm=0.3990275065104167 +[g_only s3] step 4920/12000 loss=4.7945 gn=0.77 lr=4.13e-04 tps=22763 mem=5.5G lam=None Gm=0.3951314290364583 +[g_only s3] step 4940/12000 loss=4.6561 gn=0.75 lr=4.12e-04 tps=22763 mem=5.5G lam=None Gm=0.3956502278645833 +[g_only s3] step 4960/12000 loss=4.7262 gn=0.73 lr=4.11e-04 tps=22763 mem=5.5G lam=None Gm=0.3982798258463542 +[g_only s3] step 4980/12000 loss=4.6837 gn=0.75 lr=4.09e-04 tps=22763 mem=5.5G lam=None Gm=0.3992360432942708 +[g_only s3] step 5000/12000 loss=4.5583 gn=0.71 lr=4.08e-04 tps=22764 mem=5.5G lam=None Gm=0.3993784586588542 +[g_only s3] EVAL step 5000: val_loss=4.7582 val_ppl=116.54 +[g_only s3] step 5020/12000 loss=4.7420 gn=0.71 lr=4.07e-04 tps=22747 mem=5.5G lam=None Gm=0.3994496663411458 +[g_only s3] step 5040/12000 loss=4.6063 gn=0.71 lr=4.05e-04 tps=22747 mem=5.5G lam=None Gm=0.3942159016927083 +[g_only s3] step 5060/12000 loss=4.6561 gn=0.86 lr=4.04e-04 tps=22747 mem=5.5G lam=None Gm=0.4005839029947917 +[g_only s3] step 5080/12000 loss=4.7201 gn=0.77 lr=4.02e-04 tps=22746 mem=5.5G lam=None Gm=0.3964996337890625 +[g_only s3] step 5100/12000 loss=4.6136 gn=0.74 lr=4.01e-04 tps=22745 mem=5.5G lam=None Gm=0.4021250406901042 +[g_only s3] step 5120/12000 loss=4.7301 gn=0.74 lr=4.00e-04 tps=22745 mem=5.5G lam=None Gm=0.3931172688802083 +[g_only s3] step 5140/12000 loss=4.6594 gn=0.71 lr=3.98e-04 tps=22745 mem=5.5G lam=None Gm=0.3948262532552083 +[g_only s3] step 5160/12000 loss=4.7814 gn=0.77 lr=3.97e-04 tps=22744 mem=5.5G lam=None Gm=0.3958943684895833 +[g_only s3] step 5180/12000 loss=4.5088 gn=0.72 lr=3.95e-04 tps=22744 mem=5.5G lam=None Gm=0.3923288981119792 +[g_only s3] step 5200/12000 loss=4.5296 gn=0.81 lr=3.94e-04 tps=22744 mem=5.5G lam=None Gm=0.3952128092447917 +[g_only s3] step 5220/12000 loss=4.7311 gn=0.73 lr=3.93e-04 tps=22744 mem=5.5G lam=None Gm=0.4017283121744792 +[g_only s3] step 5240/12000 loss=4.5491 gn=0.79 lr=3.91e-04 tps=22744 mem=5.5G lam=None Gm=0.3964589436848958 +[g_only s3] step 5260/12000 loss=4.6953 gn=0.78 lr=3.90e-04 tps=22743 mem=5.5G lam=None Gm=0.3982747395833333 +[g_only s3] step 5280/12000 loss=4.5585 gn=0.79 lr=3.88e-04 tps=22743 mem=5.5G lam=None Gm=0.3896280924479167 +[g_only s3] step 5300/12000 loss=4.7446 gn=0.74 lr=3.87e-04 tps=22743 mem=5.5G lam=None Gm=0.3952687581380208 +[g_only s3] step 5320/12000 loss=4.5803 gn=0.74 lr=3.86e-04 tps=22742 mem=5.5G lam=None Gm=0.3903961181640625 +[g_only s3] step 5340/12000 loss=4.5687 gn=0.71 lr=3.84e-04 tps=22742 mem=5.5G lam=None Gm=0.3938547770182292 +[g_only s3] step 5360/12000 loss=4.5956 gn=0.76 lr=3.83e-04 tps=22743 mem=5.5G lam=None Gm=0.3990020751953125 +[g_only s3] step 5380/12000 loss=4.7207 gn=0.79 lr=3.81e-04 tps=22743 mem=5.5G lam=None Gm=0.3990325927734375 +[g_only s3] step 5400/12000 loss=4.6609 gn=0.76 lr=3.80e-04 tps=22743 mem=5.5G lam=None Gm=0.3939107259114583 +[g_only s3] step 5420/12000 loss=4.7914 gn=0.82 lr=3.79e-04 tps=22743 mem=5.5G lam=None Gm=0.3936055501302083 +[g_only s3] step 5440/12000 loss=4.5323 gn=0.76 lr=3.77e-04 tps=22744 mem=5.5G lam=None Gm=0.3960723876953125 +[g_only s3] step 5460/12000 loss=4.6079 gn=0.74 lr=3.76e-04 tps=22744 mem=5.5G lam=None Gm=0.3955739339192708 +[g_only s3] step 5480/12000 loss=4.5727 gn=0.75 lr=3.74e-04 tps=22744 mem=5.5G lam=None Gm=0.3958841959635417 +[g_only s3] step 5500/12000 loss=4.7783 gn=0.76 lr=3.73e-04 tps=22745 mem=5.5G lam=None Gm=0.3992258707682292 +[g_only s3] step 5520/12000 loss=4.7900 gn=0.78 lr=3.72e-04 tps=22744 mem=5.5G lam=None Gm=0.3965098063151042 +[g_only s3] step 5540/12000 loss=4.6677 gn=0.77 lr=3.70e-04 tps=22745 mem=5.5G lam=None Gm=0.3919016520182292 +[g_only s3] step 5560/12000 loss=4.6853 gn=0.83 lr=3.69e-04 tps=22745 mem=5.5G lam=None Gm=0.3936665852864583 +[g_only s3] step 5580/12000 loss=4.5037 gn=0.74 lr=3.67e-04 tps=22744 mem=5.5G lam=None Gm=0.3902028401692708 +[g_only s3] step 5600/12000 loss=4.7160 gn=0.80 lr=3.66e-04 tps=22745 mem=5.5G lam=None Gm=0.3973846435546875 +[g_only s3] step 5620/12000 loss=4.5954 gn=0.78 lr=3.64e-04 tps=22744 mem=5.5G lam=None Gm=0.3925882975260417 +[g_only s3] step 5640/12000 loss=4.6001 gn=0.78 lr=3.63e-04 tps=22744 mem=5.5G lam=None Gm=0.3956756591796875 +[g_only s3] step 5660/12000 loss=4.6431 gn=0.80 lr=3.62e-04 tps=22745 mem=5.5G lam=None Gm=0.39141845703125 +[g_only s3] step 5680/12000 loss=4.7360 gn=0.79 lr=3.60e-04 tps=22745 mem=5.5G lam=None Gm=0.3893585205078125 +[g_only s3] step 5700/12000 loss=4.6155 gn=0.78 lr=3.59e-04 tps=22744 mem=5.5G lam=None Gm=0.3962758382161458 +[g_only s3] step 5720/12000 loss=4.6280 gn=0.74 lr=3.57e-04 tps=22744 mem=5.5G lam=None Gm=0.3903859456380208 +[g_only s3] step 5740/12000 loss=4.6421 gn=0.78 lr=3.56e-04 tps=22744 mem=5.5G lam=None Gm=0.3833669026692708 +[g_only s3] step 5760/12000 loss=4.5578 gn=0.73 lr=3.54e-04 tps=22744 mem=5.5G lam=None Gm=0.3831431070963542 +[g_only s3] step 5780/12000 loss=4.6537 gn=0.77 lr=3.53e-04 tps=22744 mem=5.5G lam=None Gm=0.3888905843098958 +[g_only s3] step 5800/12000 loss=4.6344 gn=0.80 lr=3.52e-04 tps=22744 mem=5.5G lam=None Gm=0.3884175618489583 +[g_only s3] step 5820/12000 loss=4.4703 gn=0.80 lr=3.50e-04 tps=22744 mem=5.5G lam=None Gm=0.391265869140625 +[g_only s3] step 5840/12000 loss=4.5582 gn=0.83 lr=3.49e-04 tps=22744 mem=5.5G lam=None Gm=0.3889923095703125 +[g_only s3] step 5860/12000 loss=4.4693 gn=0.76 lr=3.47e-04 tps=22744 mem=5.5G lam=None Gm=0.3809254964192708 +[g_only s3] step 5880/12000 loss=4.4989 gn=0.75 lr=3.46e-04 tps=22744 mem=5.5G lam=None Gm=0.3909505208333333 +[g_only s3] step 5900/12000 loss=4.5420 gn=0.76 lr=3.44e-04 tps=22745 mem=5.5G lam=None Gm=0.3935089111328125 +[g_only s3] step 5920/12000 loss=4.5172 gn=0.77 lr=3.43e-04 tps=22745 mem=5.5G lam=None Gm=0.3875630696614583 +[g_only s3] step 5940/12000 loss=4.6886 gn=0.76 lr=3.41e-04 tps=22742 mem=5.5G lam=None Gm=0.3858286539713542 +[g_only s3] step 5960/12000 loss=4.6095 gn=0.78 lr=3.40e-04 tps=22742 mem=5.5G lam=None Gm=0.3858795166015625 +[g_only s3] step 5980/12000 loss=4.4465 gn=0.78 lr=3.39e-04 tps=22741 mem=5.5G lam=None Gm=0.3843536376953125 +[g_only s3] step 6000/12000 loss=4.6886 gn=0.81 lr=3.37e-04 tps=22742 mem=5.5G lam=None Gm=0.3870442708333333 +[g_only s3] EVAL step 6000: val_loss=4.7161 val_ppl=111.74 +[g_only s3] step 6020/12000 loss=4.7727 gn=0.79 lr=3.36e-04 tps=22728 mem=5.5G lam=None Gm=0.3819630940755208 +[g_only s3] step 6040/12000 loss=4.5588 gn=0.73 lr=3.34e-04 tps=22727 mem=5.5G lam=None Gm=0.3833211263020833 +[g_only s3] step 6060/12000 loss=4.4900 gn=0.79 lr=3.33e-04 tps=22728 mem=5.5G lam=None Gm=0.3892873128255208 +[g_only s3] step 6080/12000 loss=4.5244 gn=0.78 lr=3.31e-04 tps=22728 mem=5.5G lam=None Gm=0.38330078125 +[g_only s3] step 6100/12000 loss=4.6396 gn=0.74 lr=3.30e-04 tps=22728 mem=5.5G lam=None Gm=0.3903299967447917 +[g_only s3] step 6120/12000 loss=4.6753 gn=0.81 lr=3.29e-04 tps=22728 mem=5.5G lam=None Gm=0.3945414225260417 +[g_only s3] step 6140/12000 loss=4.6470 gn=0.83 lr=3.27e-04 tps=22728 mem=5.5G lam=None Gm=0.3904673258463542 +[g_only s3] step 6160/12000 loss=4.5085 gn=0.79 lr=3.26e-04 tps=22728 mem=5.5G lam=None Gm=0.3872528076171875 +[g_only s3] step 6180/12000 loss=4.6077 gn=0.81 lr=3.24e-04 tps=22728 mem=5.5G lam=None Gm=0.3845926920572917 +[g_only s3] step 6200/12000 loss=4.5373 gn=0.91 lr=3.23e-04 tps=22728 mem=5.5G lam=None Gm=0.3866729736328125 +[g_only s3] step 6220/12000 loss=4.5757 gn=0.84 lr=3.21e-04 tps=22728 mem=5.5G lam=None Gm=0.3878377278645833 +[g_only s3] step 6240/12000 loss=4.6667 gn=0.79 lr=3.20e-04 tps=22729 mem=5.5G lam=None Gm=0.3870086669921875 +[g_only s3] step 6260/12000 loss=4.5456 gn=0.78 lr=3.19e-04 tps=22728 mem=5.5G lam=None Gm=0.3788045247395833 +[g_only s3] step 6280/12000 loss=4.5359 gn=0.80 lr=3.17e-04 tps=22728 mem=5.5G lam=None Gm=0.383392333984375 +[g_only s3] step 6300/12000 loss=4.6699 gn=0.77 lr=3.16e-04 tps=22729 mem=5.5G lam=None Gm=0.3852488199869792 +[g_only s3] step 6320/12000 loss=4.5134 gn=0.79 lr=3.14e-04 tps=22729 mem=5.5G lam=None Gm=0.3807576497395833 +[g_only s3] step 6340/12000 loss=4.6241 gn=0.82 lr=3.13e-04 tps=22729 mem=5.5G lam=None Gm=0.3834330240885417 +[g_only s3] step 6360/12000 loss=4.7472 gn=0.82 lr=3.11e-04 tps=22729 mem=5.5G lam=None Gm=0.3865814208984375 +[g_only s3] step 6380/12000 loss=4.3125 gn=0.78 lr=3.10e-04 tps=22728 mem=5.5G lam=None Gm=0.3859049479166667 +[g_only s3] step 6400/12000 loss=4.5335 gn=0.76 lr=3.08e-04 tps=22728 mem=5.5G lam=None Gm=0.3792317708333333 +[g_only s3] step 6420/12000 loss=4.4739 gn=0.83 lr=3.07e-04 tps=22729 mem=5.5G lam=None Gm=0.3814900716145833 +[g_only s3] step 6440/12000 loss=4.6360 gn=0.77 lr=3.06e-04 tps=22729 mem=5.5G lam=None Gm=0.3899943033854167 +[g_only s3] step 6460/12000 loss=4.5093 gn=0.80 lr=3.04e-04 tps=22729 mem=5.5G lam=None Gm=0.3787129720052083 +[g_only s3] step 6480/12000 loss=4.5283 gn=0.82 lr=3.03e-04 tps=22729 mem=5.5G lam=None Gm=0.3933817545572917 +[g_only s3] step 6500/12000 loss=4.4823 gn=0.78 lr=3.01e-04 tps=22729 mem=5.5G lam=None Gm=0.379913330078125 +[g_only s3] step 6520/12000 loss=4.6026 gn=0.81 lr=3.00e-04 tps=22729 mem=5.5G lam=None Gm=0.3870798746744792 +[g_only s3] step 6540/12000 loss=4.5234 gn=0.81 lr=2.98e-04 tps=22728 mem=5.5G lam=None Gm=0.3813069661458333 +[g_only s3] step 6560/12000 loss=4.6617 gn=0.83 lr=2.97e-04 tps=22728 mem=5.5G lam=None Gm=0.3834940592447917 +[g_only s3] step 6580/12000 loss=4.6068 gn=0.80 lr=2.96e-04 tps=22728 mem=5.5G lam=None Gm=0.383575439453125 +[g_only s3] step 6600/12000 loss=4.4840 gn=0.78 lr=2.94e-04 tps=22728 mem=5.5G lam=None Gm=0.3834228515625 +[g_only s3] step 6620/12000 loss=4.5165 gn=0.85 lr=2.93e-04 tps=22728 mem=5.5G lam=None Gm=0.3857828776041667 +[g_only s3] step 6640/12000 loss=4.3960 gn=0.80 lr=2.91e-04 tps=22728 mem=5.5G lam=None Gm=0.384735107421875 +[g_only s3] step 6660/12000 loss=4.4616 gn=0.79 lr=2.90e-04 tps=22728 mem=5.5G lam=None Gm=0.3804728190104167 +[g_only s3] step 6680/12000 loss=4.4670 gn=0.84 lr=2.88e-04 tps=22728 mem=5.5G lam=None Gm=0.384002685546875 +[g_only s3] step 6700/12000 loss=4.7762 gn=0.87 lr=2.87e-04 tps=22728 mem=5.5G lam=None Gm=0.3881632486979167 +[g_only s3] step 6720/12000 loss=4.4343 gn=0.77 lr=2.86e-04 tps=22728 mem=5.5G lam=None Gm=0.38323974609375 +[g_only s3] step 6740/12000 loss=4.4785 gn=0.82 lr=2.84e-04 tps=22728 mem=5.5G lam=None Gm=0.3837025960286458 +[g_only s3] step 6760/12000 loss=4.5794 gn=0.83 lr=2.83e-04 tps=22729 mem=5.5G lam=None Gm=0.3863118489583333 +[g_only s3] step 6780/12000 loss=4.5330 gn=0.80 lr=2.81e-04 tps=22729 mem=5.5G lam=None Gm=0.3832804361979167 +[g_only s3] step 6800/12000 loss=4.6786 gn=0.80 lr=2.80e-04 tps=22729 mem=5.5G lam=None Gm=0.3823801676432292 +[g_only s3] step 6820/12000 loss=4.5864 gn=0.82 lr=2.79e-04 tps=22729 mem=5.5G lam=None Gm=0.3745168050130208 +[g_only s3] step 6840/12000 loss=4.5585 gn=0.81 lr=2.77e-04 tps=22729 mem=5.5G lam=None Gm=0.3865966796875 +[g_only s3] step 6860/12000 loss=4.5649 gn=0.83 lr=2.76e-04 tps=22729 mem=5.5G lam=None Gm=0.3837127685546875 +[g_only s3] step 6880/12000 loss=4.5600 gn=0.81 lr=2.74e-04 tps=22729 mem=5.5G lam=None Gm=0.3737894694010417 +[g_only s3] step 6900/12000 loss=4.5518 gn=0.87 lr=2.73e-04 tps=22729 mem=5.5G lam=None Gm=0.3813629150390625 +[g_only s3] step 6920/12000 loss=4.4856 gn=0.86 lr=2.72e-04 tps=22729 mem=5.5G lam=None Gm=0.376678466796875 +[g_only s3] step 6940/12000 loss=4.3673 gn=0.78 lr=2.70e-04 tps=22729 mem=5.5G lam=None Gm=0.3726857503255208 +[g_only s3] step 6960/12000 loss=4.4202 gn=0.81 lr=2.69e-04 tps=22729 mem=5.5G lam=None Gm=0.3791402180989583 +[g_only s3] step 6980/12000 loss=4.6130 gn=0.81 lr=2.67e-04 tps=22729 mem=5.5G lam=None Gm=0.3798065185546875 +[g_only s3] step 7000/12000 loss=4.3990 gn=0.81 lr=2.66e-04 tps=22729 mem=5.5G lam=None Gm=0.3794403076171875 +[g_only s3] EVAL step 7000: val_loss=4.6073 val_ppl=100.21 +[g_only s3] step 7020/12000 loss=4.4983 gn=0.78 lr=2.65e-04 tps=22717 mem=5.5G lam=None Gm=0.3795674641927083 +[g_only s3] step 7040/12000 loss=4.4684 gn=0.84 lr=2.63e-04 tps=22717 mem=5.5G lam=None Gm=0.3767598470052083 +[g_only s3] step 7060/12000 loss=4.5283 gn=0.82 lr=2.62e-04 tps=22717 mem=5.5G lam=None Gm=0.3750864664713542 +[g_only s3] step 7080/12000 loss=4.4782 gn=0.83 lr=2.60e-04 tps=22717 mem=5.5G lam=None Gm=0.3853556315104167 +[g_only s3] step 7100/12000 loss=4.4878 gn=0.84 lr=2.59e-04 tps=22717 mem=5.5G lam=None Gm=0.3734842936197917 +[g_only s3] step 7120/12000 loss=4.5989 gn=0.86 lr=2.58e-04 tps=22717 mem=5.5G lam=None Gm=0.37896728515625 +[g_only s3] step 7140/12000 loss=4.5239 gn=0.85 lr=2.56e-04 tps=22717 mem=5.5G lam=None Gm=0.380523681640625 +[g_only s3] step 7160/12000 loss=4.6800 gn=0.90 lr=2.55e-04 tps=22717 mem=5.5G lam=None Gm=0.3810068766276042 +[g_only s3] step 7180/12000 loss=4.4990 gn=0.84 lr=2.53e-04 tps=22718 mem=5.5G lam=None Gm=0.3803761800130208 +[g_only s3] step 7200/12000 loss=4.4070 gn=0.81 lr=2.52e-04 tps=22717 mem=5.5G lam=None Gm=0.3753407796223958 +[g_only s3] step 7220/12000 loss=4.4240 gn=0.83 lr=2.51e-04 tps=22717 mem=5.5G lam=None Gm=0.3765411376953125 +[g_only s3] step 7240/12000 loss=4.5756 gn=0.79 lr=2.49e-04 tps=22717 mem=5.5G lam=None Gm=0.3759613037109375 +[g_only s3] step 7260/12000 loss=4.4005 gn=0.88 lr=2.48e-04 tps=22717 mem=5.5G lam=None Gm=0.3831634521484375 +[g_only s3] step 7280/12000 loss=4.3639 gn=0.83 lr=2.47e-04 tps=22717 mem=5.5G lam=None Gm=0.3759206136067708 +[g_only s3] step 7300/12000 loss=4.6232 gn=0.83 lr=2.45e-04 tps=22718 mem=5.5G lam=None Gm=0.3779500325520833 +[g_only s3] step 7320/12000 loss=4.6372 gn=0.87 lr=2.44e-04 tps=22717 mem=5.5G lam=None Gm=0.3831278483072917 +[g_only s3] step 7340/12000 loss=4.4336 gn=0.84 lr=2.42e-04 tps=22717 mem=5.5G lam=None Gm=0.3787027994791667 +[g_only s3] step 7360/12000 loss=4.4133 gn=0.84 lr=2.41e-04 tps=22716 mem=5.5G lam=None Gm=0.3726806640625 +[g_only s3] step 7380/12000 loss=4.4580 gn=0.82 lr=2.40e-04 tps=22714 mem=5.5G lam=None Gm=0.3747100830078125 +[g_only s3] step 7400/12000 loss=4.5705 gn=0.82 lr=2.38e-04 tps=22712 mem=5.5G lam=None Gm=0.3762919108072917 +[g_only s3] step 7420/12000 loss=4.4516 gn=0.81 lr=2.37e-04 tps=22710 mem=5.5G lam=None Gm=0.3722279866536458 +[g_only s3] step 7440/12000 loss=4.5163 gn=0.86 lr=2.36e-04 tps=22708 mem=5.5G lam=None Gm=0.3838348388671875 +[g_only s3] step 7460/12000 loss=4.5935 gn=0.85 lr=2.34e-04 tps=22706 mem=5.5G lam=None Gm=0.3823293050130208 +[g_only s3] step 7480/12000 loss=4.5010 gn=0.86 lr=2.33e-04 tps=22704 mem=5.5G lam=None Gm=0.383392333984375 +[g_only s3] step 7500/12000 loss=4.5813 gn=0.84 lr=2.32e-04 tps=22701 mem=5.5G lam=None Gm=0.3749542236328125 +[g_only s3] step 7520/12000 loss=4.3874 gn=0.83 lr=2.30e-04 tps=22698 mem=5.5G lam=None Gm=0.3734079996744792 +[g_only s3] step 7540/12000 loss=4.4987 gn=0.79 lr=2.29e-04 tps=22695 mem=5.5G lam=None Gm=0.3767496744791667 +[g_only s3] step 7560/12000 loss=4.5280 gn=0.85 lr=2.28e-04 tps=22691 mem=5.5G lam=None Gm=0.3740081787109375 +[g_only s3] step 7580/12000 loss=4.4417 gn=0.87 lr=2.26e-04 tps=22688 mem=5.5G lam=None Gm=0.3764597574869792 +[g_only s3] step 7600/12000 loss=4.3916 gn=0.82 lr=2.25e-04 tps=22686 mem=5.5G lam=None Gm=0.3762664794921875 +[g_only s3] step 7620/12000 loss=4.3074 gn=0.85 lr=2.24e-04 tps=22683 mem=5.5G lam=None Gm=0.3821767171223958 +[g_only s3] step 7640/12000 loss=4.5042 gn=0.82 lr=2.22e-04 tps=22681 mem=5.5G lam=None Gm=0.3756510416666667 +[g_only s3] step 7660/12000 loss=4.4130 gn=0.84 lr=2.21e-04 tps=22679 mem=5.5G lam=None Gm=0.381439208984375 +[g_only s3] step 7680/12000 loss=4.5208 gn=0.87 lr=2.20e-04 tps=22678 mem=5.5G lam=None Gm=0.3848012288411458 +[g_only s3] step 7700/12000 loss=4.4457 gn=0.88 lr=2.18e-04 tps=22677 mem=5.5G lam=None Gm=0.3770955403645833 +[g_only s3] step 7720/12000 loss=4.3283 gn=0.86 lr=2.17e-04 tps=22677 mem=5.5G lam=None Gm=0.3810017903645833 +[g_only s3] step 7740/12000 loss=4.3273 gn=0.84 lr=2.16e-04 tps=22677 mem=5.5G lam=None Gm=0.372467041015625 +[g_only s3] step 7760/12000 loss=4.4337 gn=0.83 lr=2.15e-04 tps=22678 mem=5.5G lam=None Gm=0.3781026204427083 +[g_only s3] step 7780/12000 loss=4.4374 gn=0.80 lr=2.13e-04 tps=22678 mem=5.5G lam=None Gm=0.3799540201822917 +[g_only s3] step 7800/12000 loss=4.4557 gn=0.96 lr=2.12e-04 tps=22678 mem=5.5G lam=None Gm=0.3725433349609375 +[g_only s3] step 7820/12000 loss=4.5421 gn=0.88 lr=2.11e-04 tps=22678 mem=5.5G lam=None Gm=0.3810933430989583 +[g_only s3] step 7840/12000 loss=4.4842 gn=0.88 lr=2.09e-04 tps=22678 mem=5.5G lam=None Gm=0.3730316162109375 +[g_only s3] step 7860/12000 loss=4.2321 gn=0.79 lr=2.08e-04 tps=22678 mem=5.5G lam=None Gm=0.3708546956380208 +[g_only s3] step 7880/12000 loss=4.5301 gn=0.86 lr=2.07e-04 tps=22679 mem=5.5G lam=None Gm=0.3728993733723958 +[g_only s3] step 7900/12000 loss=4.5343 gn=0.92 lr=2.06e-04 tps=22679 mem=5.5G lam=None Gm=0.3773651123046875 +[g_only s3] step 7920/12000 loss=4.3147 gn=0.82 lr=2.04e-04 tps=22679 mem=5.5G lam=None Gm=0.3729705810546875 +[g_only s3] step 7940/12000 loss=4.4699 gn=0.84 lr=2.03e-04 tps=22679 mem=5.5G lam=None Gm=0.3780059814453125 +[g_only s3] step 7960/12000 loss=4.5313 gn=0.86 lr=2.02e-04 tps=22679 mem=5.5G lam=None Gm=0.3740488688151042 +[g_only s3] step 7980/12000 loss=4.3976 gn=0.86 lr=2.00e-04 tps=22679 mem=5.5G lam=None Gm=0.3771514892578125 +[g_only s3] step 8000/12000 loss=4.4712 gn=0.84 lr=1.99e-04 tps=22680 mem=5.5G lam=None Gm=0.3735148111979167 +[g_only s3] EVAL step 8000: val_loss=4.4905 val_ppl=89.17 +[g_only s3] step 8020/12000 loss=4.5152 gn=0.88 lr=1.98e-04 tps=22670 mem=5.5G lam=None Gm=0.3769989013671875 +[g_only s3] step 8040/12000 loss=4.5283 gn=0.84 lr=1.97e-04 tps=22670 mem=5.5G lam=None Gm=0.3826243082682292 +[g_only s3] step 8060/12000 loss=4.4756 gn=0.85 lr=1.95e-04 tps=22671 mem=5.5G lam=None Gm=0.3795928955078125 +[g_only s3] step 8080/12000 loss=4.4040 gn=0.85 lr=1.94e-04 tps=22671 mem=5.5G lam=None Gm=0.3771769205729167 +[g_only s3] step 8100/12000 loss=4.2775 gn=0.86 lr=1.93e-04 tps=22671 mem=5.5G lam=None Gm=0.3647206624348958 +[g_only s3] step 8120/12000 loss=4.7194 gn=0.88 lr=1.92e-04 tps=22671 mem=5.5G lam=None Gm=0.3743998209635417 +[g_only s3] step 8140/12000 loss=4.4057 gn=0.88 lr=1.90e-04 tps=22671 mem=5.5G lam=None Gm=0.3664296468098958 +[g_only s3] step 8160/12000 loss=4.3936 gn=0.89 lr=1.89e-04 tps=22672 mem=5.5G lam=None Gm=0.374359130859375 +[g_only s3] step 8180/12000 loss=4.4995 gn=0.86 lr=1.88e-04 tps=22672 mem=5.5G lam=None Gm=0.3762563069661458 +[g_only s3] step 8200/12000 loss=4.4503 gn=0.91 lr=1.87e-04 tps=22671 mem=5.5G lam=None Gm=0.3770650227864583 +[g_only s3] step 8220/12000 loss=4.4615 gn=0.85 lr=1.86e-04 tps=22671 mem=5.5G lam=None Gm=0.372772216796875 +[g_only s3] step 8240/12000 loss=4.5878 gn=0.87 lr=1.84e-04 tps=22672 mem=5.5G lam=None Gm=0.3772837320963542 +[g_only s3] step 8260/12000 loss=4.4317 gn=0.85 lr=1.83e-04 tps=22672 mem=5.5G lam=None Gm=0.3746185302734375 +[g_only s3] step 8280/12000 loss=4.3672 gn=0.85 lr=1.82e-04 tps=22672 mem=5.5G lam=None Gm=0.3727315266927083 +[g_only s3] step 8300/12000 loss=4.4199 gn=0.88 lr=1.81e-04 tps=22673 mem=5.5G lam=None Gm=0.3763427734375 +[g_only s3] step 8320/12000 loss=4.3800 gn=0.89 lr=1.80e-04 tps=22673 mem=5.5G lam=None Gm=0.3743235270182292 +[g_only s3] step 8340/12000 loss=4.4642 gn=0.87 lr=1.78e-04 tps=22673 mem=5.5G lam=None Gm=0.3761494954427083 +[g_only s3] step 8360/12000 loss=4.3160 gn=0.83 lr=1.77e-04 tps=22673 mem=5.5G lam=None Gm=0.3705088297526042 +[g_only s3] step 8380/12000 loss=4.4239 gn=0.85 lr=1.76e-04 tps=22673 mem=5.5G lam=None Gm=0.3683929443359375 +[g_only s3] step 8400/12000 loss=4.4760 gn=0.89 lr=1.75e-04 tps=22674 mem=5.5G lam=None Gm=0.375885009765625 +[g_only s3] step 8420/12000 loss=4.4821 gn=0.94 lr=1.74e-04 tps=22674 mem=5.5G lam=None Gm=0.3737284342447917 +[g_only s3] step 8440/12000 loss=4.4104 gn=0.92 lr=1.72e-04 tps=22674 mem=5.5G lam=None Gm=0.377044677734375 +[g_only s3] step 8460/12000 loss=4.4919 gn=0.90 lr=1.71e-04 tps=22674 mem=5.5G lam=None Gm=0.3751627604166667 +[g_only s3] step 8480/12000 loss=4.4518 gn=0.91 lr=1.70e-04 tps=22675 mem=5.5G lam=None Gm=0.374542236328125 +[g_only s3] step 8500/12000 loss=4.4123 gn=0.93 lr=1.69e-04 tps=22675 mem=5.5G lam=None Gm=0.3782806396484375 +[g_only s3] step 8520/12000 loss=4.4317 gn=0.92 lr=1.68e-04 tps=22675 mem=5.5G lam=None Gm=0.3721516927083333 +[g_only s3] step 8540/12000 loss=4.4099 gn=0.85 lr=1.67e-04 tps=22676 mem=5.5G lam=None Gm=0.3791249593098958 +[g_only s3] step 8560/12000 loss=4.3816 gn=0.90 lr=1.66e-04 tps=22676 mem=5.5G lam=None Gm=0.3693084716796875 +[g_only s3] step 8580/12000 loss=4.4923 gn=0.89 lr=1.64e-04 tps=22676 mem=5.5G lam=None Gm=0.3680928548177083 +[g_only s3] step 8600/12000 loss=4.3309 gn=0.90 lr=1.63e-04 tps=22676 mem=5.5G lam=None Gm=0.3773651123046875 +[g_only s3] step 8620/12000 loss=4.4205 gn=0.88 lr=1.62e-04 tps=22676 mem=5.5G lam=None Gm=0.3772989908854167 +[g_only s3] step 8640/12000 loss=4.4445 gn=0.89 lr=1.61e-04 tps=22677 mem=5.5G lam=None Gm=0.3663686116536458 +[g_only s3] step 8660/12000 loss=4.4426 gn=0.93 lr=1.60e-04 tps=22677 mem=5.5G lam=None Gm=0.371124267578125 +[g_only s3] step 8680/12000 loss=4.4651 gn=0.85 lr=1.59e-04 tps=22677 mem=5.5G lam=None Gm=0.3765716552734375 +[g_only s3] step 8700/12000 loss=4.4316 gn=0.85 lr=1.58e-04 tps=22678 mem=5.5G lam=None Gm=0.372650146484375 +[g_only s3] step 8720/12000 loss=4.4538 gn=0.89 lr=1.57e-04 tps=22678 mem=5.5G lam=None Gm=0.3636932373046875 +[g_only s3] step 8740/12000 loss=4.3575 gn=0.90 lr=1.55e-04 tps=22678 mem=5.5G lam=None Gm=0.3772328694661458 +[g_only s3] step 8760/12000 loss=4.4229 gn=0.85 lr=1.54e-04 tps=22679 mem=5.5G lam=None Gm=0.3719482421875 +[g_only s3] step 8780/12000 loss=4.2923 gn=0.87 lr=1.53e-04 tps=22679 mem=5.5G lam=None Gm=0.3763580322265625 +[g_only s3] step 8800/12000 loss=4.4657 gn=0.92 lr=1.52e-04 tps=22680 mem=5.5G lam=None Gm=0.3725484212239583 +[g_only s3] step 8820/12000 loss=4.5059 gn=0.89 lr=1.51e-04 tps=22680 mem=5.5G lam=None Gm=0.3707377115885417 +[g_only s3] step 8840/12000 loss=4.4294 gn=0.91 lr=1.50e-04 tps=22680 mem=5.5G lam=None Gm=0.3718312581380208 +[g_only s3] step 8860/12000 loss=4.3588 gn=0.87 lr=1.49e-04 tps=22680 mem=5.5G lam=None Gm=0.3722585042317708 +[g_only s3] step 8880/12000 loss=4.5618 gn=0.94 lr=1.48e-04 tps=22681 mem=5.5G lam=None Gm=0.3762766520182292 +[g_only s3] step 8900/12000 loss=4.4189 gn=0.86 lr=1.47e-04 tps=22681 mem=5.5G lam=None Gm=0.3704427083333333 +[g_only s3] step 8920/12000 loss=4.4225 gn=0.92 lr=1.46e-04 tps=22681 mem=5.5G lam=None Gm=0.3715871175130208 +[g_only s3] step 8940/12000 loss=4.2861 gn=0.88 lr=1.45e-04 tps=22681 mem=5.5G lam=None Gm=0.3650410970052083 +[g_only s3] step 8960/12000 loss=4.4437 gn=0.85 lr=1.44e-04 tps=22681 mem=5.5G lam=None Gm=0.3709004720052083 +[g_only s3] step 8980/12000 loss=4.4344 gn=0.91 lr=1.43e-04 tps=22681 mem=5.5G lam=None Gm=0.3726247151692708 +[g_only s3] step 9000/12000 loss=4.5361 gn=0.95 lr=1.42e-04 tps=22682 mem=5.5G lam=None Gm=0.3698170979817708 +[g_only s3] EVAL step 9000: val_loss=4.4701 val_ppl=87.36 +[g_only s3] step 9020/12000 loss=4.3452 gn=0.91 lr=1.41e-04 tps=22673 mem=5.5G lam=None Gm=0.3695831298828125 +[g_only s3] step 9040/12000 loss=4.4616 gn=0.92 lr=1.40e-04 tps=22673 mem=5.5G lam=None Gm=0.3685302734375 +[g_only s3] step 9060/12000 loss=4.2308 gn=0.85 lr=1.39e-04 tps=22673 mem=5.5G lam=None Gm=0.3624216715494792 +[g_only s3] step 9080/12000 loss=4.4131 gn=0.91 lr=1.38e-04 tps=22673 mem=5.5G lam=None Gm=0.3714701334635417 +[g_only s3] step 9100/12000 loss=4.5026 gn=0.92 lr=1.37e-04 tps=22673 mem=5.5G lam=None Gm=0.3695170084635417 +[g_only s3] step 9120/12000 loss=4.4740 gn=0.91 lr=1.36e-04 tps=22673 mem=5.5G lam=None Gm=0.3664449055989583 +[g_only s3] step 9140/12000 loss=4.2622 gn=0.88 lr=1.35e-04 tps=22674 mem=5.5G lam=None Gm=0.365478515625 +[g_only s3] step 9160/12000 loss=4.5407 gn=1.01 lr=1.34e-04 tps=22674 mem=5.5G lam=None Gm=0.3798268636067708 +[g_only s3] step 9180/12000 loss=4.3448 gn=0.92 lr=1.33e-04 tps=22674 mem=5.5G lam=None Gm=0.3727010091145833 +[g_only s3] step 9200/12000 loss=4.4396 gn=0.88 lr=1.32e-04 tps=22674 mem=5.5G lam=None Gm=0.3686981201171875 +[g_only s3] step 9220/12000 loss=4.3774 gn=0.97 lr=1.31e-04 tps=22675 mem=5.5G lam=None Gm=0.3670501708984375 +[g_only s3] step 9240/12000 loss=4.4117 gn=0.99 lr=1.30e-04 tps=22675 mem=5.5G lam=None Gm=0.371063232421875 +[g_only s3] step 9260/12000 loss=4.4326 gn=0.94 lr=1.29e-04 tps=22675 mem=5.5G lam=None Gm=0.3677571614583333 +[g_only s3] step 9280/12000 loss=4.3977 gn=0.90 lr=1.28e-04 tps=22675 mem=5.5G lam=None Gm=0.3720855712890625 +[g_only s3] step 9300/12000 loss=4.2963 gn=0.96 lr=1.27e-04 tps=22675 mem=5.5G lam=None Gm=0.3665262858072917 +[g_only s3] step 9320/12000 loss=4.5333 gn=0.89 lr=1.26e-04 tps=22675 mem=5.5G lam=None Gm=0.3734537760416667 +[g_only s3] step 9340/12000 loss=4.5381 gn=0.93 lr=1.25e-04 tps=22676 mem=5.5G lam=None Gm=0.3754119873046875 +[g_only s3] step 9360/12000 loss=4.3603 gn=0.94 lr=1.24e-04 tps=22676 mem=5.5G lam=None Gm=0.3715260823567708 +[g_only s3] step 9380/12000 loss=4.4278 gn=0.91 lr=1.23e-04 tps=22676 mem=5.5G lam=None Gm=0.3667551676432292 +[g_only s3] step 9400/12000 loss=4.4073 gn=0.95 lr=1.22e-04 tps=22676 mem=5.5G lam=None Gm=0.3707173665364583 +[g_only s3] step 9420/12000 loss=4.3738 gn=0.93 lr=1.21e-04 tps=22677 mem=5.5G lam=None Gm=0.3679707845052083 +[g_only s3] step 9440/12000 loss=4.2721 gn=0.88 lr=1.20e-04 tps=22677 mem=5.5G lam=None Gm=0.3691965738932292 +[g_only s3] step 9460/12000 loss=4.4930 gn=0.87 lr=1.19e-04 tps=22677 mem=5.5G lam=None Gm=0.3658447265625 +[g_only s3] step 9480/12000 loss=4.5436 gn=0.95 lr=1.19e-04 tps=22677 mem=5.5G lam=None Gm=0.3715362548828125 +[g_only s3] step 9500/12000 loss=4.2983 gn=0.86 lr=1.18e-04 tps=22677 mem=5.5G lam=None Gm=0.3653106689453125 +[g_only s3] step 9520/12000 loss=4.4104 gn=0.96 lr=1.17e-04 tps=22677 mem=5.5G lam=None Gm=0.3721415201822917 +[g_only s3] step 9540/12000 loss=4.3642 gn=0.92 lr=1.16e-04 tps=22678 mem=5.5G lam=None Gm=0.3672637939453125 +[g_only s3] step 9560/12000 loss=4.2943 gn=0.93 lr=1.15e-04 tps=22678 mem=5.5G lam=None Gm=0.3706003824869792 +[g_only s3] step 9580/12000 loss=4.3902 gn=0.92 lr=1.14e-04 tps=22678 mem=5.5G lam=None Gm=0.36669921875 +[g_only s3] step 9600/12000 loss=4.5957 gn=0.92 lr=1.13e-04 tps=22677 mem=5.5G lam=None Gm=0.3691965738932292 +[g_only s3] step 9620/12000 loss=4.4245 gn=0.94 lr=1.12e-04 tps=22677 mem=5.5G lam=None Gm=0.36663818359375 +[g_only s3] step 9640/12000 loss=4.2526 gn=0.93 lr=1.12e-04 tps=22674 mem=5.5G lam=None Gm=0.3705952962239583 +[g_only s3] step 9660/12000 loss=4.4964 gn=0.93 lr=1.11e-04 tps=22673 mem=5.5G lam=None Gm=0.3722279866536458 +[g_only s3] step 9680/12000 loss=4.4042 gn=0.97 lr=1.10e-04 tps=22672 mem=5.5G lam=None Gm=0.3695831298828125 +[g_only s3] step 9700/12000 loss=4.2638 gn=0.87 lr=1.09e-04 tps=22671 mem=5.5G lam=None Gm=0.3623148600260417 +[g_only s3] step 9720/12000 loss=4.3462 gn=0.96 lr=1.08e-04 tps=22669 mem=5.5G lam=None Gm=0.3690592447916667 +[g_only s3] step 9740/12000 loss=4.3093 gn=0.92 lr=1.07e-04 tps=22669 mem=5.5G lam=None Gm=0.3722381591796875 +[g_only s3] step 9760/12000 loss=4.3991 gn=0.91 lr=1.07e-04 tps=22669 mem=5.5G lam=None Gm=0.3691253662109375 +[g_only s3] step 9780/12000 loss=4.3215 gn=0.90 lr=1.06e-04 tps=22669 mem=5.5G lam=None Gm=0.3692067464192708 +[g_only s3] step 9800/12000 loss=4.3329 gn=0.94 lr=1.05e-04 tps=22668 mem=5.5G lam=None Gm=0.3688151041666667 +[g_only s3] step 9820/12000 loss=4.3061 gn=0.98 lr=1.04e-04 tps=22668 mem=5.5G lam=None Gm=0.36859130859375 +[g_only s3] step 9840/12000 loss=4.3289 gn=0.91 lr=1.03e-04 tps=22667 mem=5.5G lam=None Gm=0.3630167643229167 +[g_only s3] step 9860/12000 loss=4.3260 gn=0.96 lr=1.03e-04 tps=22667 mem=5.5G lam=None Gm=0.3692677815755208 +[g_only s3] step 9880/12000 loss=4.2408 gn=0.93 lr=1.02e-04 tps=22665 mem=5.5G lam=None Gm=0.3681488037109375 +[g_only s3] step 9900/12000 loss=4.3366 gn=0.98 lr=1.01e-04 tps=22665 mem=5.5G lam=None Gm=0.3659413655598958 +[g_only s3] step 9920/12000 loss=4.3324 gn=0.93 lr=1.00e-04 tps=22665 mem=5.5G lam=None Gm=0.3649800618489583 +[g_only s3] step 9940/12000 loss=4.2965 gn=0.99 lr=9.96e-05 tps=22664 mem=5.5G lam=None Gm=0.3681742350260417 +[g_only s3] step 9960/12000 loss=4.4851 gn=1.01 lr=9.89e-05 tps=22664 mem=5.5G lam=None Gm=0.3689829508463542 +[g_only s3] step 9980/12000 loss=4.2495 gn=0.94 lr=9.81e-05 tps=22665 mem=5.5G lam=None Gm=0.3687082926432292 +[g_only s3] step 10000/12000 loss=4.3884 gn=0.96 lr=9.74e-05 tps=22664 mem=5.5G lam=None Gm=0.3716837565104167 +[g_only s3] EVAL step 10000: val_loss=4.4620 val_ppl=86.66 +[g_only s3] step 10020/12000 loss=4.2487 gn=0.99 lr=9.67e-05 tps=22656 mem=5.5G lam=None Gm=0.3633982340494792 +[g_only s3] step 10040/12000 loss=4.3566 gn=0.94 lr=9.59e-05 tps=22656 mem=5.5G lam=None Gm=0.3728841145833333 +[g_only s3] step 10060/12000 loss=4.2574 gn=0.93 lr=9.52e-05 tps=22656 mem=5.5G lam=None Gm=0.3645477294921875 +[g_only s3] step 10080/12000 loss=4.1857 gn=0.89 lr=9.45e-05 tps=22656 mem=5.5G lam=None Gm=0.3615875244140625 +[g_only s3] step 10100/12000 loss=4.4980 gn=0.94 lr=9.38e-05 tps=22656 mem=5.5G lam=None Gm=0.3669230143229167 +[g_only s3] step 10120/12000 loss=4.4120 gn=0.98 lr=9.31e-05 tps=22656 mem=5.5G lam=None Gm=0.3633371988932292 +[g_only s3] step 10140/12000 loss=4.2732 gn=0.92 lr=9.24e-05 tps=22656 mem=5.5G lam=None Gm=0.366363525390625 +[g_only s3] step 10160/12000 loss=4.4233 gn=0.89 lr=9.18e-05 tps=22655 mem=5.5G lam=None Gm=0.3649698893229167 +[g_only s3] step 10180/12000 loss=4.5189 gn=0.93 lr=9.11e-05 tps=22655 mem=5.5G lam=None Gm=0.3680775960286458 +[g_only s3] step 10200/12000 loss=4.4958 gn=0.95 lr=9.04e-05 tps=22655 mem=5.5G lam=None Gm=0.3674774169921875 +[g_only s3] step 10220/12000 loss=4.3975 gn=0.96 lr=8.98e-05 tps=22655 mem=5.5G lam=None Gm=0.3677927652994792 +[g_only s3] step 10240/12000 loss=4.2588 gn=0.92 lr=8.91e-05 tps=22655 mem=5.5G lam=None Gm=0.3619232177734375 +[g_only s3] step 10260/12000 loss=4.3823 gn=0.97 lr=8.85e-05 tps=22655 mem=5.5G lam=None Gm=0.3626047770182292 +[g_only s3] step 10280/12000 loss=4.4145 gn=1.02 lr=8.78e-05 tps=22655 mem=5.5G lam=None Gm=0.36920166015625 +[g_only s3] step 10300/12000 loss=4.2979 gn=0.92 lr=8.72e-05 tps=22655 mem=5.5G lam=None Gm=0.3662160237630208 +[g_only s3] step 10320/12000 loss=4.4157 gn=0.91 lr=8.66e-05 tps=22655 mem=5.5G lam=None Gm=0.369171142578125 +[g_only s3] step 10340/12000 loss=4.2686 gn=0.97 lr=8.59e-05 tps=22655 mem=5.5G lam=None Gm=0.3629608154296875 +[g_only s3] step 10360/12000 loss=4.3007 gn=0.92 lr=8.53e-05 tps=22655 mem=5.5G lam=None Gm=0.3646341959635417 +[g_only s3] step 10380/12000 loss=4.4458 gn=0.94 lr=8.47e-05 tps=22655 mem=5.5G lam=None Gm=0.368072509765625 +[g_only s3] step 10400/12000 loss=4.3960 gn=1.01 lr=8.41e-05 tps=22655 mem=5.5G lam=None Gm=0.365753173828125 +[g_only s3] step 10420/12000 loss=4.2438 gn=0.92 lr=8.35e-05 tps=22655 mem=5.5G lam=None Gm=0.3626556396484375 +[g_only s3] step 10440/12000 loss=4.2462 gn=1.01 lr=8.30e-05 tps=22655 mem=5.5G lam=None Gm=0.3632863362630208 +[g_only s3] step 10460/12000 loss=4.4451 gn=0.97 lr=8.24e-05 tps=22655 mem=5.5G lam=None Gm=0.3657480875651042 +[g_only s3] step 10480/12000 loss=4.5629 gn=0.97 lr=8.18e-05 tps=22655 mem=5.5G lam=None Gm=0.3670603434244792 +[g_only s3] step 10500/12000 loss=4.4072 gn=1.04 lr=8.12e-05 tps=22655 mem=5.5G lam=None Gm=0.3679097493489583 +[g_only s3] step 10520/12000 loss=4.3021 gn=0.92 lr=8.07e-05 tps=22655 mem=5.5G lam=None Gm=0.3633321126302083 +[g_only s3] step 10540/12000 loss=4.2649 gn=0.93 lr=8.01e-05 tps=22655 mem=5.5G lam=None Gm=0.3676656087239583 +[g_only s3] step 10560/12000 loss=4.2300 gn=0.96 lr=7.96e-05 tps=22655 mem=5.5G lam=None Gm=0.3632303873697917 +[g_only s3] step 10580/12000 loss=4.4451 gn=0.99 lr=7.91e-05 tps=22656 mem=5.5G lam=None Gm=0.3671671549479167 +[g_only s3] step 10600/12000 loss=4.2397 gn=0.91 lr=7.85e-05 tps=22655 mem=5.5G lam=None Gm=0.3660634358723958 +[g_only s3] step 10620/12000 loss=4.4111 gn=0.99 lr=7.80e-05 tps=22655 mem=5.5G lam=None Gm=0.3642374674479167 +[g_only s3] step 10640/12000 loss=4.2494 gn=0.95 lr=7.75e-05 tps=22655 mem=5.5G lam=None Gm=0.3695322672526042 +[g_only s3] step 10660/12000 loss=4.3231 gn=0.94 lr=7.70e-05 tps=22655 mem=5.5G lam=None Gm=0.3609568277994792 +[g_only s3] step 10680/12000 loss=4.3883 gn=1.00 lr=7.65e-05 tps=22655 mem=5.5G lam=None Gm=0.3651784261067708 +[g_only s3] step 10700/12000 loss=4.3970 gn=0.91 lr=7.60e-05 tps=22655 mem=5.5G lam=None Gm=0.3610331217447917 +[g_only s3] step 10720/12000 loss=4.3034 gn=0.96 lr=7.55e-05 tps=22655 mem=5.5G lam=None Gm=0.3668263753255208 +[g_only s3] step 10740/12000 loss=4.2562 gn=0.97 lr=7.50e-05 tps=22655 mem=5.5G lam=None Gm=0.3658498128255208 +[g_only s3] step 10760/12000 loss=4.3574 gn=0.95 lr=7.46e-05 tps=22655 mem=5.5G lam=None Gm=0.3659515380859375 +[g_only s3] step 10780/12000 loss=4.3391 gn=0.99 lr=7.41e-05 tps=22656 mem=5.5G lam=None Gm=0.3632303873697917 +[g_only s3] step 10800/12000 loss=4.3746 gn=0.96 lr=7.37e-05 tps=22656 mem=5.5G lam=None Gm=0.3606821695963542 +[g_only s3] step 10820/12000 loss=4.3087 gn=0.98 lr=7.32e-05 tps=22656 mem=5.5G lam=None Gm=0.365509033203125 +[g_only s3] step 10840/12000 loss=4.4658 gn=1.03 lr=7.28e-05 tps=22656 mem=5.5G lam=None Gm=0.3691660563151042 +[g_only s3] step 10860/12000 loss=4.4150 gn=1.00 lr=7.23e-05 tps=22657 mem=5.5G lam=None Gm=0.3668874104817708 +[g_only s3] step 10880/12000 loss=4.4079 gn=0.98 lr=7.19e-05 tps=22656 mem=5.5G lam=None Gm=0.3654327392578125 +[g_only s3] step 10900/12000 loss=4.2797 gn=0.96 lr=7.15e-05 tps=22655 mem=5.5G lam=None Gm=0.3603871663411458 +[g_only s3] step 10920/12000 loss=4.1499 gn=0.95 lr=7.11e-05 tps=22655 mem=5.5G lam=None Gm=0.3616689046223958 +[g_only s3] step 10940/12000 loss=4.4406 gn=0.99 lr=7.07e-05 tps=22655 mem=5.5G lam=None Gm=0.367767333984375 +[g_only s3] step 10960/12000 loss=4.3681 gn=1.00 lr=7.03e-05 tps=22655 mem=5.5G lam=None Gm=0.3626556396484375 +[g_only s3] step 10980/12000 loss=4.3273 gn=0.99 lr=6.99e-05 tps=22655 mem=5.5G lam=None Gm=0.3602142333984375 +[g_only s3] step 11000/12000 loss=4.3178 gn=0.96 lr=6.95e-05 tps=22655 mem=5.5G lam=None Gm=0.3674163818359375 +[g_only s3] EVAL step 11000: val_loss=4.3747 val_ppl=79.42 +[g_only s3] step 11020/12000 loss=4.2720 gn=0.97 lr=6.91e-05 tps=22648 mem=5.5G lam=None Gm=0.3669281005859375 +[g_only s3] step 11040/12000 loss=4.2831 gn=0.90 lr=6.88e-05 tps=22648 mem=5.5G lam=None Gm=0.3624369303385417 +[g_only s3] step 11060/12000 loss=4.4100 gn=0.95 lr=6.84e-05 tps=22648 mem=5.5G lam=None Gm=0.366363525390625 +[g_only s3] step 11080/12000 loss=4.1369 gn=0.97 lr=6.81e-05 tps=22648 mem=5.5G lam=None Gm=0.3610585530598958 +[g_only s3] step 11100/12000 loss=4.3848 gn=1.06 lr=6.77e-05 tps=22648 mem=5.5G lam=None Gm=0.36492919921875 +[g_only s3] step 11120/12000 loss=4.3021 gn=0.96 lr=6.74e-05 tps=22648 mem=5.5G lam=None Gm=0.3612467447916667 +[g_only s3] step 11140/12000 loss=4.3407 gn=0.97 lr=6.70e-05 tps=22649 mem=5.5G lam=None Gm=0.3577321370442708 +[g_only s3] step 11160/12000 loss=4.3537 gn=0.95 lr=6.67e-05 tps=22649 mem=5.5G lam=None Gm=0.3684844970703125 +[g_only s3] step 11180/12000 loss=4.3074 gn=0.99 lr=6.64e-05 tps=22649 mem=5.5G lam=None Gm=0.3610483805338542 +[g_only s3] step 11200/12000 loss=4.2307 gn=0.99 lr=6.61e-05 tps=22649 mem=5.5G lam=None Gm=0.3655192057291667 +[g_only s3] step 11220/12000 loss=4.2901 gn=1.02 lr=6.58e-05 tps=22649 mem=5.5G lam=None Gm=0.363433837890625 +[g_only s3] step 11240/12000 loss=4.3353 gn=0.97 lr=6.55e-05 tps=22649 mem=5.5G lam=None Gm=0.3656717936197917 +[g_only s3] step 11260/12000 loss=4.3704 gn=1.02 lr=6.52e-05 tps=22650 mem=5.5G lam=None Gm=0.3710072835286458 +[g_only s3] step 11280/12000 loss=4.2877 gn=0.96 lr=6.49e-05 tps=22650 mem=5.5G lam=None Gm=0.3667144775390625 +[g_only s3] step 11300/12000 loss=4.2735 gn=1.02 lr=6.47e-05 tps=22649 mem=5.5G lam=None Gm=0.3613688151041667 +[g_only s3] step 11320/12000 loss=4.3263 gn=0.95 lr=6.44e-05 tps=22649 mem=5.5G lam=None Gm=0.3657989501953125 +[g_only s3] step 11340/12000 loss=4.3643 gn=0.97 lr=6.42e-05 tps=22650 mem=5.5G lam=None Gm=0.3616790771484375 +[g_only s3] step 11360/12000 loss=4.4750 gn=1.01 lr=6.39e-05 tps=22650 mem=5.5G lam=None Gm=0.3610127766927083 +[g_only s3] step 11380/12000 loss=4.3095 gn=0.97 lr=6.37e-05 tps=22650 mem=5.5G lam=None Gm=0.3640187581380208 +[g_only s3] step 11400/12000 loss=4.3967 gn=0.99 lr=6.34e-05 tps=22651 mem=5.5G lam=None Gm=0.3656209309895833 +[g_only s3] step 11420/12000 loss=4.3088 gn=0.96 lr=6.32e-05 tps=22651 mem=5.5G lam=None Gm=0.3613637288411458 +[g_only s3] step 11440/12000 loss=4.3267 gn=1.00 lr=6.30e-05 tps=22651 mem=5.5G lam=None Gm=0.3671468098958333 +[g_only s3] step 11460/12000 loss=4.3676 gn=1.01 lr=6.28e-05 tps=22651 mem=5.5G lam=None Gm=0.3670501708984375 +[g_only s3] step 11480/12000 loss=4.3677 gn=0.94 lr=6.26e-05 tps=22652 mem=5.5G lam=None Gm=0.3619791666666667 +[g_only s3] step 11500/12000 loss=4.2611 gn=0.99 lr=6.24e-05 tps=22652 mem=5.5G lam=None Gm=0.363677978515625 +[g_only s3] step 11520/12000 loss=4.1762 gn=1.00 lr=6.22e-05 tps=22652 mem=5.5G lam=None Gm=0.3643951416015625 +[g_only s3] step 11540/12000 loss=4.4177 gn=0.97 lr=6.20e-05 tps=22652 mem=5.5G lam=None Gm=0.3654836018880208 +[g_only s3] step 11560/12000 loss=4.2979 gn=1.05 lr=6.19e-05 tps=22652 mem=5.5G lam=None Gm=0.3654530843098958 +[g_only s3] step 11580/12000 loss=4.1734 gn=0.90 lr=6.17e-05 tps=22652 mem=5.5G lam=None Gm=0.36224365234375 +[g_only s3] step 11600/12000 loss=4.3299 gn=0.95 lr=6.15e-05 tps=22652 mem=5.5G lam=None Gm=0.3593292236328125 +[g_only s3] step 11620/12000 loss=4.2794 gn=1.01 lr=6.14e-05 tps=22653 mem=5.5G lam=None Gm=0.3654581705729167 +[g_only s3] step 11640/12000 loss=4.3418 gn=0.95 lr=6.12e-05 tps=22653 mem=5.5G lam=None Gm=0.3652598063151042 +[g_only s3] step 11660/12000 loss=4.4274 gn=1.08 lr=6.11e-05 tps=22653 mem=5.5G lam=None Gm=0.3618672688802083 +[g_only s3] step 11680/12000 loss=4.2104 gn=1.03 lr=6.10e-05 tps=22653 mem=5.5G lam=None Gm=0.3650054931640625 +[g_only s3] step 11700/12000 loss=4.3507 gn=1.04 lr=6.09e-05 tps=22653 mem=5.5G lam=None Gm=0.3597768147786458 +[g_only s3] step 11720/12000 loss=4.2738 gn=0.98 lr=6.07e-05 tps=22653 mem=5.5G lam=None Gm=0.3616129557291667 +[g_only s3] step 11740/12000 loss=4.3231 gn=1.03 lr=6.06e-05 tps=22653 mem=5.5G lam=None Gm=0.3645477294921875 +[g_only s3] step 11760/12000 loss=4.3053 gn=0.98 lr=6.06e-05 tps=22653 mem=5.5G lam=None Gm=0.3596343994140625 +[g_only s3] step 11780/12000 loss=4.3584 gn=0.97 lr=6.05e-05 tps=22653 mem=5.5G lam=None Gm=0.3619791666666667 +[g_only s3] step 11800/12000 loss=4.3077 gn=0.96 lr=6.04e-05 tps=22654 mem=5.5G lam=None Gm=0.3606363932291667 +[g_only s3] step 11820/12000 loss=4.3634 gn=1.01 lr=6.03e-05 tps=22654 mem=5.5G lam=None Gm=0.3610890706380208 +[g_only s3] step 11840/12000 loss=4.2361 gn=0.98 lr=6.02e-05 tps=22654 mem=5.5G lam=None Gm=0.3582661946614583 +[g_only s3] step 11860/12000 loss=4.2741 gn=0.97 lr=6.02e-05 tps=22654 mem=5.5G lam=None Gm=0.3607889811197917 +[g_only s3] step 11880/12000 loss=4.4497 gn=1.03 lr=6.01e-05 tps=22654 mem=5.5G lam=None Gm=0.3670857747395833 +[g_only s3] step 11900/12000 loss=4.2731 gn=0.99 lr=6.01e-05 tps=22654 mem=5.5G lam=None Gm=0.35687255859375 +[g_only s3] step 11920/12000 loss=4.2581 gn=0.99 lr=6.01e-05 tps=22654 mem=5.5G lam=None Gm=0.3612060546875 +[g_only s3] step 11940/12000 loss=4.3984 gn=1.03 lr=6.00e-05 tps=22655 mem=5.5G lam=None Gm=0.359466552734375 +[g_only s3] step 11960/12000 loss=4.2547 gn=1.01 lr=6.00e-05 tps=22655 mem=5.5G lam=None Gm=0.3640034993489583 +[g_only s3] step 11980/12000 loss=4.2862 gn=0.97 lr=6.00e-05 tps=22655 mem=5.5G lam=None Gm=0.362548828125 +[g_only s3] EVAL step 11999: val_loss=4.3565 val_ppl=77.98 +[init] variant=full seed=3 params=63.98M +[init] use_S=True use_G=True +[full s3] step 0/12000 loss=10.9221 gn=11.56 lr=3.00e-06 tps=7926 mem=5.2G lam=0.12692801654338837 Gm=0.4967380625506242 +[full s3] step 20/12000 loss=9.8363 gn=1.91 lr=6.30e-05 tps=17542 mem=5.7G lam=0.12691842702527842 Gm=0.45481953024864197 +[full s3] step 40/12000 loss=9.0962 gn=1.72 lr=1.23e-04 tps=18045 mem=5.7G lam=0.12683101060489813 Gm=0.43924688175320625 +[full s3] step 60/12000 loss=8.1738 gn=1.58 lr=1.83e-04 tps=18219 mem=5.7G lam=0.12671193418403467 Gm=0.44859347864985466 +[full s3] step 80/12000 loss=7.5189 gn=0.80 lr=2.43e-04 tps=18314 mem=5.7G lam=0.12658455595374107 Gm=0.4408634069065253 +[full s3] step 100/12000 loss=7.0934 gn=0.62 lr=3.03e-04 tps=18380 mem=5.7G lam=0.12652622039119402 Gm=0.43364975104729336 +[full s3] step 120/12000 loss=7.1862 gn=0.81 lr=3.63e-04 tps=18424 mem=5.7G lam=0.12645522008339563 Gm=0.4304923477272193 +[full s3] step 140/12000 loss=6.8931 gn=0.62 lr=4.23e-04 tps=18435 mem=5.7G lam=0.12640980879465738 Gm=0.4268057569861412 +[full s3] step 160/12000 loss=6.8667 gn=0.71 lr=4.83e-04 tps=18453 mem=5.7G lam=0.1264190897345543 Gm=0.41893654440840083 +[full s3] step 180/12000 loss=6.7397 gn=0.56 lr=5.43e-04 tps=18473 mem=5.7G lam=0.1264035999774933 Gm=0.40707895904779434 +[full s3] step 200/12000 loss=6.6726 gn=0.65 lr=6.00e-04 tps=18476 mem=5.7G lam=0.12636210831503072 Gm=0.4164770406981309 +[full s3] step 220/12000 loss=6.5862 gn=0.51 lr=6.00e-04 tps=18487 mem=5.7G lam=0.12634787956873575 Gm=0.40114111577471095 +[full s3] step 240/12000 loss=6.6082 gn=0.49 lr=6.00e-04 tps=18485 mem=5.7G lam=0.12632736812035242 Gm=0.40214137981335324 +[full s3] step 260/12000 loss=6.6109 gn=0.47 lr=6.00e-04 tps=18484 mem=5.7G lam=0.12627975145975748 Gm=0.41223492845892906 +[full s3] step 280/12000 loss=6.4416 gn=0.57 lr=6.00e-04 tps=18500 mem=5.7G lam=0.12631403158108392 Gm=0.43195637812217075 +[full s3] step 300/12000 loss=6.3335 gn=0.53 lr=6.00e-04 tps=18501 mem=5.7G lam=0.1263508622845014 Gm=0.4020000249147415 +[full s3] step 320/12000 loss=6.4145 gn=0.45 lr=6.00e-04 tps=18505 mem=5.7G lam=0.1264127374937137 Gm=0.41024623562892276 +[full s3] step 340/12000 loss=6.4789 gn=0.51 lr=6.00e-04 tps=18511 mem=5.7G lam=0.12639792325596014 Gm=0.42827024807532627 +[full s3] step 360/12000 loss=6.2552 gn=0.45 lr=6.00e-04 tps=18509 mem=5.7G lam=0.1264340846488873 Gm=0.40848731125394505 +[full s3] step 380/12000 loss=6.2941 gn=0.43 lr=6.00e-04 tps=18508 mem=5.7G lam=0.12651421253879866 Gm=0.42361511290073395 +[full s3] step 400/12000 loss=6.3704 gn=0.51 lr=6.00e-04 tps=18512 mem=5.7G lam=0.1266118629525105 Gm=0.45276952038208645 +[full s3] step 420/12000 loss=6.2788 gn=0.44 lr=6.00e-04 tps=18513 mem=5.7G lam=0.1267273494352897 Gm=0.4435681216418743 +[full s3] step 440/12000 loss=6.2219 gn=0.49 lr=5.99e-04 tps=18518 mem=5.7G lam=0.12678298354148865 Gm=0.44803983097275096 +[full s3] step 460/12000 loss=6.3442 gn=0.55 lr=5.99e-04 tps=18519 mem=5.7G lam=0.12695602948466936 Gm=0.44404230763514835 +[full s3] step 480/12000 loss=6.2082 gn=0.49 lr=5.99e-04 tps=18522 mem=5.7G lam=0.12705691469212374 Gm=0.4475450664758682 +[full s3] step 500/12000 loss=6.2355 gn=0.48 lr=5.99e-04 tps=18524 mem=5.7G lam=0.1271813033769528 Gm=0.48572882637381554 +[full s3] step 520/12000 loss=6.2590 gn=0.50 lr=5.99e-04 tps=18527 mem=5.7G lam=0.12735133493940035 Gm=0.5092406285305818 +[full s3] step 540/12000 loss=6.1617 gn=0.50 lr=5.99e-04 tps=18529 mem=5.7G lam=0.12745306454598904 Gm=0.49537043645977974 +[full s3] step 560/12000 loss=6.1975 gn=0.52 lr=5.99e-04 tps=18535 mem=5.7G lam=0.12762730879088244 Gm=0.5030427513023218 +[full s3] step 580/12000 loss=6.1768 gn=0.46 lr=5.99e-04 tps=18538 mem=5.7G lam=0.1278144766887029 Gm=0.5059734769165516 +[full s3] step 600/12000 loss=6.0907 gn=0.45 lr=5.98e-04 tps=18543 mem=5.7G lam=0.12795594334602356 Gm=0.5297425587972006 +[full s3] step 620/12000 loss=5.9970 gn=0.51 lr=5.98e-04 tps=18544 mem=5.7G lam=0.1280880911896626 Gm=0.5520224173863729 +[full s3] step 640/12000 loss=6.0872 gn=0.44 lr=5.98e-04 tps=18546 mem=5.7G lam=0.12824489859243235 Gm=0.5449822805821896 +[full s3] step 660/12000 loss=5.8833 gn=0.45 lr=5.98e-04 tps=18549 mem=5.7G lam=0.1284874894966682 Gm=0.54122061530749 +[full s3] step 680/12000 loss=6.0015 gn=0.47 lr=5.98e-04 tps=18553 mem=5.7G lam=0.12871527733902136 Gm=0.5474922843277454 +[full s3] step 700/12000 loss=6.0539 gn=0.48 lr=5.98e-04 tps=18554 mem=5.7G lam=0.12895583423475424 Gm=0.5579936169087887 +[full s3] step 720/12000 loss=5.8573 gn=0.46 lr=5.97e-04 tps=18554 mem=5.7G lam=0.12920383301874003 Gm=0.5700867759684721 +[full s3] step 740/12000 loss=6.0212 gn=0.50 lr=5.97e-04 tps=18554 mem=5.7G lam=0.1294959963609775 Gm=0.5704271011054516 +[full s3] step 760/12000 loss=5.9113 gn=0.45 lr=5.97e-04 tps=18556 mem=5.7G lam=0.12978538746635118 Gm=0.5685371079792579 +[full s3] step 780/12000 loss=5.9955 gn=0.50 lr=5.97e-04 tps=18556 mem=5.7G lam=0.13009096619983515 Gm=0.579496718943119 +[full s3] step 800/12000 loss=5.9230 gn=0.48 lr=5.97e-04 tps=18553 mem=5.7G lam=0.13039981697996458 Gm=0.5970512684434652 +[full s3] step 820/12000 loss=5.8392 gn=0.55 lr=5.96e-04 tps=18555 mem=5.7G lam=0.13074643537402153 Gm=0.5568996276706457 +[full s3] step 840/12000 loss=5.8622 gn=0.57 lr=5.96e-04 tps=18553 mem=5.7G lam=0.1310335509479046 Gm=0.5853832804908355 +[full s3] step 860/12000 loss=5.8306 gn=0.51 lr=5.96e-04 tps=18554 mem=5.7G lam=0.13133961614221334 Gm=0.568771556019783 +[full s3] step 880/12000 loss=5.9744 gn=0.54 lr=5.96e-04 tps=18556 mem=5.7G lam=0.13164737168699503 Gm=0.5888839264710745 +[full s3] step 900/12000 loss=5.7480 gn=0.51 lr=5.95e-04 tps=18558 mem=5.7G lam=0.1319771527002255 Gm=0.5817632128794988 +[full s3] step 920/12000 loss=5.7861 gn=0.56 lr=5.95e-04 tps=18560 mem=5.7G lam=0.13231594612201056 Gm=0.5684681963175535 +[full s3] step 940/12000 loss=5.6304 gn=0.48 lr=5.95e-04 tps=18556 mem=5.7G lam=0.1326611020291845 Gm=0.5581475893656412 +[full s3] step 960/12000 loss=5.7929 gn=0.51 lr=5.94e-04 tps=18557 mem=5.7G lam=0.13294798384110132 Gm=0.5835227544109026 +[full s3] step 980/12000 loss=5.7140 gn=0.50 lr=5.94e-04 tps=18555 mem=5.7G lam=0.13330694070706764 Gm=0.5766928369800249 +[full s3] step 1000/12000 loss=5.8928 gn=0.58 lr=5.94e-04 tps=18550 mem=5.7G lam=0.13373626861721277 Gm=0.5736776161938906 +[full s3] EVAL step 1000: val_loss=5.8245 val_ppl=338.47 +[full s3] step 1020/12000 loss=5.7277 gn=0.49 lr=5.94e-04 tps=18480 mem=5.7G lam=0.13412510665754476 Gm=0.5719081349670887 +[full s3] step 1040/12000 loss=5.6511 gn=0.50 lr=5.93e-04 tps=18482 mem=5.7G lam=0.13450418909390768 Gm=0.5661529749631882 +[full s3] step 1060/12000 loss=5.6267 gn=0.56 lr=5.93e-04 tps=18480 mem=5.7G lam=0.13486077108730873 Gm=0.5509572364389896 +[full s3] step 1080/12000 loss=5.6786 gn=0.49 lr=5.93e-04 tps=18483 mem=5.7G lam=0.13524920276055732 Gm=0.559166923786203 +[full s3] step 1100/12000 loss=5.6341 gn=0.55 lr=5.92e-04 tps=18487 mem=5.7G lam=0.13556802769502005 Gm=0.5507545315970978 +[full s3] step 1120/12000 loss=5.7392 gn=0.50 lr=5.92e-04 tps=18489 mem=5.7G lam=0.13594675871233144 Gm=0.5532472642759482 +[full s3] step 1140/12000 loss=5.6199 gn=0.50 lr=5.92e-04 tps=18493 mem=5.7G lam=0.13633934780955315 Gm=0.5475799143314362 +[full s3] step 1160/12000 loss=5.6870 gn=0.52 lr=5.91e-04 tps=18496 mem=5.7G lam=0.13672193915893635 Gm=0.5607337603966395 +[full s3] step 1180/12000 loss=5.6099 gn=0.54 lr=5.91e-04 tps=18496 mem=5.7G lam=0.1370984259992838 Gm=0.5429668767998616 +[full s3] step 1200/12000 loss=5.6717 gn=0.57 lr=5.90e-04 tps=18494 mem=5.7G lam=0.13747288659214973 Gm=0.5394938569515944 +[full s3] step 1220/12000 loss=5.4860 gn=0.49 lr=5.90e-04 tps=18494 mem=5.7G lam=0.13780508190393448 Gm=0.5513620283454657 +[full s3] step 1240/12000 loss=5.5283 gn=0.51 lr=5.90e-04 tps=18496 mem=5.7G lam=0.1382291903719306 Gm=0.545398968582352 +[full s3] step 1260/12000 loss=5.6972 gn=0.50 lr=5.89e-04 tps=18497 mem=5.7G lam=0.1386366734902064 Gm=0.547384524717927 +[full s3] step 1280/12000 loss=5.5513 gn=0.52 lr=5.89e-04 tps=18498 mem=5.7G lam=0.13910071024050316 Gm=0.5477355650315682 +[full s3] step 1300/12000 loss=5.6446 gn=0.51 lr=5.89e-04 tps=18498 mem=5.7G lam=0.13955739016334215 Gm=0.5341927769283453 +[full s3] step 1320/12000 loss=5.5351 gn=0.55 lr=5.88e-04 tps=18499 mem=5.7G lam=0.13994202421357235 Gm=0.5416294460495313 +[full s3] step 1340/12000 loss=5.6093 gn=0.52 lr=5.88e-04 tps=18499 mem=5.7G lam=0.1404077528665463 Gm=0.5324197448790073 +[full s3] step 1360/12000 loss=5.4576 gn=0.55 lr=5.87e-04 tps=18500 mem=5.7G lam=0.14087652632345757 Gm=0.5330395766844352 +[full s3] step 1380/12000 loss=5.4112 gn=0.52 lr=5.87e-04 tps=18499 mem=5.7G lam=0.1412687636911869 Gm=0.5407715359081825 +[full s3] step 1400/12000 loss=5.5917 gn=0.54 lr=5.86e-04 tps=18501 mem=5.7G lam=0.14171015564352274 Gm=0.5295054685945312 +[full s3] step 1420/12000 loss=5.4604 gn=0.60 lr=5.86e-04 tps=18503 mem=5.7G lam=0.14217153377830982 Gm=0.5367607300480207 +[full s3] step 1440/12000 loss=5.3540 gn=0.53 lr=5.85e-04 tps=18501 mem=5.7G lam=0.14260988806684813 Gm=0.5345920709272226 +[full s3] step 1460/12000 loss=5.5268 gn=0.56 lr=5.85e-04 tps=18500 mem=5.7G lam=0.14297471288591623 Gm=0.5240606237202883 +[full s3] step 1480/12000 loss=5.3797 gn=0.56 lr=5.84e-04 tps=18498 mem=5.7G lam=0.14344882282118002 Gm=0.5139474067837 +[full s3] step 1500/12000 loss=5.4974 gn=0.63 lr=5.84e-04 tps=18492 mem=5.7G lam=0.14391555730253458 Gm=0.5151703739538789 +[full s3] step 1520/12000 loss=5.4817 gn=0.57 lr=5.83e-04 tps=18491 mem=5.7G lam=0.14436287060379982 Gm=0.5185290277004242 +[full s3] step 1540/12000 loss=5.3302 gn=0.55 lr=5.83e-04 tps=18489 mem=5.7G lam=0.14483378641307354 Gm=0.5218359949067235 +[full s3] step 1560/12000 loss=5.4051 gn=0.56 lr=5.82e-04 tps=18488 mem=5.7G lam=0.14527197554707527 Gm=0.5139392595738173 +[full s3] step 1580/12000 loss=5.4814 gn=0.61 lr=5.82e-04 tps=18487 mem=5.7G lam=0.1456199719881018 Gm=0.5130662703886628 +[full s3] step 1600/12000 loss=5.2485 gn=0.54 lr=5.81e-04 tps=18486 mem=5.7G lam=0.14608451072126627 Gm=0.5000439692909519 +[full s3] step 1620/12000 loss=5.4540 gn=0.59 lr=5.81e-04 tps=18482 mem=5.7G lam=0.1465468524644772 Gm=0.5083229613179961 +[full s3] step 1640/12000 loss=5.3403 gn=0.57 lr=5.80e-04 tps=18481 mem=5.7G lam=0.1469801211108764 Gm=0.4969669912631313 +[full s3] step 1660/12000 loss=5.3544 gn=0.60 lr=5.80e-04 tps=18480 mem=5.7G lam=0.1474189537887772 Gm=0.5045342991749445 +[full s3] step 1680/12000 loss=5.3426 gn=0.54 lr=5.79e-04 tps=18478 mem=5.7G lam=0.147878828458488 Gm=0.5035647389789423 +[full s3] step 1700/12000 loss=5.5207 gn=0.60 lr=5.79e-04 tps=18476 mem=5.7G lam=0.14831875450909138 Gm=0.4974391457314293 +[full s3] step 1720/12000 loss=5.3931 gn=0.57 lr=5.78e-04 tps=18473 mem=5.7G lam=0.1487974381695191 Gm=0.5127755999565125 +[full s3] step 1740/12000 loss=5.3023 gn=0.63 lr=5.78e-04 tps=18471 mem=5.7G lam=0.14924805238842964 Gm=0.5032419729977846 +[full s3] step 1760/12000 loss=5.2440 gn=0.58 lr=5.77e-04 tps=18470 mem=5.7G lam=0.14969127997756004 Gm=0.49920669632653397 +[full s3] step 1780/12000 loss=5.3962 gn=0.56 lr=5.76e-04 tps=18524 mem=5.7G lam=0.15015929068128267 Gm=0.5004452212403218 +[full s3] step 1800/12000 loss=5.3039 gn=0.55 lr=5.76e-04 tps=18520 mem=5.7G lam=0.15061559341847897 Gm=0.5037053988004724 +[full s3] step 1820/12000 loss=5.2531 gn=0.55 lr=5.75e-04 tps=18517 mem=5.7G lam=0.15110963489860296 Gm=0.4981577967604001 +[full s3] step 1840/12000 loss=5.4158 gn=0.58 lr=5.75e-04 tps=18514 mem=5.7G lam=0.15161037848641476 Gm=0.4946722537279129 +[full s3] step 1860/12000 loss=5.3799 gn=0.61 lr=5.74e-04 tps=18512 mem=5.7G lam=0.15209209950019917 Gm=0.5029975604265928 +[full s3] step 1880/12000 loss=5.2691 gn=0.56 lr=5.73e-04 tps=18509 mem=5.7G lam=0.15254663086185852 Gm=0.486190989613533 +[full s3] step 1900/12000 loss=5.1921 gn=0.54 lr=5.73e-04 tps=18508 mem=5.7G lam=0.15295248478651047 Gm=0.49598497586945695 +[full s3] step 1920/12000 loss=5.2902 gn=0.55 lr=5.72e-04 tps=18504 mem=5.7G lam=0.1533584324643016 Gm=0.47944036809106666 +[full s3] step 1940/12000 loss=5.4075 gn=0.59 lr=5.72e-04 tps=18501 mem=5.7G lam=0.15379353209088245 Gm=0.47853625441590947 +[full s3] step 1960/12000 loss=5.3451 gn=0.60 lr=5.71e-04 tps=18499 mem=5.7G lam=0.15428681982060274 Gm=0.4896544289464752 +[full s3] step 1980/12000 loss=5.0859 gn=0.56 lr=5.70e-04 tps=18498 mem=5.7G lam=0.154761364373068 Gm=0.47815420882155496 +[full s3] step 2000/12000 loss=5.3270 gn=0.56 lr=5.70e-04 tps=18497 mem=5.7G lam=0.15527928372224173 Gm=0.47678756589690846 +[full s3] EVAL step 2000: val_loss=5.3391 val_ppl=208.32 +[full s3] step 2020/12000 loss=5.3478 gn=0.57 lr=5.69e-04 tps=18461 mem=5.7G lam=0.15576473933955035 Gm=0.47277219593524933 +[full s3] step 2040/12000 loss=5.3751 gn=0.56 lr=5.68e-04 tps=18460 mem=5.7G lam=0.15621777468671402 Gm=0.482533598318696 +[full s3] step 2060/12000 loss=5.2761 gn=0.62 lr=5.68e-04 tps=18458 mem=5.7G lam=0.156664385770758 Gm=0.478086088784039 +[full s3] step 2080/12000 loss=5.1824 gn=0.65 lr=5.67e-04 tps=18456 mem=5.7G lam=0.1571388371909658 Gm=0.47430708911269903 +[full s3] step 2100/12000 loss=5.2639 gn=0.58 lr=5.66e-04 tps=18455 mem=5.7G lam=0.15766878860692182 Gm=0.4709221289182703 +[full s3] step 2120/12000 loss=5.3145 gn=0.59 lr=5.65e-04 tps=18454 mem=5.7G lam=0.15816919784992933 Gm=0.47507063873733085 +[full s3] step 2140/12000 loss=5.1219 gn=0.57 lr=5.65e-04 tps=18454 mem=5.7G lam=0.15857153727362552 Gm=0.46388923687239486 +[full s3] step 2160/12000 loss=5.1022 gn=0.60 lr=5.64e-04 tps=18453 mem=5.7G lam=0.15898399893194437 Gm=0.4685168449456493 +[full s3] step 2180/12000 loss=5.1413 gn=0.60 lr=5.63e-04 tps=18454 mem=5.7G lam=0.15945726167410612 Gm=0.46697180656095344 +[full s3] step 2200/12000 loss=5.3150 gn=0.60 lr=5.63e-04 tps=18453 mem=5.7G lam=0.16001113317906857 Gm=0.4637915963927905 +[full s3] step 2220/12000 loss=5.2502 gn=0.62 lr=5.62e-04 tps=18452 mem=5.7G lam=0.1605066688110431 Gm=0.45119408269723255 +[full s3] step 2240/12000 loss=5.2440 gn=0.65 lr=5.61e-04 tps=18452 mem=5.7G lam=0.16095415378610292 Gm=0.4665068353836735 +[full s3] step 2260/12000 loss=5.1073 gn=0.63 lr=5.60e-04 tps=18450 mem=5.7G lam=0.16141158497581878 Gm=0.4652114389464259 +[full s3] step 2280/12000 loss=5.1985 gn=0.78 lr=5.60e-04 tps=18448 mem=5.7G lam=0.16190082424630722 Gm=0.46641289566953975 +[full s3] step 2300/12000 loss=5.2486 gn=0.61 lr=5.59e-04 tps=18446 mem=5.7G lam=0.16239101563890776 Gm=0.46345187289019424 +[full s3] step 2320/12000 loss=5.1644 gn=0.67 lr=5.58e-04 tps=18446 mem=5.7G lam=0.1628605875497063 Gm=0.4583744714036584 +[full s3] step 2340/12000 loss=4.9674 gn=0.61 lr=5.57e-04 tps=18445 mem=5.7G lam=0.16333289972196022 Gm=0.4556826410504679 +[full s3] step 2360/12000 loss=5.2003 gn=0.62 lr=5.57e-04 tps=18445 mem=5.7G lam=0.16382271641244492 Gm=0.4499886967241764 +[full s3] step 2380/12000 loss=5.1162 gn=0.66 lr=5.56e-04 tps=18443 mem=5.7G lam=0.16429761486748853 Gm=0.4636841270451744 +[full s3] step 2400/12000 loss=5.0642 gn=0.59 lr=5.55e-04 tps=18441 mem=5.7G lam=0.1647541873777906 Gm=0.4609002306436499 +[full s3] step 2420/12000 loss=5.0770 gn=0.66 lr=5.54e-04 tps=18439 mem=5.7G lam=0.16524210603286824 Gm=0.4543470572680235 +[full s3] step 2440/12000 loss=5.1239 gn=0.64 lr=5.53e-04 tps=18439 mem=5.7G lam=0.1656868401914835 Gm=0.45704827333490056 +[full s3] step 2460/12000 loss=5.0653 gn=0.59 lr=5.53e-04 tps=18439 mem=5.7G lam=0.16611077170819044 Gm=0.4449260321756204 +[full s3] step 2480/12000 loss=5.0797 gn=0.60 lr=5.52e-04 tps=18437 mem=5.7G lam=0.16660322217891613 Gm=0.4482532699281971 +[full s3] step 2500/12000 loss=5.1023 gn=0.64 lr=5.51e-04 tps=18437 mem=5.7G lam=0.167030718177557 Gm=0.44357009946058196 +[full s3] step 2520/12000 loss=5.2055 gn=0.60 lr=5.50e-04 tps=18436 mem=5.7G lam=0.16749307047575712 Gm=0.43692943174391985 +[full s3] step 2540/12000 loss=5.0409 gn=0.61 lr=5.49e-04 tps=18435 mem=5.7G lam=0.16796364728361368 Gm=0.44534921009714407 +[full s3] step 2560/12000 loss=5.1719 gn=0.71 lr=5.48e-04 tps=18436 mem=5.7G lam=0.16845977244277796 Gm=0.46042316406965256 +[full s3] step 2580/12000 loss=5.0208 gn=0.64 lr=5.48e-04 tps=18433 mem=5.7G lam=0.168928362429142 Gm=0.44091110738615197 +[full s3] step 2600/12000 loss=5.0850 gn=0.67 lr=5.47e-04 tps=18432 mem=5.7G lam=0.1694660515834888 Gm=0.4511012779548764 +[full s3] step 2620/12000 loss=4.9871 gn=0.60 lr=5.46e-04 tps=18432 mem=5.7G lam=0.16993656195700169 Gm=0.44406686040262383 +[full s3] step 2640/12000 loss=4.9481 gn=0.71 lr=5.45e-04 tps=18430 mem=5.7G lam=0.17036662995815277 Gm=0.4365022995819648 +[full s3] step 2660/12000 loss=5.0516 gn=0.61 lr=5.44e-04 tps=18430 mem=5.7G lam=0.1708889218668143 Gm=0.44255409700175125 +[full s3] step 2680/12000 loss=5.0366 gn=0.61 lr=5.43e-04 tps=18430 mem=5.7G lam=0.17134492316593727 Gm=0.45262953514854115 +[full s3] step 2700/12000 loss=5.0263 gn=0.65 lr=5.42e-04 tps=18430 mem=5.7G lam=0.17183905560523272 Gm=0.43809363214919966 +[full s3] step 2720/12000 loss=5.1050 gn=0.74 lr=5.41e-04 tps=18430 mem=5.7G lam=0.17232291772961617 Gm=0.444405479511867 +[full s3] step 2740/12000 loss=5.0752 gn=0.64 lr=5.41e-04 tps=18429 mem=5.7G lam=0.17284341311703125 Gm=0.43679656243572634 +[full s3] step 2760/12000 loss=5.0744 gn=0.67 lr=5.40e-04 tps=18429 mem=5.7G lam=0.1733005562176307 Gm=0.43310785355667275 +[full s3] step 2780/12000 loss=5.0182 gn=0.65 lr=5.39e-04 tps=18429 mem=5.7G lam=0.17377315492679676 Gm=0.43439330408970517 +[full s3] step 2800/12000 loss=4.9795 gn=0.70 lr=5.38e-04 tps=18428 mem=5.7G lam=0.1742748220761617 Gm=0.4304962479509413 +[full s3] step 2820/12000 loss=5.1096 gn=0.64 lr=5.37e-04 tps=18427 mem=5.7G lam=0.17479864849398533 Gm=0.43422481960927445 +[full s3] step 2840/12000 loss=5.0696 gn=0.62 lr=5.36e-04 tps=18426 mem=5.7G lam=0.17533959603557983 Gm=0.4356282519487043 +[full s3] step 2860/12000 loss=4.9277 gn=0.65 lr=5.35e-04 tps=18425 mem=5.7G lam=0.17586671592046818 Gm=0.4279988951360186 +[full s3] step 2880/12000 loss=4.9756 gn=0.67 lr=5.34e-04 tps=18424 mem=5.7G lam=0.17634254538764557 Gm=0.43101070417712134 +[full s3] step 2900/12000 loss=4.8463 gn=0.63 lr=5.33e-04 tps=18424 mem=5.7G lam=0.1768145483608047 Gm=0.43138939049094915 +[full s3] step 2920/12000 loss=4.9634 gn=0.64 lr=5.32e-04 tps=18422 mem=5.7G lam=0.1772152759755651 Gm=0.42836759286001325 +[full s3] step 2940/12000 loss=5.0101 gn=0.64 lr=5.31e-04 tps=18421 mem=5.7G lam=0.17764333511392275 Gm=0.42725764277080697 +[full s3] step 2960/12000 loss=4.8495 gn=0.73 lr=5.30e-04 tps=18419 mem=5.7G lam=0.17811867905159792 Gm=0.43923400063067675 +[full s3] step 2980/12000 loss=4.7751 gn=0.87 lr=5.29e-04 tps=18419 mem=5.7G lam=0.1786124793191751 Gm=0.41878064069896936 +[full s3] step 3000/12000 loss=5.0391 gn=0.69 lr=5.28e-04 tps=18419 mem=5.7G lam=0.1790725439786911 Gm=0.42783546323577565 +[full s3] EVAL step 3000: val_loss=4.9961 val_ppl=147.83 +[full s3] step 3020/12000 loss=5.0987 gn=0.72 lr=5.27e-04 tps=18395 mem=5.7G lam=0.17960365768522024 Gm=0.42852017330005765 +[full s3] step 3040/12000 loss=4.8173 gn=0.62 lr=5.26e-04 tps=18394 mem=5.7G lam=0.18008054979145527 Gm=0.4196814966077606 +[full s3] step 3060/12000 loss=4.9054 gn=0.68 lr=5.25e-04 tps=18393 mem=5.7G lam=0.18055547835926214 Gm=0.429230282548815 +[full s3] step 3080/12000 loss=4.8502 gn=0.65 lr=5.24e-04 tps=18391 mem=5.7G lam=0.1811045091599226 Gm=0.4260876369662583 +[full s3] step 3100/12000 loss=5.0223 gn=0.80 lr=5.23e-04 tps=18390 mem=5.7G lam=0.18162075243890285 Gm=0.4182443098785977 +[full s3] step 3120/12000 loss=4.7491 gn=0.73 lr=5.22e-04 tps=18384 mem=5.7G lam=0.18212779456128678 Gm=0.4146326758588354 +[full s3] step 3140/12000 loss=4.8993 gn=0.71 lr=5.21e-04 tps=18384 mem=5.7G lam=0.1825281037017703 Gm=0.43241047812625766 +[full s3] step 3160/12000 loss=4.8269 gn=0.70 lr=5.20e-04 tps=18384 mem=5.7G lam=0.18301562933872143 Gm=0.431648048106581 +[full s3] step 3180/12000 loss=4.8694 gn=0.71 lr=5.19e-04 tps=18383 mem=5.7G lam=0.18356924504041672 Gm=0.4205333637073636 +[full s3] step 3200/12000 loss=4.8479 gn=0.69 lr=5.18e-04 tps=18384 mem=5.7G lam=0.18405138359715542 Gm=0.4139159945771098 +[full s3] step 3220/12000 loss=4.9539 gn=0.79 lr=5.17e-04 tps=18383 mem=5.7G lam=0.18462168363233408 Gm=0.4255977653277417 +[full s3] step 3240/12000 loss=4.9271 gn=0.64 lr=5.16e-04 tps=18381 mem=5.7G lam=0.18509670936812958 Gm=0.42780584655702114 +[full s3] step 3260/12000 loss=4.9817 gn=0.67 lr=5.15e-04 tps=18380 mem=5.7G lam=0.1855221272756656 Gm=0.4261376409170528 +[full s3] step 3280/12000 loss=4.7987 gn=0.62 lr=5.14e-04 tps=18380 mem=5.7G lam=0.18598971391717592 Gm=0.4201998362938563 +[full s3] step 3300/12000 loss=5.0654 gn=0.72 lr=5.13e-04 tps=18380 mem=5.7G lam=0.186452383796374 Gm=0.4143383130431175 +[full s3] step 3320/12000 loss=4.9073 gn=0.72 lr=5.12e-04 tps=18381 mem=5.7G lam=0.1868442486350735 Gm=0.4195767054334283 +[full s3] step 3340/12000 loss=4.8597 gn=0.67 lr=5.11e-04 tps=18381 mem=5.7G lam=0.18725741716722646 Gm=0.4150834276030461 +[full s3] step 3360/12000 loss=4.8975 gn=0.69 lr=5.10e-04 tps=18381 mem=5.7G lam=0.1877978149180611 Gm=0.4116564186600347 +[full s3] step 3380/12000 loss=5.0335 gn=0.69 lr=5.09e-04 tps=18381 mem=5.7G lam=0.18821058298150697 Gm=0.4227605794245998 +[full s3] step 3400/12000 loss=4.9734 gn=0.71 lr=5.08e-04 tps=18381 mem=5.7G lam=0.1887191260854403 Gm=0.4203516594134271 +[full s3] step 3420/12000 loss=4.9126 gn=0.68 lr=5.07e-04 tps=18381 mem=5.7G lam=0.18919295879701772 Gm=0.4176974833632509 +[full s3] step 3440/12000 loss=4.7012 gn=0.65 lr=5.06e-04 tps=18381 mem=5.7G lam=0.1896978716055552 Gm=0.405692917915682 +[full s3] step 3460/12000 loss=4.9447 gn=0.74 lr=5.05e-04 tps=18380 mem=5.7G lam=0.19015980946520963 Gm=0.4095097441847126 +[full s3] step 3480/12000 loss=4.9385 gn=0.69 lr=5.03e-04 tps=18379 mem=5.7G lam=0.1906739752739668 Gm=0.41404276915515464 +[full s3] step 3500/12000 loss=5.0513 gn=0.70 lr=5.02e-04 tps=18379 mem=5.7G lam=0.19113452328989902 Gm=0.4243252792706092 +[full s3] step 3520/12000 loss=4.8190 gn=0.74 lr=5.01e-04 tps=18379 mem=5.7G lam=0.19158356202145418 Gm=0.41158488346263766 +[full s3] step 3540/12000 loss=4.8354 gn=0.72 lr=5.00e-04 tps=18379 mem=5.7G lam=0.19206356847037873 Gm=0.40897127287462354 +[full s3] step 3560/12000 loss=4.8870 gn=0.75 lr=4.99e-04 tps=18378 mem=5.7G lam=0.19251792194942632 Gm=0.4083167288141946 +[full s3] step 3580/12000 loss=4.9020 gn=0.70 lr=4.98e-04 tps=18378 mem=5.7G lam=0.19298928106824556 Gm=0.4070359130079548 +[full s3] step 3600/12000 loss=4.8822 gn=0.70 lr=4.97e-04 tps=18377 mem=5.7G lam=0.1934200127919515 Gm=0.4068047176115215 +[full s3] step 3620/12000 loss=4.8811 gn=0.67 lr=4.96e-04 tps=18376 mem=5.7G lam=0.19393691793084145 Gm=0.4084977060556412 +[full s3] step 3640/12000 loss=4.7919 gn=0.67 lr=4.94e-04 tps=18376 mem=5.7G lam=0.19448356175174317 Gm=0.41052263385305804 +[full s3] step 3660/12000 loss=4.9937 gn=0.72 lr=4.93e-04 tps=18376 mem=5.7G lam=0.1949152760207653 Gm=0.41558714692170423 +[full s3] step 3680/12000 loss=4.8332 gn=0.69 lr=4.92e-04 tps=18376 mem=5.7G lam=0.19535534580548605 Gm=0.40799606319827336 +[full s3] step 3700/12000 loss=4.9856 gn=0.69 lr=4.91e-04 tps=18377 mem=5.7G lam=0.19586002423117557 Gm=0.4145556873021026 +[full s3] step 3720/12000 loss=4.9368 gn=0.73 lr=4.90e-04 tps=18376 mem=5.7G lam=0.19633463863283396 Gm=0.4131518246140331 +[full s3] step 3740/12000 loss=4.8810 gn=0.74 lr=4.89e-04 tps=18376 mem=5.7G lam=0.19679298376043639 Gm=0.41025348271553713 +[full s3] step 3760/12000 loss=4.7850 gn=0.70 lr=4.88e-04 tps=18377 mem=5.7G lam=0.19731919560581446 Gm=0.40633049444295466 +[full s3] step 3780/12000 loss=4.7481 gn=0.73 lr=4.86e-04 tps=18376 mem=5.7G lam=0.19779325587054095 Gm=0.40265938530986506 +[full s3] step 3800/12000 loss=4.9055 gn=0.73 lr=4.85e-04 tps=18377 mem=5.7G lam=0.19823697892328104 Gm=0.40212862682528794 +[full s3] step 3820/12000 loss=5.0040 gn=0.71 lr=4.84e-04 tps=18378 mem=5.7G lam=0.19875236134976149 Gm=0.4106232187865923 +[full s3] step 3840/12000 loss=4.8965 gn=0.72 lr=4.83e-04 tps=18378 mem=5.7G lam=0.19917875186850628 Gm=0.4088990224214892 +[full s3] step 3860/12000 loss=4.8482 gn=0.69 lr=4.82e-04 tps=18377 mem=5.7G lam=0.19970943654576936 Gm=0.39758297164614004 +[full s3] step 3880/12000 loss=4.8795 gn=0.73 lr=4.80e-04 tps=18378 mem=5.7G lam=0.2001916148389379 Gm=0.4108955553577592 +[full s3] step 3900/12000 loss=4.8836 gn=0.71 lr=4.79e-04 tps=18378 mem=5.7G lam=0.20069120544940233 Gm=0.4026264944113791 +[full s3] step 3920/12000 loss=4.8662 gn=0.67 lr=4.78e-04 tps=18377 mem=5.7G lam=0.20115812650571266 Gm=0.39675399924938876 +[full s3] step 3940/12000 loss=4.7416 gn=0.76 lr=4.77e-04 tps=18377 mem=5.7G lam=0.2015723322207729 Gm=0.3938269172795117 +[full s3] step 3960/12000 loss=4.7346 gn=0.75 lr=4.76e-04 tps=18377 mem=5.7G lam=0.20200621957580248 Gm=0.4071318036876619 +[full s3] step 3980/12000 loss=4.8148 gn=0.73 lr=4.74e-04 tps=18377 mem=5.7G lam=0.20242205758889517 Gm=0.4006751264290263 +[full s3] step 4000/12000 loss=4.7571 gn=0.75 lr=4.73e-04 tps=18377 mem=5.7G lam=0.20283319521695375 Gm=0.40065090971377987 +[full s3] EVAL step 4000: val_loss=4.8735 val_ppl=130.77 +[full s3] step 4020/12000 loss=4.6965 gn=0.72 lr=4.72e-04 tps=18359 mem=5.7G lam=0.20320955539743105 Gm=0.3962544760045906 +[full s3] step 4040/12000 loss=5.0916 gn=0.76 lr=4.71e-04 tps=18360 mem=5.7G lam=0.20365866584082445 Gm=0.3968217431101948 +[full s3] step 4060/12000 loss=4.7331 gn=0.75 lr=4.70e-04 tps=18359 mem=5.7G lam=0.20415812730789185 Gm=0.4043332018579046 +[full s3] step 4080/12000 loss=4.8772 gn=0.76 lr=4.68e-04 tps=18358 mem=5.7G lam=0.2046098724628488 Gm=0.407872352283448 +[full s3] step 4100/12000 loss=4.8054 gn=0.70 lr=4.67e-04 tps=18358 mem=5.7G lam=0.20507123911132416 Gm=0.40093642271434266 +[full s3] step 4120/12000 loss=4.7767 gn=0.74 lr=4.66e-04 tps=18359 mem=5.7G lam=0.20554391325761875 Gm=0.40159052874272066 +[full s3] step 4140/12000 loss=4.7974 gn=0.69 lr=4.65e-04 tps=18359 mem=5.7G lam=0.20597126117597023 Gm=0.3990220078267157 +[full s3] step 4160/12000 loss=4.7530 gn=0.81 lr=4.63e-04 tps=18358 mem=5.7G lam=0.206331388403972 Gm=0.4015410087692241 +[full s3] step 4180/12000 loss=4.7186 gn=0.70 lr=4.62e-04 tps=18358 mem=5.7G lam=0.20676300271103779 Gm=0.39857537516703206 +[full s3] step 4200/12000 loss=4.8276 gn=0.77 lr=4.61e-04 tps=18358 mem=5.7G lam=0.207265869093438 Gm=0.39742020634002984 +[full s3] step 4220/12000 loss=4.6635 gn=0.73 lr=4.60e-04 tps=18357 mem=5.7G lam=0.20770766430844864 Gm=0.3966012925375253 +[full s3] step 4240/12000 loss=4.8334 gn=0.73 lr=4.58e-04 tps=18357 mem=5.7G lam=0.20812945843984684 Gm=0.4100186175201088 +[full s3] step 4260/12000 loss=4.8704 gn=0.77 lr=4.57e-04 tps=18357 mem=5.7G lam=0.2086132268110911 Gm=0.3995835743844509 +[full s3] step 4280/12000 loss=4.7730 gn=0.73 lr=4.56e-04 tps=18356 mem=5.7G lam=0.20894492821147045 Gm=0.3933955443402131 +[full s3] step 4300/12000 loss=4.7082 gn=0.69 lr=4.54e-04 tps=18356 mem=5.7G lam=0.20945907942950726 Gm=0.39098724676296115 +[full s3] step 4320/12000 loss=4.7473 gn=0.76 lr=4.53e-04 tps=18356 mem=5.7G lam=0.2099128346890211 Gm=0.3920088961410026 +[full s3] step 4340/12000 loss=4.8596 gn=0.75 lr=4.52e-04 tps=18356 mem=5.7G lam=0.2103918936724464 Gm=0.39370348770171404 +[full s3] step 4360/12000 loss=4.8900 gn=0.74 lr=4.51e-04 tps=18356 mem=5.7G lam=0.21091007503370443 Gm=0.3958664995928605 +[full s3] step 4380/12000 loss=4.7657 gn=0.74 lr=4.49e-04 tps=18356 mem=5.7G lam=0.21134600788354874 Gm=0.3922643482219428 +[full s3] step 4400/12000 loss=4.6855 gn=0.78 lr=4.48e-04 tps=18356 mem=5.7G lam=0.21184203401207924 Gm=0.3896613187001397 +[full s3] step 4420/12000 loss=4.9974 gn=0.74 lr=4.47e-04 tps=18356 mem=5.7G lam=0.21227172575891018 Gm=0.3894556436377267 +[full s3] step 4440/12000 loss=4.8741 gn=0.78 lr=4.45e-04 tps=18357 mem=5.7G lam=0.21272388442109028 Gm=0.38657534200077254 +[full s3] step 4460/12000 loss=4.7743 gn=0.73 lr=4.44e-04 tps=18356 mem=5.7G lam=0.2132046694556872 Gm=0.39148894837126136 +[full s3] step 4480/12000 loss=4.9626 gn=0.76 lr=4.43e-04 tps=18357 mem=5.7G lam=0.21365310779462257 Gm=0.392027894500643 +[full s3] step 4500/12000 loss=4.8968 gn=0.78 lr=4.42e-04 tps=18356 mem=5.7G lam=0.21416930512835583 Gm=0.3883055094629526 +[full s3] step 4520/12000 loss=4.7403 gn=0.78 lr=4.40e-04 tps=18357 mem=5.7G lam=0.21459874076147875 Gm=0.3920795447193086 +[full s3] step 4540/12000 loss=4.7852 gn=0.71 lr=4.39e-04 tps=18356 mem=5.7G lam=0.2150325250501434 Gm=0.3865039592298369 +[full s3] step 4560/12000 loss=4.9322 gn=0.75 lr=4.38e-04 tps=18355 mem=5.7G lam=0.21543435441950956 Gm=0.392306405507649 +[full s3] step 4580/12000 loss=4.7773 gn=0.73 lr=4.36e-04 tps=18355 mem=5.7G lam=0.2158889714628458 Gm=0.3827276194933802 +[full s3] step 4600/12000 loss=4.7930 gn=0.74 lr=4.35e-04 tps=18354 mem=5.7G lam=0.21627711411565542 Gm=0.3918572817929089 +[full s3] step 4620/12000 loss=4.7366 gn=0.78 lr=4.34e-04 tps=18355 mem=5.7G lam=0.2167399593939384 Gm=0.38438381456459564 +[full s3] step 4640/12000 loss=4.7423 gn=0.71 lr=4.32e-04 tps=18355 mem=5.7G lam=0.21717609371989965 Gm=0.3827361154835671 +[full s3] step 4660/12000 loss=4.9305 gn=0.77 lr=4.31e-04 tps=18355 mem=5.7G lam=0.21760499477386475 Gm=0.39626438659615815 +[full s3] step 4680/12000 loss=4.6033 gn=0.76 lr=4.30e-04 tps=18355 mem=5.7G lam=0.2180883390828967 Gm=0.38149609393440187 +[full s3] step 4700/12000 loss=4.6179 gn=0.74 lr=4.28e-04 tps=18355 mem=5.7G lam=0.21843452968945107 Gm=0.38429449902226526 +[full s3] step 4720/12000 loss=4.8733 gn=0.77 lr=4.27e-04 tps=18354 mem=5.7G lam=0.21881385706365108 Gm=0.38831159425899386 +[full s3] step 4740/12000 loss=4.6356 gn=0.82 lr=4.26e-04 tps=18354 mem=5.7G lam=0.21927434174964824 Gm=0.38864193111658096 +[full s3] step 4760/12000 loss=4.7867 gn=0.72 lr=4.24e-04 tps=18354 mem=5.7G lam=0.21966844238340855 Gm=0.38316686016817886 +[full s3] step 4780/12000 loss=4.7397 gn=0.77 lr=4.23e-04 tps=18354 mem=5.7G lam=0.2201324449852109 Gm=0.3937210123209904 +[full s3] step 4800/12000 loss=4.7462 gn=0.72 lr=4.22e-04 tps=18354 mem=5.7G lam=0.2205323874950409 Gm=0.3796710352568577 +[full s3] step 4820/12000 loss=4.6148 gn=0.76 lr=4.20e-04 tps=18354 mem=5.7G lam=0.2209743189935883 Gm=0.38393849817415077 +[full s3] step 4840/12000 loss=4.7098 gn=0.71 lr=4.19e-04 tps=18354 mem=5.7G lam=0.22137274065365395 Gm=0.38313210910807055 +[full s3] step 4860/12000 loss=4.7590 gn=0.73 lr=4.18e-04 tps=18354 mem=5.7G lam=0.22181025240570307 Gm=0.38630286340291303 +[full s3] step 4880/12000 loss=4.7662 gn=0.74 lr=4.16e-04 tps=18354 mem=5.7G lam=0.22223842795938253 Gm=0.38024405265847844 +[full s3] step 4900/12000 loss=4.5607 gn=0.79 lr=4.15e-04 tps=18355 mem=5.7G lam=0.22269884124398232 Gm=0.38250514888204634 +[full s3] step 4920/12000 loss=4.7827 gn=0.82 lr=4.13e-04 tps=18355 mem=5.7G lam=0.223149956514438 Gm=0.3767259134911001 +[full s3] step 4940/12000 loss=4.6405 gn=0.75 lr=4.12e-04 tps=18355 mem=5.7G lam=0.22359248157590628 Gm=0.37783804497060675 +[full s3] step 4960/12000 loss=4.7256 gn=0.75 lr=4.11e-04 tps=18354 mem=5.7G lam=0.22397184185683727 Gm=0.3848319853035112 +[full s3] step 4980/12000 loss=4.6801 gn=0.78 lr=4.09e-04 tps=18354 mem=5.7G lam=0.2243904114390413 Gm=0.3784740023159732 +[full s3] step 5000/12000 loss=4.5578 gn=0.80 lr=4.08e-04 tps=18354 mem=5.7G lam=0.22476688865572214 Gm=0.3760076643278201 +[full s3] EVAL step 5000: val_loss=4.7535 val_ppl=115.99 +[full s3] step 5020/12000 loss=4.7391 gn=0.72 lr=4.07e-04 tps=18340 mem=5.7G lam=0.2251187264919281 Gm=0.3876501211586098 +[full s3] step 5040/12000 loss=4.6113 gn=0.73 lr=4.05e-04 tps=18340 mem=5.7G lam=0.22546842818458876 Gm=0.377045548055321 +[full s3] step 5060/12000 loss=4.6489 gn=0.84 lr=4.04e-04 tps=18340 mem=5.7G lam=0.22586799009392658 Gm=0.3862920430644105 +[full s3] step 5080/12000 loss=4.7155 gn=0.78 lr=4.02e-04 tps=18340 mem=5.7G lam=0.22629977824787298 Gm=0.3779819467260192 +[full s3] step 5100/12000 loss=4.6149 gn=0.73 lr=4.01e-04 tps=18340 mem=5.7G lam=0.22678093736370405 Gm=0.38296296711390215 +[full s3] step 5120/12000 loss=4.7242 gn=0.72 lr=4.00e-04 tps=18339 mem=5.7G lam=0.22719127995272478 Gm=0.37534866326798993 +[full s3] step 5140/12000 loss=4.6539 gn=0.76 lr=3.98e-04 tps=18339 mem=5.7G lam=0.22765192917237678 Gm=0.376819768998151 +[full s3] step 5160/12000 loss=4.7825 gn=0.74 lr=3.97e-04 tps=18340 mem=5.7G lam=0.2279865968351563 Gm=0.3827038930418591 +[full s3] step 5180/12000 loss=4.4998 gn=0.76 lr=3.95e-04 tps=18340 mem=5.7G lam=0.22841981643189987 Gm=0.3739876184457292 +[full s3] step 5200/12000 loss=4.5233 gn=0.93 lr=3.94e-04 tps=18340 mem=5.7G lam=0.2288213592643539 Gm=0.3792594246721516 +[full s3] step 5220/12000 loss=4.7243 gn=0.79 lr=3.93e-04 tps=18340 mem=5.7G lam=0.22922184442480406 Gm=0.38024860286774737 +[full s3] step 5240/12000 loss=4.5415 gn=0.75 lr=3.91e-04 tps=18340 mem=5.7G lam=0.2296016135563453 Gm=0.38339143246412277 +[full s3] step 5260/12000 loss=4.6997 gn=0.81 lr=3.90e-04 tps=18339 mem=5.7G lam=0.22997133309642473 Gm=0.38412390757973 +[full s3] step 5280/12000 loss=4.5396 gn=0.78 lr=3.88e-04 tps=18338 mem=5.7G lam=0.23045416766156754 Gm=0.37054836718986434 +[full s3] step 5300/12000 loss=4.7387 gn=0.77 lr=3.87e-04 tps=18338 mem=5.7G lam=0.23085753619670868 Gm=0.3778844013189276 +[full s3] step 5320/12000 loss=4.5725 gn=0.72 lr=3.86e-04 tps=18338 mem=5.7G lam=0.23125094237426916 Gm=0.37145498480337363 +[full s3] step 5340/12000 loss=4.5588 gn=0.73 lr=3.84e-04 tps=18338 mem=5.7G lam=0.23167960718274117 Gm=0.3761174362152815 +[full s3] step 5360/12000 loss=4.5940 gn=0.78 lr=3.83e-04 tps=18337 mem=5.7G lam=0.23204128226886192 Gm=0.37326604236538213 +[full s3] step 5380/12000 loss=4.7233 gn=0.80 lr=3.81e-04 tps=18338 mem=5.7G lam=0.23242796305567026 Gm=0.38310300139710307 +[full s3] step 5400/12000 loss=4.6554 gn=0.77 lr=3.80e-04 tps=18338 mem=5.7G lam=0.2328367323304216 Gm=0.37658443907275796 +[full s3] step 5420/12000 loss=4.7920 gn=0.76 lr=3.79e-04 tps=18339 mem=5.7G lam=0.23328816114614406 Gm=0.3740104866058876 +[full s3] step 5440/12000 loss=4.5230 gn=0.82 lr=3.77e-04 tps=18339 mem=5.7G lam=0.2337073109423121 Gm=0.3782471634913236 +[full s3] step 5460/12000 loss=4.5742 gn=0.75 lr=3.76e-04 tps=18339 mem=5.7G lam=0.23415192868560553 Gm=0.3781446445112427 +[full s3] step 5480/12000 loss=4.5431 gn=0.75 lr=3.74e-04 tps=18339 mem=5.7G lam=0.2345329439267516 Gm=0.38066133623942733 +[full s3] step 5500/12000 loss=4.7786 gn=0.83 lr=3.73e-04 tps=18338 mem=5.7G lam=0.2349592304478089 Gm=0.3821127105038613 +[full s3] step 5520/12000 loss=4.7748 gn=0.78 lr=3.72e-04 tps=18338 mem=5.7G lam=0.23543389576176801 Gm=0.37501552964871127 +[full s3] step 5540/12000 loss=4.6661 gn=0.77 lr=3.70e-04 tps=18337 mem=5.7G lam=0.23586657115568718 Gm=0.3711177813820541 +[full s3] step 5560/12000 loss=4.6788 gn=0.84 lr=3.69e-04 tps=18337 mem=5.7G lam=0.23617792191604772 Gm=0.37177359134269256 +[full s3] step 5580/12000 loss=4.5124 gn=0.74 lr=3.67e-04 tps=18337 mem=5.7G lam=0.23648266214877367 Gm=0.36999211196477216 +[full s3] step 5600/12000 loss=4.7176 gn=0.86 lr=3.66e-04 tps=18337 mem=5.7G lam=0.236950953801473 Gm=0.3828983975884815 +[full s3] step 5620/12000 loss=4.5773 gn=0.79 lr=3.64e-04 tps=18337 mem=5.7G lam=0.23739150445908308 Gm=0.37806196689295274 +[full s3] step 5640/12000 loss=4.5847 gn=0.77 lr=3.63e-04 tps=18336 mem=5.7G lam=0.23773668085535368 Gm=0.37807010137476027 +[full s3] step 5660/12000 loss=4.6393 gn=0.78 lr=3.62e-04 tps=18336 mem=5.7G lam=0.23809135208527246 Gm=0.3763364808789144 +[full s3] step 5680/12000 loss=4.7397 gn=0.80 lr=3.60e-04 tps=18336 mem=5.7G lam=0.23844033479690552 Gm=0.3738628790403406 +[full s3] step 5700/12000 loss=4.6054 gn=0.79 lr=3.59e-04 tps=18336 mem=5.7G lam=0.23875730050106844 Gm=0.3788930573500693 +[full s3] step 5720/12000 loss=4.6029 gn=0.72 lr=3.57e-04 tps=18336 mem=5.7G lam=0.23920176457613707 Gm=0.36759191115076345 +[full s3] step 5740/12000 loss=4.6247 gn=0.79 lr=3.56e-04 tps=18336 mem=5.7G lam=0.2395819534237186 Gm=0.3680620277300477 +[full s3] step 5760/12000 loss=4.5421 gn=0.74 lr=3.54e-04 tps=18336 mem=5.7G lam=0.2398928158606092 Gm=0.3635546457177649 +[full s3] step 5780/12000 loss=4.6556 gn=0.79 lr=3.53e-04 tps=18336 mem=5.7G lam=0.24029029409090677 Gm=0.3740282092864315 +[full s3] step 5800/12000 loss=4.6346 gn=0.78 lr=3.52e-04 tps=18336 mem=5.7G lam=0.2407055295382937 Gm=0.3686633388667057 +[full s3] step 5820/12000 loss=4.4696 gn=0.85 lr=3.50e-04 tps=18336 mem=5.7G lam=0.2410897510126233 Gm=0.3790580452575038 +[full s3] step 5840/12000 loss=4.5281 gn=0.83 lr=3.49e-04 tps=18336 mem=5.7G lam=0.24152207809189954 Gm=0.3736343678707878 +[full s3] step 5860/12000 loss=4.4757 gn=0.74 lr=3.47e-04 tps=18336 mem=5.7G lam=0.24188381309310594 Gm=0.36535973629603785 +[full s3] step 5880/12000 loss=4.4795 gn=0.77 lr=3.46e-04 tps=18336 mem=5.7G lam=0.24223223390678564 Gm=0.3701937525377919 +[full s3] step 5900/12000 loss=4.5267 gn=0.78 lr=3.44e-04 tps=18336 mem=5.7G lam=0.24261655813703933 Gm=0.37502494268119335 +[full s3] step 5920/12000 loss=4.5339 gn=0.75 lr=3.43e-04 tps=18336 mem=5.7G lam=0.24299323682983717 Gm=0.37229222897440195 +[full s3] step 5940/12000 loss=4.6771 gn=0.78 lr=3.41e-04 tps=18337 mem=5.7G lam=0.24334117801239094 Gm=0.3706366525342067 +[full s3] step 5960/12000 loss=4.6107 gn=0.77 lr=3.40e-04 tps=18337 mem=5.7G lam=0.2436943125600616 Gm=0.36438395276976127 +[full s3] step 5980/12000 loss=4.4434 gn=0.75 lr=3.39e-04 tps=18337 mem=5.7G lam=0.24405634496361017 Gm=0.3691699397750199 +[full s3] step 6000/12000 loss=4.6710 gn=0.80 lr=3.37e-04 tps=18337 mem=5.7G lam=0.2443741594130794 Gm=0.37089095381088555 +[full s3] EVAL step 6000: val_loss=4.7104 val_ppl=111.10 +[full s3] step 6020/12000 loss=4.7625 gn=0.78 lr=3.36e-04 tps=18325 mem=5.7G lam=0.2447616591428717 Gm=0.36853502426917356 +[full s3] step 6040/12000 loss=4.5437 gn=0.78 lr=3.34e-04 tps=18324 mem=5.7G lam=0.2451704159999887 Gm=0.3647017839054267 +[full s3] step 6060/12000 loss=4.4885 gn=0.83 lr=3.33e-04 tps=18324 mem=5.7G lam=0.2455772248407205 Gm=0.3752304071870943 +[full s3] step 6080/12000 loss=4.5084 gn=0.77 lr=3.31e-04 tps=18324 mem=5.7G lam=0.24604046189536652 Gm=0.3645928835806747 +[full s3] step 6100/12000 loss=4.6385 gn=0.77 lr=3.30e-04 tps=18324 mem=5.7G lam=0.24637730636944374 Gm=0.3722895449803521 +[full s3] step 6120/12000 loss=4.6766 gn=0.81 lr=3.29e-04 tps=18324 mem=5.7G lam=0.24672406073659658 Gm=0.374088972962151 +[full s3] step 6140/12000 loss=4.6383 gn=0.84 lr=3.27e-04 tps=18324 mem=5.7G lam=0.24704495444893837 Gm=0.3757207386661321 +[full s3] step 6160/12000 loss=4.5124 gn=0.80 lr=3.26e-04 tps=18325 mem=5.7G lam=0.24736965789149204 Gm=0.37236811057664454 +[full s3] step 6180/12000 loss=4.5943 gn=0.81 lr=3.24e-04 tps=18324 mem=5.7G lam=0.24769688645998636 Gm=0.36542558840786415 +[full s3] step 6200/12000 loss=4.5565 gn=0.98 lr=3.23e-04 tps=18324 mem=5.7G lam=0.24799491185694933 Gm=0.37386235866385203 +[full s3] step 6220/12000 loss=4.5758 gn=0.86 lr=3.21e-04 tps=18325 mem=5.7G lam=0.24830244295299053 Gm=0.3705524099059403 +[full s3] step 6240/12000 loss=4.6727 gn=0.82 lr=3.20e-04 tps=18324 mem=5.7G lam=0.24865479643146196 Gm=0.3735965151960651 +[full s3] step 6260/12000 loss=4.5419 gn=0.78 lr=3.19e-04 tps=18324 mem=5.7G lam=0.2489823174352447 Gm=0.3607375754509121 +[full s3] step 6280/12000 loss=4.5397 gn=0.80 lr=3.17e-04 tps=18324 mem=5.7G lam=0.24932513013482094 Gm=0.36319390738693375 +[full s3] step 6300/12000 loss=4.6550 gn=0.79 lr=3.16e-04 tps=18324 mem=5.7G lam=0.24966280286510786 Gm=0.37563608470372856 +[full s3] step 6320/12000 loss=4.4942 gn=0.80 lr=3.14e-04 tps=18324 mem=5.7G lam=0.24997488564501205 Gm=0.3644452599498133 +[full s3] step 6340/12000 loss=4.6238 gn=0.85 lr=3.13e-04 tps=18324 mem=5.7G lam=0.2503116171186169 Gm=0.3648172076791525 +[full s3] step 6360/12000 loss=4.7458 gn=0.84 lr=3.11e-04 tps=18324 mem=5.7G lam=0.25062658451497555 Gm=0.3705599857494235 +[full s3] step 6380/12000 loss=4.3022 gn=0.77 lr=3.10e-04 tps=18324 mem=5.7G lam=0.25100943353027105 Gm=0.36804948141798377 +[full s3] step 6400/12000 loss=4.5179 gn=0.79 lr=3.08e-04 tps=18324 mem=5.7G lam=0.25131553318351507 Gm=0.35871486730563146 +[full s3] step 6420/12000 loss=4.4469 gn=0.80 lr=3.07e-04 tps=18325 mem=5.7G lam=0.2515677126745383 Gm=0.3666742652033766 +[full s3] step 6440/12000 loss=4.6422 gn=0.78 lr=3.06e-04 tps=18325 mem=5.7G lam=0.25197612183789414 Gm=0.3664530764023463 +[full s3] step 6460/12000 loss=4.5016 gn=0.80 lr=3.04e-04 tps=18325 mem=5.7G lam=0.2522820355370641 Gm=0.3585635678221782 +[full s3] step 6480/12000 loss=4.5203 gn=0.86 lr=3.03e-04 tps=18325 mem=5.7G lam=0.2526124917591612 Gm=0.3723067866327862 +[full s3] step 6500/12000 loss=4.4861 gn=0.78 lr=3.01e-04 tps=18325 mem=5.7G lam=0.25299744835744303 Gm=0.3608428945299238 +[full s3] step 6520/12000 loss=4.5998 gn=0.81 lr=3.00e-04 tps=18325 mem=5.7G lam=0.2533393809571862 Gm=0.36904907459393144 +[full s3] step 6540/12000 loss=4.5193 gn=0.77 lr=2.98e-04 tps=18326 mem=5.7G lam=0.25364147406071424 Gm=0.35961776599287987 +[full s3] step 6560/12000 loss=4.6568 gn=0.81 lr=2.97e-04 tps=18326 mem=5.7G lam=0.2539324639365077 Gm=0.36233421647921205 +[full s3] step 6580/12000 loss=4.5935 gn=0.82 lr=2.96e-04 tps=18326 mem=5.7G lam=0.2541594185555975 Gm=0.3654542335619529 +[full s3] step 6600/12000 loss=4.4982 gn=0.81 lr=2.94e-04 tps=18327 mem=5.7G lam=0.2544409865513444 Gm=0.361358394070218 +[full s3] step 6620/12000 loss=4.4985 gn=0.89 lr=2.93e-04 tps=18327 mem=5.7G lam=0.25474490380535525 Gm=0.3651293655857444 +[full s3] step 6640/12000 loss=4.4051 gn=0.82 lr=2.91e-04 tps=18327 mem=5.7G lam=0.25505856797099113 Gm=0.36830374016426504 +[full s3] step 6660/12000 loss=4.4440 gn=0.78 lr=2.90e-04 tps=18328 mem=5.7G lam=0.2554361637060841 Gm=0.36484195897355676 +[full s3] step 6680/12000 loss=4.4669 gn=0.85 lr=2.88e-04 tps=18328 mem=5.7G lam=0.255720604211092 Gm=0.36581984798734385 +[full s3] step 6700/12000 loss=4.7730 gn=0.89 lr=2.87e-04 tps=18328 mem=5.7G lam=0.2560019101947546 Gm=0.3671158990667512 +[full s3] step 6720/12000 loss=4.4173 gn=0.79 lr=2.86e-04 tps=18328 mem=5.7G lam=0.2563060764223337 Gm=0.36336848194090027 +[full s3] step 6740/12000 loss=4.4790 gn=0.83 lr=2.84e-04 tps=18328 mem=5.7G lam=0.25663555692881346 Gm=0.3700487726988892 +[full s3] step 6760/12000 loss=4.5540 gn=0.83 lr=2.83e-04 tps=18328 mem=5.7G lam=0.25692305869112414 Gm=0.36684143954577547 +[full s3] step 6780/12000 loss=4.5290 gn=0.77 lr=2.81e-04 tps=18328 mem=5.7G lam=0.2572859919940432 Gm=0.3671871934396525 +[full s3] step 6800/12000 loss=4.6607 gn=0.80 lr=2.80e-04 tps=18328 mem=5.7G lam=0.2576060987388094 Gm=0.3643089454465856 +[full s3] step 6820/12000 loss=4.5679 gn=0.85 lr=2.79e-04 tps=18328 mem=5.7G lam=0.25792034653325874 Gm=0.3542603384703398 +[full s3] step 6840/12000 loss=4.5557 gn=0.84 lr=2.77e-04 tps=18328 mem=5.7G lam=0.2582073525215189 Gm=0.36102844029664993 +[full s3] step 6860/12000 loss=4.5502 gn=0.83 lr=2.76e-04 tps=18328 mem=5.7G lam=0.25846464342127246 Gm=0.367682463179032 +[full s3] step 6880/12000 loss=4.5469 gn=0.82 lr=2.74e-04 tps=18329 mem=5.7G lam=0.2587856752797961 Gm=0.35594909607122344 +[full s3] step 6900/12000 loss=4.5427 gn=0.86 lr=2.73e-04 tps=18328 mem=5.7G lam=0.2591222968573372 Gm=0.35906137991696596 +[full s3] step 6920/12000 loss=4.4828 gn=0.88 lr=2.72e-04 tps=18329 mem=5.7G lam=0.2594361972684662 Gm=0.3619955932566275 +[full s3] step 6940/12000 loss=4.3729 gn=0.80 lr=2.70e-04 tps=18328 mem=5.7G lam=0.25967891917874414 Gm=0.3594913366250694 +[full s3] step 6960/12000 loss=4.4222 gn=0.80 lr=2.69e-04 tps=18328 mem=5.7G lam=0.2599240705991785 Gm=0.36103813668402535 +[full s3] step 6980/12000 loss=4.6111 gn=0.84 lr=2.67e-04 tps=18329 mem=5.7G lam=0.260183507266144 Gm=0.36581425050583977 +[full s3] step 7000/12000 loss=4.4009 gn=0.80 lr=2.66e-04 tps=18329 mem=5.7G lam=0.26047360183050233 Gm=0.3618803847736369 +[full s3] EVAL step 7000: val_loss=4.6035 val_ppl=99.84 +[full s3] step 7020/12000 loss=4.4800 gn=0.79 lr=2.65e-04 tps=18319 mem=5.7G lam=0.2607491910457611 Gm=0.36203518657324213 +[full s3] step 7040/12000 loss=4.4663 gn=0.82 lr=2.63e-04 tps=18320 mem=5.7G lam=0.26100581884384155 Gm=0.3612016891905417 +[full s3] step 7060/12000 loss=4.5269 gn=0.86 lr=2.62e-04 tps=18320 mem=5.7G lam=0.2611930112664898 Gm=0.3575824243016541 +[full s3] step 7080/12000 loss=4.4766 gn=0.89 lr=2.60e-04 tps=18320 mem=5.7G lam=0.26153569140781957 Gm=0.36780355296408135 +[full s3] step 7100/12000 loss=4.4771 gn=0.86 lr=2.59e-04 tps=18320 mem=5.7G lam=0.2617917073269685 Gm=0.36062288431761164 +[full s3] step 7120/12000 loss=4.6050 gn=0.86 lr=2.58e-04 tps=18320 mem=5.7G lam=0.26204233927031356 Gm=0.3581040167870621 +[full s3] step 7140/12000 loss=4.5004 gn=0.86 lr=2.56e-04 tps=18318 mem=5.7G lam=0.26232007797807455 Gm=0.3623841015311579 +[full s3] step 7160/12000 loss=4.6677 gn=0.84 lr=2.55e-04 tps=18317 mem=5.7G lam=0.2625846279164155 Gm=0.36276452980625135 +[full s3] step 7180/12000 loss=4.4850 gn=0.83 lr=2.53e-04 tps=18317 mem=5.7G lam=0.2628077395881216 Gm=0.36237675611240167 +[full s3] step 7200/12000 loss=4.4098 gn=0.83 lr=2.52e-04 tps=18318 mem=5.7G lam=0.26308702925841015 Gm=0.35581129835918546 +[full s3] step 7220/12000 loss=4.4061 gn=0.81 lr=2.51e-04 tps=18317 mem=5.7G lam=0.26343684053669375 Gm=0.3613181870896369 +[full s3] step 7240/12000 loss=4.5625 gn=0.83 lr=2.49e-04 tps=18318 mem=5.7G lam=0.26374314818531275 Gm=0.3562360542515914 +[full s3] step 7260/12000 loss=4.3922 gn=0.90 lr=2.48e-04 tps=18318 mem=5.7G lam=0.26396917179226875 Gm=0.3630204804552098 +[full s3] step 7280/12000 loss=4.3664 gn=0.83 lr=2.47e-04 tps=18318 mem=5.7G lam=0.2642477198193471 Gm=0.359071810496971 +[full s3] step 7300/12000 loss=4.6205 gn=0.84 lr=2.45e-04 tps=18317 mem=5.7G lam=0.26445930792639655 Gm=0.35940942090625566 +[full s3] step 7320/12000 loss=4.6208 gn=0.89 lr=2.44e-04 tps=18317 mem=5.7G lam=0.2647029170766473 Gm=0.3659455721111347 +[full s3] step 7340/12000 loss=4.4282 gn=0.95 lr=2.42e-04 tps=18317 mem=5.7G lam=0.2650334605326255 Gm=0.3622627378596614 +[full s3] step 7360/12000 loss=4.4035 gn=0.83 lr=2.41e-04 tps=18317 mem=5.7G lam=0.2653195010498166 Gm=0.3551031503205498 +[full s3] step 7380/12000 loss=4.4485 gn=0.82 lr=2.40e-04 tps=18317 mem=5.7G lam=0.2656047359729807 Gm=0.35867582975576323 +[full s3] step 7400/12000 loss=4.5751 gn=0.85 lr=2.38e-04 tps=18317 mem=5.7G lam=0.26588967877129716 Gm=0.36235731467604637 +[full s3] step 7420/12000 loss=4.4311 gn=0.81 lr=2.37e-04 tps=18317 mem=5.7G lam=0.2660984347263972 Gm=0.3553557097135733 +[full s3] step 7440/12000 loss=4.5047 gn=0.88 lr=2.36e-04 tps=18317 mem=5.7G lam=0.2663535649577777 Gm=0.36501626390963793 +[full s3] step 7460/12000 loss=4.5825 gn=0.81 lr=2.34e-04 tps=18317 mem=5.7G lam=0.2665923833847046 Gm=0.3595247194947054 +[full s3] step 7480/12000 loss=4.5052 gn=0.86 lr=2.33e-04 tps=18317 mem=5.7G lam=0.2668519727885723 Gm=0.365518368082121 +[full s3] step 7500/12000 loss=4.5768 gn=0.88 lr=2.32e-04 tps=18318 mem=5.7G lam=0.2671386795118451 Gm=0.3575469844508916 +[full s3] step 7520/12000 loss=4.3935 gn=0.84 lr=2.30e-04 tps=18318 mem=5.7G lam=0.2674341605355342 Gm=0.3529654696273307 +[full s3] step 7540/12000 loss=4.4898 gn=0.83 lr=2.29e-04 tps=18318 mem=5.7G lam=0.26766815936813754 Gm=0.35181507871796686 +[full s3] step 7560/12000 loss=4.5311 gn=0.84 lr=2.28e-04 tps=18318 mem=5.7G lam=0.26784639277805883 Gm=0.3544789324514568 +[full s3] step 7580/12000 loss=4.4293 gn=0.91 lr=2.26e-04 tps=18319 mem=5.7G lam=0.268088989580671 Gm=0.35859119535113376 +[full s3] step 7600/12000 loss=4.3795 gn=0.85 lr=2.25e-04 tps=18319 mem=5.7G lam=0.26832651533186436 Gm=0.35889100974115234 +[full s3] step 7620/12000 loss=4.2890 gn=0.89 lr=2.24e-04 tps=18319 mem=5.7G lam=0.2684954299281041 Gm=0.36144600676683086 +[full s3] step 7640/12000 loss=4.5022 gn=0.84 lr=2.22e-04 tps=18319 mem=5.7G lam=0.26877264585345984 Gm=0.35688538864875835 +[full s3] step 7660/12000 loss=4.4107 gn=0.89 lr=2.21e-04 tps=18319 mem=5.7G lam=0.26903496589511633 Gm=0.3646016671943168 +[full s3] step 7680/12000 loss=4.5207 gn=0.88 lr=2.20e-04 tps=18319 mem=5.7G lam=0.2692575032512347 Gm=0.3663042043335736 +[full s3] step 7700/12000 loss=4.4420 gn=0.87 lr=2.18e-04 tps=18320 mem=5.7G lam=0.2694818538924058 Gm=0.3576083799513678 +[full s3] step 7720/12000 loss=4.3248 gn=0.93 lr=2.17e-04 tps=18320 mem=5.7G lam=0.26970157648126286 Gm=0.3640928976237774 +[full s3] step 7740/12000 loss=4.3165 gn=0.82 lr=2.16e-04 tps=18320 mem=5.7G lam=0.26994194462895393 Gm=0.3571810941211879 +[full s3] step 7760/12000 loss=4.4180 gn=0.85 lr=2.15e-04 tps=18320 mem=5.7G lam=0.27017108506212634 Gm=0.3607874579417209 +[full s3] step 7780/12000 loss=4.4365 gn=0.80 lr=2.13e-04 tps=18320 mem=5.7G lam=0.27032163242499035 Gm=0.35984506939227384 +[full s3] step 7800/12000 loss=4.4370 gn=0.94 lr=2.12e-04 tps=18320 mem=5.7G lam=0.27051810671885806 Gm=0.3564123718533665 +[full s3] step 7820/12000 loss=4.5254 gn=0.86 lr=2.11e-04 tps=18320 mem=5.7G lam=0.27074479032307863 Gm=0.3639300264573346 +[full s3] step 7840/12000 loss=4.4603 gn=0.86 lr=2.09e-04 tps=18321 mem=5.7G lam=0.2710071870436271 Gm=0.35547690900663537 +[full s3] step 7860/12000 loss=4.2302 gn=0.83 lr=2.08e-04 tps=18321 mem=5.7G lam=0.27123791351914406 Gm=0.353484148470064 +[full s3] step 7880/12000 loss=4.5280 gn=0.88 lr=2.07e-04 tps=18321 mem=5.7G lam=0.271484666193525 Gm=0.35691421704056364 +[full s3] step 7900/12000 loss=4.5224 gn=0.92 lr=2.06e-04 tps=18322 mem=5.7G lam=0.2716900072991848 Gm=0.35698518824453157 +[full s3] step 7920/12000 loss=4.3013 gn=0.82 lr=2.04e-04 tps=18322 mem=5.7G lam=0.2719091434652607 Gm=0.3541745458108683 +[full s3] step 7940/12000 loss=4.4830 gn=0.86 lr=2.03e-04 tps=18322 mem=5.7G lam=0.2721147056048115 Gm=0.356257867688934 +[full s3] step 7960/12000 loss=4.5237 gn=0.85 lr=2.02e-04 tps=18322 mem=5.7G lam=0.2723151324316859 Gm=0.35259134446581203 +[full s3] step 7980/12000 loss=4.3992 gn=0.87 lr=2.00e-04 tps=18322 mem=5.7G lam=0.2724986433361967 Gm=0.3558991802080224 +[full s3] step 8000/12000 loss=4.4620 gn=0.83 lr=1.99e-04 tps=18323 mem=5.7G lam=0.2727404066051046 Gm=0.35677089379169047 +[full s3] EVAL step 8000: val_loss=4.4880 val_ppl=88.94 +[full s3] step 8020/12000 loss=4.5158 gn=0.88 lr=1.98e-04 tps=18315 mem=5.7G lam=0.273012169636786 Gm=0.36113277124240994 +[full s3] step 8040/12000 loss=4.5293 gn=0.87 lr=1.97e-04 tps=18315 mem=5.7G lam=0.2732170249025027 Gm=0.36325217977476615 +[full s3] step 8060/12000 loss=4.4842 gn=0.88 lr=1.95e-04 tps=18315 mem=5.7G lam=0.27344741610189277 Gm=0.36031649021121365 +[full s3] step 8080/12000 loss=4.4040 gn=0.87 lr=1.94e-04 tps=18315 mem=5.7G lam=0.27367102075368166 Gm=0.3604877424271156 +[full s3] step 8100/12000 loss=4.2707 gn=0.85 lr=1.93e-04 tps=18316 mem=5.7G lam=0.2738702005396287 Gm=0.349901888674746 +[full s3] step 8120/12000 loss=4.7112 gn=0.91 lr=1.92e-04 tps=18316 mem=5.7G lam=0.2740819612517953 Gm=0.356723407516256 +[full s3] step 8140/12000 loss=4.4090 gn=0.93 lr=1.90e-04 tps=18316 mem=5.7G lam=0.2743065633500616 Gm=0.3510019814129919 +[full s3] step 8160/12000 loss=4.3738 gn=0.86 lr=1.89e-04 tps=18317 mem=5.7G lam=0.27453477028757334 Gm=0.355182617281874 +[full s3] step 8180/12000 loss=4.4945 gn=0.87 lr=1.88e-04 tps=18317 mem=5.7G lam=0.2747163601840536 Gm=0.3580277485307306 +[full s3] step 8200/12000 loss=4.4428 gn=0.89 lr=1.87e-04 tps=18316 mem=5.7G lam=0.27490822163720924 Gm=0.35885201146205264 +[full s3] step 8220/12000 loss=4.4474 gn=0.88 lr=1.86e-04 tps=18316 mem=5.7G lam=0.2751280531908075 Gm=0.35241165997770924 +[full s3] step 8240/12000 loss=4.5847 gn=0.95 lr=1.84e-04 tps=18317 mem=5.7G lam=0.2753311734025677 Gm=0.356693366697679 +[full s3] step 8260/12000 loss=4.4265 gn=0.87 lr=1.83e-04 tps=18315 mem=5.7G lam=0.27550910351177055 Gm=0.35543801775202155 +[full s3] step 8280/12000 loss=4.3671 gn=0.85 lr=1.82e-04 tps=18313 mem=5.7G lam=0.2757022924100359 Gm=0.3557431463462611 +[full s3] step 8300/12000 loss=4.4147 gn=0.86 lr=1.81e-04 tps=18312 mem=5.7G lam=0.2758780065923929 Gm=0.3571136014846464 +[full s3] step 8320/12000 loss=4.3671 gn=0.85 lr=1.80e-04 tps=18309 mem=5.7G lam=0.27606100620081025 Gm=0.35937180331287283 +[full s3] step 8340/12000 loss=4.4631 gn=0.87 lr=1.78e-04 tps=18306 mem=5.7G lam=0.2761922487989068 Gm=0.35712250353147584 +[full s3] step 8360/12000 loss=4.3100 gn=0.84 lr=1.77e-04 tps=18302 mem=5.7G lam=0.2763822330161929 Gm=0.3528022455672423 +[full s3] step 8380/12000 loss=4.4090 gn=0.93 lr=1.76e-04 tps=18299 mem=5.7G lam=0.27663516501585644 Gm=0.3559374023073663 +[full s3] step 8400/12000 loss=4.4740 gn=0.91 lr=1.75e-04 tps=18296 mem=5.7G lam=0.276860348880291 Gm=0.3563720517947028 +[full s3] step 8420/12000 loss=4.4768 gn=0.94 lr=1.74e-04 tps=18293 mem=5.7G lam=0.2770481885721286 Gm=0.35559853445738554 +[full s3] step 8440/12000 loss=4.4087 gn=0.90 lr=1.72e-04 tps=18293 mem=5.7G lam=0.27720764372497797 Gm=0.3593213196533422 +[full s3] step 8460/12000 loss=4.4818 gn=0.87 lr=1.71e-04 tps=18292 mem=5.7G lam=0.2773886499926448 Gm=0.3614325935486704 +[full s3] step 8480/12000 loss=4.4394 gn=0.88 lr=1.70e-04 tps=18291 mem=5.7G lam=0.2775308834388852 Gm=0.35719790984876454 +[full s3] step 8500/12000 loss=4.4154 gn=0.92 lr=1.69e-04 tps=18291 mem=5.7G lam=0.2777457196886341 Gm=0.36430941817040247 +[full s3] step 8520/12000 loss=4.4121 gn=0.95 lr=1.68e-04 tps=18291 mem=5.7G lam=0.27790119281659525 Gm=0.3538981336168945 +[full s3] step 8540/12000 loss=4.4046 gn=0.87 lr=1.67e-04 tps=18291 mem=5.7G lam=0.2780565644303958 Gm=0.3579304240799199 +[full s3] step 8560/12000 loss=4.3768 gn=0.92 lr=1.66e-04 tps=18292 mem=5.7G lam=0.27820424269884825 Gm=0.3534997010913988 +[full s3] step 8580/12000 loss=4.4819 gn=0.94 lr=1.64e-04 tps=18292 mem=5.7G lam=0.27834729260454577 Gm=0.3533222808813055 +[full s3] step 8600/12000 loss=4.3223 gn=0.90 lr=1.63e-04 tps=18292 mem=5.7G lam=0.2785165250922243 Gm=0.3584717948300143 +[full s3] step 8620/12000 loss=4.4187 gn=0.89 lr=1.62e-04 tps=18293 mem=5.7G lam=0.27866464977463085 Gm=0.35663530440069735 +[full s3] step 8640/12000 loss=4.4237 gn=0.89 lr=1.61e-04 tps=18293 mem=5.7G lam=0.2788654025644064 Gm=0.34952424427804846 +[full s3] step 8660/12000 loss=4.4359 gn=0.88 lr=1.60e-04 tps=18293 mem=5.7G lam=0.2790245621775587 Gm=0.35005378754188615 +[full s3] step 8680/12000 loss=4.4694 gn=0.91 lr=1.59e-04 tps=18294 mem=5.7G lam=0.27916327863931656 Gm=0.35912969793813926 +[full s3] step 8700/12000 loss=4.4170 gn=0.85 lr=1.58e-04 tps=18294 mem=5.7G lam=0.2793611378098528 Gm=0.3561600255779922 +[full s3] step 8720/12000 loss=4.4565 gn=0.88 lr=1.57e-04 tps=18294 mem=5.7G lam=0.2795328112939994 Gm=0.3502627472237994 +[full s3] step 8740/12000 loss=4.3469 gn=0.94 lr=1.55e-04 tps=18295 mem=5.7G lam=0.2797471219673753 Gm=0.3606275020477672 +[full s3] step 8760/12000 loss=4.4211 gn=0.86 lr=1.54e-04 tps=18294 mem=5.7G lam=0.27992266820122796 Gm=0.3572605062897007 +[full s3] step 8780/12000 loss=4.2791 gn=0.87 lr=1.53e-04 tps=18295 mem=5.7G lam=0.2800466225792964 Gm=0.357064885708193 +[full s3] step 8800/12000 loss=4.4594 gn=0.92 lr=1.52e-04 tps=18297 mem=5.7G lam=0.28015340336908895 Gm=0.35757391519534093 +[full s3] step 8820/12000 loss=4.5017 gn=0.91 lr=1.51e-04 tps=18297 mem=5.7G lam=0.28032159867386025 Gm=0.35366191947832704 +[full s3] step 8840/12000 loss=4.4118 gn=0.91 lr=1.50e-04 tps=18297 mem=5.7G lam=0.2804915237550934 Gm=0.35293584464428324 +[full s3] step 8860/12000 loss=4.3505 gn=0.89 lr=1.49e-04 tps=18297 mem=5.7G lam=0.28066204208880663 Gm=0.3551287646405399 +[full s3] step 8880/12000 loss=4.5632 gn=0.91 lr=1.48e-04 tps=18297 mem=5.7G lam=0.28078710070500773 Gm=0.3576138530236979 +[full s3] step 8900/12000 loss=4.3984 gn=0.84 lr=1.47e-04 tps=18297 mem=5.7G lam=0.2809343704332908 Gm=0.3556620665670683 +[full s3] step 8920/12000 loss=4.4094 gn=0.90 lr=1.46e-04 tps=18297 mem=5.7G lam=0.2811157278095682 Gm=0.35252772682967287 +[full s3] step 8940/12000 loss=4.2839 gn=0.91 lr=1.45e-04 tps=18297 mem=5.7G lam=0.28128890817364055 Gm=0.3453702407423407 +[full s3] step 8960/12000 loss=4.4481 gn=0.91 lr=1.44e-04 tps=18297 mem=5.7G lam=0.2814262059206764 Gm=0.35315877889903885 +[full s3] step 8980/12000 loss=4.4407 gn=0.96 lr=1.43e-04 tps=18297 mem=5.7G lam=0.2815959909930825 Gm=0.3529163314184795 +[full s3] step 9000/12000 loss=4.5282 gn=0.87 lr=1.42e-04 tps=18297 mem=5.7G lam=0.28172016764680546 Gm=0.3579088693950325 +[full s3] EVAL step 9000: val_loss=4.4629 val_ppl=86.74 +[full s3] step 9020/12000 loss=4.3433 gn=0.91 lr=1.41e-04 tps=18289 mem=5.7G lam=0.2818328182523449 Gm=0.35178415616974235 +[full s3] step 9040/12000 loss=4.4291 gn=0.90 lr=1.40e-04 tps=18289 mem=5.7G lam=0.28196887268374365 Gm=0.35286782938055694 +[full s3] step 9060/12000 loss=4.2427 gn=0.89 lr=1.39e-04 tps=18289 mem=5.7G lam=0.2821289875234167 Gm=0.3456715321323524 +[full s3] step 9080/12000 loss=4.3879 gn=0.92 lr=1.38e-04 tps=18289 mem=5.7G lam=0.2822478556384643 Gm=0.35451657449205715 +[full s3] step 9100/12000 loss=4.5005 gn=0.86 lr=1.37e-04 tps=18289 mem=5.7G lam=0.28241043258458376 Gm=0.3522710995748639 +[full s3] step 9120/12000 loss=4.4697 gn=0.91 lr=1.36e-04 tps=18290 mem=5.7G lam=0.28252696214864653 Gm=0.35066289082169533 +[full s3] step 9140/12000 loss=4.2457 gn=0.88 lr=1.35e-04 tps=18290 mem=5.7G lam=0.28268453913430375 Gm=0.34992530872114 +[full s3] step 9160/12000 loss=4.5323 gn=1.06 lr=1.34e-04 tps=18290 mem=5.7G lam=0.2828342750047644 Gm=0.3583304548325638 +[full s3] step 9180/12000 loss=4.3359 gn=0.92 lr=1.33e-04 tps=18291 mem=5.7G lam=0.2829824403549234 Gm=0.35510068930064637 +[full s3] step 9200/12000 loss=4.4327 gn=0.89 lr=1.32e-04 tps=18291 mem=5.7G lam=0.28314910798023146 Gm=0.3491475930592666 +[full s3] step 9220/12000 loss=4.3753 gn=0.99 lr=1.31e-04 tps=18291 mem=5.7G lam=0.2833139232049386 Gm=0.3494332314003259 +[full s3] step 9240/12000 loss=4.3960 gn=0.98 lr=1.30e-04 tps=18291 mem=5.7G lam=0.28345864980171126 Gm=0.35085476787450415 +[full s3] step 9260/12000 loss=4.4264 gn=0.90 lr=1.29e-04 tps=18291 mem=5.7G lam=0.28358458417157334 Gm=0.35199599697565037 +[full s3] step 9280/12000 loss=4.4100 gn=0.94 lr=1.28e-04 tps=18291 mem=5.7G lam=0.2837104744588335 Gm=0.3574292487464845 +[full s3] step 9300/12000 loss=4.2934 gn=0.97 lr=1.27e-04 tps=18290 mem=5.7G lam=0.28383142842600745 Gm=0.3506297883577645 +[full s3] step 9320/12000 loss=4.5324 gn=0.90 lr=1.26e-04 tps=18290 mem=5.7G lam=0.2839677256221573 Gm=0.35241075480977696 +[full s3] step 9340/12000 loss=4.5263 gn=0.97 lr=1.25e-04 tps=18290 mem=5.7G lam=0.28410009760409594 Gm=0.35862439568154514 +[full s3] step 9360/12000 loss=4.3575 gn=0.97 lr=1.24e-04 tps=18290 mem=5.7G lam=0.2842389636983474 Gm=0.3513757574837655 +[full s3] step 9380/12000 loss=4.4194 gn=0.89 lr=1.23e-04 tps=18290 mem=5.7G lam=0.2843957481284936 Gm=0.35033719257141155 +[full s3] step 9400/12000 loss=4.3876 gn=0.96 lr=1.22e-04 tps=18290 mem=5.7G lam=0.28453630798806745 Gm=0.3539602153468877 +[full s3] step 9420/12000 loss=4.3812 gn=0.93 lr=1.21e-04 tps=18290 mem=5.7G lam=0.28466417888800305 Gm=0.35042750649154186 +[full s3] step 9440/12000 loss=4.2422 gn=0.88 lr=1.20e-04 tps=18290 mem=5.7G lam=0.2847707948336999 Gm=0.35526701738126576 +[full s3] step 9460/12000 loss=4.4833 gn=0.91 lr=1.19e-04 tps=18290 mem=5.7G lam=0.28490042717506486 Gm=0.35027493233792484 +[full s3] step 9480/12000 loss=4.5536 gn=0.97 lr=1.19e-04 tps=18290 mem=5.7G lam=0.28500588176151115 Gm=0.3535595720944305 +[full s3] step 9500/12000 loss=4.2856 gn=0.88 lr=1.18e-04 tps=18290 mem=5.7G lam=0.2851064608742793 Gm=0.3493980606241773 +[full s3] step 9520/12000 loss=4.4019 gn=1.00 lr=1.17e-04 tps=18290 mem=5.7G lam=0.28521721189220745 Gm=0.36052481713704765 +[full s3] step 9540/12000 loss=4.3531 gn=0.92 lr=1.16e-04 tps=18290 mem=5.7G lam=0.2853229319055875 Gm=0.35144722725575167 +[full s3] step 9560/12000 loss=4.2905 gn=0.88 lr=1.15e-04 tps=18290 mem=5.7G lam=0.2854468484098713 Gm=0.35453080967999995 +[full s3] step 9580/12000 loss=4.3894 gn=0.94 lr=1.14e-04 tps=18290 mem=5.7G lam=0.28555870801210403 Gm=0.3502388385434945 +[full s3] step 9600/12000 loss=4.5958 gn=0.94 lr=1.13e-04 tps=18291 mem=5.7G lam=0.28568361699581146 Gm=0.35032409251046676 +[full s3] step 9620/12000 loss=4.4064 gn=0.97 lr=1.12e-04 tps=18291 mem=5.7G lam=0.28582616802304983 Gm=0.3507808418944478 +[full s3] step 9640/12000 loss=4.2402 gn=0.95 lr=1.12e-04 tps=18291 mem=5.7G lam=0.28598050959408283 Gm=0.35422945573615533 +[full s3] step 9660/12000 loss=4.4774 gn=0.95 lr=1.11e-04 tps=18291 mem=5.7G lam=0.2860807413235307 Gm=0.3541374326062699 +[full s3] step 9680/12000 loss=4.3924 gn=1.01 lr=1.10e-04 tps=18292 mem=5.7G lam=0.28621187899261713 Gm=0.35010652026782435 +[full s3] step 9700/12000 loss=4.2538 gn=0.93 lr=1.09e-04 tps=18292 mem=5.7G lam=0.2863213500628869 Gm=0.3442042422636102 +[full s3] step 9720/12000 loss=4.3329 gn=0.93 lr=1.08e-04 tps=18292 mem=5.7G lam=0.2864219282443325 Gm=0.35216054460033774 +[full s3] step 9740/12000 loss=4.3190 gn=0.95 lr=1.07e-04 tps=18293 mem=5.7G lam=0.28650954955567914 Gm=0.34929375528978807 +[full s3] step 9760/12000 loss=4.3901 gn=0.93 lr=1.07e-04 tps=18293 mem=5.7G lam=0.28662734292447567 Gm=0.34823179555435974 +[full s3] step 9780/12000 loss=4.3316 gn=0.92 lr=1.06e-04 tps=18293 mem=5.7G lam=0.2867491543292999 Gm=0.351815311005339 +[full s3] step 9800/12000 loss=4.3316 gn=0.93 lr=1.05e-04 tps=18293 mem=5.7G lam=0.2868690388277173 Gm=0.3507756139151752 +[full s3] step 9820/12000 loss=4.2924 gn=0.93 lr=1.04e-04 tps=18293 mem=5.7G lam=0.2869994780048728 Gm=0.35083743215849 +[full s3] step 9840/12000 loss=4.3264 gn=0.92 lr=1.03e-04 tps=18294 mem=5.7G lam=0.2870847520728906 Gm=0.3460657140240073 +[full s3] step 9860/12000 loss=4.3193 gn=0.98 lr=1.03e-04 tps=18294 mem=5.7G lam=0.2871894808486104 Gm=0.3540173589717597 +[full s3] step 9880/12000 loss=4.2288 gn=0.91 lr=1.02e-04 tps=18294 mem=5.7G lam=0.28727624099701643 Gm=0.35040352881575626 +[full s3] step 9900/12000 loss=4.3377 gn=0.96 lr=1.01e-04 tps=18294 mem=5.7G lam=0.287381986156106 Gm=0.3488711613851289 +[full s3] step 9920/12000 loss=4.3303 gn=0.89 lr=1.00e-04 tps=18294 mem=5.7G lam=0.2874921889354785 Gm=0.34826913064656156 +[full s3] step 9940/12000 loss=4.3009 gn=1.01 lr=9.96e-05 tps=18294 mem=5.7G lam=0.2875727778300643 Gm=0.3496007982175797 +[full s3] step 9960/12000 loss=4.4780 gn=0.98 lr=9.89e-05 tps=18294 mem=5.7G lam=0.2876891726627946 Gm=0.3487881791467468 +[full s3] step 9980/12000 loss=4.2374 gn=0.95 lr=9.81e-05 tps=18294 mem=5.7G lam=0.28780122691144544 Gm=0.3509773470771809 +[full s3] step 10000/12000 loss=4.3870 gn=1.03 lr=9.74e-05 tps=18294 mem=5.7G lam=0.2879038928076625 Gm=0.3539778118332227 +[full s3] EVAL step 10000: val_loss=4.4536 val_ppl=85.94 +[full s3] step 10020/12000 loss=4.2533 gn=0.95 lr=9.67e-05 tps=18287 mem=5.7G lam=0.2880248427391052 Gm=0.348458306863904 +[full s3] step 10040/12000 loss=4.3540 gn=0.94 lr=9.59e-05 tps=18287 mem=5.7G lam=0.28811350588997203 Gm=0.35458426635401946 +[full s3] step 10060/12000 loss=4.2497 gn=0.98 lr=9.52e-05 tps=18288 mem=5.7G lam=0.28822563712795574 Gm=0.3482105729635805 +[full s3] step 10080/12000 loss=4.1905 gn=0.90 lr=9.45e-05 tps=18288 mem=5.7G lam=0.28830761307229597 Gm=0.34428616147488356 +[full s3] step 10100/12000 loss=4.4835 gn=1.04 lr=9.38e-05 tps=18288 mem=5.7G lam=0.28839656213919324 Gm=0.3511688280850649 +[full s3] step 10120/12000 loss=4.3871 gn=0.95 lr=9.31e-05 tps=18288 mem=5.7G lam=0.2884990470483899 Gm=0.34762357563401264 +[full s3] step 10140/12000 loss=4.2763 gn=0.93 lr=9.24e-05 tps=18288 mem=5.7G lam=0.288606157215933 Gm=0.34878535677368444 +[full s3] step 10160/12000 loss=4.4121 gn=0.94 lr=9.18e-05 tps=18288 mem=5.7G lam=0.28870289990057546 Gm=0.3464270392432809 +[full s3] step 10180/12000 loss=4.5039 gn=0.96 lr=9.11e-05 tps=18289 mem=5.7G lam=0.2887761077533166 Gm=0.35000671764525276 +[full s3] step 10200/12000 loss=4.4847 gn=0.97 lr=9.04e-05 tps=18289 mem=5.7G lam=0.2888695476576686 Gm=0.34927926554034155 +[full s3] step 10220/12000 loss=4.3882 gn=0.96 lr=8.98e-05 tps=18289 mem=5.7G lam=0.28896275473137695 Gm=0.3504551756195724 +[full s3] step 10240/12000 loss=4.2606 gn=0.97 lr=8.91e-05 tps=18289 mem=5.7G lam=0.2890955038989584 Gm=0.3490292942151427 +[full s3] step 10260/12000 loss=4.3947 gn=0.99 lr=8.85e-05 tps=18289 mem=5.7G lam=0.2892116916676362 Gm=0.34765375514204305 +[full s3] step 10280/12000 loss=4.3979 gn=1.04 lr=8.78e-05 tps=18290 mem=5.7G lam=0.2893339504177372 Gm=0.3496775613166392 +[full s3] step 10300/12000 loss=4.2973 gn=0.94 lr=8.72e-05 tps=18290 mem=5.7G lam=0.28941441761950654 Gm=0.3531998172402382 +[full s3] step 10320/12000 loss=4.4206 gn=0.95 lr=8.66e-05 tps=18290 mem=5.7G lam=0.2895001145079732 Gm=0.35077680779310566 +[full s3] step 10340/12000 loss=4.2640 gn=0.93 lr=8.59e-05 tps=18290 mem=5.7G lam=0.28957391685495776 Gm=0.34559814166277647 +[full s3] step 10360/12000 loss=4.2959 gn=0.92 lr=8.53e-05 tps=18290 mem=5.7G lam=0.2896354825546344 Gm=0.3475181677689155 +[full s3] step 10380/12000 loss=4.4295 gn=1.01 lr=8.47e-05 tps=18291 mem=5.7G lam=0.2896922367314498 Gm=0.354249512621512 +[full s3] step 10400/12000 loss=4.3854 gn=0.96 lr=8.41e-05 tps=18291 mem=5.7G lam=0.28975357456753653 Gm=0.3485590770530204 +[full s3] step 10420/12000 loss=4.2386 gn=0.95 lr=8.35e-05 tps=18291 mem=5.7G lam=0.28985659312456846 Gm=0.34802380825082463 +[full s3] step 10440/12000 loss=4.2359 gn=0.93 lr=8.30e-05 tps=18291 mem=5.7G lam=0.28993251143644255 Gm=0.3480362536696096 +[full s3] step 10460/12000 loss=4.4410 gn=1.01 lr=8.24e-05 tps=18291 mem=5.7G lam=0.29002055681000155 Gm=0.35174041070664924 +[full s3] step 10480/12000 loss=4.5702 gn=0.98 lr=8.18e-05 tps=18291 mem=5.7G lam=0.2901010476052761 Gm=0.3486231987675031 +[full s3] step 10500/12000 loss=4.3973 gn=0.99 lr=8.12e-05 tps=18291 mem=5.7G lam=0.2901807675758998 Gm=0.34877660490262014 +[full s3] step 10520/12000 loss=4.3030 gn=0.99 lr=8.07e-05 tps=18291 mem=5.7G lam=0.2902714600786567 Gm=0.3472804863316317 +[full s3] step 10540/12000 loss=4.2597 gn=0.94 lr=8.01e-05 tps=18292 mem=5.7G lam=0.2903604970003168 Gm=0.35422316162536543 +[full s3] step 10560/12000 loss=4.2291 gn=0.94 lr=7.96e-05 tps=18292 mem=5.7G lam=0.290447188851734 Gm=0.345743351848796 +[full s3] step 10580/12000 loss=4.4353 gn=0.97 lr=7.91e-05 tps=18292 mem=5.7G lam=0.29052525293082 Gm=0.35062486802538234 +[full s3] step 10600/12000 loss=4.2329 gn=0.94 lr=7.85e-05 tps=18292 mem=5.7G lam=0.29059709639598924 Gm=0.35067679717515904 +[full s3] step 10620/12000 loss=4.3925 gn=0.95 lr=7.80e-05 tps=18293 mem=5.7G lam=0.29067428316920996 Gm=0.34875851597947377 +[full s3] step 10640/12000 loss=4.2333 gn=0.92 lr=7.75e-05 tps=18293 mem=5.7G lam=0.2907598266998927 Gm=0.35635501115272444 +[full s3] step 10660/12000 loss=4.3226 gn=0.96 lr=7.70e-05 tps=18293 mem=5.7G lam=0.2908205473795533 Gm=0.3437254980672151 +[full s3] step 10680/12000 loss=4.3787 gn=1.02 lr=7.65e-05 tps=18293 mem=5.7G lam=0.29090423447390396 Gm=0.3481454163168867 +[full s3] step 10700/12000 loss=4.3840 gn=1.15 lr=7.60e-05 tps=18293 mem=5.7G lam=0.2910111201927066 Gm=0.344527550196896 +[full s3] step 10720/12000 loss=4.3073 gn=0.92 lr=7.55e-05 tps=18294 mem=5.7G lam=0.29109403925637406 Gm=0.34858141707566875 +[full s3] step 10740/12000 loss=4.2420 gn=0.95 lr=7.50e-05 tps=18294 mem=5.7G lam=0.29116902127861977 Gm=0.3501182604425897 +[full s3] step 10760/12000 loss=4.3562 gn=0.97 lr=7.46e-05 tps=18295 mem=5.7G lam=0.29123306771119434 Gm=0.3490511835552752 +[full s3] step 10780/12000 loss=4.3322 gn=0.96 lr=7.41e-05 tps=18295 mem=5.7G lam=0.2913126712664962 Gm=0.3471968839876354 +[full s3] step 10800/12000 loss=4.3719 gn=0.99 lr=7.37e-05 tps=18295 mem=5.7G lam=0.29138096049427986 Gm=0.34559186338447034 +[full s3] step 10820/12000 loss=4.3104 gn=0.99 lr=7.32e-05 tps=18295 mem=5.7G lam=0.2914893363292019 Gm=0.3520235445660849 +[full s3] step 10840/12000 loss=4.4673 gn=0.97 lr=7.28e-05 tps=18296 mem=5.7G lam=0.29156228620558977 Gm=0.3486957602823774 +[full s3] step 10860/12000 loss=4.3992 gn=0.99 lr=7.23e-05 tps=18296 mem=5.7G lam=0.29162190947681665 Gm=0.3492409452640762 +[full s3] step 10880/12000 loss=4.4121 gn=0.99 lr=7.19e-05 tps=18296 mem=5.7G lam=0.2916844558591644 Gm=0.34933063744877774 +[full s3] step 10900/12000 loss=4.2647 gn=0.96 lr=7.15e-05 tps=18296 mem=5.7G lam=0.2917492634927233 Gm=0.34308602986857295 +[full s3] step 10920/12000 loss=4.1621 gn=0.95 lr=7.11e-05 tps=18296 mem=5.7G lam=0.2918381628890832 Gm=0.3437670817753921 +[full s3] step 10940/12000 loss=4.4333 gn=0.97 lr=7.07e-05 tps=18296 mem=5.7G lam=0.2919292940447728 Gm=0.35436840712403256 +[full s3] step 10960/12000 loss=4.3468 gn=0.98 lr=7.03e-05 tps=18296 mem=5.7G lam=0.29202997280905646 Gm=0.3468780176093181 +[full s3] step 10980/12000 loss=4.3204 gn=1.01 lr=6.99e-05 tps=18296 mem=5.7G lam=0.2921091364696622 Gm=0.3481501841451973 +[full s3] step 11000/12000 loss=4.3267 gn=0.99 lr=6.95e-05 tps=18296 mem=5.7G lam=0.29220844618976116 Gm=0.3521840472240001 +[full s3] EVAL step 11000: val_loss=4.3724 val_ppl=79.24 +[full s3] step 11020/12000 loss=4.2843 gn=1.01 lr=6.91e-05 tps=18290 mem=5.7G lam=0.2922888156026602 Gm=0.35106598570321995 +[full s3] step 11040/12000 loss=4.2733 gn=0.94 lr=6.88e-05 tps=18290 mem=5.7G lam=0.29234910539040965 Gm=0.34253992000594735 +[full s3] step 11060/12000 loss=4.4118 gn=0.94 lr=6.84e-05 tps=18290 mem=5.7G lam=0.29243073302010697 Gm=0.34784327959641814 +[full s3] step 11080/12000 loss=4.1219 gn=0.99 lr=6.81e-05 tps=18291 mem=5.7G lam=0.2925067792336146 Gm=0.34792202594690025 +[full s3] step 11100/12000 loss=4.3700 gn=1.03 lr=6.77e-05 tps=18291 mem=5.7G lam=0.2925861269856493 Gm=0.3476062035188079 +[full s3] step 11120/12000 loss=4.3120 gn=0.93 lr=6.74e-05 tps=18291 mem=5.7G lam=0.29266684347142774 Gm=0.34260160150006413 +[full s3] step 11140/12000 loss=4.3310 gn=0.95 lr=6.70e-05 tps=18290 mem=5.7G lam=0.2927453387528658 Gm=0.3406032353329162 +[full s3] step 11160/12000 loss=4.3531 gn=0.97 lr=6.67e-05 tps=18290 mem=5.7G lam=0.29279179343332845 Gm=0.3500772487993042 +[full s3] step 11180/12000 loss=4.3058 gn=0.97 lr=6.64e-05 tps=18290 mem=5.7G lam=0.2928537093102932 Gm=0.3445511457199852 +[full s3] step 11200/12000 loss=4.2144 gn=0.97 lr=6.61e-05 tps=18291 mem=5.7G lam=0.2929401332512498 Gm=0.34925990148137015 +[full s3] step 11220/12000 loss=4.2764 gn=0.96 lr=6.58e-05 tps=18291 mem=5.7G lam=0.2930014804005623 Gm=0.34661903271141153 +[full s3] step 11240/12000 loss=4.3262 gn=1.00 lr=6.55e-05 tps=18291 mem=5.7G lam=0.2930523691078027 Gm=0.34840430443485576 +[full s3] step 11260/12000 loss=4.3560 gn=1.06 lr=6.52e-05 tps=18291 mem=5.7G lam=0.2931395623212059 Gm=0.3550705541856587 +[full s3] step 11280/12000 loss=4.2733 gn=0.98 lr=6.49e-05 tps=18291 mem=5.7G lam=0.2931939549744129 Gm=0.34917017646754783 +[full s3] step 11300/12000 loss=4.2685 gn=1.00 lr=6.47e-05 tps=18292 mem=5.7G lam=0.293256051838398 Gm=0.3460798390830557 +[full s3] step 11320/12000 loss=4.3095 gn=0.97 lr=6.44e-05 tps=18292 mem=5.7G lam=0.2933314563706517 Gm=0.3468385186667244 +[full s3] step 11340/12000 loss=4.3607 gn=0.97 lr=6.42e-05 tps=18291 mem=5.7G lam=0.29338633300115663 Gm=0.3440479977677266 +[full s3] step 11360/12000 loss=4.4655 gn=1.04 lr=6.39e-05 tps=18291 mem=5.7G lam=0.29345695953816175 Gm=0.34573800178865594 +[full s3] step 11380/12000 loss=4.2971 gn=0.98 lr=6.37e-05 tps=18292 mem=5.7G lam=0.29350832911829156 Gm=0.34761730387496453 +[full s3] step 11400/12000 loss=4.3955 gn=1.01 lr=6.34e-05 tps=18292 mem=5.7G lam=0.2935714243600766 Gm=0.35067992261610925 +[full s3] step 11420/12000 loss=4.2968 gn=1.03 lr=6.32e-05 tps=18292 mem=5.7G lam=0.2936336289470394 Gm=0.34616675072660047 +[full s3] step 11440/12000 loss=4.3291 gn=0.97 lr=6.30e-05 tps=18292 mem=5.7G lam=0.29370351228863 Gm=0.34957460857306916 +[full s3] step 11460/12000 loss=4.3528 gn=0.98 lr=6.28e-05 tps=18293 mem=5.7G lam=0.2937691028540333 Gm=0.3479409112284581 +[full s3] step 11480/12000 loss=4.3519 gn=0.94 lr=6.26e-05 tps=18292 mem=5.7G lam=0.29381995151440304 Gm=0.34382960743581253 +[full s3] step 11500/12000 loss=4.2588 gn=0.98 lr=6.24e-05 tps=18292 mem=5.7G lam=0.2938999254256487 Gm=0.35056097619235516 +[full s3] step 11520/12000 loss=4.1578 gn=0.98 lr=6.22e-05 tps=18292 mem=5.7G lam=0.2939649348457654 Gm=0.3493179399520159 +[full s3] step 11540/12000 loss=4.4020 gn=0.97 lr=6.20e-05 tps=18292 mem=5.7G lam=0.2940227873623371 Gm=0.3503534502815455 +[full s3] step 11560/12000 loss=4.3186 gn=0.99 lr=6.19e-05 tps=18292 mem=5.7G lam=0.29409127434094745 Gm=0.3488332185273369 +[full s3] step 11580/12000 loss=4.1802 gn=0.95 lr=6.17e-05 tps=18293 mem=5.7G lam=0.2941502497221033 Gm=0.3452120421764751 +[full s3] step 11600/12000 loss=4.3339 gn=0.97 lr=6.15e-05 tps=18293 mem=5.7G lam=0.29423069426168996 Gm=0.344015776955833 +[full s3] step 11620/12000 loss=4.2809 gn=0.96 lr=6.14e-05 tps=18293 mem=5.7G lam=0.2942902386809389 Gm=0.3498778151503454 +[full s3] step 11640/12000 loss=4.3339 gn=0.97 lr=6.12e-05 tps=18293 mem=5.7G lam=0.2943502801160018 Gm=0.34863538093244034 +[full s3] step 11660/12000 loss=4.4188 gn=1.04 lr=6.11e-05 tps=18293 mem=5.7G lam=0.29441843337068957 Gm=0.3469986109994352 +[full s3] step 11680/12000 loss=4.2024 gn=1.02 lr=6.10e-05 tps=18293 mem=5.7G lam=0.2944743921980262 Gm=0.3484587719819198 +[full s3] step 11700/12000 loss=4.3528 gn=1.02 lr=6.09e-05 tps=18293 mem=5.7G lam=0.2945383774737517 Gm=0.34353929773593944 +[full s3] step 11720/12000 loss=4.2713 gn=1.02 lr=6.07e-05 tps=18293 mem=5.7G lam=0.2945883894960086 Gm=0.3450699915799002 +[full s3] step 11740/12000 loss=4.3121 gn=1.02 lr=6.06e-05 tps=18293 mem=5.7G lam=0.294662047487994 Gm=0.3498500226996839 +[full s3] step 11760/12000 loss=4.3104 gn=0.96 lr=6.06e-05 tps=18294 mem=5.7G lam=0.2947271605953574 Gm=0.34226030328621465 +[full s3] step 11780/12000 loss=4.3392 gn=0.96 lr=6.05e-05 tps=18294 mem=5.7G lam=0.29479620171089965 Gm=0.34503818380956847 +[full s3] step 11800/12000 loss=4.2899 gn=1.00 lr=6.04e-05 tps=18294 mem=5.7G lam=0.2948615749677022 Gm=0.3434130629369368 +[full s3] step 11820/12000 loss=4.3426 gn=1.04 lr=6.03e-05 tps=18295 mem=5.7G lam=0.29493217697987956 Gm=0.3436102520984908 +[full s3] step 11840/12000 loss=4.2383 gn=0.99 lr=6.02e-05 tps=18295 mem=5.7G lam=0.29499644103149575 Gm=0.34280447103083134 +[full s3] step 11860/12000 loss=4.2479 gn=0.97 lr=6.02e-05 tps=18295 mem=5.7G lam=0.2950252477700512 Gm=0.3454628235194832 +[full s3] step 11880/12000 loss=4.4525 gn=1.11 lr=6.01e-05 tps=18295 mem=5.7G lam=0.2950887664531668 Gm=0.3487356298137456 +[full s3] step 11900/12000 loss=4.2639 gn=1.09 lr=6.01e-05 tps=18296 mem=5.7G lam=0.29515782619516057 Gm=0.34090893948450685 +[full s3] step 11920/12000 loss=4.2608 gn=1.01 lr=6.01e-05 tps=18296 mem=5.7G lam=0.2952043395489454 Gm=0.34289556586494047 +[full s3] step 11940/12000 loss=4.4004 gn=1.25 lr=6.00e-05 tps=18296 mem=5.7G lam=0.29525130180021125 Gm=0.34044637670740485 +[full s3] step 11960/12000 loss=4.2410 gn=1.02 lr=6.00e-05 tps=18296 mem=5.7G lam=0.29530744316677254 Gm=0.34899194926644367 +[full s3] step 11980/12000 loss=4.2859 gn=0.97 lr=6.00e-05 tps=18296 mem=5.7G lam=0.29537375799069804 Gm=0.3447236562302957 +[full s3] EVAL step 11999: val_loss=4.3529 val_ppl=77.70 +[init] variant=baseline seed=4 params=63.38M +[init] use_S=False use_G=False +[baseline s4] step 0/12000 loss=10.9199 gn=14.71 lr=3.00e-06 tps=7714 mem=4.8G lam=None Gm=None +[baseline s4] step 20/12000 loss=9.8133 gn=1.96 lr=6.30e-05 tps=22784 mem=5.4G lam=None Gm=None +[baseline s4] step 40/12000 loss=9.0611 gn=1.73 lr=1.23e-04 tps=23908 mem=5.4G lam=None Gm=None +[baseline s4] step 60/12000 loss=8.1784 gn=1.44 lr=1.83e-04 tps=24304 mem=5.4G lam=None Gm=None +[baseline s4] step 80/12000 loss=7.5532 gn=0.70 lr=2.43e-04 tps=24489 mem=5.4G lam=None Gm=None +[baseline s4] step 100/12000 loss=7.0018 gn=0.76 lr=3.03e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 120/12000 loss=7.1853 gn=0.67 lr=3.63e-04 tps=24651 mem=5.4G lam=None Gm=None +[baseline s4] step 140/12000 loss=6.8237 gn=0.88 lr=4.23e-04 tps=24689 mem=5.4G lam=None Gm=None +[baseline s4] step 160/12000 loss=6.7339 gn=0.66 lr=4.83e-04 tps=24720 mem=5.4G lam=None Gm=None +[baseline s4] step 180/12000 loss=6.7584 gn=0.60 lr=5.43e-04 tps=24741 mem=5.4G lam=None Gm=None +[baseline s4] step 200/12000 loss=6.7070 gn=0.70 lr=6.00e-04 tps=24756 mem=5.4G lam=None Gm=None +[baseline s4] step 220/12000 loss=6.7465 gn=0.63 lr=6.00e-04 tps=24762 mem=5.4G lam=None Gm=None +[baseline s4] step 240/12000 loss=6.5478 gn=0.74 lr=6.00e-04 tps=24769 mem=5.4G lam=None Gm=None +[baseline s4] step 260/12000 loss=6.6226 gn=0.51 lr=6.00e-04 tps=24772 mem=5.4G lam=None Gm=None +[baseline s4] step 280/12000 loss=6.6808 gn=0.58 lr=6.00e-04 tps=24775 mem=5.4G lam=None Gm=None +[baseline s4] step 300/12000 loss=6.3562 gn=0.53 lr=6.00e-04 tps=24775 mem=5.4G lam=None Gm=None +[baseline s4] step 320/12000 loss=6.4046 gn=0.51 lr=6.00e-04 tps=24778 mem=5.4G lam=None Gm=None +[baseline s4] step 340/12000 loss=6.4278 gn=0.50 lr=6.00e-04 tps=24773 mem=5.4G lam=None Gm=None +[baseline s4] step 360/12000 loss=6.4817 gn=0.58 lr=6.00e-04 tps=24775 mem=5.4G lam=None Gm=None +[baseline s4] step 380/12000 loss=6.3962 gn=0.55 lr=6.00e-04 tps=24771 mem=5.4G lam=None Gm=None +[baseline s4] step 400/12000 loss=6.2915 gn=0.54 lr=6.00e-04 tps=24771 mem=5.4G lam=None Gm=None +[baseline s4] step 420/12000 loss=6.1846 gn=0.58 lr=6.00e-04 tps=24772 mem=5.4G lam=None Gm=None +[baseline s4] step 440/12000 loss=6.2905 gn=0.59 lr=5.99e-04 tps=24771 mem=5.4G lam=None Gm=None +[baseline s4] step 460/12000 loss=6.2958 gn=0.49 lr=5.99e-04 tps=24773 mem=5.4G lam=None Gm=None +[baseline s4] step 480/12000 loss=6.3081 gn=0.56 lr=5.99e-04 tps=24773 mem=5.4G lam=None Gm=None +[baseline s4] step 500/12000 loss=6.2311 gn=0.55 lr=5.99e-04 tps=24772 mem=5.4G lam=None Gm=None +[baseline s4] step 520/12000 loss=6.2747 gn=0.50 lr=5.99e-04 tps=24769 mem=5.4G lam=None Gm=None +[baseline s4] step 540/12000 loss=6.0884 gn=0.56 lr=5.99e-04 tps=24760 mem=5.4G lam=None Gm=None +[baseline s4] step 560/12000 loss=6.2252 gn=0.60 lr=5.99e-04 tps=24759 mem=5.4G lam=None Gm=None +[baseline s4] step 580/12000 loss=6.0781 gn=0.50 lr=5.99e-04 tps=24759 mem=5.4G lam=None Gm=None +[baseline s4] step 600/12000 loss=6.0251 gn=0.55 lr=5.98e-04 tps=24756 mem=5.4G lam=None Gm=None +[baseline s4] step 620/12000 loss=6.1343 gn=0.50 lr=5.98e-04 tps=24755 mem=5.4G lam=None Gm=None +[baseline s4] step 640/12000 loss=6.2399 gn=0.56 lr=5.98e-04 tps=24753 mem=5.4G lam=None Gm=None +[baseline s4] step 660/12000 loss=6.0785 gn=0.56 lr=5.98e-04 tps=24752 mem=5.4G lam=None Gm=None +[baseline s4] step 680/12000 loss=6.0802 gn=0.59 lr=5.98e-04 tps=24751 mem=5.4G lam=None Gm=None +[baseline s4] step 700/12000 loss=5.9757 gn=0.52 lr=5.98e-04 tps=24749 mem=5.4G lam=None Gm=None +[baseline s4] step 720/12000 loss=5.9375 gn=0.53 lr=5.97e-04 tps=24748 mem=5.4G lam=None Gm=None +[baseline s4] step 740/12000 loss=6.0433 gn=0.51 lr=5.97e-04 tps=24744 mem=5.4G lam=None Gm=None +[baseline s4] step 760/12000 loss=5.9957 gn=0.52 lr=5.97e-04 tps=24744 mem=5.4G lam=None Gm=None +[baseline s4] step 780/12000 loss=6.1450 gn=0.62 lr=5.97e-04 tps=24742 mem=5.4G lam=None Gm=None +[baseline s4] step 800/12000 loss=5.8629 gn=0.50 lr=5.97e-04 tps=24740 mem=5.4G lam=None Gm=None +[baseline s4] step 820/12000 loss=5.9215 gn=0.54 lr=5.96e-04 tps=24740 mem=5.4G lam=None Gm=None +[baseline s4] step 840/12000 loss=5.8786 gn=0.55 lr=5.96e-04 tps=24739 mem=5.4G lam=None Gm=None +[baseline s4] step 860/12000 loss=5.9390 gn=0.61 lr=5.96e-04 tps=24737 mem=5.4G lam=None Gm=None +[baseline s4] step 880/12000 loss=5.7823 gn=0.58 lr=5.96e-04 tps=24735 mem=5.4G lam=None Gm=None +[baseline s4] step 900/12000 loss=5.7792 gn=0.55 lr=5.95e-04 tps=24732 mem=5.4G lam=None Gm=None +[baseline s4] step 920/12000 loss=5.8564 gn=0.56 lr=5.95e-04 tps=24732 mem=5.4G lam=None Gm=None +[baseline s4] step 940/12000 loss=5.7620 gn=0.56 lr=5.95e-04 tps=24731 mem=5.4G lam=None Gm=None +[baseline s4] step 960/12000 loss=5.8163 gn=0.55 lr=5.94e-04 tps=24728 mem=5.4G lam=None Gm=None +[baseline s4] step 980/12000 loss=5.8196 gn=0.61 lr=5.94e-04 tps=24727 mem=5.4G lam=None Gm=None +[baseline s4] step 1000/12000 loss=5.7348 gn=0.58 lr=5.94e-04 tps=24726 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 1000: val_loss=5.7911 val_ppl=327.36 +[baseline s4] step 1020/12000 loss=5.7897 gn=0.69 lr=5.94e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1040/12000 loss=5.7318 gn=0.55 lr=5.93e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1060/12000 loss=5.6146 gn=0.51 lr=5.93e-04 tps=24640 mem=5.4G lam=None Gm=None +[baseline s4] step 1080/12000 loss=5.6692 gn=0.60 lr=5.93e-04 tps=24640 mem=5.4G lam=None Gm=None +[baseline s4] step 1100/12000 loss=5.8375 gn=0.54 lr=5.92e-04 tps=24641 mem=5.4G lam=None Gm=None +[baseline s4] step 1120/12000 loss=5.7865 gn=0.53 lr=5.92e-04 tps=24643 mem=5.4G lam=None Gm=None +[baseline s4] step 1140/12000 loss=5.5917 gn=0.56 lr=5.92e-04 tps=24643 mem=5.4G lam=None Gm=None +[baseline s4] step 1160/12000 loss=5.6418 gn=0.55 lr=5.91e-04 tps=24643 mem=5.4G lam=None Gm=None +[baseline s4] step 1180/12000 loss=5.7181 gn=0.57 lr=5.91e-04 tps=24642 mem=5.4G lam=None Gm=None +[baseline s4] step 1200/12000 loss=5.6787 gn=0.54 lr=5.90e-04 tps=24641 mem=5.4G lam=None Gm=None +[baseline s4] step 1220/12000 loss=5.9080 gn=0.54 lr=5.90e-04 tps=24640 mem=5.4G lam=None Gm=None +[baseline s4] step 1240/12000 loss=5.5653 gn=0.55 lr=5.90e-04 tps=24641 mem=5.4G lam=None Gm=None +[baseline s4] step 1260/12000 loss=5.5745 gn=0.55 lr=5.89e-04 tps=24640 mem=5.4G lam=None Gm=None +[baseline s4] step 1280/12000 loss=5.4629 gn=0.53 lr=5.89e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1300/12000 loss=5.7751 gn=0.57 lr=5.89e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1320/12000 loss=5.5345 gn=0.57 lr=5.88e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1340/12000 loss=5.5003 gn=0.53 lr=5.88e-04 tps=24639 mem=5.4G lam=None Gm=None +[baseline s4] step 1360/12000 loss=5.6130 gn=0.57 lr=5.87e-04 tps=24638 mem=5.4G lam=None Gm=None +[baseline s4] step 1380/12000 loss=5.5657 gn=0.56 lr=5.87e-04 tps=24637 mem=5.4G lam=None Gm=None +[baseline s4] step 1400/12000 loss=5.4329 gn=0.61 lr=5.86e-04 tps=24634 mem=5.4G lam=None Gm=None +[baseline s4] step 1420/12000 loss=5.5534 gn=0.55 lr=5.86e-04 tps=24634 mem=5.4G lam=None Gm=None +[baseline s4] step 1440/12000 loss=5.5047 gn=0.60 lr=5.85e-04 tps=24633 mem=5.4G lam=None Gm=None +[baseline s4] step 1460/12000 loss=5.4255 gn=0.56 lr=5.85e-04 tps=24633 mem=5.4G lam=None Gm=None +[baseline s4] step 1480/12000 loss=5.4540 gn=0.58 lr=5.84e-04 tps=24632 mem=5.4G lam=None Gm=None +[baseline s4] step 1500/12000 loss=5.5036 gn=0.57 lr=5.84e-04 tps=24631 mem=5.4G lam=None Gm=None +[baseline s4] step 1520/12000 loss=5.3491 gn=0.57 lr=5.83e-04 tps=24631 mem=5.4G lam=None Gm=None +[baseline s4] step 1540/12000 loss=5.4310 gn=0.58 lr=5.83e-04 tps=24631 mem=5.4G lam=None Gm=None +[baseline s4] step 1560/12000 loss=5.5050 gn=0.58 lr=5.82e-04 tps=24631 mem=5.4G lam=None Gm=None +[baseline s4] step 1580/12000 loss=5.5791 gn=0.62 lr=5.82e-04 tps=24630 mem=5.4G lam=None Gm=None +[baseline s4] step 1600/12000 loss=5.2702 gn=0.59 lr=5.81e-04 tps=24630 mem=5.4G lam=None Gm=None +[baseline s4] step 1620/12000 loss=5.4029 gn=0.58 lr=5.81e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1640/12000 loss=5.4298 gn=0.63 lr=5.80e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s4] step 1660/12000 loss=5.3792 gn=0.56 lr=5.80e-04 tps=24626 mem=5.4G lam=None Gm=None +[baseline s4] step 1680/12000 loss=5.2611 gn=0.58 lr=5.79e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s4] step 1700/12000 loss=5.3251 gn=0.59 lr=5.79e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s4] step 1720/12000 loss=5.4073 gn=0.60 lr=5.78e-04 tps=24627 mem=5.4G lam=None Gm=None +[baseline s4] step 1740/12000 loss=5.5021 gn=0.66 lr=5.78e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] step 1760/12000 loss=5.3029 gn=0.59 lr=5.77e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] step 1780/12000 loss=5.4733 gn=0.61 lr=5.76e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] step 1800/12000 loss=5.2094 gn=0.58 lr=5.76e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1820/12000 loss=5.3099 gn=0.57 lr=5.75e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1840/12000 loss=5.3591 gn=0.65 lr=5.75e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1860/12000 loss=5.5566 gn=0.59 lr=5.74e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1880/12000 loss=5.4269 gn=0.59 lr=5.73e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1900/12000 loss=5.3382 gn=0.61 lr=5.73e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1920/12000 loss=5.3266 gn=0.57 lr=5.72e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1940/12000 loss=5.3417 gn=0.59 lr=5.72e-04 tps=24629 mem=5.4G lam=None Gm=None +[baseline s4] step 1960/12000 loss=5.4523 gn=0.60 lr=5.71e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] step 1980/12000 loss=5.2294 gn=0.58 lr=5.70e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] step 2000/12000 loss=5.2820 gn=0.61 lr=5.70e-04 tps=24628 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 2000: val_loss=5.3369 val_ppl=207.87 +[baseline s4] step 2020/12000 loss=5.2357 gn=0.66 lr=5.69e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2040/12000 loss=5.4265 gn=0.79 lr=5.68e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2060/12000 loss=5.3380 gn=0.65 lr=5.68e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2080/12000 loss=5.2412 gn=0.61 lr=5.67e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 2100/12000 loss=5.1439 gn=0.63 lr=5.66e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 2120/12000 loss=5.2546 gn=0.63 lr=5.65e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 2140/12000 loss=5.2123 gn=0.62 lr=5.65e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 2160/12000 loss=5.1908 gn=0.61 lr=5.64e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2180/12000 loss=5.3326 gn=0.66 lr=5.63e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2200/12000 loss=5.2235 gn=0.64 lr=5.63e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 2220/12000 loss=5.1817 gn=0.61 lr=5.62e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 2240/12000 loss=5.1067 gn=0.63 lr=5.61e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 2260/12000 loss=5.2847 gn=0.67 lr=5.60e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 2280/12000 loss=5.1751 gn=0.62 lr=5.60e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 2300/12000 loss=5.1534 gn=0.62 lr=5.59e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 2320/12000 loss=5.0332 gn=0.63 lr=5.58e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 2340/12000 loss=5.2042 gn=0.66 lr=5.57e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 2360/12000 loss=4.9370 gn=0.66 lr=5.57e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2380/12000 loss=5.1180 gn=0.69 lr=5.56e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2400/12000 loss=5.1021 gn=0.64 lr=5.55e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2420/12000 loss=5.0647 gn=0.64 lr=5.54e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2440/12000 loss=5.1101 gn=0.65 lr=5.53e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2460/12000 loss=5.0794 gn=0.66 lr=5.53e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2480/12000 loss=5.2079 gn=0.75 lr=5.52e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2500/12000 loss=5.1309 gn=0.64 lr=5.51e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2520/12000 loss=5.0199 gn=0.69 lr=5.50e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2540/12000 loss=5.0463 gn=0.64 lr=5.49e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2560/12000 loss=5.2180 gn=0.64 lr=5.48e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2580/12000 loss=5.0871 gn=0.69 lr=5.48e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2600/12000 loss=5.0437 gn=0.63 lr=5.47e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2620/12000 loss=5.0921 gn=0.68 lr=5.46e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 2640/12000 loss=5.0896 gn=0.69 lr=5.45e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2660/12000 loss=5.2174 gn=0.66 lr=5.44e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2680/12000 loss=4.9315 gn=0.71 lr=5.43e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2700/12000 loss=5.1900 gn=0.62 lr=5.42e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2720/12000 loss=5.1510 gn=0.65 lr=5.41e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2740/12000 loss=5.1250 gn=0.73 lr=5.41e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2760/12000 loss=5.0446 gn=0.69 lr=5.40e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2780/12000 loss=5.0986 gn=0.74 lr=5.39e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2800/12000 loss=5.0164 gn=0.74 lr=5.38e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2820/12000 loss=4.9681 gn=0.71 lr=5.37e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2840/12000 loss=4.9469 gn=0.72 lr=5.36e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2860/12000 loss=5.0463 gn=0.65 lr=5.35e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2880/12000 loss=5.0233 gn=0.66 lr=5.34e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2900/12000 loss=5.0380 gn=0.80 lr=5.33e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2920/12000 loss=5.0701 gn=0.67 lr=5.32e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 2940/12000 loss=4.9986 gn=0.68 lr=5.31e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2960/12000 loss=4.9925 gn=0.71 lr=5.30e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 2980/12000 loss=5.0011 gn=0.74 lr=5.29e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 3000/12000 loss=4.9787 gn=0.72 lr=5.28e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 3000: val_loss=5.0064 val_ppl=149.37 +[baseline s4] step 3020/12000 loss=4.9065 gn=0.72 lr=5.27e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 3040/12000 loss=4.8671 gn=0.69 lr=5.26e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 3060/12000 loss=5.1465 gn=0.72 lr=5.25e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 3080/12000 loss=5.0022 gn=0.71 lr=5.24e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 3100/12000 loss=5.0355 gn=0.75 lr=5.23e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 3120/12000 loss=4.9155 gn=0.70 lr=5.22e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 3140/12000 loss=4.9168 gn=0.70 lr=5.21e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 3160/12000 loss=5.0226 gn=0.71 lr=5.20e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 3180/12000 loss=4.9155 gn=0.73 lr=5.19e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3200/12000 loss=4.9501 gn=0.78 lr=5.18e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3220/12000 loss=5.0989 gn=0.75 lr=5.17e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3240/12000 loss=4.9614 gn=0.72 lr=5.16e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3260/12000 loss=4.9523 gn=0.70 lr=5.15e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3280/12000 loss=4.8772 gn=0.79 lr=5.14e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3300/12000 loss=4.9497 gn=0.73 lr=5.13e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3320/12000 loss=4.8569 gn=0.72 lr=5.12e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3340/12000 loss=4.9707 gn=0.75 lr=5.11e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3360/12000 loss=4.9141 gn=0.74 lr=5.10e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3380/12000 loss=4.8371 gn=0.70 lr=5.09e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 3400/12000 loss=4.8363 gn=0.76 lr=5.08e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3420/12000 loss=4.8390 gn=0.66 lr=5.07e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3440/12000 loss=4.7840 gn=0.70 lr=5.06e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3460/12000 loss=4.7952 gn=0.68 lr=5.05e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3480/12000 loss=5.0341 gn=0.69 lr=5.03e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3500/12000 loss=4.9443 gn=0.74 lr=5.02e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3520/12000 loss=4.7855 gn=0.73 lr=5.01e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3540/12000 loss=5.0520 gn=0.76 lr=5.00e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3560/12000 loss=4.9251 gn=0.72 lr=4.99e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3580/12000 loss=5.1562 gn=0.76 lr=4.98e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3600/12000 loss=4.9147 gn=0.75 lr=4.97e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 3620/12000 loss=4.7493 gn=0.79 lr=4.96e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3640/12000 loss=4.8946 gn=0.72 lr=4.94e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3660/12000 loss=4.8192 gn=1.05 lr=4.93e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3680/12000 loss=4.9056 gn=0.81 lr=4.92e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3700/12000 loss=4.8906 gn=0.81 lr=4.91e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3720/12000 loss=4.8090 gn=0.76 lr=4.90e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3740/12000 loss=4.6541 gn=0.73 lr=4.89e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3760/12000 loss=4.7997 gn=0.73 lr=4.88e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3780/12000 loss=4.7733 gn=0.75 lr=4.86e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3800/12000 loss=4.8490 gn=0.74 lr=4.85e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3820/12000 loss=4.8110 gn=0.74 lr=4.84e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3840/12000 loss=4.6256 gn=0.74 lr=4.83e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3860/12000 loss=4.9362 gn=0.76 lr=4.82e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3880/12000 loss=4.8516 gn=0.75 lr=4.80e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3900/12000 loss=4.8894 gn=0.78 lr=4.79e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 3920/12000 loss=4.8735 gn=0.79 lr=4.78e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3940/12000 loss=4.7612 gn=0.75 lr=4.77e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 3960/12000 loss=4.9650 gn=0.95 lr=4.76e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 3980/12000 loss=4.8160 gn=0.73 lr=4.74e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 4000/12000 loss=4.7417 gn=0.70 lr=4.73e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 4000: val_loss=4.9018 val_ppl=134.53 +[baseline s4] step 4020/12000 loss=4.6787 gn=0.75 lr=4.72e-04 tps=24546 mem=5.4G lam=None Gm=None +[baseline s4] step 4040/12000 loss=4.7961 gn=0.69 lr=4.71e-04 tps=24547 mem=5.4G lam=None Gm=None +[baseline s4] step 4060/12000 loss=4.7962 gn=0.79 lr=4.70e-04 tps=24547 mem=5.4G lam=None Gm=None +[baseline s4] step 4080/12000 loss=4.8178 gn=0.82 lr=4.68e-04 tps=24548 mem=5.4G lam=None Gm=None +[baseline s4] step 4100/12000 loss=4.8648 gn=0.81 lr=4.67e-04 tps=24548 mem=5.4G lam=None Gm=None +[baseline s4] step 4120/12000 loss=4.8236 gn=0.77 lr=4.66e-04 tps=24549 mem=5.4G lam=None Gm=None +[baseline s4] step 4140/12000 loss=4.8268 gn=0.80 lr=4.65e-04 tps=24549 mem=5.4G lam=None Gm=None +[baseline s4] step 4160/12000 loss=4.9243 gn=0.87 lr=4.63e-04 tps=24550 mem=5.4G lam=None Gm=None +[baseline s4] step 4180/12000 loss=4.8235 gn=0.79 lr=4.62e-04 tps=24550 mem=5.4G lam=None Gm=None +[baseline s4] step 4200/12000 loss=4.8059 gn=0.78 lr=4.61e-04 tps=24551 mem=5.4G lam=None Gm=None +[baseline s4] step 4220/12000 loss=4.7655 gn=0.76 lr=4.60e-04 tps=24551 mem=5.4G lam=None Gm=None +[baseline s4] step 4240/12000 loss=4.6866 gn=0.75 lr=4.58e-04 tps=24552 mem=5.4G lam=None Gm=None +[baseline s4] step 4260/12000 loss=4.7645 gn=0.77 lr=4.57e-04 tps=24552 mem=5.4G lam=None Gm=None +[baseline s4] step 4280/12000 loss=4.8843 gn=0.81 lr=4.56e-04 tps=24553 mem=5.4G lam=None Gm=None +[baseline s4] step 4300/12000 loss=4.9124 gn=0.86 lr=4.54e-04 tps=24553 mem=5.4G lam=None Gm=None +[baseline s4] step 4320/12000 loss=4.7102 gn=0.86 lr=4.53e-04 tps=24554 mem=5.4G lam=None Gm=None +[baseline s4] step 4340/12000 loss=4.9459 gn=0.74 lr=4.52e-04 tps=24554 mem=5.4G lam=None Gm=None +[baseline s4] step 4360/12000 loss=4.7484 gn=0.82 lr=4.51e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s4] step 4380/12000 loss=4.4682 gn=0.78 lr=4.49e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s4] step 4400/12000 loss=4.9148 gn=0.79 lr=4.48e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s4] step 4420/12000 loss=4.7500 gn=0.75 lr=4.47e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s4] step 4440/12000 loss=4.9259 gn=0.81 lr=4.45e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s4] step 4460/12000 loss=4.6151 gn=0.79 lr=4.44e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s4] step 4480/12000 loss=4.7581 gn=0.76 lr=4.43e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s4] step 4500/12000 loss=4.8501 gn=0.75 lr=4.42e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s4] step 4520/12000 loss=4.5478 gn=0.80 lr=4.40e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s4] step 4540/12000 loss=4.6697 gn=0.81 lr=4.39e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s4] step 4560/12000 loss=4.6699 gn=0.75 lr=4.38e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s4] step 4580/12000 loss=4.7233 gn=0.79 lr=4.36e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s4] step 4600/12000 loss=4.6124 gn=0.81 lr=4.35e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s4] step 4620/12000 loss=4.7150 gn=0.80 lr=4.34e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s4] step 4640/12000 loss=4.7547 gn=0.82 lr=4.32e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 4660/12000 loss=4.6894 gn=0.81 lr=4.31e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 4680/12000 loss=4.6272 gn=0.83 lr=4.30e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 4700/12000 loss=4.6689 gn=0.85 lr=4.28e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 4720/12000 loss=4.6952 gn=0.84 lr=4.27e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 4740/12000 loss=4.7366 gn=0.81 lr=4.26e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 4760/12000 loss=4.7997 gn=0.82 lr=4.24e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 4780/12000 loss=4.6588 gn=0.82 lr=4.23e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 4800/12000 loss=4.8540 gn=0.79 lr=4.22e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 4820/12000 loss=4.6585 gn=0.82 lr=4.20e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 4840/12000 loss=4.5724 gn=0.78 lr=4.19e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 4860/12000 loss=4.6603 gn=0.80 lr=4.18e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 4880/12000 loss=4.6289 gn=0.76 lr=4.16e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 4900/12000 loss=4.6974 gn=0.79 lr=4.15e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 4920/12000 loss=4.6839 gn=0.89 lr=4.13e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 4940/12000 loss=4.6363 gn=0.85 lr=4.12e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 4960/12000 loss=4.7014 gn=0.80 lr=4.11e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 4980/12000 loss=4.6500 gn=0.85 lr=4.09e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] step 5000/12000 loss=4.8028 gn=0.83 lr=4.08e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 5000: val_loss=4.7747 val_ppl=118.48 +[baseline s4] step 5020/12000 loss=4.7614 gn=0.82 lr=4.07e-04 tps=24554 mem=5.4G lam=None Gm=None +[baseline s4] step 5040/12000 loss=4.8145 gn=0.83 lr=4.05e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s4] step 5060/12000 loss=4.6651 gn=0.93 lr=4.04e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s4] step 5080/12000 loss=4.7594 gn=0.82 lr=4.02e-04 tps=24555 mem=5.4G lam=None Gm=None +[baseline s4] step 5100/12000 loss=4.7128 gn=0.95 lr=4.01e-04 tps=24556 mem=5.4G lam=None Gm=None +[baseline s4] step 5120/12000 loss=4.7347 gn=0.84 lr=4.00e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s4] step 5140/12000 loss=4.6590 gn=0.83 lr=3.98e-04 tps=24557 mem=5.4G lam=None Gm=None +[baseline s4] step 5160/12000 loss=4.7638 gn=0.90 lr=3.97e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s4] step 5180/12000 loss=4.5715 gn=0.86 lr=3.95e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s4] step 5200/12000 loss=4.5327 gn=0.83 lr=3.94e-04 tps=24558 mem=5.4G lam=None Gm=None +[baseline s4] step 5220/12000 loss=4.7171 gn=0.82 lr=3.93e-04 tps=24559 mem=5.4G lam=None Gm=None +[baseline s4] step 5240/12000 loss=4.7494 gn=0.86 lr=3.91e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s4] step 5260/12000 loss=4.6332 gn=0.82 lr=3.90e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s4] step 5280/12000 loss=4.6151 gn=0.79 lr=3.88e-04 tps=24560 mem=5.4G lam=None Gm=None +[baseline s4] step 5300/12000 loss=4.6370 gn=0.80 lr=3.87e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s4] step 5320/12000 loss=4.6303 gn=0.77 lr=3.86e-04 tps=24561 mem=5.4G lam=None Gm=None +[baseline s4] step 5340/12000 loss=4.7477 gn=0.83 lr=3.84e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 5360/12000 loss=4.6497 gn=0.76 lr=3.83e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 5380/12000 loss=4.6203 gn=0.86 lr=3.81e-04 tps=24562 mem=5.4G lam=None Gm=None +[baseline s4] step 5400/12000 loss=4.7184 gn=0.85 lr=3.80e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 5420/12000 loss=4.6015 gn=0.93 lr=3.79e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 5440/12000 loss=4.7630 gn=0.87 lr=3.77e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 5460/12000 loss=4.5841 gn=0.87 lr=3.76e-04 tps=24563 mem=5.4G lam=None Gm=None +[baseline s4] step 5480/12000 loss=4.7028 gn=0.87 lr=3.74e-04 tps=24564 mem=5.4G lam=None Gm=None +[baseline s4] step 5500/12000 loss=4.7198 gn=0.84 lr=3.73e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 5520/12000 loss=4.8270 gn=0.89 lr=3.72e-04 tps=24565 mem=5.4G lam=None Gm=None +[baseline s4] step 5540/12000 loss=4.7370 gn=0.83 lr=3.70e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 5560/12000 loss=4.5307 gn=0.84 lr=3.69e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 5580/12000 loss=4.5629 gn=0.86 lr=3.67e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 5600/12000 loss=4.6880 gn=0.85 lr=3.66e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 5620/12000 loss=4.4339 gn=0.80 lr=3.64e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 5640/12000 loss=4.6906 gn=0.82 lr=3.63e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 5660/12000 loss=4.7155 gn=0.84 lr=3.62e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 5680/12000 loss=4.5247 gn=0.88 lr=3.60e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] step 5700/12000 loss=4.4790 gn=0.81 lr=3.59e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] step 5720/12000 loss=4.6227 gn=0.88 lr=3.57e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s4] step 5740/12000 loss=4.6659 gn=0.85 lr=3.56e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s4] step 5760/12000 loss=4.5773 gn=0.91 lr=3.54e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s4] step 5780/12000 loss=4.5426 gn=0.83 lr=3.53e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s4] step 5800/12000 loss=4.5159 gn=0.84 lr=3.52e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s4] step 5820/12000 loss=4.5585 gn=0.91 lr=3.50e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s4] step 5840/12000 loss=4.7504 gn=0.86 lr=3.49e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s4] step 5860/12000 loss=4.5676 gn=0.86 lr=3.47e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s4] step 5880/12000 loss=4.6737 gn=0.91 lr=3.46e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s4] step 5900/12000 loss=4.5000 gn=0.90 lr=3.44e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s4] step 5920/12000 loss=4.4547 gn=0.86 lr=3.43e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s4] step 5940/12000 loss=4.6201 gn=0.85 lr=3.41e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s4] step 5960/12000 loss=4.5415 gn=0.86 lr=3.40e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s4] step 5980/12000 loss=4.6403 gn=0.85 lr=3.39e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s4] step 6000/12000 loss=4.7052 gn=0.88 lr=3.37e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 6000: val_loss=4.6554 val_ppl=105.15 +[baseline s4] step 6020/12000 loss=4.6898 gn=0.88 lr=3.36e-04 tps=24566 mem=5.4G lam=None Gm=None +[baseline s4] step 6040/12000 loss=4.4851 gn=0.93 lr=3.34e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 6060/12000 loss=4.5566 gn=0.84 lr=3.33e-04 tps=24567 mem=5.4G lam=None Gm=None +[baseline s4] step 6080/12000 loss=4.7458 gn=0.84 lr=3.31e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 6100/12000 loss=4.4018 gn=0.82 lr=3.30e-04 tps=24568 mem=5.4G lam=None Gm=None +[baseline s4] step 6120/12000 loss=4.6648 gn=0.84 lr=3.29e-04 tps=24569 mem=5.4G lam=None Gm=None +[baseline s4] step 6140/12000 loss=4.4871 gn=0.86 lr=3.27e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] step 6160/12000 loss=4.5784 gn=0.87 lr=3.26e-04 tps=24570 mem=5.4G lam=None Gm=None +[baseline s4] step 6180/12000 loss=4.4950 gn=0.90 lr=3.24e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s4] step 6200/12000 loss=4.6385 gn=0.90 lr=3.23e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s4] step 6220/12000 loss=4.6810 gn=0.87 lr=3.21e-04 tps=24571 mem=5.4G lam=None Gm=None +[baseline s4] step 6240/12000 loss=4.6225 gn=0.89 lr=3.20e-04 tps=24572 mem=5.4G lam=None Gm=None +[baseline s4] step 6260/12000 loss=4.4988 gn=0.91 lr=3.19e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s4] step 6280/12000 loss=4.4849 gn=0.87 lr=3.17e-04 tps=24573 mem=5.4G lam=None Gm=None +[baseline s4] step 6300/12000 loss=4.6008 gn=0.95 lr=3.16e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s4] step 6320/12000 loss=4.3598 gn=0.84 lr=3.14e-04 tps=24574 mem=5.4G lam=None Gm=None +[baseline s4] step 6340/12000 loss=4.5866 gn=0.84 lr=3.13e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s4] step 6360/12000 loss=4.4580 gn=0.86 lr=3.11e-04 tps=24575 mem=5.4G lam=None Gm=None +[baseline s4] step 6380/12000 loss=4.6310 gn=0.89 lr=3.10e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s4] step 6400/12000 loss=4.5897 gn=0.92 lr=3.08e-04 tps=24576 mem=5.4G lam=None Gm=None +[baseline s4] step 6420/12000 loss=4.6369 gn=0.89 lr=3.07e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s4] step 6440/12000 loss=4.6539 gn=0.91 lr=3.06e-04 tps=24577 mem=5.4G lam=None Gm=None +[baseline s4] step 6460/12000 loss=4.5316 gn=0.95 lr=3.04e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s4] step 6480/12000 loss=4.7253 gn=0.93 lr=3.03e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s4] step 6500/12000 loss=4.4697 gn=0.91 lr=3.01e-04 tps=24578 mem=5.4G lam=None Gm=None +[baseline s4] step 6520/12000 loss=4.5358 gn=0.85 lr=3.00e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s4] step 6540/12000 loss=4.6180 gn=0.88 lr=2.98e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s4] step 6560/12000 loss=4.3860 gn=0.88 lr=2.97e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s4] step 6580/12000 loss=4.6981 gn=0.95 lr=2.96e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s4] step 6600/12000 loss=4.5112 gn=0.92 lr=2.94e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s4] step 6620/12000 loss=4.7794 gn=1.13 lr=2.93e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s4] step 6640/12000 loss=4.4545 gn=1.04 lr=2.91e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s4] step 6660/12000 loss=4.4719 gn=0.89 lr=2.90e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s4] step 6680/12000 loss=4.4524 gn=0.90 lr=2.88e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s4] step 6700/12000 loss=4.6114 gn=0.89 lr=2.87e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s4] step 6720/12000 loss=4.4973 gn=0.91 lr=2.86e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s4] step 6740/12000 loss=4.4989 gn=0.86 lr=2.84e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s4] step 6760/12000 loss=4.6124 gn=0.96 lr=2.83e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 6780/12000 loss=4.4671 gn=0.90 lr=2.81e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 6800/12000 loss=4.4159 gn=0.99 lr=2.80e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 6820/12000 loss=4.6007 gn=0.92 lr=2.79e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 6840/12000 loss=4.4306 gn=0.87 lr=2.77e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 6860/12000 loss=4.6199 gn=0.83 lr=2.76e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 6880/12000 loss=4.5455 gn=0.92 lr=2.74e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 6900/12000 loss=4.4625 gn=0.93 lr=2.73e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 6920/12000 loss=4.4978 gn=0.96 lr=2.72e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 6940/12000 loss=4.5458 gn=0.91 lr=2.70e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 6960/12000 loss=4.5256 gn=0.93 lr=2.69e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 6980/12000 loss=4.4847 gn=0.89 lr=2.67e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 7000/12000 loss=4.3344 gn=0.93 lr=2.66e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 7000: val_loss=4.6244 val_ppl=101.94 +[baseline s4] step 7020/12000 loss=4.6909 gn=0.92 lr=2.65e-04 tps=24579 mem=5.4G lam=None Gm=None +[baseline s4] step 7040/12000 loss=4.6820 gn=1.02 lr=2.63e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s4] step 7060/12000 loss=4.5751 gn=0.93 lr=2.62e-04 tps=24580 mem=5.4G lam=None Gm=None +[baseline s4] step 7080/12000 loss=4.5356 gn=0.94 lr=2.60e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s4] step 7100/12000 loss=4.5618 gn=0.91 lr=2.59e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s4] step 7120/12000 loss=4.7470 gn=1.03 lr=2.58e-04 tps=24581 mem=5.4G lam=None Gm=None +[baseline s4] step 7140/12000 loss=4.4672 gn=0.96 lr=2.56e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s4] step 7160/12000 loss=4.4817 gn=0.90 lr=2.55e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s4] step 7180/12000 loss=4.5522 gn=0.98 lr=2.53e-04 tps=24582 mem=5.4G lam=None Gm=None +[baseline s4] step 7200/12000 loss=4.5541 gn=0.96 lr=2.52e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s4] step 7220/12000 loss=4.3515 gn=0.92 lr=2.51e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s4] step 7240/12000 loss=4.4362 gn=0.85 lr=2.49e-04 tps=24583 mem=5.4G lam=None Gm=None +[baseline s4] step 7260/12000 loss=4.5513 gn=0.93 lr=2.48e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s4] step 7280/12000 loss=4.4350 gn=0.91 lr=2.47e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s4] step 7300/12000 loss=4.5853 gn=0.92 lr=2.45e-04 tps=24584 mem=5.4G lam=None Gm=None +[baseline s4] step 7320/12000 loss=4.4152 gn=0.92 lr=2.44e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 7340/12000 loss=4.5057 gn=0.97 lr=2.42e-04 tps=24585 mem=5.4G lam=None Gm=None +[baseline s4] step 7360/12000 loss=4.5653 gn=0.96 lr=2.41e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 7380/12000 loss=4.5419 gn=1.04 lr=2.40e-04 tps=24586 mem=5.4G lam=None Gm=None +[baseline s4] step 7400/12000 loss=4.4007 gn=0.94 lr=2.38e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 7420/12000 loss=4.4840 gn=0.97 lr=2.37e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 7440/12000 loss=4.3933 gn=0.94 lr=2.36e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 7460/12000 loss=4.5824 gn=0.91 lr=2.34e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 7480/12000 loss=4.4201 gn=0.96 lr=2.33e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 7500/12000 loss=4.4787 gn=0.94 lr=2.32e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 7520/12000 loss=4.5220 gn=0.90 lr=2.30e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 7540/12000 loss=4.4062 gn=0.95 lr=2.29e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 7560/12000 loss=4.3158 gn=0.97 lr=2.28e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 7580/12000 loss=4.5891 gn=0.96 lr=2.26e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 7600/12000 loss=4.3540 gn=0.91 lr=2.25e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 7620/12000 loss=4.4448 gn=0.90 lr=2.24e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 7640/12000 loss=4.4149 gn=0.94 lr=2.22e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 7660/12000 loss=4.4429 gn=0.93 lr=2.21e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 7680/12000 loss=4.6261 gn=1.00 lr=2.20e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 7700/12000 loss=4.5830 gn=0.93 lr=2.18e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 7720/12000 loss=4.5809 gn=0.95 lr=2.17e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 7740/12000 loss=4.4936 gn=0.94 lr=2.16e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 7760/12000 loss=4.4873 gn=0.88 lr=2.15e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 7780/12000 loss=4.5644 gn=1.03 lr=2.13e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 7800/12000 loss=4.4832 gn=0.97 lr=2.12e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 7820/12000 loss=4.3474 gn=0.92 lr=2.11e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 7840/12000 loss=4.3891 gn=0.94 lr=2.09e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 7860/12000 loss=4.3877 gn=0.97 lr=2.08e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 7880/12000 loss=4.3906 gn=0.95 lr=2.07e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 7900/12000 loss=4.3523 gn=0.92 lr=2.06e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 7920/12000 loss=4.5125 gn=1.08 lr=2.04e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 7940/12000 loss=4.4992 gn=0.96 lr=2.03e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 7960/12000 loss=4.5004 gn=1.00 lr=2.02e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 7980/12000 loss=4.5053 gn=0.97 lr=2.00e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 8000/12000 loss=4.5533 gn=1.01 lr=1.99e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 8000: val_loss=4.5469 val_ppl=94.34 +[baseline s4] step 8020/12000 loss=4.5203 gn=1.03 lr=1.98e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 8040/12000 loss=4.4192 gn=1.02 lr=1.97e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 8060/12000 loss=4.4576 gn=0.93 lr=1.95e-04 tps=24587 mem=5.4G lam=None Gm=None +[baseline s4] step 8080/12000 loss=4.5475 gn=0.95 lr=1.94e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 8100/12000 loss=4.5275 gn=0.96 lr=1.93e-04 tps=24588 mem=5.4G lam=None Gm=None +[baseline s4] step 8120/12000 loss=4.6647 gn=0.99 lr=1.92e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 8140/12000 loss=4.3218 gn=0.95 lr=1.90e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 8160/12000 loss=4.4011 gn=0.97 lr=1.89e-04 tps=24589 mem=5.4G lam=None Gm=None +[baseline s4] step 8180/12000 loss=4.4860 gn=1.00 lr=1.88e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 8200/12000 loss=4.3460 gn=0.95 lr=1.87e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 8220/12000 loss=4.4537 gn=0.98 lr=1.86e-04 tps=24590 mem=5.4G lam=None Gm=None +[baseline s4] step 8240/12000 loss=4.3773 gn=0.95 lr=1.84e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 8260/12000 loss=4.3520 gn=1.05 lr=1.83e-04 tps=24591 mem=5.4G lam=None Gm=None +[baseline s4] step 8280/12000 loss=4.3406 gn=0.90 lr=1.82e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 8300/12000 loss=4.5207 gn=0.92 lr=1.81e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 8320/12000 loss=4.3983 gn=0.96 lr=1.80e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 8340/12000 loss=4.4513 gn=0.97 lr=1.78e-04 tps=24592 mem=5.4G lam=None Gm=None +[baseline s4] step 8360/12000 loss=4.5378 gn=1.03 lr=1.77e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 8380/12000 loss=4.3802 gn=1.05 lr=1.76e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 8400/12000 loss=4.4597 gn=1.00 lr=1.75e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 8420/12000 loss=4.3396 gn=0.98 lr=1.74e-04 tps=24593 mem=5.4G lam=None Gm=None +[baseline s4] step 8440/12000 loss=4.3068 gn=0.94 lr=1.72e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 8460/12000 loss=4.3742 gn=1.11 lr=1.71e-04 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 8480/12000 loss=4.3680 gn=0.95 lr=1.70e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 8500/12000 loss=4.3906 gn=0.95 lr=1.69e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 8520/12000 loss=4.4852 gn=1.02 lr=1.68e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 8540/12000 loss=4.5515 gn=1.02 lr=1.67e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 8560/12000 loss=4.3697 gn=0.96 lr=1.66e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 8580/12000 loss=4.4705 gn=1.03 lr=1.64e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 8600/12000 loss=4.2951 gn=1.06 lr=1.63e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 8620/12000 loss=4.4203 gn=1.04 lr=1.62e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 8640/12000 loss=4.4869 gn=0.99 lr=1.61e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 8660/12000 loss=4.3043 gn=1.05 lr=1.60e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 8680/12000 loss=4.4061 gn=0.92 lr=1.59e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 8700/12000 loss=4.2667 gn=1.08 lr=1.58e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 8720/12000 loss=4.5120 gn=1.09 lr=1.57e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 8740/12000 loss=4.4294 gn=1.03 lr=1.55e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 8760/12000 loss=4.3508 gn=0.99 lr=1.54e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 8780/12000 loss=4.4105 gn=1.01 lr=1.53e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 8800/12000 loss=4.3472 gn=1.01 lr=1.52e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 8820/12000 loss=4.2989 gn=0.97 lr=1.51e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 8840/12000 loss=4.5025 gn=1.07 lr=1.50e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 8860/12000 loss=4.3546 gn=1.04 lr=1.49e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 8880/12000 loss=4.3920 gn=0.95 lr=1.48e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 8900/12000 loss=4.2014 gn=0.96 lr=1.47e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 8920/12000 loss=4.3841 gn=0.98 lr=1.46e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 8940/12000 loss=4.3882 gn=0.95 lr=1.45e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 8960/12000 loss=4.4081 gn=0.92 lr=1.44e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 8980/12000 loss=4.4546 gn=1.04 lr=1.43e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9000/12000 loss=4.3931 gn=1.05 lr=1.42e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 9000: val_loss=4.4615 val_ppl=86.62 +[baseline s4] step 9020/12000 loss=4.4698 gn=1.04 lr=1.41e-04 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 9040/12000 loss=4.4173 gn=1.04 lr=1.40e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 9060/12000 loss=4.1491 gn=1.04 lr=1.39e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 9080/12000 loss=4.4883 gn=1.01 lr=1.38e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 9100/12000 loss=4.4311 gn=1.02 lr=1.37e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 9120/12000 loss=4.2795 gn=1.04 lr=1.36e-04 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 9140/12000 loss=4.2335 gn=1.00 lr=1.35e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 9160/12000 loss=4.4655 gn=1.07 lr=1.34e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 9180/12000 loss=4.4697 gn=0.94 lr=1.33e-04 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 9200/12000 loss=4.2646 gn=1.09 lr=1.32e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 9220/12000 loss=4.4609 gn=1.09 lr=1.31e-04 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 9240/12000 loss=4.4814 gn=1.10 lr=1.30e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9260/12000 loss=4.3766 gn=1.06 lr=1.29e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9280/12000 loss=4.3604 gn=1.13 lr=1.28e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9300/12000 loss=4.5090 gn=1.12 lr=1.27e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9320/12000 loss=4.3059 gn=1.04 lr=1.26e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9340/12000 loss=4.4976 gn=1.04 lr=1.25e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9360/12000 loss=4.3045 gn=0.95 lr=1.24e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9380/12000 loss=4.4417 gn=1.02 lr=1.23e-04 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 9400/12000 loss=4.2521 gn=1.11 lr=1.22e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9420/12000 loss=4.2689 gn=1.01 lr=1.21e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9440/12000 loss=4.3305 gn=1.24 lr=1.20e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9460/12000 loss=4.3977 gn=0.94 lr=1.19e-04 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 9480/12000 loss=4.3888 gn=1.06 lr=1.19e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 9500/12000 loss=4.5384 gn=1.13 lr=1.18e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 9520/12000 loss=4.2233 gn=1.08 lr=1.17e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 9540/12000 loss=4.3896 gn=1.06 lr=1.16e-04 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 9560/12000 loss=4.5605 gn=1.05 lr=1.15e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 9580/12000 loss=4.2846 gn=1.07 lr=1.14e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 9600/12000 loss=4.4194 gn=1.00 lr=1.13e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 9620/12000 loss=4.3658 gn=1.02 lr=1.12e-04 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 9640/12000 loss=4.5257 gn=1.13 lr=1.12e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 9660/12000 loss=4.3446 gn=1.01 lr=1.11e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 9680/12000 loss=4.5249 gn=1.10 lr=1.10e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 9700/12000 loss=4.2826 gn=1.07 lr=1.09e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 9720/12000 loss=4.3255 gn=1.17 lr=1.08e-04 tps=24603 mem=5.4G lam=None Gm=None +[baseline s4] step 9740/12000 loss=4.3797 gn=1.04 lr=1.07e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9760/12000 loss=4.4052 gn=1.00 lr=1.07e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9780/12000 loss=4.1333 gn=1.07 lr=1.06e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9800/12000 loss=4.3037 gn=1.06 lr=1.05e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9820/12000 loss=4.1839 gn=0.98 lr=1.04e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9840/12000 loss=4.5019 gn=1.00 lr=1.03e-04 tps=24604 mem=5.4G lam=None Gm=None +[baseline s4] step 9860/12000 loss=4.3223 gn=1.12 lr=1.03e-04 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9880/12000 loss=4.3504 gn=1.01 lr=1.02e-04 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9900/12000 loss=4.2691 gn=1.07 lr=1.01e-04 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9920/12000 loss=4.3176 gn=1.09 lr=1.00e-04 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9940/12000 loss=4.5210 gn=1.04 lr=9.96e-05 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9960/12000 loss=4.4109 gn=1.08 lr=9.89e-05 tps=24605 mem=5.4G lam=None Gm=None +[baseline s4] step 9980/12000 loss=4.3786 gn=1.03 lr=9.81e-05 tps=24606 mem=5.4G lam=None Gm=None +[baseline s4] step 10000/12000 loss=4.3567 gn=0.96 lr=9.74e-05 tps=24606 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 10000: val_loss=4.4265 val_ppl=83.64 +[baseline s4] step 10020/12000 loss=4.5097 gn=1.07 lr=9.67e-05 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 10040/12000 loss=4.3275 gn=1.03 lr=9.59e-05 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 10060/12000 loss=4.3184 gn=1.10 lr=9.52e-05 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 10080/12000 loss=4.3542 gn=1.55 lr=9.45e-05 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 10100/12000 loss=4.3856 gn=1.06 lr=9.38e-05 tps=24597 mem=5.4G lam=None Gm=None +[baseline s4] step 10120/12000 loss=4.3743 gn=0.95 lr=9.31e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10140/12000 loss=4.1586 gn=1.01 lr=9.24e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10160/12000 loss=4.2873 gn=1.18 lr=9.18e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10180/12000 loss=4.4518 gn=1.04 lr=9.11e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10200/12000 loss=4.3280 gn=1.05 lr=9.04e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10220/12000 loss=4.4111 gn=1.14 lr=8.98e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10240/12000 loss=4.2416 gn=1.14 lr=8.91e-05 tps=24598 mem=5.4G lam=None Gm=None +[baseline s4] step 10260/12000 loss=4.4092 gn=1.21 lr=8.85e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10280/12000 loss=4.4107 gn=1.08 lr=8.78e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10300/12000 loss=4.2900 gn=1.09 lr=8.72e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10320/12000 loss=4.4488 gn=1.12 lr=8.66e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10340/12000 loss=4.3524 gn=1.16 lr=8.59e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10360/12000 loss=4.4406 gn=1.14 lr=8.53e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10380/12000 loss=4.4022 gn=1.14 lr=8.47e-05 tps=24599 mem=5.4G lam=None Gm=None +[baseline s4] step 10400/12000 loss=4.4503 gn=1.11 lr=8.41e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10420/12000 loss=4.4130 gn=1.07 lr=8.35e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10440/12000 loss=4.2722 gn=1.18 lr=8.30e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10460/12000 loss=4.4104 gn=1.11 lr=8.24e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10480/12000 loss=4.2685 gn=1.06 lr=8.18e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10500/12000 loss=4.3509 gn=1.13 lr=8.12e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10520/12000 loss=4.4262 gn=1.11 lr=8.07e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10540/12000 loss=4.3451 gn=1.17 lr=8.01e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10560/12000 loss=4.3485 gn=1.06 lr=7.96e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10580/12000 loss=4.3821 gn=1.25 lr=7.91e-05 tps=24600 mem=5.4G lam=None Gm=None +[baseline s4] step 10600/12000 loss=4.2389 gn=1.00 lr=7.85e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10620/12000 loss=4.2676 gn=1.15 lr=7.80e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10640/12000 loss=4.2806 gn=1.03 lr=7.75e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10660/12000 loss=4.4381 gn=1.13 lr=7.70e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10680/12000 loss=4.3318 gn=1.24 lr=7.65e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10700/12000 loss=4.3179 gn=1.10 lr=7.60e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10720/12000 loss=4.4433 gn=1.18 lr=7.55e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10740/12000 loss=4.4193 gn=1.21 lr=7.50e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10760/12000 loss=4.2094 gn=1.20 lr=7.46e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10780/12000 loss=4.2631 gn=1.37 lr=7.41e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10800/12000 loss=4.3327 gn=1.17 lr=7.37e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10820/12000 loss=4.2858 gn=1.14 lr=7.32e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10840/12000 loss=4.2975 gn=1.14 lr=7.28e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10860/12000 loss=4.2774 gn=1.22 lr=7.23e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10880/12000 loss=4.2591 gn=1.20 lr=7.19e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10900/12000 loss=4.1824 gn=1.05 lr=7.15e-05 tps=24601 mem=5.4G lam=None Gm=None +[baseline s4] step 10920/12000 loss=4.3505 gn=1.07 lr=7.11e-05 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 10940/12000 loss=4.3244 gn=1.18 lr=7.07e-05 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 10960/12000 loss=4.3126 gn=1.12 lr=7.03e-05 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 10980/12000 loss=4.4738 gn=1.19 lr=6.99e-05 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] step 11000/12000 loss=4.3937 gn=1.04 lr=6.95e-05 tps=24602 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 11000: val_loss=4.3780 val_ppl=79.68 +[baseline s4] step 11020/12000 loss=4.2131 gn=1.10 lr=6.91e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11040/12000 loss=4.2987 gn=1.10 lr=6.88e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11060/12000 loss=4.4624 gn=1.21 lr=6.84e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11080/12000 loss=4.3256 gn=1.05 lr=6.81e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11100/12000 loss=4.3603 gn=1.23 lr=6.77e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11120/12000 loss=4.4071 gn=1.10 lr=6.74e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11140/12000 loss=4.3703 gn=1.18 lr=6.70e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11160/12000 loss=4.2571 gn=1.05 lr=6.67e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11180/12000 loss=4.3690 gn=1.28 lr=6.64e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11200/12000 loss=4.3607 gn=1.14 lr=6.61e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11220/12000 loss=4.2135 gn=1.11 lr=6.58e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11240/12000 loss=4.4120 gn=1.07 lr=6.55e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11260/12000 loss=4.3977 gn=1.06 lr=6.52e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11280/12000 loss=4.4274 gn=1.22 lr=6.49e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11300/12000 loss=4.2730 gn=1.21 lr=6.47e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11320/12000 loss=4.3537 gn=1.03 lr=6.44e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11340/12000 loss=4.5851 gn=1.20 lr=6.42e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11360/12000 loss=4.2724 gn=1.08 lr=6.39e-05 tps=24594 mem=5.4G lam=None Gm=None +[baseline s4] step 11380/12000 loss=4.3523 gn=1.25 lr=6.37e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11400/12000 loss=4.3299 gn=1.29 lr=6.34e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11420/12000 loss=4.2841 gn=1.13 lr=6.32e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11440/12000 loss=4.3702 gn=1.13 lr=6.30e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11460/12000 loss=4.2920 gn=1.18 lr=6.28e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11480/12000 loss=4.5037 gn=1.12 lr=6.26e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11500/12000 loss=4.2734 gn=1.13 lr=6.24e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11520/12000 loss=4.4505 gn=1.20 lr=6.22e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11540/12000 loss=4.3780 gn=1.28 lr=6.20e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11560/12000 loss=4.3517 gn=1.12 lr=6.19e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11580/12000 loss=4.4554 gn=1.14 lr=6.17e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11600/12000 loss=4.3126 gn=1.29 lr=6.15e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11620/12000 loss=4.2206 gn=1.25 lr=6.14e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11640/12000 loss=4.3181 gn=1.20 lr=6.12e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11660/12000 loss=4.2125 gn=1.19 lr=6.11e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11680/12000 loss=4.3541 gn=1.12 lr=6.10e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11700/12000 loss=4.2539 gn=1.19 lr=6.09e-05 tps=24595 mem=5.4G lam=None Gm=None +[baseline s4] step 11720/12000 loss=4.3097 gn=1.28 lr=6.07e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11740/12000 loss=4.2739 gn=1.10 lr=6.06e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11760/12000 loss=4.3402 gn=1.12 lr=6.06e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11780/12000 loss=4.5873 gn=1.56 lr=6.05e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11800/12000 loss=4.4274 gn=1.18 lr=6.04e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11820/12000 loss=4.4448 gn=1.28 lr=6.03e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11840/12000 loss=4.4330 gn=1.16 lr=6.02e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11860/12000 loss=4.2737 gn=1.23 lr=6.02e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11880/12000 loss=4.3180 gn=1.14 lr=6.01e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11900/12000 loss=4.3165 gn=1.24 lr=6.01e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11920/12000 loss=4.3017 gn=1.23 lr=6.01e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11940/12000 loss=4.4711 gn=1.27 lr=6.00e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11960/12000 loss=4.3874 gn=1.29 lr=6.00e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] step 11980/12000 loss=4.2869 gn=1.13 lr=6.00e-05 tps=24596 mem=5.4G lam=None Gm=None +[baseline s4] EVAL step 11999: val_loss=4.3534 val_ppl=77.74 +[init] variant=s_only seed=4 params=63.97M +[init] use_S=True use_G=False +[s_only s4] step 0/12000 loss=10.8749 gn=13.18 lr=3.00e-06 tps=8231 mem=5.0G lam=0.12692801654338837 Gm=None +[s_only s4] step 20/12000 loss=9.8635 gn=1.97 lr=6.30e-05 tps=19663 mem=5.5G lam=0.1269354640195767 Gm=None +[s_only s4] step 40/12000 loss=9.1046 gn=1.75 lr=1.23e-04 tps=20320 mem=5.5G lam=0.1268776822835207 Gm=None +[s_only s4] step 60/12000 loss=8.2392 gn=1.51 lr=1.83e-04 tps=20640 mem=5.5G lam=0.12680154169599214 Gm=None +[s_only s4] step 80/12000 loss=7.5625 gn=0.78 lr=2.43e-04 tps=20770 mem=5.5G lam=0.12670300155878067 Gm=None +[s_only s4] step 100/12000 loss=7.0056 gn=0.68 lr=3.03e-04 tps=20858 mem=5.5G lam=0.12663045898079872 Gm=None +[s_only s4] step 120/12000 loss=7.1905 gn=0.59 lr=3.63e-04 tps=20907 mem=5.5G lam=0.12653953147431216 Gm=None +[s_only s4] step 140/12000 loss=6.8227 gn=0.60 lr=4.23e-04 tps=20943 mem=5.5G lam=0.12645861444373926 Gm=None +[s_only s4] step 160/12000 loss=6.7416 gn=0.71 lr=4.83e-04 tps=20977 mem=5.5G lam=0.1264731672902902 Gm=None +[s_only s4] step 180/12000 loss=6.7637 gn=0.61 lr=5.43e-04 tps=20998 mem=5.5G lam=0.12646111225088438 Gm=None +[s_only s4] step 200/12000 loss=6.7242 gn=0.98 lr=6.00e-04 tps=21024 mem=5.5G lam=0.126350407799085 Gm=None +[s_only s4] step 220/12000 loss=6.7364 gn=0.62 lr=6.00e-04 tps=21020 mem=5.5G lam=0.1263274255519112 Gm=None +[s_only s4] step 240/12000 loss=6.5418 gn=0.66 lr=6.00e-04 tps=21037 mem=5.5G lam=0.1262963181361556 Gm=None +[s_only s4] step 260/12000 loss=6.6101 gn=0.46 lr=6.00e-04 tps=21048 mem=5.5G lam=0.12624366860836744 Gm=None +[s_only s4] step 280/12000 loss=6.6734 gn=0.56 lr=6.00e-04 tps=21062 mem=5.5G lam=0.12626296592255434 Gm=None +[s_only s4] step 300/12000 loss=6.3615 gn=0.55 lr=6.00e-04 tps=21072 mem=5.5G lam=0.12622429430484772 Gm=None +[s_only s4] step 320/12000 loss=6.4086 gn=0.57 lr=6.00e-04 tps=21084 mem=5.5G lam=0.1262331005806724 Gm=None +[s_only s4] step 340/12000 loss=6.4347 gn=0.52 lr=6.00e-04 tps=21083 mem=5.5G lam=0.1262821014970541 Gm=None +[s_only s4] step 360/12000 loss=6.4866 gn=0.52 lr=6.00e-04 tps=21089 mem=5.5G lam=0.12626108154654503 Gm=None +[s_only s4] step 380/12000 loss=6.4070 gn=0.58 lr=6.00e-04 tps=21087 mem=5.5G lam=0.12636485882103443 Gm=None +[s_only s4] step 400/12000 loss=6.3030 gn=0.53 lr=6.00e-04 tps=21086 mem=5.5G lam=0.12637504500647387 Gm=None +[s_only s4] step 420/12000 loss=6.1821 gn=0.56 lr=6.00e-04 tps=21089 mem=5.5G lam=0.1264394416163365 Gm=None +[s_only s4] step 440/12000 loss=6.2792 gn=0.56 lr=5.99e-04 tps=21095 mem=5.5G lam=0.1264825069035093 Gm=None +[s_only s4] step 460/12000 loss=6.3112 gn=0.56 lr=5.99e-04 tps=21097 mem=5.5G lam=0.1265180641785264 Gm=None +[s_only s4] step 480/12000 loss=6.2986 gn=0.53 lr=5.99e-04 tps=21103 mem=5.5G lam=0.12657696443299452 Gm=None +[s_only s4] step 500/12000 loss=6.2349 gn=0.54 lr=5.99e-04 tps=21104 mem=5.5G lam=0.1267233146354556 Gm=None +[s_only s4] step 520/12000 loss=6.2714 gn=0.50 lr=5.99e-04 tps=21108 mem=5.5G lam=0.12685844643662372 Gm=None +[s_only s4] step 540/12000 loss=6.0896 gn=0.56 lr=5.99e-04 tps=21111 mem=5.5G lam=0.1269174808015426 Gm=None +[s_only s4] step 560/12000 loss=6.2320 gn=0.60 lr=5.99e-04 tps=21112 mem=5.5G lam=0.1270720042909185 Gm=None +[s_only s4] step 580/12000 loss=6.0924 gn=0.52 lr=5.99e-04 tps=21113 mem=5.5G lam=0.12728003381441036 Gm=None +[s_only s4] step 600/12000 loss=6.0363 gn=0.59 lr=5.98e-04 tps=21110 mem=5.5G lam=0.12754694713900486 Gm=None +[s_only s4] step 620/12000 loss=6.1409 gn=0.47 lr=5.98e-04 tps=21111 mem=5.5G lam=0.12779868704577288 Gm=None +[s_only s4] step 640/12000 loss=6.2377 gn=0.52 lr=5.98e-04 tps=21112 mem=5.5G lam=0.12804412438223758 Gm=None +[s_only s4] step 660/12000 loss=6.0785 gn=0.56 lr=5.98e-04 tps=21116 mem=5.5G lam=0.12834056839346886 Gm=None +[s_only s4] step 680/12000 loss=6.0807 gn=0.66 lr=5.98e-04 tps=21117 mem=5.5G lam=0.12852566440900168 Gm=None +[s_only s4] step 700/12000 loss=5.9925 gn=0.52 lr=5.98e-04 tps=21124 mem=5.5G lam=0.1287958218405644 Gm=None +[s_only s4] step 720/12000 loss=5.9510 gn=0.59 lr=5.97e-04 tps=21123 mem=5.5G lam=0.1290908350298802 Gm=None +[s_only s4] step 740/12000 loss=6.0543 gn=0.54 lr=5.97e-04 tps=21125 mem=5.5G lam=0.12948724099745354 Gm=None +[s_only s4] step 760/12000 loss=6.0089 gn=0.50 lr=5.97e-04 tps=21127 mem=5.5G lam=0.1299094914769133 Gm=None +[s_only s4] step 780/12000 loss=6.1386 gn=0.58 lr=5.97e-04 tps=21128 mem=5.5G lam=0.13032151044656834 Gm=None +[s_only s4] step 800/12000 loss=5.8521 gn=0.55 lr=5.97e-04 tps=21125 mem=5.5G lam=0.13065565843135118 Gm=None +[s_only s4] step 820/12000 loss=5.9238 gn=0.51 lr=5.96e-04 tps=21125 mem=5.5G lam=0.13110509204367796 Gm=None +[s_only s4] step 840/12000 loss=5.8719 gn=0.59 lr=5.96e-04 tps=21128 mem=5.5G lam=0.1314980968212088 Gm=None +[s_only s4] step 860/12000 loss=5.9279 gn=0.59 lr=5.96e-04 tps=21125 mem=5.5G lam=0.13195150438696146 Gm=None +[s_only s4] step 880/12000 loss=5.7863 gn=0.53 lr=5.96e-04 tps=21128 mem=5.5G lam=0.13244060271730027 Gm=None +[s_only s4] step 900/12000 loss=5.7835 gn=0.55 lr=5.95e-04 tps=21130 mem=5.5G lam=0.13282673278202614 Gm=None +[s_only s4] step 920/12000 loss=5.8573 gn=0.57 lr=5.95e-04 tps=21132 mem=5.5G lam=0.13325486580530801 Gm=None +[s_only s4] step 940/12000 loss=5.7695 gn=0.56 lr=5.95e-04 tps=21134 mem=5.5G lam=0.13370732621600231 Gm=None +[s_only s4] step 960/12000 loss=5.8165 gn=0.55 lr=5.94e-04 tps=21135 mem=5.5G lam=0.13413415849208832 Gm=None +[s_only s4] step 980/12000 loss=5.8184 gn=0.57 lr=5.94e-04 tps=21138 mem=5.5G lam=0.1346285796413819 Gm=None +[s_only s4] step 1000/12000 loss=5.7310 gn=0.63 lr=5.94e-04 tps=21138 mem=5.5G lam=0.13513978415479264 Gm=None +[s_only s4] EVAL step 1000: val_loss=5.7897 val_ppl=326.91 +[s_only s4] step 1020/12000 loss=5.7814 gn=0.57 lr=5.94e-04 tps=21064 mem=5.5G lam=0.1356215492511789 Gm=None +[s_only s4] step 1040/12000 loss=5.7328 gn=0.55 lr=5.93e-04 tps=21064 mem=5.5G lam=0.13606574945151806 Gm=None +[s_only s4] step 1060/12000 loss=5.6228 gn=0.52 lr=5.93e-04 tps=21062 mem=5.5G lam=0.1365930556009213 Gm=None +[s_only s4] step 1080/12000 loss=5.6708 gn=0.58 lr=5.93e-04 tps=21065 mem=5.5G lam=0.1370550518234571 Gm=None +[s_only s4] step 1100/12000 loss=5.8341 gn=0.55 lr=5.92e-04 tps=21066 mem=5.5G lam=0.13754963502287865 Gm=None +[s_only s4] step 1120/12000 loss=5.7695 gn=0.52 lr=5.92e-04 tps=21067 mem=5.5G lam=0.13799921981990337 Gm=None +[s_only s4] step 1140/12000 loss=5.5920 gn=0.54 lr=5.92e-04 tps=21068 mem=5.5G lam=0.13847187533974648 Gm=None +[s_only s4] step 1160/12000 loss=5.6270 gn=0.52 lr=5.91e-04 tps=21071 mem=5.5G lam=0.13897408482929072 Gm=None +[s_only s4] step 1180/12000 loss=5.7183 gn=0.57 lr=5.91e-04 tps=21073 mem=5.5G lam=0.13945387521137795 Gm=None +[s_only s4] step 1200/12000 loss=5.6840 gn=0.54 lr=5.90e-04 tps=21072 mem=5.5G lam=0.1399195995181799 Gm=None +[s_only s4] step 1220/12000 loss=5.9013 gn=0.57 lr=5.90e-04 tps=21075 mem=5.5G lam=0.14036059007048607 Gm=None +[s_only s4] step 1240/12000 loss=5.5504 gn=0.52 lr=5.90e-04 tps=21077 mem=5.5G lam=0.14083771035075188 Gm=None +[s_only s4] step 1260/12000 loss=5.5868 gn=0.56 lr=5.89e-04 tps=21078 mem=5.5G lam=0.14135889802128077 Gm=None +[s_only s4] step 1280/12000 loss=5.4536 gn=0.51 lr=5.89e-04 tps=21080 mem=5.5G lam=0.14184506703168154 Gm=None +[s_only s4] step 1300/12000 loss=5.7682 gn=0.56 lr=5.89e-04 tps=21082 mem=5.5G lam=0.14232502474139133 Gm=None +[s_only s4] step 1320/12000 loss=5.5360 gn=0.59 lr=5.88e-04 tps=21086 mem=5.5G lam=0.14278402210523686 Gm=None +[s_only s4] step 1340/12000 loss=5.4999 gn=0.54 lr=5.88e-04 tps=21086 mem=5.5G lam=0.14332549987981716 Gm=None +[s_only s4] step 1360/12000 loss=5.6200 gn=0.59 lr=5.87e-04 tps=21087 mem=5.5G lam=0.14378527738153934 Gm=None +[s_only s4] step 1380/12000 loss=5.5561 gn=0.57 lr=5.87e-04 tps=21088 mem=5.5G lam=0.14423525892198086 Gm=None +[s_only s4] step 1400/12000 loss=5.4464 gn=0.59 lr=5.86e-04 tps=21089 mem=5.5G lam=0.14473018236458302 Gm=None +[s_only s4] step 1420/12000 loss=5.5575 gn=0.55 lr=5.86e-04 tps=21089 mem=5.5G lam=0.14516903273761272 Gm=None +[s_only s4] step 1440/12000 loss=5.5046 gn=0.65 lr=5.85e-04 tps=21090 mem=5.5G lam=0.14563736350586018 Gm=None +[s_only s4] step 1460/12000 loss=5.4280 gn=0.59 lr=5.85e-04 tps=21091 mem=5.5G lam=0.1461489892875155 Gm=None +[s_only s4] step 1480/12000 loss=5.4680 gn=0.60 lr=5.84e-04 tps=21092 mem=5.5G lam=0.14668714224050441 Gm=None +[s_only s4] step 1500/12000 loss=5.5081 gn=0.61 lr=5.84e-04 tps=21095 mem=5.5G lam=0.14718262509753308 Gm=None +[s_only s4] step 1520/12000 loss=5.3453 gn=0.58 lr=5.83e-04 tps=21096 mem=5.5G lam=0.14758962796380123 Gm=None +[s_only s4] step 1540/12000 loss=5.4387 gn=0.60 lr=5.83e-04 tps=21098 mem=5.5G lam=0.14806531059245268 Gm=None +[s_only s4] step 1560/12000 loss=5.5024 gn=0.56 lr=5.82e-04 tps=21099 mem=5.5G lam=0.14854030528416237 Gm=None +[s_only s4] step 1580/12000 loss=5.5804 gn=0.58 lr=5.82e-04 tps=21100 mem=5.5G lam=0.14898411153505245 Gm=None +[s_only s4] step 1600/12000 loss=5.2680 gn=0.56 lr=5.81e-04 tps=21101 mem=5.5G lam=0.1495009878029426 Gm=None +[s_only s4] step 1620/12000 loss=5.4041 gn=0.59 lr=5.81e-04 tps=21104 mem=5.5G lam=0.14998728719850382 Gm=None +[s_only s4] step 1640/12000 loss=5.4315 gn=0.60 lr=5.80e-04 tps=21105 mem=5.5G lam=0.15047285022834936 Gm=None +[s_only s4] step 1660/12000 loss=5.3782 gn=0.55 lr=5.80e-04 tps=21106 mem=5.5G lam=0.15097498365988335 Gm=None +[s_only s4] step 1680/12000 loss=5.2615 gn=0.57 lr=5.79e-04 tps=21108 mem=5.5G lam=0.15139164589345455 Gm=None +[s_only s4] step 1700/12000 loss=5.3314 gn=0.57 lr=5.79e-04 tps=21109 mem=5.5G lam=0.15190307578692833 Gm=None +[s_only s4] step 1720/12000 loss=5.4281 gn=0.62 lr=5.78e-04 tps=21110 mem=5.5G lam=0.1523720740030209 Gm=None +[s_only s4] step 1740/12000 loss=5.5031 gn=0.60 lr=5.78e-04 tps=21110 mem=5.5G lam=0.15287526790052652 Gm=None +[s_only s4] step 1760/12000 loss=5.3176 gn=0.57 lr=5.77e-04 tps=21110 mem=5.5G lam=0.15336126772065958 Gm=None +[s_only s4] step 1780/12000 loss=5.4728 gn=0.60 lr=5.76e-04 tps=21111 mem=5.5G lam=0.15387929572413364 Gm=None +[s_only s4] step 1800/12000 loss=5.2239 gn=0.60 lr=5.76e-04 tps=21110 mem=5.5G lam=0.15444793365895748 Gm=None +[s_only s4] step 1820/12000 loss=5.3111 gn=0.60 lr=5.75e-04 tps=21112 mem=5.5G lam=0.1549294845511516 Gm=None +[s_only s4] step 1840/12000 loss=5.3633 gn=0.63 lr=5.75e-04 tps=21112 mem=5.5G lam=0.15544309156636396 Gm=None +[s_only s4] step 1860/12000 loss=5.5619 gn=0.59 lr=5.74e-04 tps=21114 mem=5.5G lam=0.15590902976691723 Gm=None +[s_only s4] step 1880/12000 loss=5.4245 gn=0.61 lr=5.73e-04 tps=21114 mem=5.5G lam=0.15646408963948488 Gm=None +[s_only s4] step 1900/12000 loss=5.3333 gn=0.60 lr=5.73e-04 tps=21115 mem=5.5G lam=0.15700544199595848 Gm=None +[s_only s4] step 1920/12000 loss=5.3343 gn=0.61 lr=5.72e-04 tps=21116 mem=5.5G lam=0.15751903286824623 Gm=None +[s_only s4] step 1940/12000 loss=5.3381 gn=0.58 lr=5.72e-04 tps=21117 mem=5.5G lam=0.1580360767741998 Gm=None +[s_only s4] step 1960/12000 loss=5.4636 gn=0.60 lr=5.71e-04 tps=21118 mem=5.5G lam=0.15865825271854797 Gm=None +[s_only s4] step 1980/12000 loss=5.2308 gn=0.56 lr=5.70e-04 tps=21118 mem=5.5G lam=0.15921222263326248 Gm=None +[s_only s4] step 2000/12000 loss=5.2902 gn=0.64 lr=5.70e-04 tps=21119 mem=5.5G lam=0.15977114625275135 Gm=None +[s_only s4] EVAL step 2000: val_loss=5.3427 val_ppl=209.08 +[s_only s4] step 2020/12000 loss=5.2527 gn=0.69 lr=5.69e-04 tps=21080 mem=5.5G lam=0.1603714128335317 Gm=None +[s_only s4] step 2040/12000 loss=5.4215 gn=0.69 lr=5.68e-04 tps=21081 mem=5.5G lam=0.16094677615910769 Gm=None +[s_only s4] step 2060/12000 loss=5.3400 gn=0.65 lr=5.68e-04 tps=21081 mem=5.5G lam=0.1614952559272448 Gm=None +[s_only s4] step 2080/12000 loss=5.2401 gn=0.62 lr=5.67e-04 tps=21082 mem=5.5G lam=0.16203524420658746 Gm=None +[s_only s4] step 2100/12000 loss=5.1656 gn=0.69 lr=5.66e-04 tps=21084 mem=5.5G lam=0.1625828609491388 Gm=None +[s_only s4] step 2120/12000 loss=5.2565 gn=0.63 lr=5.65e-04 tps=21084 mem=5.5G lam=0.16315384022891521 Gm=None +[s_only s4] step 2140/12000 loss=5.2210 gn=0.61 lr=5.65e-04 tps=21084 mem=5.5G lam=0.16369951236993074 Gm=None +[s_only s4] step 2160/12000 loss=5.1868 gn=0.59 lr=5.64e-04 tps=21086 mem=5.5G lam=0.16433575997749963 Gm=None +[s_only s4] step 2180/12000 loss=5.3207 gn=0.60 lr=5.63e-04 tps=21086 mem=5.5G lam=0.1649023313075304 Gm=None +[s_only s4] step 2200/12000 loss=5.2279 gn=0.64 lr=5.63e-04 tps=21087 mem=5.5G lam=0.16543951289107403 Gm=None +[s_only s4] step 2220/12000 loss=5.1789 gn=0.58 lr=5.62e-04 tps=21088 mem=5.5G lam=0.16608269046992064 Gm=None +[s_only s4] step 2240/12000 loss=5.1014 gn=0.62 lr=5.61e-04 tps=21088 mem=5.5G lam=0.1666701715439558 Gm=None +[s_only s4] step 2260/12000 loss=5.2904 gn=0.75 lr=5.60e-04 tps=21088 mem=5.5G lam=0.16722732999672493 Gm=None +[s_only s4] step 2280/12000 loss=5.1815 gn=0.63 lr=5.60e-04 tps=21089 mem=5.5G lam=0.16780183495332798 Gm=None +[s_only s4] step 2300/12000 loss=5.1394 gn=0.60 lr=5.59e-04 tps=21089 mem=5.5G lam=0.16832317505031824 Gm=None +[s_only s4] step 2320/12000 loss=5.0390 gn=0.68 lr=5.58e-04 tps=21089 mem=5.5G lam=0.1689060271407167 Gm=None +[s_only s4] step 2340/12000 loss=5.1985 gn=0.62 lr=5.57e-04 tps=21088 mem=5.5G lam=0.16950232225159803 Gm=None +[s_only s4] step 2360/12000 loss=4.9491 gn=0.67 lr=5.57e-04 tps=21080 mem=5.5G lam=0.16999093784640232 Gm=None +[s_only s4] step 2380/12000 loss=5.1323 gn=0.67 lr=5.56e-04 tps=21080 mem=5.5G lam=0.17052102275192738 Gm=None +[s_only s4] step 2400/12000 loss=5.1140 gn=0.66 lr=5.55e-04 tps=21081 mem=5.5G lam=0.17110408221681914 Gm=None +[s_only s4] step 2420/12000 loss=5.0624 gn=0.65 lr=5.54e-04 tps=21081 mem=5.5G lam=0.17163931609441838 Gm=None +[s_only s4] step 2440/12000 loss=5.1157 gn=0.65 lr=5.53e-04 tps=21083 mem=5.5G lam=0.17223171424120665 Gm=None +[s_only s4] step 2460/12000 loss=5.0795 gn=0.64 lr=5.53e-04 tps=21083 mem=5.5G lam=0.1728110620751977 Gm=None +[s_only s4] step 2480/12000 loss=5.2171 gn=0.69 lr=5.52e-04 tps=21085 mem=5.5G lam=0.17339391137162843 Gm=None +[s_only s4] step 2500/12000 loss=5.1378 gn=0.64 lr=5.51e-04 tps=21086 mem=5.5G lam=0.17406232127298912 Gm=None +[s_only s4] step 2520/12000 loss=5.0238 gn=0.69 lr=5.50e-04 tps=21087 mem=5.5G lam=0.17463181478281817 Gm=None +[s_only s4] step 2540/12000 loss=5.0404 gn=0.64 lr=5.49e-04 tps=21087 mem=5.5G lam=0.1752179025982817 Gm=None +[s_only s4] step 2560/12000 loss=5.2294 gn=0.61 lr=5.48e-04 tps=21088 mem=5.5G lam=0.1758183343335986 Gm=None +[s_only s4] step 2580/12000 loss=5.0742 gn=0.63 lr=5.48e-04 tps=21091 mem=5.5G lam=0.1764581153790156 Gm=None +[s_only s4] step 2600/12000 loss=5.0489 gn=0.61 lr=5.47e-04 tps=21093 mem=5.5G lam=0.17703604511916637 Gm=None +[s_only s4] step 2620/12000 loss=5.1020 gn=0.70 lr=5.46e-04 tps=21096 mem=5.5G lam=0.1775686483209332 Gm=None +[s_only s4] step 2640/12000 loss=5.1127 gn=0.69 lr=5.45e-04 tps=21098 mem=5.5G lam=0.17812064538399378 Gm=None +[s_only s4] step 2660/12000 loss=5.2103 gn=0.63 lr=5.44e-04 tps=21101 mem=5.5G lam=0.17868295001486936 Gm=None +[s_only s4] step 2680/12000 loss=4.9332 gn=0.68 lr=5.43e-04 tps=21103 mem=5.5G lam=0.17930863921840987 Gm=None +[s_only s4] step 2700/12000 loss=5.1937 gn=0.69 lr=5.42e-04 tps=21106 mem=5.5G lam=0.17990991535286108 Gm=None +[s_only s4] step 2720/12000 loss=5.1548 gn=0.65 lr=5.41e-04 tps=21107 mem=5.5G lam=0.1804751018062234 Gm=None +[s_only s4] step 2740/12000 loss=5.1408 gn=0.70 lr=5.41e-04 tps=21107 mem=5.5G lam=0.18107955250889063 Gm=None +[s_only s4] step 2760/12000 loss=5.0487 gn=0.70 lr=5.40e-04 tps=21107 mem=5.5G lam=0.18168274872004986 Gm=None +[s_only s4] step 2780/12000 loss=5.1346 gn=0.75 lr=5.39e-04 tps=21108 mem=5.5G lam=0.18222266001005968 Gm=None +[s_only s4] step 2800/12000 loss=5.0341 gn=0.66 lr=5.38e-04 tps=21108 mem=5.5G lam=0.18276780874778828 Gm=None +[s_only s4] step 2820/12000 loss=4.9559 gn=0.68 lr=5.37e-04 tps=21109 mem=5.5G lam=0.1833235820134481 Gm=None +[s_only s4] step 2840/12000 loss=4.9435 gn=0.70 lr=5.36e-04 tps=21110 mem=5.5G lam=0.1838997658342123 Gm=None +[s_only s4] step 2860/12000 loss=5.0496 gn=0.68 lr=5.35e-04 tps=21111 mem=5.5G lam=0.18451861975093684 Gm=None +[s_only s4] step 2880/12000 loss=5.0327 gn=0.68 lr=5.34e-04 tps=21113 mem=5.5G lam=0.18510191577176252 Gm=None +[s_only s4] step 2900/12000 loss=5.0322 gn=0.68 lr=5.33e-04 tps=21115 mem=5.5G lam=0.18572899606078863 Gm=None +[s_only s4] step 2920/12000 loss=5.0817 gn=0.64 lr=5.32e-04 tps=21116 mem=5.5G lam=0.18633599765598774 Gm=None +[s_only s4] step 2940/12000 loss=4.9894 gn=0.67 lr=5.31e-04 tps=21118 mem=5.5G lam=0.18699054171641669 Gm=None +[s_only s4] step 2960/12000 loss=5.0075 gn=0.69 lr=5.30e-04 tps=21118 mem=5.5G lam=0.1876399659862121 Gm=None +[s_only s4] step 2980/12000 loss=5.0037 gn=0.75 lr=5.29e-04 tps=21117 mem=5.5G lam=0.1881393746783336 Gm=None +[s_only s4] step 3000/12000 loss=4.9787 gn=0.67 lr=5.28e-04 tps=21117 mem=5.5G lam=0.18870916217565536 Gm=None +[s_only s4] EVAL step 3000: val_loss=5.0046 val_ppl=149.10 +[s_only s4] step 3020/12000 loss=4.9113 gn=0.70 lr=5.27e-04 tps=21092 mem=5.5G lam=0.18933993360648552 Gm=None +[s_only s4] step 3040/12000 loss=4.8570 gn=0.70 lr=5.26e-04 tps=21090 mem=5.5G lam=0.18986332695931196 Gm=None +[s_only s4] step 3060/12000 loss=5.1400 gn=0.76 lr=5.25e-04 tps=21087 mem=5.5G lam=0.19043198755631843 Gm=None +[s_only s4] step 3080/12000 loss=5.0153 gn=0.70 lr=5.24e-04 tps=21086 mem=5.5G lam=0.19102840746442476 Gm=None +[s_only s4] step 3100/12000 loss=5.0429 gn=0.73 lr=5.23e-04 tps=21083 mem=5.5G lam=0.19160251008967558 Gm=None +[s_only s4] step 3120/12000 loss=4.9241 gn=0.68 lr=5.22e-04 tps=21083 mem=5.5G lam=0.19221791283537945 Gm=None +[s_only s4] step 3140/12000 loss=4.9193 gn=0.72 lr=5.21e-04 tps=21082 mem=5.5G lam=0.1928134939322869 Gm=None +[s_only s4] step 3160/12000 loss=5.0320 gn=0.69 lr=5.20e-04 tps=21082 mem=5.5G lam=0.19343890187640986 Gm=None +[s_only s4] step 3180/12000 loss=4.9278 gn=0.74 lr=5.19e-04 tps=21081 mem=5.5G lam=0.1939571142817537 Gm=None +[s_only s4] step 3200/12000 loss=4.9377 gn=0.71 lr=5.18e-04 tps=21080 mem=5.5G lam=0.19447297789156437 Gm=None +[s_only s4] step 3220/12000 loss=5.1159 gn=0.78 lr=5.17e-04 tps=21079 mem=5.5G lam=0.19506021651128927 Gm=None +[s_only s4] step 3240/12000 loss=4.9603 gn=0.72 lr=5.16e-04 tps=21079 mem=5.5G lam=0.19565477998306355 Gm=None +[s_only s4] step 3260/12000 loss=4.9659 gn=0.70 lr=5.15e-04 tps=21079 mem=5.5G lam=0.19619773235172033 Gm=None +[s_only s4] step 3280/12000 loss=4.8596 gn=0.74 lr=5.14e-04 tps=21079 mem=5.5G lam=0.19677575336148342 Gm=None +[s_only s4] step 3300/12000 loss=4.9474 gn=0.71 lr=5.13e-04 tps=21077 mem=5.5G lam=0.19731864022711912 Gm=None +[s_only s4] step 3320/12000 loss=4.8484 gn=0.69 lr=5.12e-04 tps=21076 mem=5.5G lam=0.1978702029834191 Gm=None +[s_only s4] step 3340/12000 loss=4.9560 gn=0.72 lr=5.11e-04 tps=21076 mem=5.5G lam=0.19837298958251873 Gm=None +[s_only s4] step 3360/12000 loss=4.9193 gn=0.77 lr=5.10e-04 tps=21076 mem=5.5G lam=0.1989029347896576 Gm=None +[s_only s4] step 3380/12000 loss=4.8163 gn=0.67 lr=5.09e-04 tps=21076 mem=5.5G lam=0.1994598532716433 Gm=None +[s_only s4] step 3400/12000 loss=4.8375 gn=0.75 lr=5.08e-04 tps=21075 mem=5.5G lam=0.19993178887913624 Gm=None +[s_only s4] step 3420/12000 loss=4.8553 gn=0.68 lr=5.07e-04 tps=21075 mem=5.5G lam=0.20046847872436047 Gm=None +[s_only s4] step 3440/12000 loss=4.7811 gn=0.66 lr=5.06e-04 tps=21076 mem=5.5G lam=0.20100337794671455 Gm=None +[s_only s4] step 3460/12000 loss=4.8056 gn=0.70 lr=5.05e-04 tps=21076 mem=5.5G lam=0.20160925357292095 Gm=None +[s_only s4] step 3480/12000 loss=5.0322 gn=0.73 lr=5.03e-04 tps=21076 mem=5.5G lam=0.20220927459498247 Gm=None +[s_only s4] step 3500/12000 loss=4.9472 gn=0.68 lr=5.02e-04 tps=21075 mem=5.5G lam=0.2027640532081326 Gm=None +[s_only s4] step 3520/12000 loss=4.7838 gn=0.74 lr=5.01e-04 tps=21075 mem=5.5G lam=0.2032819570352634 Gm=None +[s_only s4] step 3540/12000 loss=5.0479 gn=0.74 lr=5.00e-04 tps=21074 mem=5.5G lam=0.203931444324553 Gm=None +[s_only s4] step 3560/12000 loss=4.9291 gn=0.71 lr=4.99e-04 tps=21074 mem=5.5G lam=0.20451344860096773 Gm=None +[s_only s4] step 3580/12000 loss=5.1633 gn=0.75 lr=4.98e-04 tps=21074 mem=5.5G lam=0.20512024126946926 Gm=None +[s_only s4] step 3600/12000 loss=4.9099 gn=0.73 lr=4.97e-04 tps=21074 mem=5.5G lam=0.20564792398363352 Gm=None +[s_only s4] step 3620/12000 loss=4.7675 gn=0.75 lr=4.96e-04 tps=21074 mem=5.5G lam=0.20622948867579302 Gm=None +[s_only s4] step 3640/12000 loss=4.9019 gn=0.74 lr=4.94e-04 tps=21072 mem=5.5G lam=0.2067316916460792 Gm=None +[s_only s4] step 3660/12000 loss=4.8361 gn=0.93 lr=4.93e-04 tps=21066 mem=5.5G lam=0.20736028160899878 Gm=None +[s_only s4] step 3680/12000 loss=4.9078 gn=0.73 lr=4.92e-04 tps=21061 mem=5.5G lam=0.20791446417570114 Gm=None +[s_only s4] step 3700/12000 loss=4.8940 gn=0.75 lr=4.91e-04 tps=21055 mem=5.5G lam=0.20850591889272133 Gm=None +[s_only s4] step 3720/12000 loss=4.7992 gn=0.70 lr=4.90e-04 tps=21048 mem=5.5G lam=0.20908387936651707 Gm=None +[s_only s4] step 3740/12000 loss=4.6355 gn=0.69 lr=4.89e-04 tps=21044 mem=5.5G lam=0.2096655803422133 Gm=None +[s_only s4] step 3760/12000 loss=4.7957 gn=0.74 lr=4.88e-04 tps=21038 mem=5.5G lam=0.21020421975602707 Gm=None +[s_only s4] step 3780/12000 loss=4.7928 gn=0.82 lr=4.86e-04 tps=21030 mem=5.5G lam=0.21075239218771458 Gm=None +[s_only s4] step 3800/12000 loss=4.8613 gn=0.75 lr=4.85e-04 tps=21025 mem=5.5G lam=0.2112573158616821 Gm=None +[s_only s4] step 3820/12000 loss=4.8201 gn=0.70 lr=4.84e-04 tps=21018 mem=5.5G lam=0.21174217698474726 Gm=None +[s_only s4] step 3840/12000 loss=4.6248 gn=0.70 lr=4.83e-04 tps=21010 mem=5.5G lam=0.21226417956252894 Gm=None +[s_only s4] step 3860/12000 loss=4.9207 gn=0.74 lr=4.82e-04 tps=21005 mem=5.5G lam=0.2128626083334287 Gm=None +[s_only s4] step 3880/12000 loss=4.8584 gn=0.73 lr=4.80e-04 tps=21002 mem=5.5G lam=0.213404826199015 Gm=None +[s_only s4] step 3900/12000 loss=4.8731 gn=0.77 lr=4.79e-04 tps=20999 mem=5.5G lam=0.21396617560336986 Gm=None +[s_only s4] step 3920/12000 loss=4.8731 gn=0.79 lr=4.78e-04 tps=20999 mem=5.5G lam=0.2144981399178505 Gm=None +[s_only s4] step 3940/12000 loss=4.7590 gn=0.76 lr=4.77e-04 tps=20999 mem=5.5G lam=0.21507881892224154 Gm=None +[s_only s4] step 3960/12000 loss=4.9632 gn=0.95 lr=4.76e-04 tps=21000 mem=5.5G lam=0.21565426296244064 Gm=None +[s_only s4] step 3980/12000 loss=4.8157 gn=0.82 lr=4.74e-04 tps=21000 mem=5.5G lam=0.21620505085835853 Gm=None +[s_only s4] step 4000/12000 loss=4.7307 gn=0.72 lr=4.73e-04 tps=21001 mem=5.5G lam=0.21682525301973024 Gm=None +[s_only s4] EVAL step 4000: val_loss=4.9045 val_ppl=134.90 +[s_only s4] step 4020/12000 loss=4.6654 gn=0.73 lr=4.72e-04 tps=20981 mem=5.5G lam=0.21743608793864647 Gm=None +[s_only s4] step 4040/12000 loss=4.7871 gn=0.71 lr=4.71e-04 tps=20982 mem=5.5G lam=0.21796236063043276 Gm=None +[s_only s4] step 4060/12000 loss=4.7835 gn=0.74 lr=4.70e-04 tps=20982 mem=5.5G lam=0.21855009098847708 Gm=None +[s_only s4] step 4080/12000 loss=4.8085 gn=0.82 lr=4.68e-04 tps=20983 mem=5.5G lam=0.21906473767012358 Gm=None +[s_only s4] step 4100/12000 loss=4.8587 gn=0.77 lr=4.67e-04 tps=20983 mem=5.5G lam=0.2196924618134896 Gm=None +[s_only s4] step 4120/12000 loss=4.8184 gn=0.77 lr=4.66e-04 tps=20984 mem=5.5G lam=0.22030672958741584 Gm=None +[s_only s4] step 4140/12000 loss=4.8325 gn=0.74 lr=4.65e-04 tps=20984 mem=5.5G lam=0.22091180055091778 Gm=None +[s_only s4] step 4160/12000 loss=4.9304 gn=0.99 lr=4.63e-04 tps=20984 mem=5.5G lam=0.2215104770536224 Gm=None +[s_only s4] step 4180/12000 loss=4.8146 gn=0.74 lr=4.62e-04 tps=20984 mem=5.5G lam=0.22200161932657161 Gm=None +[s_only s4] step 4200/12000 loss=4.8228 gn=0.79 lr=4.61e-04 tps=20984 mem=5.5G lam=0.2225548354908824 Gm=None +[s_only s4] step 4220/12000 loss=4.7708 gn=0.74 lr=4.60e-04 tps=20984 mem=5.5G lam=0.22313200837622085 Gm=None +[s_only s4] step 4240/12000 loss=4.6757 gn=0.77 lr=4.58e-04 tps=20984 mem=5.5G lam=0.22368351370096207 Gm=None +[s_only s4] step 4260/12000 loss=4.7780 gn=0.87 lr=4.57e-04 tps=20985 mem=5.5G lam=0.2243651651466886 Gm=None +[s_only s4] step 4280/12000 loss=4.8849 gn=0.83 lr=4.56e-04 tps=20986 mem=5.5G lam=0.22488130753238997 Gm=None +[s_only s4] step 4300/12000 loss=4.9048 gn=0.80 lr=4.54e-04 tps=20986 mem=5.5G lam=0.22540627575169006 Gm=None +[s_only s4] step 4320/12000 loss=4.7181 gn=0.89 lr=4.53e-04 tps=20987 mem=5.5G lam=0.22602208610624075 Gm=None +[s_only s4] step 4340/12000 loss=4.9537 gn=0.73 lr=4.52e-04 tps=20987 mem=5.5G lam=0.2265478310485681 Gm=None +[s_only s4] step 4360/12000 loss=4.7464 gn=0.78 lr=4.51e-04 tps=20988 mem=5.5G lam=0.22713151387870312 Gm=None +[s_only s4] step 4380/12000 loss=4.4526 gn=0.73 lr=4.49e-04 tps=20988 mem=5.5G lam=0.22765629583348831 Gm=None +[s_only s4] step 4400/12000 loss=4.9151 gn=0.77 lr=4.48e-04 tps=20989 mem=5.5G lam=0.22817335526148477 Gm=None +[s_only s4] step 4420/12000 loss=4.7537 gn=0.78 lr=4.47e-04 tps=20989 mem=5.5G lam=0.22865010052919388 Gm=None +[s_only s4] step 4440/12000 loss=4.9116 gn=0.83 lr=4.45e-04 tps=20988 mem=5.5G lam=0.22913893653700748 Gm=None +[s_only s4] step 4460/12000 loss=4.6216 gn=0.80 lr=4.44e-04 tps=20989 mem=5.5G lam=0.22961131514360508 Gm=None +[s_only s4] step 4480/12000 loss=4.7642 gn=0.73 lr=4.43e-04 tps=20989 mem=5.5G lam=0.23012209497392178 Gm=None +[s_only s4] step 4500/12000 loss=4.8535 gn=0.77 lr=4.42e-04 tps=20989 mem=5.5G lam=0.2305930865307649 Gm=None +[s_only s4] step 4520/12000 loss=4.5538 gn=0.82 lr=4.40e-04 tps=20990 mem=5.5G lam=0.23102025563518205 Gm=None +[s_only s4] step 4540/12000 loss=4.6553 gn=0.73 lr=4.39e-04 tps=20990 mem=5.5G lam=0.231504092613856 Gm=None +[s_only s4] step 4560/12000 loss=4.6518 gn=0.74 lr=4.38e-04 tps=20990 mem=5.5G lam=0.23208291052530208 Gm=None +[s_only s4] step 4580/12000 loss=4.7152 gn=0.78 lr=4.36e-04 tps=20991 mem=5.5G lam=0.2325651136537393 Gm=None +[s_only s4] step 4600/12000 loss=4.6074 gn=0.80 lr=4.35e-04 tps=20992 mem=5.5G lam=0.23304426887383065 Gm=None +[s_only s4] step 4620/12000 loss=4.7065 gn=0.78 lr=4.34e-04 tps=20993 mem=5.5G lam=0.23353508673608303 Gm=None +[s_only s4] step 4640/12000 loss=4.7561 gn=0.81 lr=4.32e-04 tps=20993 mem=5.5G lam=0.23411860689520836 Gm=None +[s_only s4] step 4660/12000 loss=4.6936 gn=0.75 lr=4.31e-04 tps=20994 mem=5.5G lam=0.2345980784545342 Gm=None +[s_only s4] step 4680/12000 loss=4.6362 gn=0.84 lr=4.30e-04 tps=20995 mem=5.5G lam=0.23515196641286215 Gm=None +[s_only s4] step 4700/12000 loss=4.6690 gn=0.76 lr=4.28e-04 tps=20995 mem=5.5G lam=0.23574292287230492 Gm=None +[s_only s4] step 4720/12000 loss=4.6917 gn=0.85 lr=4.27e-04 tps=20996 mem=5.5G lam=0.23628123849630356 Gm=None +[s_only s4] step 4740/12000 loss=4.7486 gn=0.80 lr=4.26e-04 tps=20996 mem=5.5G lam=0.2367440415546298 Gm=None +[s_only s4] step 4760/12000 loss=4.8019 gn=0.81 lr=4.24e-04 tps=20997 mem=5.5G lam=0.23729485180228949 Gm=None +[s_only s4] step 4780/12000 loss=4.6605 gn=0.80 lr=4.23e-04 tps=20997 mem=5.5G lam=0.23778675279269615 Gm=None +[s_only s4] step 4800/12000 loss=4.8341 gn=0.80 lr=4.22e-04 tps=20997 mem=5.5G lam=0.23825581992665926 Gm=None +[s_only s4] step 4820/12000 loss=4.6603 gn=0.80 lr=4.20e-04 tps=20996 mem=5.5G lam=0.23877115019907555 Gm=None +[s_only s4] step 4840/12000 loss=4.5737 gn=0.85 lr=4.19e-04 tps=20996 mem=5.5G lam=0.2393283365915219 Gm=None +[s_only s4] step 4860/12000 loss=4.6444 gn=0.79 lr=4.18e-04 tps=20996 mem=5.5G lam=0.2398453870167335 Gm=None +[s_only s4] step 4880/12000 loss=4.6371 gn=0.75 lr=4.16e-04 tps=20997 mem=5.5G lam=0.24035916198045015 Gm=None +[s_only s4] step 4900/12000 loss=4.7000 gn=0.79 lr=4.15e-04 tps=20997 mem=5.5G lam=0.24080803555746874 Gm=None +[s_only s4] step 4920/12000 loss=4.6720 gn=0.80 lr=4.13e-04 tps=20998 mem=5.5G lam=0.2412382693340381 Gm=None +[s_only s4] step 4940/12000 loss=4.6424 gn=0.79 lr=4.12e-04 tps=20998 mem=5.5G lam=0.24167581057796875 Gm=None +[s_only s4] step 4960/12000 loss=4.7026 gn=0.80 lr=4.11e-04 tps=20999 mem=5.5G lam=0.2422363112370173 Gm=None +[s_only s4] step 4980/12000 loss=4.6313 gn=0.77 lr=4.09e-04 tps=20994 mem=5.5G lam=0.24277615236739317 Gm=None +[s_only s4] step 5000/12000 loss=4.7962 gn=0.86 lr=4.08e-04 tps=20993 mem=5.5G lam=0.2432463833441337 Gm=None +[s_only s4] EVAL step 5000: val_loss=4.7647 val_ppl=117.29 +[s_only s4] step 5020/12000 loss=4.7468 gn=0.77 lr=4.07e-04 tps=20977 mem=5.5G lam=0.2437811428681016 Gm=None +[s_only s4] step 5040/12000 loss=4.8233 gn=0.78 lr=4.05e-04 tps=20975 mem=5.5G lam=0.24429051919529834 Gm=None +[s_only s4] step 5060/12000 loss=4.6645 gn=0.91 lr=4.04e-04 tps=20975 mem=5.5G lam=0.24483079059670368 Gm=None +[s_only s4] step 5080/12000 loss=4.7603 gn=0.80 lr=4.02e-04 tps=20975 mem=5.5G lam=0.2453694473952055 Gm=None +[s_only s4] step 5100/12000 loss=4.7002 gn=0.83 lr=4.01e-04 tps=20975 mem=5.5G lam=0.24584175242731968 Gm=None +[s_only s4] step 5120/12000 loss=4.7367 gn=0.81 lr=4.00e-04 tps=20976 mem=5.5G lam=0.24632523177812496 Gm=None +[s_only s4] step 5140/12000 loss=4.6617 gn=0.82 lr=3.98e-04 tps=20976 mem=5.5G lam=0.24676122547437748 Gm=None +[s_only s4] step 5160/12000 loss=4.7587 gn=0.83 lr=3.97e-04 tps=20977 mem=5.5G lam=0.24725271513064703 Gm=None +[s_only s4] step 5180/12000 loss=4.5634 gn=0.76 lr=3.95e-04 tps=20977 mem=5.5G lam=0.24776360609879097 Gm=None +[s_only s4] step 5200/12000 loss=4.5394 gn=0.79 lr=3.94e-04 tps=20977 mem=5.5G lam=0.2482512422526876 Gm=None +[s_only s4] step 5220/12000 loss=4.7138 gn=0.81 lr=3.93e-04 tps=20977 mem=5.5G lam=0.2487087231129408 Gm=None +[s_only s4] step 5240/12000 loss=4.7524 gn=0.85 lr=3.91e-04 tps=20978 mem=5.5G lam=0.24908532047023377 Gm=None +[s_only s4] step 5260/12000 loss=4.6329 gn=0.78 lr=3.90e-04 tps=20978 mem=5.5G lam=0.24966076792528233 Gm=None +[s_only s4] step 5280/12000 loss=4.6220 gn=0.79 lr=3.88e-04 tps=20979 mem=5.5G lam=0.25018677581101656 Gm=None +[s_only s4] step 5300/12000 loss=4.6398 gn=0.74 lr=3.87e-04 tps=20979 mem=5.5G lam=0.25063683837652206 Gm=None +[s_only s4] step 5320/12000 loss=4.6333 gn=0.80 lr=3.86e-04 tps=20980 mem=5.5G lam=0.25106844616432983 Gm=None +[s_only s4] step 5340/12000 loss=4.7286 gn=0.79 lr=3.84e-04 tps=20980 mem=5.5G lam=0.25150423776358366 Gm=None +[s_only s4] step 5360/12000 loss=4.6586 gn=0.78 lr=3.83e-04 tps=20981 mem=5.5G lam=0.25195325290163356 Gm=None +[s_only s4] step 5380/12000 loss=4.6156 gn=0.80 lr=3.81e-04 tps=20981 mem=5.5G lam=0.2524067560831706 Gm=None +[s_only s4] step 5400/12000 loss=4.7247 gn=0.82 lr=3.80e-04 tps=20981 mem=5.5G lam=0.2528624360760053 Gm=None +[s_only s4] step 5420/12000 loss=4.5929 gn=0.98 lr=3.79e-04 tps=20981 mem=5.5G lam=0.25325274746865034 Gm=None +[s_only s4] step 5440/12000 loss=4.7693 gn=0.83 lr=3.77e-04 tps=20981 mem=5.5G lam=0.25368431645135087 Gm=None +[s_only s4] step 5460/12000 loss=4.5821 gn=0.80 lr=3.76e-04 tps=20982 mem=5.5G lam=0.2540362663567066 Gm=None +[s_only s4] step 5480/12000 loss=4.7040 gn=0.93 lr=3.74e-04 tps=20982 mem=5.5G lam=0.2545130178332329 Gm=None +[s_only s4] step 5500/12000 loss=4.7230 gn=0.82 lr=3.73e-04 tps=20983 mem=5.5G lam=0.25493588640044135 Gm=None +[s_only s4] step 5520/12000 loss=4.8381 gn=0.86 lr=3.72e-04 tps=20983 mem=5.5G lam=0.2554251039400697 Gm=None +[s_only s4] step 5540/12000 loss=4.7391 gn=0.85 lr=3.70e-04 tps=20983 mem=5.5G lam=0.2558932552735011 Gm=None +[s_only s4] step 5560/12000 loss=4.5322 gn=0.82 lr=3.69e-04 tps=20983 mem=5.5G lam=0.2563313903907935 Gm=None +[s_only s4] step 5580/12000 loss=4.5731 gn=0.85 lr=3.67e-04 tps=20983 mem=5.5G lam=0.2568667586892843 Gm=None +[s_only s4] step 5600/12000 loss=4.6761 gn=0.87 lr=3.66e-04 tps=20984 mem=5.5G lam=0.2573286993429065 Gm=None +[s_only s4] step 5620/12000 loss=4.4332 gn=0.79 lr=3.64e-04 tps=20983 mem=5.5G lam=0.2578336897616585 Gm=None +[s_only s4] step 5640/12000 loss=4.6877 gn=0.81 lr=3.63e-04 tps=20982 mem=5.5G lam=0.2582383931924899 Gm=None +[s_only s4] step 5660/12000 loss=4.7160 gn=0.84 lr=3.62e-04 tps=20982 mem=5.5G lam=0.2586594820022583 Gm=None +[s_only s4] step 5680/12000 loss=4.5369 gn=0.89 lr=3.60e-04 tps=20983 mem=5.5G lam=0.25910444495578605 Gm=None +[s_only s4] step 5700/12000 loss=4.4757 gn=0.78 lr=3.59e-04 tps=20983 mem=5.5G lam=0.2595475198080142 Gm=None +[s_only s4] step 5720/12000 loss=4.6282 gn=0.85 lr=3.57e-04 tps=20983 mem=5.5G lam=0.26010451031227905 Gm=None +[s_only s4] step 5740/12000 loss=4.6672 gn=0.83 lr=3.56e-04 tps=20983 mem=5.5G lam=0.2605771590024233 Gm=None +[s_only s4] step 5760/12000 loss=4.5724 gn=0.85 lr=3.54e-04 tps=20983 mem=5.5G lam=0.2610245772326986 Gm=None +[s_only s4] step 5780/12000 loss=4.5469 gn=0.86 lr=3.53e-04 tps=20983 mem=5.5G lam=0.2614086118216316 Gm=None +[s_only s4] step 5800/12000 loss=4.5267 gn=0.87 lr=3.52e-04 tps=20984 mem=5.5G lam=0.26191724340120953 Gm=None +[s_only s4] step 5820/12000 loss=4.5626 gn=0.86 lr=3.50e-04 tps=20984 mem=5.5G lam=0.26234522213538486 Gm=None +[s_only s4] step 5840/12000 loss=4.7422 gn=0.90 lr=3.49e-04 tps=20984 mem=5.5G lam=0.2627431461587548 Gm=None +[s_only s4] step 5860/12000 loss=4.5738 gn=0.83 lr=3.47e-04 tps=20984 mem=5.5G lam=0.2631686208769679 Gm=None +[s_only s4] step 5880/12000 loss=4.6712 gn=0.87 lr=3.46e-04 tps=20985 mem=5.5G lam=0.263636389747262 Gm=None +[s_only s4] step 5900/12000 loss=4.4901 gn=0.87 lr=3.44e-04 tps=20986 mem=5.5G lam=0.2640724303200841 Gm=None +[s_only s4] step 5920/12000 loss=4.4547 gn=0.83 lr=3.43e-04 tps=20986 mem=5.5G lam=0.26452096179127693 Gm=None +[s_only s4] step 5940/12000 loss=4.6198 gn=0.80 lr=3.41e-04 tps=20987 mem=5.5G lam=0.26494669169187546 Gm=None +[s_only s4] step 5960/12000 loss=4.5376 gn=0.80 lr=3.40e-04 tps=20987 mem=5.5G lam=0.2653238885104656 Gm=None +[s_only s4] step 5980/12000 loss=4.6554 gn=0.87 lr=3.39e-04 tps=20987 mem=5.5G lam=0.26579016136626404 Gm=None +[s_only s4] step 6000/12000 loss=4.7039 gn=0.93 lr=3.37e-04 tps=20987 mem=5.5G lam=0.2661558712522189 Gm=None +[s_only s4] EVAL step 6000: val_loss=4.6618 val_ppl=105.83 +[s_only s4] step 6020/12000 loss=4.6903 gn=0.93 lr=3.36e-04 tps=20975 mem=5.5G lam=0.26657322111229104 Gm=None +[s_only s4] step 6040/12000 loss=4.4937 gn=0.90 lr=3.34e-04 tps=20975 mem=5.5G lam=0.26695321934918564 Gm=None +[s_only s4] step 6060/12000 loss=4.5561 gn=0.79 lr=3.33e-04 tps=20976 mem=5.5G lam=0.26728356288125116 Gm=None +[s_only s4] step 6080/12000 loss=4.7428 gn=0.82 lr=3.31e-04 tps=20976 mem=5.5G lam=0.2677405836681525 Gm=None +[s_only s4] step 6100/12000 loss=4.4193 gn=0.85 lr=3.30e-04 tps=20977 mem=5.5G lam=0.26817986462265253 Gm=None +[s_only s4] step 6120/12000 loss=4.6775 gn=0.88 lr=3.29e-04 tps=20977 mem=5.5G lam=0.2686259603748719 Gm=None +[s_only s4] step 6140/12000 loss=4.4985 gn=0.87 lr=3.27e-04 tps=20978 mem=5.5G lam=0.269013164875408 Gm=None +[s_only s4] step 6160/12000 loss=4.5886 gn=0.84 lr=3.26e-04 tps=20978 mem=5.5G lam=0.2694746156533559 Gm=None +[s_only s4] step 6180/12000 loss=4.4941 gn=0.90 lr=3.24e-04 tps=20978 mem=5.5G lam=0.26989477034658194 Gm=None +[s_only s4] step 6200/12000 loss=4.6311 gn=0.89 lr=3.23e-04 tps=20979 mem=5.5G lam=0.27032812871038914 Gm=None +[s_only s4] step 6220/12000 loss=4.6752 gn=0.89 lr=3.21e-04 tps=20979 mem=5.5G lam=0.27069155871868134 Gm=None +[s_only s4] step 6240/12000 loss=4.6093 gn=0.79 lr=3.20e-04 tps=20980 mem=5.5G lam=0.27111527354766923 Gm=None +[s_only s4] step 6260/12000 loss=4.4861 gn=0.83 lr=3.19e-04 tps=20980 mem=5.5G lam=0.2715427416066329 Gm=None +[s_only s4] step 6280/12000 loss=4.4703 gn=0.84 lr=3.17e-04 tps=20981 mem=5.5G lam=0.2719097708662351 Gm=None +[s_only s4] step 6300/12000 loss=4.5728 gn=0.91 lr=3.16e-04 tps=20981 mem=5.5G lam=0.27232065610587597 Gm=None +[s_only s4] step 6320/12000 loss=4.3543 gn=0.80 lr=3.14e-04 tps=20981 mem=5.5G lam=0.2726631825789809 Gm=None +[s_only s4] step 6340/12000 loss=4.5977 gn=0.82 lr=3.13e-04 tps=20982 mem=5.5G lam=0.2730106385424733 Gm=None +[s_only s4] step 6360/12000 loss=4.4591 gn=0.86 lr=3.11e-04 tps=20982 mem=5.5G lam=0.27337938764443 Gm=None +[s_only s4] step 6380/12000 loss=4.6240 gn=0.84 lr=3.10e-04 tps=20982 mem=5.5G lam=0.27381266094744205 Gm=None +[s_only s4] step 6400/12000 loss=4.5657 gn=0.86 lr=3.08e-04 tps=20983 mem=5.5G lam=0.2741311068336169 Gm=None +[s_only s4] step 6420/12000 loss=4.6275 gn=0.84 lr=3.07e-04 tps=20983 mem=5.5G lam=0.27450680422286194 Gm=None +[s_only s4] step 6440/12000 loss=4.6588 gn=0.99 lr=3.06e-04 tps=20984 mem=5.5G lam=0.27495268639177084 Gm=None +[s_only s4] step 6460/12000 loss=4.5344 gn=0.86 lr=3.04e-04 tps=20984 mem=5.5G lam=0.2753937964638074 Gm=None +[s_only s4] step 6480/12000 loss=4.7273 gn=0.92 lr=3.03e-04 tps=20983 mem=5.5G lam=0.2757017013306419 Gm=None +[s_only s4] step 6500/12000 loss=4.4730 gn=0.89 lr=3.01e-04 tps=20984 mem=5.5G lam=0.27597553593417007 Gm=None +[s_only s4] step 6520/12000 loss=4.5383 gn=0.84 lr=3.00e-04 tps=20984 mem=5.5G lam=0.27633371359358233 Gm=None +[s_only s4] step 6540/12000 loss=4.6156 gn=0.86 lr=2.98e-04 tps=20985 mem=5.5G lam=0.2767102091262738 Gm=None +[s_only s4] step 6560/12000 loss=4.3934 gn=0.90 lr=2.97e-04 tps=20984 mem=5.5G lam=0.27701200762142736 Gm=None +[s_only s4] step 6580/12000 loss=4.6978 gn=0.95 lr=2.96e-04 tps=20984 mem=5.5G lam=0.27726567722857 Gm=None +[s_only s4] step 6600/12000 loss=4.5119 gn=0.91 lr=2.94e-04 tps=20985 mem=5.5G lam=0.27763359900563955 Gm=None +[s_only s4] step 6620/12000 loss=4.7593 gn=0.97 lr=2.93e-04 tps=20985 mem=5.5G lam=0.2779787549128135 Gm=None +[s_only s4] step 6640/12000 loss=4.4529 gn=1.00 lr=2.91e-04 tps=20985 mem=5.5G lam=0.2783484359582265 Gm=None +[s_only s4] step 6660/12000 loss=4.4635 gn=0.85 lr=2.90e-04 tps=20985 mem=5.5G lam=0.27872083615511656 Gm=None +[s_only s4] step 6680/12000 loss=4.4571 gn=0.87 lr=2.88e-04 tps=20985 mem=5.5G lam=0.27911099636306363 Gm=None +[s_only s4] step 6700/12000 loss=4.6167 gn=0.88 lr=2.87e-04 tps=20986 mem=5.5G lam=0.2794766615455349 Gm=None +[s_only s4] step 6720/12000 loss=4.4996 gn=0.87 lr=2.86e-04 tps=20986 mem=5.5G lam=0.27988815276573104 Gm=None +[s_only s4] step 6740/12000 loss=4.4940 gn=0.87 lr=2.84e-04 tps=20986 mem=5.5G lam=0.2803427117566268 Gm=None +[s_only s4] step 6760/12000 loss=4.5921 gn=0.86 lr=2.83e-04 tps=20986 mem=5.5G lam=0.28070089345177013 Gm=None +[s_only s4] step 6780/12000 loss=4.4628 gn=0.92 lr=2.81e-04 tps=20987 mem=5.5G lam=0.28102158289402723 Gm=None +[s_only s4] step 6800/12000 loss=4.4023 gn=0.85 lr=2.80e-04 tps=20986 mem=5.5G lam=0.28138805242876214 Gm=None +[s_only s4] step 6820/12000 loss=4.6029 gn=0.89 lr=2.79e-04 tps=20986 mem=5.5G lam=0.28170794031272334 Gm=None +[s_only s4] step 6840/12000 loss=4.4295 gn=0.84 lr=2.77e-04 tps=20987 mem=5.5G lam=0.2820820923273762 Gm=None +[s_only s4] step 6860/12000 loss=4.6283 gn=0.87 lr=2.76e-04 tps=20987 mem=5.5G lam=0.2824389686187108 Gm=None +[s_only s4] step 6880/12000 loss=4.5306 gn=0.95 lr=2.74e-04 tps=20987 mem=5.5G lam=0.2827875229219596 Gm=None +[s_only s4] step 6900/12000 loss=4.4764 gn=0.91 lr=2.73e-04 tps=20987 mem=5.5G lam=0.2830973385522763 Gm=None +[s_only s4] step 6920/12000 loss=4.4897 gn=0.91 lr=2.72e-04 tps=20987 mem=5.5G lam=0.2834770083427429 Gm=None +[s_only s4] step 6940/12000 loss=4.5603 gn=0.95 lr=2.70e-04 tps=20986 mem=5.5G lam=0.2838409577185909 Gm=None +[s_only s4] step 6960/12000 loss=4.5392 gn=0.93 lr=2.69e-04 tps=20984 mem=5.5G lam=0.28418359470864135 Gm=None +[s_only s4] step 6980/12000 loss=4.4879 gn=0.83 lr=2.67e-04 tps=20984 mem=5.5G lam=0.2845180717607339 Gm=None +[s_only s4] step 7000/12000 loss=4.3341 gn=0.88 lr=2.66e-04 tps=20984 mem=5.5G lam=0.2848631500576933 Gm=None +[s_only s4] EVAL step 7000: val_loss=4.6221 val_ppl=101.71 +[s_only s4] step 7020/12000 loss=4.6896 gn=0.87 lr=2.65e-04 tps=20973 mem=5.5G lam=0.2851637282098333 Gm=None +[s_only s4] step 7040/12000 loss=4.6749 gn=0.98 lr=2.63e-04 tps=20973 mem=5.5G lam=0.2854777028163274 Gm=None +[s_only s4] step 7060/12000 loss=4.5714 gn=0.86 lr=2.62e-04 tps=20974 mem=5.5G lam=0.2857873331134518 Gm=None +[s_only s4] step 7080/12000 loss=4.5438 gn=0.87 lr=2.60e-04 tps=20974 mem=5.5G lam=0.28612879446397227 Gm=None +[s_only s4] step 7100/12000 loss=4.5632 gn=0.88 lr=2.59e-04 tps=20974 mem=5.5G lam=0.28651184526582557 Gm=None +[s_only s4] step 7120/12000 loss=4.7616 gn=1.08 lr=2.58e-04 tps=20975 mem=5.5G lam=0.28690661303699017 Gm=None +[s_only s4] step 7140/12000 loss=4.4686 gn=0.92 lr=2.56e-04 tps=20975 mem=5.5G lam=0.28723378758877516 Gm=None +[s_only s4] step 7160/12000 loss=4.4977 gn=0.86 lr=2.55e-04 tps=20976 mem=5.5G lam=0.2874448914080858 Gm=None +[s_only s4] step 7180/12000 loss=4.5604 gn=0.99 lr=2.53e-04 tps=20975 mem=5.5G lam=0.28778999640295905 Gm=None +[s_only s4] step 7200/12000 loss=4.5510 gn=0.97 lr=2.52e-04 tps=20975 mem=5.5G lam=0.28814688480148715 Gm=None +[s_only s4] step 7220/12000 loss=4.3480 gn=0.90 lr=2.51e-04 tps=20976 mem=5.5G lam=0.28844433433065814 Gm=None +[s_only s4] step 7240/12000 loss=4.4383 gn=0.81 lr=2.49e-04 tps=20975 mem=5.5G lam=0.2887680825466911 Gm=None +[s_only s4] step 7260/12000 loss=4.5518 gn=0.89 lr=2.48e-04 tps=20976 mem=5.5G lam=0.28912104417880374 Gm=None +[s_only s4] step 7280/12000 loss=4.4441 gn=0.86 lr=2.47e-04 tps=20976 mem=5.5G lam=0.28946252384533483 Gm=None +[s_only s4] step 7300/12000 loss=4.5898 gn=0.88 lr=2.45e-04 tps=20976 mem=5.5G lam=0.28982597899933654 Gm=None +[s_only s4] step 7320/12000 loss=4.4036 gn=0.86 lr=2.44e-04 tps=20976 mem=5.5G lam=0.29010155114034814 Gm=None +[s_only s4] step 7340/12000 loss=4.4911 gn=0.89 lr=2.42e-04 tps=20976 mem=5.5G lam=0.29039802805831033 Gm=None +[s_only s4] step 7360/12000 loss=4.5631 gn=0.89 lr=2.41e-04 tps=20976 mem=5.5G lam=0.2906757043674588 Gm=None +[s_only s4] step 7380/12000 loss=4.5363 gn=0.96 lr=2.40e-04 tps=20975 mem=5.5G lam=0.29100907283524674 Gm=None +[s_only s4] step 7400/12000 loss=4.4114 gn=0.90 lr=2.38e-04 tps=20975 mem=5.5G lam=0.29134827107191086 Gm=None +[s_only s4] step 7420/12000 loss=4.4769 gn=0.99 lr=2.37e-04 tps=20976 mem=5.5G lam=0.29161531447122496 Gm=None +[s_only s4] step 7440/12000 loss=4.4153 gn=0.92 lr=2.36e-04 tps=20976 mem=5.5G lam=0.2919024241467317 Gm=None +[s_only s4] step 7460/12000 loss=4.5716 gn=0.90 lr=2.34e-04 tps=20976 mem=5.5G lam=0.2922171751658122 Gm=None +[s_only s4] step 7480/12000 loss=4.4011 gn=0.94 lr=2.33e-04 tps=20977 mem=5.5G lam=0.2925238385796547 Gm=None +[s_only s4] step 7500/12000 loss=4.4644 gn=0.90 lr=2.32e-04 tps=20977 mem=5.5G lam=0.292859704233706 Gm=None +[s_only s4] step 7520/12000 loss=4.5225 gn=0.92 lr=2.30e-04 tps=20977 mem=5.5G lam=0.2931307752927144 Gm=None +[s_only s4] step 7540/12000 loss=4.3954 gn=0.97 lr=2.29e-04 tps=20977 mem=5.5G lam=0.2933978255217274 Gm=None +[s_only s4] step 7560/12000 loss=4.3196 gn=0.93 lr=2.28e-04 tps=20977 mem=5.5G lam=0.29366523834566277 Gm=None +[s_only s4] step 7580/12000 loss=4.6087 gn=1.03 lr=2.26e-04 tps=20977 mem=5.5G lam=0.29390792331347865 Gm=None +[s_only s4] step 7600/12000 loss=4.3619 gn=0.92 lr=2.25e-04 tps=20978 mem=5.5G lam=0.2941531104346116 Gm=None +[s_only s4] step 7620/12000 loss=4.4450 gn=0.91 lr=2.24e-04 tps=20978 mem=5.5G lam=0.2944639893248677 Gm=None +[s_only s4] step 7640/12000 loss=4.4056 gn=0.89 lr=2.22e-04 tps=20977 mem=5.5G lam=0.2947266198073824 Gm=None +[s_only s4] step 7660/12000 loss=4.4481 gn=0.92 lr=2.21e-04 tps=20978 mem=5.5G lam=0.2949335960050424 Gm=None +[s_only s4] step 7680/12000 loss=4.6220 gn=0.95 lr=2.20e-04 tps=20978 mem=5.5G lam=0.295140213953952 Gm=None +[s_only s4] step 7700/12000 loss=4.5893 gn=0.94 lr=2.18e-04 tps=20978 mem=5.5G lam=0.2954183258116245 Gm=None +[s_only s4] step 7720/12000 loss=4.5701 gn=0.92 lr=2.17e-04 tps=20978 mem=5.5G lam=0.29573597727964324 Gm=None +[s_only s4] step 7740/12000 loss=4.4939 gn=0.96 lr=2.16e-04 tps=20978 mem=5.5G lam=0.29600651655346155 Gm=None +[s_only s4] step 7760/12000 loss=4.4824 gn=0.83 lr=2.15e-04 tps=20979 mem=5.5G lam=0.29627326782792807 Gm=None +[s_only s4] step 7780/12000 loss=4.5755 gn=0.96 lr=2.13e-04 tps=20979 mem=5.5G lam=0.29656230192631483 Gm=None +[s_only s4] step 7800/12000 loss=4.4959 gn=0.96 lr=2.12e-04 tps=20979 mem=5.5G lam=0.29684832350661355 Gm=None +[s_only s4] step 7820/12000 loss=4.3513 gn=0.92 lr=2.11e-04 tps=20979 mem=5.5G lam=0.2970991426457961 Gm=None +[s_only s4] step 7840/12000 loss=4.3922 gn=0.94 lr=2.09e-04 tps=20979 mem=5.5G lam=0.2973560181756814 Gm=None +[s_only s4] step 7860/12000 loss=4.4005 gn=0.94 lr=2.08e-04 tps=20979 mem=5.5G lam=0.2976074026276668 Gm=None +[s_only s4] step 7880/12000 loss=4.3876 gn=0.95 lr=2.07e-04 tps=20980 mem=5.5G lam=0.29785167922576267 Gm=None +[s_only s4] step 7900/12000 loss=4.3514 gn=0.92 lr=2.06e-04 tps=20980 mem=5.5G lam=0.2981217736378312 Gm=None +[s_only s4] step 7920/12000 loss=4.5196 gn=0.99 lr=2.04e-04 tps=20980 mem=5.5G lam=0.29837618333597976 Gm=None +[s_only s4] step 7940/12000 loss=4.5095 gn=0.98 lr=2.03e-04 tps=20980 mem=5.5G lam=0.2986345471193393 Gm=None +[s_only s4] step 7960/12000 loss=4.4981 gn=0.93 lr=2.02e-04 tps=20981 mem=5.5G lam=0.2988432018707196 Gm=None +[s_only s4] step 7980/12000 loss=4.5046 gn=0.93 lr=2.00e-04 tps=20981 mem=5.5G lam=0.299059871584177 Gm=None +[s_only s4] step 8000/12000 loss=4.5499 gn=0.99 lr=1.99e-04 tps=20981 mem=5.5G lam=0.2992765322948496 Gm=None +[s_only s4] EVAL step 8000: val_loss=4.5488 val_ppl=94.52 +[s_only s4] step 8020/12000 loss=4.5180 gn=0.92 lr=1.98e-04 tps=20972 mem=5.5G lam=0.2995269459982713 Gm=None +[s_only s4] step 8040/12000 loss=4.4307 gn=1.09 lr=1.97e-04 tps=20972 mem=5.5G lam=0.29977660595128935 Gm=None +[s_only s4] step 8060/12000 loss=4.4570 gn=0.92 lr=1.95e-04 tps=20972 mem=5.5G lam=0.29997926236440736 Gm=None +[s_only s4] step 8080/12000 loss=4.5590 gn=0.93 lr=1.94e-04 tps=20973 mem=5.5G lam=0.3001612626637022 Gm=None +[s_only s4] step 8100/12000 loss=4.5297 gn=0.95 lr=1.93e-04 tps=20973 mem=5.5G lam=0.30047966726124287 Gm=None +[s_only s4] step 8120/12000 loss=4.6583 gn=0.97 lr=1.92e-04 tps=20973 mem=5.5G lam=0.3007390508428216 Gm=None +[s_only s4] step 8140/12000 loss=4.3280 gn=0.85 lr=1.90e-04 tps=20973 mem=5.5G lam=0.3009115243330598 Gm=None +[s_only s4] step 8160/12000 loss=4.3995 gn=0.94 lr=1.89e-04 tps=20973 mem=5.5G lam=0.3011290514841676 Gm=None +[s_only s4] step 8180/12000 loss=4.4927 gn=1.08 lr=1.88e-04 tps=20974 mem=5.5G lam=0.301312948577106 Gm=None +[s_only s4] step 8200/12000 loss=4.3444 gn=0.94 lr=1.87e-04 tps=20974 mem=5.5G lam=0.3015705257033308 Gm=None +[s_only s4] step 8220/12000 loss=4.4397 gn=1.02 lr=1.86e-04 tps=20974 mem=5.5G lam=0.3017765013501048 Gm=None +[s_only s4] step 8240/12000 loss=4.3741 gn=1.01 lr=1.84e-04 tps=20974 mem=5.5G lam=0.30202646398295957 Gm=None +[s_only s4] step 8260/12000 loss=4.3398 gn=1.00 lr=1.83e-04 tps=20975 mem=5.5G lam=0.30228884021441144 Gm=None +[s_only s4] step 8280/12000 loss=4.3337 gn=0.92 lr=1.82e-04 tps=20974 mem=5.5G lam=0.30250814743340015 Gm=None +[s_only s4] step 8300/12000 loss=4.5377 gn=0.92 lr=1.81e-04 tps=20975 mem=5.5G lam=0.30271473123381537 Gm=None +[s_only s4] step 8320/12000 loss=4.4013 gn=0.94 lr=1.80e-04 tps=20975 mem=5.5G lam=0.3028920032083988 Gm=None +[s_only s4] step 8340/12000 loss=4.4507 gn=0.94 lr=1.78e-04 tps=20976 mem=5.5G lam=0.30314979919542867 Gm=None +[s_only s4] step 8360/12000 loss=4.5463 gn=1.04 lr=1.77e-04 tps=20975 mem=5.5G lam=0.3034237399697304 Gm=None +[s_only s4] step 8380/12000 loss=4.3681 gn=0.96 lr=1.76e-04 tps=20976 mem=5.5G lam=0.3036789372563362 Gm=None +[s_only s4] step 8400/12000 loss=4.4672 gn=0.96 lr=1.75e-04 tps=20976 mem=5.5G lam=0.30391396085421246 Gm=None +[s_only s4] step 8420/12000 loss=4.3501 gn=1.00 lr=1.74e-04 tps=20975 mem=5.5G lam=0.3041386092081666 Gm=None +[s_only s4] step 8440/12000 loss=4.3032 gn=0.89 lr=1.72e-04 tps=20975 mem=5.5G lam=0.3043541644389431 Gm=None +[s_only s4] step 8460/12000 loss=4.3802 gn=1.02 lr=1.71e-04 tps=20975 mem=5.5G lam=0.3045568475499749 Gm=None +[s_only s4] step 8480/12000 loss=4.3761 gn=0.91 lr=1.70e-04 tps=20976 mem=5.5G lam=0.30478647413353127 Gm=None +[s_only s4] step 8500/12000 loss=4.3933 gn=1.05 lr=1.69e-04 tps=20976 mem=5.5G lam=0.3050056391706069 Gm=None +[s_only s4] step 8520/12000 loss=4.4921 gn=1.02 lr=1.68e-04 tps=20976 mem=5.5G lam=0.3052216734116276 Gm=None +[s_only s4] step 8540/12000 loss=4.5558 gn=0.97 lr=1.67e-04 tps=20976 mem=5.5G lam=0.3054409930482507 Gm=None +[s_only s4] step 8560/12000 loss=4.3557 gn=0.89 lr=1.66e-04 tps=20976 mem=5.5G lam=0.3056090821822484 Gm=None +[s_only s4] step 8580/12000 loss=4.4637 gn=0.98 lr=1.64e-04 tps=20976 mem=5.5G lam=0.3057541136319439 Gm=None +[s_only s4] step 8600/12000 loss=4.2850 gn=0.98 lr=1.63e-04 tps=20977 mem=5.5G lam=0.30596814459810656 Gm=None +[s_only s4] step 8620/12000 loss=4.4269 gn=1.06 lr=1.62e-04 tps=20977 mem=5.5G lam=0.306221437950929 Gm=None +[s_only s4] step 8640/12000 loss=4.4905 gn=0.99 lr=1.61e-04 tps=20977 mem=5.5G lam=0.3064166021843751 Gm=None +[s_only s4] step 8660/12000 loss=4.3049 gn=1.01 lr=1.60e-04 tps=20977 mem=5.5G lam=0.3065540681903561 Gm=None +[s_only s4] step 8680/12000 loss=4.4047 gn=0.90 lr=1.59e-04 tps=20977 mem=5.5G lam=0.306756757820646 Gm=None +[s_only s4] step 8700/12000 loss=4.2609 gn=0.98 lr=1.58e-04 tps=20977 mem=5.5G lam=0.3069349608073632 Gm=None +[s_only s4] step 8720/12000 loss=4.5107 gn=0.99 lr=1.57e-04 tps=20977 mem=5.5G lam=0.30718077967564267 Gm=None +[s_only s4] step 8740/12000 loss=4.4088 gn=0.96 lr=1.55e-04 tps=20977 mem=5.5G lam=0.3074283062790831 Gm=None +[s_only s4] step 8760/12000 loss=4.3359 gn=0.90 lr=1.54e-04 tps=20978 mem=5.5G lam=0.3076296312113603 Gm=None +[s_only s4] step 8780/12000 loss=4.4158 gn=0.98 lr=1.53e-04 tps=20978 mem=5.5G lam=0.30779602968444425 Gm=None +[s_only s4] step 8800/12000 loss=4.3511 gn=0.99 lr=1.52e-04 tps=20978 mem=5.5G lam=0.3079209827507536 Gm=None +[s_only s4] step 8820/12000 loss=4.2926 gn=1.05 lr=1.51e-04 tps=20978 mem=5.5G lam=0.3081057832265894 Gm=None +[s_only s4] step 8840/12000 loss=4.5021 gn=1.00 lr=1.50e-04 tps=20978 mem=5.5G lam=0.30831694540878135 Gm=None +[s_only s4] step 8860/12000 loss=4.3568 gn=0.95 lr=1.49e-04 tps=20978 mem=5.5G lam=0.30854168130705756 Gm=None +[s_only s4] step 8880/12000 loss=4.3856 gn=0.99 lr=1.48e-04 tps=20978 mem=5.5G lam=0.3087791108215849 Gm=None +[s_only s4] step 8900/12000 loss=4.1962 gn=1.00 lr=1.47e-04 tps=20978 mem=5.5G lam=0.308940379259487 Gm=None +[s_only s4] step 8920/12000 loss=4.3901 gn=0.94 lr=1.46e-04 tps=20978 mem=5.5G lam=0.30908972055961687 Gm=None +[s_only s4] step 8940/12000 loss=4.4112 gn=0.95 lr=1.45e-04 tps=20978 mem=5.5G lam=0.3092916930715243 Gm=None +[s_only s4] step 8960/12000 loss=4.3954 gn=0.97 lr=1.44e-04 tps=20979 mem=5.5G lam=0.3094771020114422 Gm=None +[s_only s4] step 8980/12000 loss=4.4451 gn=0.99 lr=1.43e-04 tps=20979 mem=5.5G lam=0.30964909090350073 Gm=None +[s_only s4] step 9000/12000 loss=4.3815 gn=1.04 lr=1.42e-04 tps=20979 mem=5.5G lam=0.3098092516884208 Gm=None +[s_only s4] EVAL step 9000: val_loss=4.4629 val_ppl=86.74 +[s_only s4] step 9020/12000 loss=4.4683 gn=0.94 lr=1.41e-04 tps=20970 mem=5.5G lam=0.30994386319071054 Gm=None +[s_only s4] step 9040/12000 loss=4.4105 gn=0.99 lr=1.40e-04 tps=20970 mem=5.5G lam=0.310146354449292 Gm=None +[s_only s4] step 9060/12000 loss=4.1324 gn=0.92 lr=1.39e-04 tps=20970 mem=5.5G lam=0.3103058636188507 Gm=None +[s_only s4] step 9080/12000 loss=4.4822 gn=0.97 lr=1.38e-04 tps=20970 mem=5.5G lam=0.31045226907978457 Gm=None +[s_only s4] step 9100/12000 loss=4.4339 gn=0.97 lr=1.37e-04 tps=20971 mem=5.5G lam=0.3105774127567808 Gm=None +[s_only s4] step 9120/12000 loss=4.2876 gn=0.99 lr=1.36e-04 tps=20971 mem=5.5G lam=0.31075249674419564 Gm=None +[s_only s4] step 9140/12000 loss=4.2255 gn=0.92 lr=1.35e-04 tps=20971 mem=5.5G lam=0.31095711266001064 Gm=None +[s_only s4] step 9160/12000 loss=4.4645 gn=0.95 lr=1.34e-04 tps=20971 mem=5.5G lam=0.3111248854547739 Gm=None +[s_only s4] step 9180/12000 loss=4.4635 gn=0.95 lr=1.33e-04 tps=20971 mem=5.5G lam=0.31127120833843946 Gm=None +[s_only s4] step 9200/12000 loss=4.2629 gn=1.00 lr=1.32e-04 tps=20972 mem=5.5G lam=0.3114823329572876 Gm=None +[s_only s4] step 9220/12000 loss=4.4588 gn=1.03 lr=1.31e-04 tps=20972 mem=5.5G lam=0.31166115899880725 Gm=None +[s_only s4] step 9240/12000 loss=4.4790 gn=1.00 lr=1.30e-04 tps=20972 mem=5.5G lam=0.3118564309552312 Gm=None +[s_only s4] step 9260/12000 loss=4.3526 gn=0.98 lr=1.29e-04 tps=20972 mem=5.5G lam=0.31203212061276037 Gm=None +[s_only s4] step 9280/12000 loss=4.3508 gn=0.99 lr=1.28e-04 tps=20972 mem=5.5G lam=0.3122306289151311 Gm=None +[s_only s4] step 9300/12000 loss=4.5044 gn=1.09 lr=1.27e-04 tps=20972 mem=5.5G lam=0.31237834009031457 Gm=None +[s_only s4] step 9320/12000 loss=4.3042 gn=0.96 lr=1.26e-04 tps=20972 mem=5.5G lam=0.31255405272046727 Gm=None +[s_only s4] step 9340/12000 loss=4.4998 gn=1.04 lr=1.25e-04 tps=20972 mem=5.5G lam=0.31276874151080847 Gm=None +[s_only s4] step 9360/12000 loss=4.3067 gn=0.98 lr=1.24e-04 tps=20973 mem=5.5G lam=0.3129515666514635 Gm=None +[s_only s4] step 9380/12000 loss=4.4518 gn=1.00 lr=1.23e-04 tps=20973 mem=5.5G lam=0.3131057446201642 Gm=None +[s_only s4] step 9400/12000 loss=4.2351 gn=1.04 lr=1.22e-04 tps=20973 mem=5.5G lam=0.3132893079891801 Gm=None +[s_only s4] step 9420/12000 loss=4.2721 gn=0.96 lr=1.21e-04 tps=20973 mem=5.5G lam=0.3134178513040145 Gm=None +[s_only s4] step 9440/12000 loss=4.3201 gn=1.11 lr=1.20e-04 tps=20973 mem=5.5G lam=0.3135824544976155 Gm=None +[s_only s4] step 9460/12000 loss=4.3899 gn=0.98 lr=1.19e-04 tps=20974 mem=5.5G lam=0.31373650828997296 Gm=None +[s_only s4] step 9480/12000 loss=4.3804 gn=0.98 lr=1.19e-04 tps=20974 mem=5.5G lam=0.31384786684066057 Gm=None +[s_only s4] step 9500/12000 loss=4.5424 gn=1.07 lr=1.18e-04 tps=20974 mem=5.5G lam=0.314007091956834 Gm=None +[s_only s4] step 9520/12000 loss=4.2361 gn=1.00 lr=1.17e-04 tps=20974 mem=5.5G lam=0.31414498823384446 Gm=None +[s_only s4] step 9540/12000 loss=4.3809 gn=0.97 lr=1.16e-04 tps=20975 mem=5.5G lam=0.3142385569711526 Gm=None +[s_only s4] step 9560/12000 loss=4.5588 gn=1.06 lr=1.15e-04 tps=20975 mem=5.5G lam=0.3143646822621425 Gm=None +[s_only s4] step 9580/12000 loss=4.2676 gn=1.06 lr=1.14e-04 tps=20975 mem=5.5G lam=0.31452817873408395 Gm=None +[s_only s4] step 9600/12000 loss=4.4099 gn=1.02 lr=1.13e-04 tps=20975 mem=5.5G lam=0.3146650704244773 Gm=None +[s_only s4] step 9620/12000 loss=4.3610 gn=1.11 lr=1.12e-04 tps=20975 mem=5.5G lam=0.3147767983997862 Gm=None +[s_only s4] step 9640/12000 loss=4.5248 gn=1.23 lr=1.12e-04 tps=20976 mem=5.5G lam=0.3149119419977069 Gm=None +[s_only s4] step 9660/12000 loss=4.3423 gn=1.02 lr=1.11e-04 tps=20976 mem=5.5G lam=0.31507083556304377 Gm=None +[s_only s4] step 9680/12000 loss=4.5202 gn=1.04 lr=1.10e-04 tps=20976 mem=5.5G lam=0.31520187947899103 Gm=None +[s_only s4] step 9700/12000 loss=4.2839 gn=0.93 lr=1.09e-04 tps=20976 mem=5.5G lam=0.3153287485862772 Gm=None +[s_only s4] step 9720/12000 loss=4.3373 gn=1.07 lr=1.08e-04 tps=20975 mem=5.5G lam=0.315458486477534 Gm=None +[s_only s4] step 9740/12000 loss=4.3802 gn=0.97 lr=1.07e-04 tps=20976 mem=5.5G lam=0.3155565358077486 Gm=None +[s_only s4] step 9760/12000 loss=4.3933 gn=0.97 lr=1.07e-04 tps=20976 mem=5.5G lam=0.3156759003177285 Gm=None +[s_only s4] step 9780/12000 loss=4.1168 gn=0.93 lr=1.06e-04 tps=20976 mem=5.5G lam=0.3158117051546772 Gm=None +[s_only s4] step 9800/12000 loss=4.2940 gn=0.98 lr=1.05e-04 tps=20976 mem=5.5G lam=0.3159366569792231 Gm=None +[s_only s4] step 9820/12000 loss=4.1737 gn=1.00 lr=1.04e-04 tps=20976 mem=5.5G lam=0.3160382645825545 Gm=None +[s_only s4] step 9840/12000 loss=4.5113 gn=0.97 lr=1.03e-04 tps=20976 mem=5.5G lam=0.3161689784998695 Gm=None +[s_only s4] step 9860/12000 loss=4.3119 gn=1.00 lr=1.03e-04 tps=20977 mem=5.5G lam=0.31626544675479334 Gm=None +[s_only s4] step 9880/12000 loss=4.3558 gn=0.99 lr=1.02e-04 tps=20977 mem=5.5G lam=0.3163648185630639 Gm=None +[s_only s4] step 9900/12000 loss=4.2683 gn=1.04 lr=1.01e-04 tps=20977 mem=5.5G lam=0.3164774135996898 Gm=None +[s_only s4] step 9920/12000 loss=4.3086 gn=1.02 lr=1.00e-04 tps=20977 mem=5.5G lam=0.3166297435139616 Gm=None +[s_only s4] step 9940/12000 loss=4.5109 gn=0.99 lr=9.96e-05 tps=20977 mem=5.5G lam=0.316782273662587 Gm=None +[s_only s4] step 9960/12000 loss=4.3977 gn=1.01 lr=9.89e-05 tps=20977 mem=5.5G lam=0.31691854167729616 Gm=None +[s_only s4] step 9980/12000 loss=4.3664 gn=0.97 lr=9.81e-05 tps=20977 mem=5.5G lam=0.3170410205299656 Gm=None +[s_only s4] step 10000/12000 loss=4.3605 gn=0.98 lr=9.74e-05 tps=20977 mem=5.5G lam=0.3171801393230756 Gm=None +[s_only s4] EVAL step 10000: val_loss=4.4201 val_ppl=83.10 +[s_only s4] step 10020/12000 loss=4.5106 gn=1.19 lr=9.67e-05 tps=20970 mem=5.5G lam=0.3172966210792462 Gm=None +[s_only s4] step 10040/12000 loss=4.3328 gn=1.03 lr=9.59e-05 tps=20969 mem=5.5G lam=0.3174222592885296 Gm=None +[s_only s4] step 10060/12000 loss=4.3125 gn=1.04 lr=9.52e-05 tps=20970 mem=5.5G lam=0.31750708787391585 Gm=None +[s_only s4] step 10080/12000 loss=4.3616 gn=1.08 lr=9.45e-05 tps=20970 mem=5.5G lam=0.31764108687639236 Gm=None +[s_only s4] step 10100/12000 loss=4.3685 gn=0.99 lr=9.38e-05 tps=20970 mem=5.5G lam=0.31775849498808384 Gm=None +[s_only s4] step 10120/12000 loss=4.3761 gn=0.97 lr=9.31e-05 tps=20970 mem=5.5G lam=0.3178792716935277 Gm=None +[s_only s4] step 10140/12000 loss=4.1688 gn=0.99 lr=9.24e-05 tps=20971 mem=5.5G lam=0.31797652163853246 Gm=None +[s_only s4] step 10160/12000 loss=4.2585 gn=1.02 lr=9.18e-05 tps=20971 mem=5.5G lam=0.3180946158245206 Gm=None +[s_only s4] step 10180/12000 loss=4.4580 gn=1.11 lr=9.11e-05 tps=20971 mem=5.5G lam=0.31821469341715175 Gm=None +[s_only s4] step 10200/12000 loss=4.3326 gn=1.02 lr=9.04e-05 tps=20971 mem=5.5G lam=0.31834058091044426 Gm=None +[s_only s4] step 10220/12000 loss=4.4141 gn=1.11 lr=8.98e-05 tps=20972 mem=5.5G lam=0.3184533677995205 Gm=None +[s_only s4] step 10240/12000 loss=4.2307 gn=1.15 lr=8.91e-05 tps=20972 mem=5.5G lam=0.31857854531457025 Gm=None +[s_only s4] step 10260/12000 loss=4.4007 gn=1.05 lr=8.85e-05 tps=20972 mem=5.5G lam=0.3186947737510006 Gm=None +[s_only s4] step 10280/12000 loss=4.4178 gn=1.01 lr=8.78e-05 tps=20972 mem=5.5G lam=0.31884903988490504 Gm=None +[s_only s4] step 10300/12000 loss=4.2854 gn=0.98 lr=8.72e-05 tps=20972 mem=5.5G lam=0.31895187124609947 Gm=None +[s_only s4] step 10320/12000 loss=4.4386 gn=1.07 lr=8.66e-05 tps=20972 mem=5.5G lam=0.31905988572786254 Gm=None +[s_only s4] step 10340/12000 loss=4.3666 gn=1.13 lr=8.59e-05 tps=20972 mem=5.5G lam=0.3191622418041031 Gm=None +[s_only s4] step 10360/12000 loss=4.4332 gn=1.09 lr=8.53e-05 tps=20972 mem=5.5G lam=0.3192675669367115 Gm=None +[s_only s4] step 10380/12000 loss=4.3988 gn=1.14 lr=8.47e-05 tps=20972 mem=5.5G lam=0.31937368400394917 Gm=None +[s_only s4] step 10400/12000 loss=4.4540 gn=1.01 lr=8.41e-05 tps=20972 mem=5.5G lam=0.31947052385658026 Gm=None +[s_only s4] step 10420/12000 loss=4.4046 gn=1.03 lr=8.35e-05 tps=20972 mem=5.5G lam=0.3195616928860545 Gm=None +[s_only s4] step 10440/12000 loss=4.2680 gn=0.97 lr=8.30e-05 tps=20972 mem=5.5G lam=0.31962453449765843 Gm=None +[s_only s4] step 10460/12000 loss=4.4147 gn=1.10 lr=8.24e-05 tps=20973 mem=5.5G lam=0.3197386482109626 Gm=None +[s_only s4] step 10480/12000 loss=4.2731 gn=1.01 lr=8.18e-05 tps=20972 mem=5.5G lam=0.31984360950688523 Gm=None +[s_only s4] step 10500/12000 loss=4.3532 gn=1.04 lr=8.12e-05 tps=20973 mem=5.5G lam=0.31994096903751296 Gm=None +[s_only s4] step 10520/12000 loss=4.4108 gn=1.12 lr=8.07e-05 tps=20973 mem=5.5G lam=0.3200582650800546 Gm=None +[s_only s4] step 10540/12000 loss=4.3378 gn=1.04 lr=8.01e-05 tps=20973 mem=5.5G lam=0.3201578613370657 Gm=None +[s_only s4] step 10560/12000 loss=4.3411 gn=1.10 lr=7.96e-05 tps=20973 mem=5.5G lam=0.32024145995577175 Gm=None +[s_only s4] step 10580/12000 loss=4.3666 gn=1.11 lr=7.91e-05 tps=20973 mem=5.5G lam=0.32031300974388915 Gm=None +[s_only s4] step 10600/12000 loss=4.2444 gn=1.04 lr=7.85e-05 tps=20973 mem=5.5G lam=0.3204116464282076 Gm=None +[s_only s4] step 10620/12000 loss=4.2697 gn=1.04 lr=7.80e-05 tps=20973 mem=5.5G lam=0.3204937456175685 Gm=None +[s_only s4] step 10640/12000 loss=4.2736 gn=1.10 lr=7.75e-05 tps=20973 mem=5.5G lam=0.32058369740843773 Gm=None +[s_only s4] step 10660/12000 loss=4.4331 gn=1.08 lr=7.70e-05 tps=20974 mem=5.5G lam=0.32066657052685815 Gm=None +[s_only s4] step 10680/12000 loss=4.3345 gn=1.06 lr=7.65e-05 tps=20974 mem=5.5G lam=0.3207789333537221 Gm=None +[s_only s4] step 10700/12000 loss=4.3056 gn=1.20 lr=7.60e-05 tps=20974 mem=5.5G lam=0.3208739658196767 Gm=None +[s_only s4] step 10720/12000 loss=4.4449 gn=1.32 lr=7.55e-05 tps=20974 mem=5.5G lam=0.32099065215637285 Gm=None +[s_only s4] step 10740/12000 loss=4.4162 gn=1.03 lr=7.50e-05 tps=20974 mem=5.5G lam=0.32108022769292194 Gm=None +[s_only s4] step 10760/12000 loss=4.2185 gn=1.13 lr=7.46e-05 tps=20974 mem=5.5G lam=0.3211616255963842 Gm=None +[s_only s4] step 10780/12000 loss=4.2577 gn=1.07 lr=7.41e-05 tps=20974 mem=5.5G lam=0.321255587041378 Gm=None +[s_only s4] step 10800/12000 loss=4.3308 gn=1.12 lr=7.37e-05 tps=20975 mem=5.5G lam=0.32135916501283646 Gm=None +[s_only s4] step 10820/12000 loss=4.2842 gn=1.12 lr=7.32e-05 tps=20975 mem=5.5G lam=0.32144642248749733 Gm=None +[s_only s4] step 10840/12000 loss=4.2890 gn=1.09 lr=7.28e-05 tps=20975 mem=5.5G lam=0.32150103369106847 Gm=None +[s_only s4] step 10860/12000 loss=4.2795 gn=1.13 lr=7.23e-05 tps=20975 mem=5.5G lam=0.3215882144868374 Gm=None +[s_only s4] step 10880/12000 loss=4.2544 gn=1.11 lr=7.19e-05 tps=20975 mem=5.5G lam=0.32168089381108683 Gm=None +[s_only s4] step 10900/12000 loss=4.1931 gn=1.03 lr=7.15e-05 tps=20975 mem=5.5G lam=0.321766749334832 Gm=None +[s_only s4] step 10920/12000 loss=4.3479 gn=1.10 lr=7.11e-05 tps=20975 mem=5.5G lam=0.3218550933524966 Gm=None +[s_only s4] step 10940/12000 loss=4.3439 gn=1.13 lr=7.07e-05 tps=20975 mem=5.5G lam=0.32195209587613743 Gm=None +[s_only s4] step 10960/12000 loss=4.3058 gn=1.11 lr=7.03e-05 tps=20975 mem=5.5G lam=0.32202807379265624 Gm=None +[s_only s4] step 10980/12000 loss=4.4781 gn=1.13 lr=6.99e-05 tps=20975 mem=5.5G lam=0.3221087222918868 Gm=None +[s_only s4] step 11000/12000 loss=4.3925 gn=1.07 lr=6.95e-05 tps=20975 mem=5.5G lam=0.3221934602285425 Gm=None +[s_only s4] EVAL step 11000: val_loss=4.3780 val_ppl=79.68 +[s_only s4] step 11020/12000 loss=4.1989 gn=1.08 lr=6.91e-05 tps=20968 mem=5.5G lam=0.3222708376124501 Gm=None +[s_only s4] step 11040/12000 loss=4.2988 gn=1.14 lr=6.88e-05 tps=20968 mem=5.5G lam=0.3223801553249359 Gm=None +[s_only s4] step 11060/12000 loss=4.4562 gn=1.10 lr=6.84e-05 tps=20968 mem=5.5G lam=0.32246169665207464 Gm=None +[s_only s4] step 11080/12000 loss=4.3116 gn=1.06 lr=6.81e-05 tps=20968 mem=5.5G lam=0.32255756855010986 Gm=None +[s_only s4] step 11100/12000 loss=4.3510 gn=1.30 lr=6.77e-05 tps=20968 mem=5.5G lam=0.32264599079887074 Gm=None +[s_only s4] step 11120/12000 loss=4.3932 gn=1.06 lr=6.74e-05 tps=20969 mem=5.5G lam=0.3227294208481908 Gm=None +[s_only s4] step 11140/12000 loss=4.3564 gn=1.09 lr=6.70e-05 tps=20969 mem=5.5G lam=0.3228020553166668 Gm=None +[s_only s4] step 11160/12000 loss=4.2500 gn=1.05 lr=6.67e-05 tps=20969 mem=5.5G lam=0.3228835665310423 Gm=None +[s_only s4] step 11180/12000 loss=4.3714 gn=1.09 lr=6.64e-05 tps=20969 mem=5.5G lam=0.32297529994199675 Gm=None +[s_only s4] step 11200/12000 loss=4.3507 gn=1.10 lr=6.61e-05 tps=20969 mem=5.5G lam=0.3230682422096531 Gm=None +[s_only s4] step 11220/12000 loss=4.2186 gn=1.08 lr=6.58e-05 tps=20969 mem=5.5G lam=0.32313474143544835 Gm=None +[s_only s4] step 11240/12000 loss=4.4255 gn=1.06 lr=6.55e-05 tps=20969 mem=5.5G lam=0.32321928534656763 Gm=None +[s_only s4] step 11260/12000 loss=4.4095 gn=1.09 lr=6.52e-05 tps=20970 mem=5.5G lam=0.3232995814954241 Gm=None +[s_only s4] step 11280/12000 loss=4.4153 gn=1.23 lr=6.49e-05 tps=20970 mem=5.5G lam=0.32340175782640773 Gm=None +[s_only s4] step 11300/12000 loss=4.2780 gn=1.31 lr=6.47e-05 tps=20970 mem=5.5G lam=0.3234774610027671 Gm=None +[s_only s4] step 11320/12000 loss=4.3525 gn=0.98 lr=6.44e-05 tps=20970 mem=5.5G lam=0.32354277651757 Gm=None +[s_only s4] step 11340/12000 loss=4.5899 gn=1.12 lr=6.42e-05 tps=20970 mem=5.5G lam=0.3236060934141278 Gm=None +[s_only s4] step 11360/12000 loss=4.2767 gn=1.10 lr=6.39e-05 tps=20971 mem=5.5G lam=0.32369110733270645 Gm=None +[s_only s4] step 11380/12000 loss=4.3485 gn=1.22 lr=6.37e-05 tps=20971 mem=5.5G lam=0.3237645809228222 Gm=None +[s_only s4] step 11400/12000 loss=4.3327 gn=1.06 lr=6.34e-05 tps=20971 mem=5.5G lam=0.32383892219513655 Gm=None +[s_only s4] step 11420/12000 loss=4.2769 gn=1.11 lr=6.32e-05 tps=20972 mem=5.5G lam=0.3239480024203658 Gm=None +[s_only s4] step 11440/12000 loss=4.3558 gn=1.13 lr=6.30e-05 tps=20972 mem=5.5G lam=0.3240473549813032 Gm=None +[s_only s4] step 11460/12000 loss=4.2737 gn=1.14 lr=6.28e-05 tps=20972 mem=5.5G lam=0.3241238674769799 Gm=None +[s_only s4] step 11480/12000 loss=4.5079 gn=1.08 lr=6.26e-05 tps=20972 mem=5.5G lam=0.3241839933519562 Gm=None +[s_only s4] step 11500/12000 loss=4.2647 gn=1.14 lr=6.24e-05 tps=20972 mem=5.5G lam=0.3242741447562973 Gm=None +[s_only s4] step 11520/12000 loss=4.4404 gn=1.14 lr=6.22e-05 tps=20972 mem=5.5G lam=0.3243582422534625 Gm=None +[s_only s4] step 11540/12000 loss=4.3771 gn=1.11 lr=6.20e-05 tps=20972 mem=5.5G lam=0.3244298954183857 Gm=None +[s_only s4] step 11560/12000 loss=4.3502 gn=1.08 lr=6.19e-05 tps=20970 mem=5.5G lam=0.3244766149049004 Gm=None +[s_only s4] step 11580/12000 loss=4.4545 gn=1.01 lr=6.17e-05 tps=20969 mem=5.5G lam=0.3245675954967737 Gm=None +[s_only s4] step 11600/12000 loss=4.3105 gn=1.13 lr=6.15e-05 tps=20970 mem=5.5G lam=0.3246582867577672 Gm=None +[s_only s4] step 11620/12000 loss=4.2247 gn=1.21 lr=6.14e-05 tps=20970 mem=5.5G lam=0.32472286621729535 Gm=None +[s_only s4] step 11640/12000 loss=4.3182 gn=1.08 lr=6.12e-05 tps=20970 mem=5.5G lam=0.3248024582862854 Gm=None +[s_only s4] step 11660/12000 loss=4.2223 gn=1.08 lr=6.11e-05 tps=20970 mem=5.5G lam=0.3248705168565114 Gm=None +[s_only s4] step 11680/12000 loss=4.3660 gn=1.06 lr=6.10e-05 tps=20970 mem=5.5G lam=0.3249536106983821 Gm=None +[s_only s4] step 11700/12000 loss=4.2660 gn=1.09 lr=6.09e-05 tps=20971 mem=5.5G lam=0.32502557275195915 Gm=None +[s_only s4] step 11720/12000 loss=4.3098 gn=1.12 lr=6.07e-05 tps=20971 mem=5.5G lam=0.3251111240436633 Gm=None +[s_only s4] step 11740/12000 loss=4.2554 gn=1.11 lr=6.06e-05 tps=20971 mem=5.5G lam=0.32518199644982815 Gm=None +[s_only s4] step 11760/12000 loss=4.3454 gn=1.18 lr=6.06e-05 tps=20971 mem=5.5G lam=0.3252531063432495 Gm=None +[s_only s4] step 11780/12000 loss=4.5782 gn=1.22 lr=6.05e-05 tps=20971 mem=5.5G lam=0.32534991142650443 Gm=None +[s_only s4] step 11800/12000 loss=4.4450 gn=1.08 lr=6.04e-05 tps=20971 mem=5.5G lam=0.32541584751258296 Gm=None +[s_only s4] step 11820/12000 loss=4.4534 gn=1.11 lr=6.03e-05 tps=20971 mem=5.5G lam=0.3254708554595709 Gm=None +[s_only s4] step 11840/12000 loss=4.4332 gn=1.03 lr=6.02e-05 tps=20971 mem=5.5G lam=0.3255678865437706 Gm=None +[s_only s4] step 11860/12000 loss=4.2586 gn=1.15 lr=6.02e-05 tps=20971 mem=5.5G lam=0.3256492270156741 Gm=None +[s_only s4] step 11880/12000 loss=4.3139 gn=1.16 lr=6.01e-05 tps=20971 mem=5.5G lam=0.3257286244382461 Gm=None +[s_only s4] step 11900/12000 loss=4.3083 gn=1.19 lr=6.01e-05 tps=20971 mem=5.5G lam=0.325783577747643 Gm=None +[s_only s4] step 11920/12000 loss=4.3027 gn=1.13 lr=6.01e-05 tps=20971 mem=5.5G lam=0.3258594358339906 Gm=None +[s_only s4] step 11940/12000 loss=4.4700 gn=1.10 lr=6.00e-05 tps=20972 mem=5.5G lam=0.3259110553190112 Gm=None +[s_only s4] step 11960/12000 loss=4.3791 gn=1.09 lr=6.00e-05 tps=20972 mem=5.5G lam=0.3259719687824448 Gm=None +[s_only s4] step 11980/12000 loss=4.2823 gn=1.06 lr=6.00e-05 tps=20972 mem=5.5G lam=0.3260646794612209 Gm=None +[s_only s4] EVAL step 11999: val_loss=4.3526 val_ppl=77.68 +[init] variant=g_only seed=4 params=63.39M +[init] use_S=False use_G=True +[g_only s4] step 0/12000 loss=10.8847 gn=10.45 lr=3.00e-06 tps=9140 mem=5.0G lam=None Gm=0.5079752604166666 +[g_only s4] step 20/12000 loss=9.8178 gn=1.85 lr=6.30e-05 tps=21378 mem=5.5G lam=None Gm=0.4644368489583333 +[g_only s4] step 40/12000 loss=9.0641 gn=1.76 lr=1.23e-04 tps=22109 mem=5.5G lam=None Gm=0.451904296875 +[g_only s4] step 60/12000 loss=8.1962 gn=1.48 lr=1.83e-04 tps=22372 mem=5.5G lam=None Gm=0.4474283854166667 +[g_only s4] step 80/12000 loss=7.5392 gn=0.75 lr=2.43e-04 tps=22511 mem=5.5G lam=None Gm=0.453125 +[g_only s4] step 100/12000 loss=6.9924 gn=0.87 lr=3.03e-04 tps=22556 mem=5.5G lam=None Gm=0.4402669270833333 +[g_only s4] step 120/12000 loss=7.1615 gn=0.57 lr=3.63e-04 tps=22610 mem=5.5G lam=None Gm=0.4327799479166667 +[g_only s4] step 140/12000 loss=6.8090 gn=0.54 lr=4.23e-04 tps=22657 mem=5.5G lam=None Gm=0.41162109375 +[g_only s4] step 160/12000 loss=6.7257 gn=0.57 lr=4.83e-04 tps=22678 mem=5.5G lam=None Gm=0.415283203125 +[g_only s4] step 180/12000 loss=6.7495 gn=0.55 lr=5.43e-04 tps=22703 mem=5.5G lam=None Gm=0.3943684895833333 +[g_only s4] step 200/12000 loss=6.7032 gn=0.62 lr=6.00e-04 tps=22726 mem=5.5G lam=None Gm=0.4014485677083333 +[g_only s4] step 220/12000 loss=6.7377 gn=0.57 lr=6.00e-04 tps=22712 mem=5.5G lam=None Gm=0.4039713541666667 +[g_only s4] step 240/12000 loss=6.5283 gn=0.67 lr=6.00e-04 tps=22721 mem=5.5G lam=None Gm=0.389404296875 +[g_only s4] step 260/12000 loss=6.6092 gn=0.47 lr=6.00e-04 tps=22729 mem=5.5G lam=None Gm=0.406494140625 +[g_only s4] step 280/12000 loss=6.6707 gn=0.51 lr=6.00e-04 tps=22735 mem=5.5G lam=None Gm=0.4229329427083333 +[g_only s4] step 300/12000 loss=6.3502 gn=0.48 lr=6.00e-04 tps=22743 mem=5.5G lam=None Gm=0.397705078125 +[g_only s4] step 320/12000 loss=6.3975 gn=0.54 lr=6.00e-04 tps=22745 mem=5.5G lam=None Gm=0.3961588541666667 +[g_only s4] step 340/12000 loss=6.4322 gn=0.45 lr=6.00e-04 tps=22754 mem=5.5G lam=None Gm=0.4229329427083333 +[g_only s4] step 360/12000 loss=6.4653 gn=0.45 lr=6.00e-04 tps=22763 mem=5.5G lam=None Gm=0.4234212239583333 +[g_only s4] step 380/12000 loss=6.4040 gn=0.50 lr=6.00e-04 tps=22766 mem=5.5G lam=None Gm=0.423095703125 +[g_only s4] step 400/12000 loss=6.2973 gn=0.50 lr=6.00e-04 tps=22771 mem=5.5G lam=None Gm=0.4342447916666667 +[g_only s4] step 420/12000 loss=6.1717 gn=0.48 lr=6.00e-04 tps=22781 mem=5.5G lam=None Gm=0.4220377604166667 +[g_only s4] step 440/12000 loss=6.2699 gn=0.49 lr=5.99e-04 tps=22782 mem=5.5G lam=None Gm=0.4390462239583333 +[g_only s4] step 460/12000 loss=6.2982 gn=0.47 lr=5.99e-04 tps=22788 mem=5.5G lam=None Gm=0.4405924479166667 +[g_only s4] step 480/12000 loss=6.2803 gn=0.48 lr=5.99e-04 tps=22795 mem=5.5G lam=None Gm=0.4650065104166667 +[g_only s4] step 500/12000 loss=6.2172 gn=0.48 lr=5.99e-04 tps=22794 mem=5.5G lam=None Gm=0.482666015625 +[g_only s4] step 520/12000 loss=6.2500 gn=0.49 lr=5.99e-04 tps=22799 mem=5.5G lam=None Gm=0.484619140625 +[g_only s4] step 540/12000 loss=6.0741 gn=0.52 lr=5.99e-04 tps=22803 mem=5.5G lam=None Gm=0.5409342447916666 +[g_only s4] step 560/12000 loss=6.1982 gn=0.51 lr=5.99e-04 tps=22804 mem=5.5G lam=None Gm=0.5267740885416666 +[g_only s4] step 580/12000 loss=6.0648 gn=0.51 lr=5.99e-04 tps=22809 mem=5.5G lam=None Gm=0.5101725260416666 +[g_only s4] step 600/12000 loss=6.0198 gn=0.48 lr=5.98e-04 tps=22812 mem=5.5G lam=None Gm=0.5432942708333334 +[g_only s4] step 620/12000 loss=6.1228 gn=0.47 lr=5.98e-04 tps=22814 mem=5.5G lam=None Gm=0.5410970052083334 +[g_only s4] step 640/12000 loss=6.2256 gn=0.49 lr=5.98e-04 tps=22818 mem=5.5G lam=None Gm=0.5555013020833334 +[g_only s4] step 660/12000 loss=6.0436 gn=0.49 lr=5.98e-04 tps=22823 mem=5.5G lam=None Gm=0.5693359375 +[g_only s4] step 680/12000 loss=6.0521 gn=0.50 lr=5.98e-04 tps=22827 mem=5.5G lam=None Gm=0.549560546875 +[g_only s4] step 700/12000 loss=5.9549 gn=0.46 lr=5.98e-04 tps=22832 mem=5.5G lam=None Gm=0.571533203125 +[g_only s4] step 720/12000 loss=5.9225 gn=0.53 lr=5.97e-04 tps=22835 mem=5.5G lam=None Gm=0.5694173177083334 +[g_only s4] step 740/12000 loss=6.0266 gn=0.50 lr=5.97e-04 tps=22836 mem=5.5G lam=None Gm=0.554443359375 +[g_only s4] step 760/12000 loss=5.9821 gn=0.47 lr=5.97e-04 tps=22836 mem=5.5G lam=None Gm=0.56298828125 +[g_only s4] step 780/12000 loss=6.1102 gn=0.51 lr=5.97e-04 tps=22838 mem=5.5G lam=None Gm=0.573486328125 +[g_only s4] step 800/12000 loss=5.8270 gn=0.45 lr=5.97e-04 tps=22840 mem=5.5G lam=None Gm=0.5668131510416666 +[g_only s4] step 820/12000 loss=5.8979 gn=0.51 lr=5.96e-04 tps=22841 mem=5.5G lam=None Gm=0.5858561197916666 +[g_only s4] step 840/12000 loss=5.8383 gn=0.50 lr=5.96e-04 tps=22843 mem=5.5G lam=None Gm=0.572265625 +[g_only s4] step 860/12000 loss=5.9165 gn=0.53 lr=5.96e-04 tps=22844 mem=5.5G lam=None Gm=0.581787109375 +[g_only s4] step 880/12000 loss=5.7520 gn=0.47 lr=5.96e-04 tps=22846 mem=5.5G lam=None Gm=0.5745442708333334 +[g_only s4] step 900/12000 loss=5.7682 gn=0.55 lr=5.95e-04 tps=22845 mem=5.5G lam=None Gm=0.6003824869791666 +[g_only s4] step 920/12000 loss=5.8284 gn=0.53 lr=5.95e-04 tps=22845 mem=5.5G lam=None Gm=0.5851643880208334 +[g_only s4] step 940/12000 loss=5.7421 gn=0.53 lr=5.95e-04 tps=22845 mem=5.5G lam=None Gm=0.5876871744791666 +[g_only s4] step 960/12000 loss=5.7925 gn=0.57 lr=5.94e-04 tps=22845 mem=5.5G lam=None Gm=0.5704752604166666 +[g_only s4] step 980/12000 loss=5.7961 gn=0.53 lr=5.94e-04 tps=22846 mem=5.5G lam=None Gm=0.5562744140625 +[g_only s4] step 1000/12000 loss=5.6990 gn=0.55 lr=5.94e-04 tps=22847 mem=5.5G lam=None Gm=0.5625 +[g_only s4] EVAL step 1000: val_loss=5.7604 val_ppl=317.49 +[g_only s4] step 1020/12000 loss=5.7432 gn=0.48 lr=5.94e-04 tps=22768 mem=5.5G lam=None Gm=0.5682373046875 +[g_only s4] step 1040/12000 loss=5.7107 gn=0.51 lr=5.93e-04 tps=22770 mem=5.5G lam=None Gm=0.5523274739583334 +[g_only s4] step 1060/12000 loss=5.6022 gn=0.51 lr=5.93e-04 tps=22772 mem=5.5G lam=None Gm=0.5525309244791666 +[g_only s4] step 1080/12000 loss=5.6472 gn=0.53 lr=5.93e-04 tps=22771 mem=5.5G lam=None Gm=0.550537109375 +[g_only s4] step 1100/12000 loss=5.8032 gn=0.52 lr=5.92e-04 tps=22773 mem=5.5G lam=None Gm=0.564697265625 +[g_only s4] step 1120/12000 loss=5.7690 gn=0.50 lr=5.92e-04 tps=22776 mem=5.5G lam=None Gm=0.5551350911458334 +[g_only s4] step 1140/12000 loss=5.5659 gn=0.51 lr=5.92e-04 tps=22776 mem=5.5G lam=None Gm=0.5474039713541666 +[g_only s4] step 1160/12000 loss=5.6105 gn=0.48 lr=5.91e-04 tps=22778 mem=5.5G lam=None Gm=0.54052734375 +[g_only s4] step 1180/12000 loss=5.6978 gn=0.55 lr=5.91e-04 tps=22781 mem=5.5G lam=None Gm=0.5296223958333334 +[g_only s4] step 1200/12000 loss=5.6806 gn=0.53 lr=5.90e-04 tps=22783 mem=5.5G lam=None Gm=0.5393880208333334 +[g_only s4] step 1220/12000 loss=5.8899 gn=0.53 lr=5.90e-04 tps=22786 mem=5.5G lam=None Gm=0.5517578125 +[g_only s4] step 1240/12000 loss=5.5403 gn=0.54 lr=5.90e-04 tps=22788 mem=5.5G lam=None Gm=0.5680745442708334 +[g_only s4] step 1260/12000 loss=5.5545 gn=0.52 lr=5.89e-04 tps=22790 mem=5.5G lam=None Gm=0.5355224609375 +[g_only s4] step 1280/12000 loss=5.4348 gn=0.48 lr=5.89e-04 tps=22793 mem=5.5G lam=None Gm=0.5404052734375 +[g_only s4] step 1300/12000 loss=5.7645 gn=0.55 lr=5.89e-04 tps=22796 mem=5.5G lam=None Gm=0.5497233072916666 +[g_only s4] step 1320/12000 loss=5.5060 gn=0.55 lr=5.88e-04 tps=22798 mem=5.5G lam=None Gm=0.5470377604166666 +[g_only s4] step 1340/12000 loss=5.4875 gn=0.55 lr=5.88e-04 tps=22801 mem=5.5G lam=None Gm=0.5228678385416666 +[g_only s4] step 1360/12000 loss=5.6086 gn=0.53 lr=5.87e-04 tps=22803 mem=5.5G lam=None Gm=0.5298258463541666 +[g_only s4] step 1380/12000 loss=5.5489 gn=0.54 lr=5.87e-04 tps=22805 mem=5.5G lam=None Gm=0.5417073567708334 +[g_only s4] step 1400/12000 loss=5.4192 gn=0.54 lr=5.86e-04 tps=22805 mem=5.5G lam=None Gm=0.5318603515625 +[g_only s4] step 1420/12000 loss=5.5426 gn=0.52 lr=5.86e-04 tps=22805 mem=5.5G lam=None Gm=0.5221761067708334 +[g_only s4] step 1440/12000 loss=5.4812 gn=0.60 lr=5.85e-04 tps=22807 mem=5.5G lam=None Gm=0.5205078125 +[g_only s4] step 1460/12000 loss=5.4091 gn=0.52 lr=5.85e-04 tps=22807 mem=5.5G lam=None Gm=0.5219523111979166 +[g_only s4] step 1480/12000 loss=5.4453 gn=0.55 lr=5.84e-04 tps=22808 mem=5.5G lam=None Gm=0.5081380208333334 +[g_only s4] step 1500/12000 loss=5.4893 gn=0.54 lr=5.84e-04 tps=22810 mem=5.5G lam=None Gm=0.52447509765625 +[g_only s4] step 1520/12000 loss=5.3128 gn=0.53 lr=5.83e-04 tps=22811 mem=5.5G lam=None Gm=0.5252482096354166 +[g_only s4] step 1540/12000 loss=5.4161 gn=0.59 lr=5.83e-04 tps=22812 mem=5.5G lam=None Gm=0.51239013671875 +[g_only s4] step 1560/12000 loss=5.4938 gn=0.55 lr=5.82e-04 tps=22813 mem=5.5G lam=None Gm=0.5161946614583334 +[g_only s4] step 1580/12000 loss=5.5574 gn=0.54 lr=5.82e-04 tps=22814 mem=5.5G lam=None Gm=0.5107828776041666 +[g_only s4] step 1600/12000 loss=5.2589 gn=0.57 lr=5.81e-04 tps=22815 mem=5.5G lam=None Gm=0.5063680013020834 +[g_only s4] step 1620/12000 loss=5.3871 gn=0.56 lr=5.81e-04 tps=22816 mem=5.5G lam=None Gm=0.5117390950520834 +[g_only s4] step 1640/12000 loss=5.4105 gn=0.57 lr=5.80e-04 tps=22817 mem=5.5G lam=None Gm=0.50390625 +[g_only s4] step 1660/12000 loss=5.3655 gn=0.60 lr=5.80e-04 tps=22817 mem=5.5G lam=None Gm=0.4964396158854167 +[g_only s4] step 1680/12000 loss=5.2330 gn=0.53 lr=5.79e-04 tps=22818 mem=5.5G lam=None Gm=0.4943644205729167 +[g_only s4] step 1700/12000 loss=5.2985 gn=0.56 lr=5.79e-04 tps=22819 mem=5.5G lam=None Gm=0.486572265625 +[g_only s4] step 1720/12000 loss=5.3902 gn=0.55 lr=5.78e-04 tps=22819 mem=5.5G lam=None Gm=0.50238037109375 +[g_only s4] step 1740/12000 loss=5.4661 gn=0.58 lr=5.78e-04 tps=22821 mem=5.5G lam=None Gm=0.5093180338541666 +[g_only s4] step 1760/12000 loss=5.2957 gn=0.55 lr=5.77e-04 tps=22821 mem=5.5G lam=None Gm=0.49871826171875 +[g_only s4] step 1780/12000 loss=5.4531 gn=0.55 lr=5.76e-04 tps=22821 mem=5.5G lam=None Gm=0.4818929036458333 +[g_only s4] step 1800/12000 loss=5.2045 gn=0.56 lr=5.76e-04 tps=22821 mem=5.5G lam=None Gm=0.4891764322916667 +[g_only s4] step 1820/12000 loss=5.2908 gn=0.53 lr=5.75e-04 tps=22822 mem=5.5G lam=None Gm=0.49139404296875 +[g_only s4] step 1840/12000 loss=5.3348 gn=0.60 lr=5.75e-04 tps=22822 mem=5.5G lam=None Gm=0.5031941731770834 +[g_only s4] step 1860/12000 loss=5.5271 gn=0.54 lr=5.74e-04 tps=22823 mem=5.5G lam=None Gm=0.49322509765625 +[g_only s4] step 1880/12000 loss=5.4053 gn=0.56 lr=5.73e-04 tps=22823 mem=5.5G lam=None Gm=0.5029500325520834 +[g_only s4] step 1900/12000 loss=5.3072 gn=0.66 lr=5.73e-04 tps=22824 mem=5.5G lam=None Gm=0.4894205729166667 +[g_only s4] step 1920/12000 loss=5.3275 gn=0.57 lr=5.72e-04 tps=22825 mem=5.5G lam=None Gm=0.4819742838541667 +[g_only s4] step 1940/12000 loss=5.3267 gn=0.58 lr=5.72e-04 tps=22826 mem=5.5G lam=None Gm=0.4756876627604167 +[g_only s4] step 1960/12000 loss=5.4311 gn=0.56 lr=5.71e-04 tps=22827 mem=5.5G lam=None Gm=0.48016357421875 +[g_only s4] step 1980/12000 loss=5.2089 gn=0.55 lr=5.70e-04 tps=22828 mem=5.5G lam=None Gm=0.4812418619791667 +[g_only s4] step 2000/12000 loss=5.2685 gn=0.67 lr=5.70e-04 tps=22829 mem=5.5G lam=None Gm=0.4866129557291667 +[g_only s4] EVAL step 2000: val_loss=5.3137 val_ppl=203.11 +[g_only s4] step 2020/12000 loss=5.2050 gn=0.54 lr=5.69e-04 tps=22791 mem=5.5G lam=None Gm=0.47265625 +[g_only s4] step 2040/12000 loss=5.4065 gn=0.72 lr=5.68e-04 tps=22792 mem=5.5G lam=None Gm=0.46331787109375 +[g_only s4] step 2060/12000 loss=5.3093 gn=0.62 lr=5.68e-04 tps=22793 mem=5.5G lam=None Gm=0.47100830078125 +[g_only s4] step 2080/12000 loss=5.2141 gn=0.57 lr=5.67e-04 tps=22795 mem=5.5G lam=None Gm=0.4754638671875 +[g_only s4] step 2100/12000 loss=5.1380 gn=0.65 lr=5.66e-04 tps=22795 mem=5.5G lam=None Gm=0.4699910481770833 +[g_only s4] step 2120/12000 loss=5.2374 gn=0.59 lr=5.65e-04 tps=22796 mem=5.5G lam=None Gm=0.4601949055989583 +[g_only s4] step 2140/12000 loss=5.2069 gn=0.57 lr=5.65e-04 tps=22797 mem=5.5G lam=None Gm=0.471527099609375 +[g_only s4] step 2160/12000 loss=5.1664 gn=0.56 lr=5.64e-04 tps=22798 mem=5.5G lam=None Gm=0.4743855794270833 +[g_only s4] step 2180/12000 loss=5.3064 gn=0.56 lr=5.63e-04 tps=22799 mem=5.5G lam=None Gm=0.4678955078125 +[g_only s4] step 2200/12000 loss=5.2092 gn=0.61 lr=5.63e-04 tps=22800 mem=5.5G lam=None Gm=0.475799560546875 +[g_only s4] step 2220/12000 loss=5.1468 gn=0.61 lr=5.62e-04 tps=22802 mem=5.5G lam=None Gm=0.472320556640625 +[g_only s4] step 2240/12000 loss=5.0785 gn=0.59 lr=5.61e-04 tps=22802 mem=5.5G lam=None Gm=0.468048095703125 +[g_only s4] step 2260/12000 loss=5.2566 gn=0.60 lr=5.60e-04 tps=22803 mem=5.5G lam=None Gm=0.4583638509114583 +[g_only s4] step 2280/12000 loss=5.1424 gn=0.59 lr=5.60e-04 tps=22804 mem=5.5G lam=None Gm=0.4744364420572917 +[g_only s4] step 2300/12000 loss=5.1328 gn=0.58 lr=5.59e-04 tps=22804 mem=5.5G lam=None Gm=0.4678955078125 +[g_only s4] step 2320/12000 loss=5.0136 gn=0.61 lr=5.58e-04 tps=22805 mem=5.5G lam=None Gm=0.4549153645833333 +[g_only s4] step 2340/12000 loss=5.1731 gn=0.59 lr=5.57e-04 tps=22806 mem=5.5G lam=None Gm=0.468963623046875 +[g_only s4] step 2360/12000 loss=4.9227 gn=0.64 lr=5.57e-04 tps=22807 mem=5.5G lam=None Gm=0.4698282877604167 +[g_only s4] step 2380/12000 loss=5.1071 gn=0.63 lr=5.56e-04 tps=22808 mem=5.5G lam=None Gm=0.465484619140625 +[g_only s4] step 2400/12000 loss=5.0855 gn=0.68 lr=5.55e-04 tps=22808 mem=5.5G lam=None Gm=0.4626566569010417 +[g_only s4] step 2420/12000 loss=5.0505 gn=0.60 lr=5.54e-04 tps=22807 mem=5.5G lam=None Gm=0.46380615234375 +[g_only s4] step 2440/12000 loss=5.0961 gn=0.64 lr=5.53e-04 tps=22807 mem=5.5G lam=None Gm=0.4608256022135417 +[g_only s4] step 2460/12000 loss=5.0422 gn=0.59 lr=5.53e-04 tps=22807 mem=5.5G lam=None Gm=0.4639078776041667 +[g_only s4] step 2480/12000 loss=5.1951 gn=0.68 lr=5.52e-04 tps=22807 mem=5.5G lam=None Gm=0.462371826171875 +[g_only s4] step 2500/12000 loss=5.1123 gn=0.61 lr=5.51e-04 tps=22807 mem=5.5G lam=None Gm=0.4543253580729167 +[g_only s4] step 2520/12000 loss=5.0097 gn=0.67 lr=5.50e-04 tps=22805 mem=5.5G lam=None Gm=0.447296142578125 +[g_only s4] step 2540/12000 loss=5.0359 gn=0.60 lr=5.49e-04 tps=22804 mem=5.5G lam=None Gm=0.4569905598958333 +[g_only s4] step 2560/12000 loss=5.2108 gn=0.64 lr=5.48e-04 tps=22801 mem=5.5G lam=None Gm=0.4485371907552083 +[g_only s4] step 2580/12000 loss=5.0392 gn=0.64 lr=5.48e-04 tps=22801 mem=5.5G lam=None Gm=0.4401346842447917 +[g_only s4] step 2600/12000 loss=5.0223 gn=0.62 lr=5.47e-04 tps=22800 mem=5.5G lam=None Gm=0.45465087890625 +[g_only s4] step 2620/12000 loss=5.0773 gn=0.59 lr=5.46e-04 tps=22800 mem=5.5G lam=None Gm=0.4455668131510417 +[g_only s4] step 2640/12000 loss=5.0668 gn=0.63 lr=5.45e-04 tps=22800 mem=5.5G lam=None Gm=0.44287109375 +[g_only s4] step 2660/12000 loss=5.1947 gn=0.61 lr=5.44e-04 tps=22800 mem=5.5G lam=None Gm=0.4523213704427083 +[g_only s4] step 2680/12000 loss=4.9062 gn=0.65 lr=5.43e-04 tps=22801 mem=5.5G lam=None Gm=0.4415079752604167 +[g_only s4] step 2700/12000 loss=5.1460 gn=0.60 lr=5.42e-04 tps=22801 mem=5.5G lam=None Gm=0.441131591796875 +[g_only s4] step 2720/12000 loss=5.1221 gn=0.64 lr=5.41e-04 tps=22801 mem=5.5G lam=None Gm=0.4600016276041667 +[g_only s4] step 2740/12000 loss=5.0919 gn=0.64 lr=5.41e-04 tps=22801 mem=5.5G lam=None Gm=0.4520772298177083 +[g_only s4] step 2760/12000 loss=5.0311 gn=0.65 lr=5.40e-04 tps=22800 mem=5.5G lam=None Gm=0.4421590169270833 +[g_only s4] step 2780/12000 loss=5.0814 gn=0.68 lr=5.39e-04 tps=22798 mem=5.5G lam=None Gm=0.4552408854166667 +[g_only s4] step 2800/12000 loss=5.0062 gn=0.62 lr=5.38e-04 tps=22798 mem=5.5G lam=None Gm=0.4362284342447917 +[g_only s4] step 2820/12000 loss=4.9302 gn=0.66 lr=5.37e-04 tps=22798 mem=5.5G lam=None Gm=0.441253662109375 +[g_only s4] step 2840/12000 loss=4.9154 gn=0.64 lr=5.36e-04 tps=22798 mem=5.5G lam=None Gm=0.4335530598958333 +[g_only s4] step 2860/12000 loss=5.0411 gn=0.63 lr=5.35e-04 tps=22798 mem=5.5G lam=None Gm=0.4391072591145833 +[g_only s4] step 2880/12000 loss=4.9976 gn=0.65 lr=5.34e-04 tps=22797 mem=5.5G lam=None Gm=0.4346822102864583 +[g_only s4] step 2900/12000 loss=5.0113 gn=0.66 lr=5.33e-04 tps=22798 mem=5.5G lam=None Gm=0.437591552734375 +[g_only s4] step 2920/12000 loss=5.0607 gn=0.62 lr=5.32e-04 tps=22797 mem=5.5G lam=None Gm=0.4399515787760417 +[g_only s4] step 2940/12000 loss=4.9799 gn=0.63 lr=5.31e-04 tps=22798 mem=5.5G lam=None Gm=0.4353129069010417 +[g_only s4] step 2960/12000 loss=4.9686 gn=0.65 lr=5.30e-04 tps=22797 mem=5.5G lam=None Gm=0.4320576985677083 +[g_only s4] step 2980/12000 loss=4.9789 gn=0.68 lr=5.29e-04 tps=22797 mem=5.5G lam=None Gm=0.4451497395833333 +[g_only s4] step 3000/12000 loss=4.9619 gn=0.67 lr=5.28e-04 tps=22797 mem=5.5G lam=None Gm=0.4250284830729167 +[g_only s4] EVAL step 3000: val_loss=4.9751 val_ppl=144.77 +[g_only s4] step 3020/12000 loss=4.8827 gn=0.66 lr=5.27e-04 tps=22771 mem=5.5G lam=None Gm=0.4378255208333333 +[g_only s4] step 3040/12000 loss=4.8438 gn=0.67 lr=5.26e-04 tps=22770 mem=5.5G lam=None Gm=0.433349609375 +[g_only s4] step 3060/12000 loss=5.1068 gn=0.67 lr=5.25e-04 tps=22769 mem=5.5G lam=None Gm=0.4381612141927083 +[g_only s4] step 3080/12000 loss=4.9800 gn=0.66 lr=5.24e-04 tps=22768 mem=5.5G lam=None Gm=0.4365336100260417 +[g_only s4] step 3100/12000 loss=5.0134 gn=0.74 lr=5.23e-04 tps=22767 mem=5.5G lam=None Gm=0.4365336100260417 +[g_only s4] step 3120/12000 loss=4.8894 gn=0.66 lr=5.22e-04 tps=22767 mem=5.5G lam=None Gm=0.4345703125 +[g_only s4] step 3140/12000 loss=4.9043 gn=0.68 lr=5.21e-04 tps=22767 mem=5.5G lam=None Gm=0.4269612630208333 +[g_only s4] step 3160/12000 loss=4.9954 gn=0.65 lr=5.20e-04 tps=22767 mem=5.5G lam=None Gm=0.4348246256510417 +[g_only s4] step 3180/12000 loss=4.9009 gn=0.75 lr=5.19e-04 tps=22768 mem=5.5G lam=None Gm=0.42681884765625 +[g_only s4] step 3200/12000 loss=4.9203 gn=0.67 lr=5.18e-04 tps=22768 mem=5.5G lam=None Gm=0.4274698893229167 +[g_only s4] step 3220/12000 loss=5.0735 gn=0.68 lr=5.17e-04 tps=22767 mem=5.5G lam=None Gm=0.431427001953125 +[g_only s4] step 3240/12000 loss=4.9310 gn=0.64 lr=5.16e-04 tps=22766 mem=5.5G lam=None Gm=0.4303995768229167 +[g_only s4] step 3260/12000 loss=4.9422 gn=0.64 lr=5.15e-04 tps=22767 mem=5.5G lam=None Gm=0.4198811848958333 +[g_only s4] step 3280/12000 loss=4.8363 gn=0.67 lr=5.14e-04 tps=22767 mem=5.5G lam=None Gm=0.42889404296875 +[g_only s4] step 3300/12000 loss=4.9490 gn=0.68 lr=5.13e-04 tps=22768 mem=5.5G lam=None Gm=0.4282124837239583 +[g_only s4] step 3320/12000 loss=4.8292 gn=0.67 lr=5.12e-04 tps=22768 mem=5.5G lam=None Gm=0.422271728515625 +[g_only s4] step 3340/12000 loss=4.9283 gn=0.68 lr=5.11e-04 tps=22769 mem=5.5G lam=None Gm=0.4191080729166667 +[g_only s4] step 3360/12000 loss=4.8953 gn=0.70 lr=5.10e-04 tps=22770 mem=5.5G lam=None Gm=0.42913818359375 +[g_only s4] step 3380/12000 loss=4.8036 gn=0.66 lr=5.09e-04 tps=22770 mem=5.5G lam=None Gm=0.4199117024739583 +[g_only s4] step 3400/12000 loss=4.8185 gn=0.68 lr=5.08e-04 tps=22770 mem=5.5G lam=None Gm=0.4147847493489583 +[g_only s4] step 3420/12000 loss=4.8279 gn=0.64 lr=5.07e-04 tps=22770 mem=5.5G lam=None Gm=0.4180094401041667 +[g_only s4] step 3440/12000 loss=4.7563 gn=0.70 lr=5.06e-04 tps=22769 mem=5.5G lam=None Gm=0.424835205078125 +[g_only s4] step 3460/12000 loss=4.7666 gn=0.67 lr=5.05e-04 tps=22769 mem=5.5G lam=None Gm=0.4338175455729167 +[g_only s4] step 3480/12000 loss=5.0156 gn=0.67 lr=5.03e-04 tps=22769 mem=5.5G lam=None Gm=0.4249216715494792 +[g_only s4] step 3500/12000 loss=4.9214 gn=0.65 lr=5.02e-04 tps=22770 mem=5.5G lam=None Gm=0.4190775553385417 +[g_only s4] step 3520/12000 loss=4.7734 gn=0.68 lr=5.01e-04 tps=22769 mem=5.5G lam=None Gm=0.4139302571614583 +[g_only s4] step 3540/12000 loss=5.0335 gn=0.73 lr=5.00e-04 tps=22769 mem=5.5G lam=None Gm=0.4175923665364583 +[g_only s4] step 3560/12000 loss=4.9044 gn=0.68 lr=4.99e-04 tps=22769 mem=5.5G lam=None Gm=0.41876220703125 +[g_only s4] step 3580/12000 loss=5.1355 gn=0.69 lr=4.98e-04 tps=22770 mem=5.5G lam=None Gm=0.4257710774739583 +[g_only s4] step 3600/12000 loss=4.8923 gn=0.66 lr=4.97e-04 tps=22770 mem=5.5G lam=None Gm=0.4251861572265625 +[g_only s4] step 3620/12000 loss=4.7356 gn=0.70 lr=4.96e-04 tps=22769 mem=5.5G lam=None Gm=0.4156545003255208 +[g_only s4] step 3640/12000 loss=4.8574 gn=0.69 lr=4.94e-04 tps=22769 mem=5.5G lam=None Gm=0.4140675862630208 +[g_only s4] step 3660/12000 loss=4.7921 gn=0.83 lr=4.93e-04 tps=22769 mem=5.5G lam=None Gm=0.4297688802083333 +[g_only s4] step 3680/12000 loss=4.8885 gn=0.71 lr=4.92e-04 tps=22769 mem=5.5G lam=None Gm=0.412139892578125 +[g_only s4] step 3700/12000 loss=4.8733 gn=0.72 lr=4.91e-04 tps=22770 mem=5.5G lam=None Gm=0.4247080485026042 +[g_only s4] step 3720/12000 loss=4.7819 gn=0.71 lr=4.90e-04 tps=22770 mem=5.5G lam=None Gm=0.4207865397135417 +[g_only s4] step 3740/12000 loss=4.6356 gn=0.69 lr=4.89e-04 tps=22770 mem=5.5G lam=None Gm=0.4164225260416667 +[g_only s4] step 3760/12000 loss=4.7721 gn=0.70 lr=4.88e-04 tps=22770 mem=5.5G lam=None Gm=0.4212239583333333 +[g_only s4] step 3780/12000 loss=4.7584 gn=0.74 lr=4.86e-04 tps=22770 mem=5.5G lam=None Gm=0.4118448893229167 +[g_only s4] step 3800/12000 loss=4.8375 gn=0.70 lr=4.85e-04 tps=22771 mem=5.5G lam=None Gm=0.4119313557942708 +[g_only s4] step 3820/12000 loss=4.8007 gn=0.71 lr=4.84e-04 tps=22771 mem=5.5G lam=None Gm=0.4187672932942708 +[g_only s4] step 3840/12000 loss=4.6108 gn=0.66 lr=4.83e-04 tps=22771 mem=5.5G lam=None Gm=0.4101664225260417 +[g_only s4] step 3860/12000 loss=4.9067 gn=0.68 lr=4.82e-04 tps=22771 mem=5.5G lam=None Gm=0.4245402018229167 +[g_only s4] step 3880/12000 loss=4.8399 gn=0.67 lr=4.80e-04 tps=22772 mem=5.5G lam=None Gm=0.4127451578776042 +[g_only s4] step 3900/12000 loss=4.8567 gn=0.70 lr=4.79e-04 tps=22771 mem=5.5G lam=None Gm=0.41290283203125 +[g_only s4] step 3920/12000 loss=4.8542 gn=0.71 lr=4.78e-04 tps=22771 mem=5.5G lam=None Gm=0.4188639322916667 +[g_only s4] step 3940/12000 loss=4.7645 gn=0.75 lr=4.77e-04 tps=22771 mem=5.5G lam=None Gm=0.4054819742838542 +[g_only s4] step 3960/12000 loss=4.9365 gn=0.73 lr=4.76e-04 tps=22772 mem=5.5G lam=None Gm=0.4201456705729167 +[g_only s4] step 3980/12000 loss=4.7787 gn=0.72 lr=4.74e-04 tps=22772 mem=5.5G lam=None Gm=0.4018809000651042 +[g_only s4] step 4000/12000 loss=4.7285 gn=0.65 lr=4.73e-04 tps=22773 mem=5.5G lam=None Gm=0.394866943359375 +[g_only s4] EVAL step 4000: val_loss=4.8868 val_ppl=132.52 +[g_only s4] step 4020/12000 loss=4.6593 gn=0.69 lr=4.72e-04 tps=22753 mem=5.5G lam=None Gm=0.4092661539713542 +[g_only s4] step 4040/12000 loss=4.7639 gn=0.66 lr=4.71e-04 tps=22753 mem=5.5G lam=None Gm=0.3988037109375 +[g_only s4] step 4060/12000 loss=4.7648 gn=0.70 lr=4.70e-04 tps=22753 mem=5.5G lam=None Gm=0.4037933349609375 +[g_only s4] step 4080/12000 loss=4.7903 gn=0.89 lr=4.68e-04 tps=22752 mem=5.5G lam=None Gm=0.4068044026692708 +[g_only s4] step 4100/12000 loss=4.8547 gn=0.69 lr=4.67e-04 tps=22753 mem=5.5G lam=None Gm=0.4125874837239583 +[g_only s4] step 4120/12000 loss=4.8081 gn=0.77 lr=4.66e-04 tps=22753 mem=5.5G lam=None Gm=0.4035085042317708 +[g_only s4] step 4140/12000 loss=4.8018 gn=0.71 lr=4.65e-04 tps=22753 mem=5.5G lam=None Gm=0.399688720703125 +[g_only s4] step 4160/12000 loss=4.8922 gn=0.77 lr=4.63e-04 tps=22753 mem=5.5G lam=None Gm=0.4095509847005208 +[g_only s4] step 4180/12000 loss=4.8043 gn=0.69 lr=4.62e-04 tps=22752 mem=5.5G lam=None Gm=0.4121449788411458 +[g_only s4] step 4200/12000 loss=4.8073 gn=0.75 lr=4.61e-04 tps=22753 mem=5.5G lam=None Gm=0.4057973225911458 +[g_only s4] step 4220/12000 loss=4.7637 gn=0.70 lr=4.60e-04 tps=22753 mem=5.5G lam=None Gm=0.3969523111979167 +[g_only s4] step 4240/12000 loss=4.6614 gn=0.66 lr=4.58e-04 tps=22753 mem=5.5G lam=None Gm=0.4042816162109375 +[g_only s4] step 4260/12000 loss=4.7580 gn=0.72 lr=4.57e-04 tps=22752 mem=5.5G lam=None Gm=0.4057566324869792 +[g_only s4] step 4280/12000 loss=4.8762 gn=0.76 lr=4.56e-04 tps=22752 mem=5.5G lam=None Gm=0.4034322102864583 +[g_only s4] step 4300/12000 loss=4.8905 gn=0.74 lr=4.54e-04 tps=22752 mem=5.5G lam=None Gm=0.395599365234375 +[g_only s4] step 4320/12000 loss=4.6917 gn=0.76 lr=4.53e-04 tps=22753 mem=5.5G lam=None Gm=0.3969624837239583 +[g_only s4] step 4340/12000 loss=4.9273 gn=0.73 lr=4.52e-04 tps=22752 mem=5.5G lam=None Gm=0.4102834065755208 +[g_only s4] step 4360/12000 loss=4.7230 gn=0.78 lr=4.51e-04 tps=22752 mem=5.5G lam=None Gm=0.4126536051432292 +[g_only s4] step 4380/12000 loss=4.4425 gn=0.71 lr=4.49e-04 tps=22752 mem=5.5G lam=None Gm=0.3967692057291667 +[g_only s4] step 4400/12000 loss=4.9107 gn=0.73 lr=4.48e-04 tps=22752 mem=5.5G lam=None Gm=0.4088999430338542 +[g_only s4] step 4420/12000 loss=4.7475 gn=0.72 lr=4.47e-04 tps=22752 mem=5.5G lam=None Gm=0.39776611328125 +[g_only s4] step 4440/12000 loss=4.8987 gn=0.76 lr=4.45e-04 tps=22752 mem=5.5G lam=None Gm=0.407562255859375 +[g_only s4] step 4460/12000 loss=4.6203 gn=0.79 lr=4.44e-04 tps=22752 mem=5.5G lam=None Gm=0.4121042887369792 +[g_only s4] step 4480/12000 loss=4.7630 gn=0.73 lr=4.43e-04 tps=22752 mem=5.5G lam=None Gm=0.39703369140625 +[g_only s4] step 4500/12000 loss=4.8402 gn=0.72 lr=4.42e-04 tps=22746 mem=5.5G lam=None Gm=0.39208984375 +[g_only s4] step 4520/12000 loss=4.5173 gn=0.73 lr=4.40e-04 tps=22746 mem=5.5G lam=None Gm=0.4025166829427083 +[g_only s4] step 4540/12000 loss=4.6574 gn=0.71 lr=4.39e-04 tps=22747 mem=5.5G lam=None Gm=0.3954010009765625 +[g_only s4] step 4560/12000 loss=4.6413 gn=0.71 lr=4.38e-04 tps=22746 mem=5.5G lam=None Gm=0.4017283121744792 +[g_only s4] step 4580/12000 loss=4.7107 gn=0.72 lr=4.36e-04 tps=22747 mem=5.5G lam=None Gm=0.3923848470052083 +[g_only s4] step 4600/12000 loss=4.6056 gn=0.74 lr=4.35e-04 tps=22747 mem=5.5G lam=None Gm=0.40411376953125 +[g_only s4] step 4620/12000 loss=4.6867 gn=0.72 lr=4.34e-04 tps=22748 mem=5.5G lam=None Gm=0.3933054606119792 +[g_only s4] step 4640/12000 loss=4.7392 gn=0.79 lr=4.32e-04 tps=22748 mem=5.5G lam=None Gm=0.4031219482421875 +[g_only s4] step 4660/12000 loss=4.6637 gn=0.71 lr=4.31e-04 tps=22749 mem=5.5G lam=None Gm=0.3994242350260417 +[g_only s4] step 4680/12000 loss=4.6190 gn=0.70 lr=4.30e-04 tps=22749 mem=5.5G lam=None Gm=0.4063822428385417 +[g_only s4] step 4700/12000 loss=4.6746 gn=0.73 lr=4.28e-04 tps=22749 mem=5.5G lam=None Gm=0.3968556722005208 +[g_only s4] step 4720/12000 loss=4.6786 gn=0.76 lr=4.27e-04 tps=22750 mem=5.5G lam=None Gm=0.3924306233723958 +[g_only s4] step 4740/12000 loss=4.7281 gn=0.75 lr=4.26e-04 tps=22750 mem=5.5G lam=None Gm=0.3916269938151042 +[g_only s4] step 4760/12000 loss=4.7693 gn=0.72 lr=4.24e-04 tps=22750 mem=5.5G lam=None Gm=0.3899739583333333 +[g_only s4] step 4780/12000 loss=4.6542 gn=0.79 lr=4.23e-04 tps=22750 mem=5.5G lam=None Gm=0.4073486328125 +[g_only s4] step 4800/12000 loss=4.8343 gn=0.73 lr=4.22e-04 tps=22750 mem=5.5G lam=None Gm=0.3903147379557292 +[g_only s4] step 4820/12000 loss=4.6296 gn=0.72 lr=4.20e-04 tps=22750 mem=5.5G lam=None Gm=0.3924407958984375 +[g_only s4] step 4840/12000 loss=4.5485 gn=0.76 lr=4.19e-04 tps=22750 mem=5.5G lam=None Gm=0.3968607584635417 +[g_only s4] step 4860/12000 loss=4.6479 gn=0.73 lr=4.18e-04 tps=22750 mem=5.5G lam=None Gm=0.3960215250651042 +[g_only s4] step 4880/12000 loss=4.6168 gn=0.72 lr=4.16e-04 tps=22750 mem=5.5G lam=None Gm=0.3918355305989583 +[g_only s4] step 4900/12000 loss=4.6817 gn=0.74 lr=4.15e-04 tps=22750 mem=5.5G lam=None Gm=0.3927663167317708 +[g_only s4] step 4920/12000 loss=4.6579 gn=0.87 lr=4.13e-04 tps=22750 mem=5.5G lam=None Gm=0.3901824951171875 +[g_only s4] step 4940/12000 loss=4.6316 gn=0.74 lr=4.12e-04 tps=22749 mem=5.5G lam=None Gm=0.3929901123046875 +[g_only s4] step 4960/12000 loss=4.6870 gn=0.71 lr=4.11e-04 tps=22750 mem=5.5G lam=None Gm=0.3910776774088542 +[g_only s4] step 4980/12000 loss=4.6360 gn=0.81 lr=4.09e-04 tps=22749 mem=5.5G lam=None Gm=0.3902587890625 +[g_only s4] step 5000/12000 loss=4.7756 gn=0.73 lr=4.08e-04 tps=22749 mem=5.5G lam=None Gm=0.3961995442708333 +[g_only s4] EVAL step 5000: val_loss=4.7554 val_ppl=116.21 +[g_only s4] step 5020/12000 loss=4.7389 gn=0.74 lr=4.07e-04 tps=22733 mem=5.5G lam=None Gm=0.38897705078125 +[g_only s4] step 5040/12000 loss=4.8068 gn=0.77 lr=4.05e-04 tps=22733 mem=5.5G lam=None Gm=0.3918100992838542 +[g_only s4] step 5060/12000 loss=4.6609 gn=0.77 lr=4.04e-04 tps=22733 mem=5.5G lam=None Gm=0.3990071614583333 +[g_only s4] step 5080/12000 loss=4.7291 gn=0.75 lr=4.02e-04 tps=22732 mem=5.5G lam=None Gm=0.3879038492838542 +[g_only s4] step 5100/12000 loss=4.6873 gn=0.79 lr=4.01e-04 tps=22731 mem=5.5G lam=None Gm=0.3881429036458333 +[g_only s4] step 5120/12000 loss=4.7216 gn=0.75 lr=4.00e-04 tps=22731 mem=5.5G lam=None Gm=0.3898213704427083 +[g_only s4] step 5140/12000 loss=4.6473 gn=0.71 lr=3.98e-04 tps=22732 mem=5.5G lam=None Gm=0.386627197265625 +[g_only s4] step 5160/12000 loss=4.7333 gn=0.79 lr=3.97e-04 tps=22732 mem=5.5G lam=None Gm=0.3922220865885417 +[g_only s4] step 5180/12000 loss=4.5679 gn=0.74 lr=3.95e-04 tps=22732 mem=5.5G lam=None Gm=0.3943532307942708 +[g_only s4] step 5200/12000 loss=4.5251 gn=0.75 lr=3.94e-04 tps=22732 mem=5.5G lam=None Gm=0.3850148518880208 +[g_only s4] step 5220/12000 loss=4.6908 gn=0.76 lr=3.93e-04 tps=22731 mem=5.5G lam=None Gm=0.3876546223958333 +[g_only s4] step 5240/12000 loss=4.7395 gn=0.80 lr=3.91e-04 tps=22732 mem=5.5G lam=None Gm=0.4030100504557292 +[g_only s4] step 5260/12000 loss=4.6200 gn=0.72 lr=3.90e-04 tps=22732 mem=5.5G lam=None Gm=0.3902842203776042 +[g_only s4] step 5280/12000 loss=4.6009 gn=0.79 lr=3.88e-04 tps=22732 mem=5.5G lam=None Gm=0.3784535725911458 +[g_only s4] step 5300/12000 loss=4.6135 gn=0.75 lr=3.87e-04 tps=22733 mem=5.5G lam=None Gm=0.3802541097005208 +[g_only s4] step 5320/12000 loss=4.6288 gn=0.74 lr=3.86e-04 tps=22733 mem=5.5G lam=None Gm=0.3906351725260417 +[g_only s4] step 5340/12000 loss=4.7267 gn=0.77 lr=3.84e-04 tps=22733 mem=5.5G lam=None Gm=0.3845062255859375 +[g_only s4] step 5360/12000 loss=4.6463 gn=0.71 lr=3.83e-04 tps=22734 mem=5.5G lam=None Gm=0.3881479899088542 +[g_only s4] step 5380/12000 loss=4.5938 gn=0.74 lr=3.81e-04 tps=22734 mem=5.5G lam=None Gm=0.3803660074869792 +[g_only s4] step 5400/12000 loss=4.7048 gn=0.77 lr=3.80e-04 tps=22735 mem=5.5G lam=None Gm=0.3941294352213542 +[g_only s4] step 5420/12000 loss=4.5726 gn=0.86 lr=3.79e-04 tps=22735 mem=5.5G lam=None Gm=0.3951517740885417 +[g_only s4] step 5440/12000 loss=4.7490 gn=0.77 lr=3.77e-04 tps=22735 mem=5.5G lam=None Gm=0.3949330647786458 +[g_only s4] step 5460/12000 loss=4.5538 gn=0.76 lr=3.76e-04 tps=22735 mem=5.5G lam=None Gm=0.3893839518229167 +[g_only s4] step 5480/12000 loss=4.6783 gn=0.78 lr=3.74e-04 tps=22736 mem=5.5G lam=None Gm=0.3796895345052083 +[g_only s4] step 5500/12000 loss=4.7213 gn=0.75 lr=3.73e-04 tps=22736 mem=5.5G lam=None Gm=0.3897196451822917 +[g_only s4] step 5520/12000 loss=4.8121 gn=0.77 lr=3.72e-04 tps=22736 mem=5.5G lam=None Gm=0.3932545979817708 +[g_only s4] step 5540/12000 loss=4.7143 gn=0.78 lr=3.70e-04 tps=22736 mem=5.5G lam=None Gm=0.3931986490885417 +[g_only s4] step 5560/12000 loss=4.5221 gn=0.77 lr=3.69e-04 tps=22736 mem=5.5G lam=None Gm=0.3875223795572917 +[g_only s4] step 5580/12000 loss=4.5566 gn=0.78 lr=3.67e-04 tps=22736 mem=5.5G lam=None Gm=0.3899637858072917 +[g_only s4] step 5600/12000 loss=4.6783 gn=0.79 lr=3.66e-04 tps=22737 mem=5.5G lam=None Gm=0.3900858561197917 +[g_only s4] step 5620/12000 loss=4.4038 gn=0.73 lr=3.64e-04 tps=22737 mem=5.5G lam=None Gm=0.3845062255859375 +[g_only s4] step 5640/12000 loss=4.6801 gn=0.75 lr=3.63e-04 tps=22737 mem=5.5G lam=None Gm=0.382659912109375 +[g_only s4] step 5660/12000 loss=4.7022 gn=0.75 lr=3.62e-04 tps=22737 mem=5.5G lam=None Gm=0.3870951334635417 +[g_only s4] step 5680/12000 loss=4.5167 gn=0.78 lr=3.60e-04 tps=22737 mem=5.5G lam=None Gm=0.3829294840494792 +[g_only s4] step 5700/12000 loss=4.4835 gn=0.78 lr=3.59e-04 tps=22737 mem=5.5G lam=None Gm=0.3800862630208333 +[g_only s4] step 5720/12000 loss=4.6146 gn=0.77 lr=3.57e-04 tps=22738 mem=5.5G lam=None Gm=0.3812611897786458 +[g_only s4] step 5740/12000 loss=4.6698 gn=0.83 lr=3.56e-04 tps=22738 mem=5.5G lam=None Gm=0.3901214599609375 +[g_only s4] step 5760/12000 loss=4.5692 gn=0.81 lr=3.54e-04 tps=22738 mem=5.5G lam=None Gm=0.38909912109375 +[g_only s4] step 5780/12000 loss=4.5360 gn=0.82 lr=3.53e-04 tps=22738 mem=5.5G lam=None Gm=0.3803660074869792 +[g_only s4] step 5800/12000 loss=4.4926 gn=0.78 lr=3.52e-04 tps=22738 mem=5.5G lam=None Gm=0.3903656005859375 +[g_only s4] step 5820/12000 loss=4.5341 gn=0.87 lr=3.50e-04 tps=22739 mem=5.5G lam=None Gm=0.3813120524088542 +[g_only s4] step 5840/12000 loss=4.7283 gn=0.80 lr=3.49e-04 tps=22739 mem=5.5G lam=None Gm=0.3820597330729167 +[g_only s4] step 5860/12000 loss=4.5423 gn=0.75 lr=3.47e-04 tps=22739 mem=5.5G lam=None Gm=0.3785807291666667 +[g_only s4] step 5880/12000 loss=4.6536 gn=0.78 lr=3.46e-04 tps=22739 mem=5.5G lam=None Gm=0.3838399251302083 +[g_only s4] step 5900/12000 loss=4.4806 gn=0.80 lr=3.44e-04 tps=22739 mem=5.5G lam=None Gm=0.3787689208984375 +[g_only s4] step 5920/12000 loss=4.4425 gn=0.81 lr=3.43e-04 tps=22739 mem=5.5G lam=None Gm=0.3802388509114583 +[g_only s4] step 5940/12000 loss=4.6035 gn=0.79 lr=3.41e-04 tps=22739 mem=5.5G lam=None Gm=0.3855489095052083 +[g_only s4] step 5960/12000 loss=4.5320 gn=0.80 lr=3.40e-04 tps=22739 mem=5.5G lam=None Gm=0.3875071207682292 +[g_only s4] step 5980/12000 loss=4.6455 gn=0.83 lr=3.39e-04 tps=22739 mem=5.5G lam=None Gm=0.3857981363932292 +[g_only s4] step 6000/12000 loss=4.6878 gn=0.82 lr=3.37e-04 tps=22739 mem=5.5G lam=None Gm=0.3807729085286458 +[g_only s4] EVAL step 6000: val_loss=4.6436 val_ppl=103.92 +[g_only s4] step 6020/12000 loss=4.6709 gn=0.81 lr=3.36e-04 tps=22726 mem=5.5G lam=None Gm=0.3762715657552083 +[g_only s4] step 6040/12000 loss=4.4744 gn=0.82 lr=3.34e-04 tps=22726 mem=5.5G lam=None Gm=0.3778025309244792 +[g_only s4] step 6060/12000 loss=4.5523 gn=0.79 lr=3.33e-04 tps=22726 mem=5.5G lam=None Gm=0.3732706705729167 +[g_only s4] step 6080/12000 loss=4.7318 gn=0.76 lr=3.31e-04 tps=22726 mem=5.5G lam=None Gm=0.3853658040364583 +[g_only s4] step 6100/12000 loss=4.3904 gn=0.76 lr=3.30e-04 tps=22726 mem=5.5G lam=None Gm=0.3747660319010417 +[g_only s4] step 6120/12000 loss=4.6573 gn=0.76 lr=3.29e-04 tps=22727 mem=5.5G lam=None Gm=0.3853556315104167 +[g_only s4] step 6140/12000 loss=4.4794 gn=0.76 lr=3.27e-04 tps=22727 mem=5.5G lam=None Gm=0.3821919759114583 +[g_only s4] step 6160/12000 loss=4.5644 gn=0.77 lr=3.26e-04 tps=22727 mem=5.5G lam=None Gm=0.3832753499348958 +[g_only s4] step 6180/12000 loss=4.4820 gn=0.76 lr=3.24e-04 tps=22727 mem=5.5G lam=None Gm=0.3716786702473958 +[g_only s4] step 6200/12000 loss=4.6234 gn=0.79 lr=3.23e-04 tps=22727 mem=5.5G lam=None Gm=0.3826548258463542 +[g_only s4] step 6220/12000 loss=4.6658 gn=0.79 lr=3.21e-04 tps=22727 mem=5.5G lam=None Gm=0.3864237467447917 +[g_only s4] step 6240/12000 loss=4.6099 gn=0.77 lr=3.20e-04 tps=22727 mem=5.5G lam=None Gm=0.3831227620442708 +[g_only s4] step 6260/12000 loss=4.4939 gn=0.77 lr=3.19e-04 tps=22727 mem=5.5G lam=None Gm=0.3737589518229167 +[g_only s4] step 6280/12000 loss=4.4623 gn=0.76 lr=3.17e-04 tps=22727 mem=5.5G lam=None Gm=0.37420654296875 +[g_only s4] step 6300/12000 loss=4.5937 gn=0.83 lr=3.16e-04 tps=22726 mem=5.5G lam=None Gm=0.3807118733723958 +[g_only s4] step 6320/12000 loss=4.3439 gn=0.75 lr=3.14e-04 tps=22726 mem=5.5G lam=None Gm=0.3752187093098958 +[g_only s4] step 6340/12000 loss=4.5990 gn=0.77 lr=3.13e-04 tps=22726 mem=5.5G lam=None Gm=0.3767191569010417 +[g_only s4] step 6360/12000 loss=4.4614 gn=0.79 lr=3.11e-04 tps=22726 mem=5.5G lam=None Gm=0.3798878987630208 +[g_only s4] step 6380/12000 loss=4.6117 gn=0.79 lr=3.10e-04 tps=22726 mem=5.5G lam=None Gm=0.383209228515625 +[g_only s4] step 6400/12000 loss=4.5694 gn=0.83 lr=3.08e-04 tps=22726 mem=5.5G lam=None Gm=0.3851064046223958 +[g_only s4] step 6420/12000 loss=4.6132 gn=0.81 lr=3.07e-04 tps=22726 mem=5.5G lam=None Gm=0.3783009847005208 +[g_only s4] step 6440/12000 loss=4.6458 gn=0.82 lr=3.06e-04 tps=22726 mem=5.5G lam=None Gm=0.3756205240885417 +[g_only s4] step 6460/12000 loss=4.5356 gn=0.80 lr=3.04e-04 tps=22726 mem=5.5G lam=None Gm=0.3831787109375 +[g_only s4] step 6480/12000 loss=4.7135 gn=0.84 lr=3.03e-04 tps=22726 mem=5.5G lam=None Gm=0.386627197265625 +[g_only s4] step 6500/12000 loss=4.4473 gn=0.81 lr=3.01e-04 tps=22725 mem=5.5G lam=None Gm=0.3734385172526042 +[g_only s4] step 6520/12000 loss=4.5224 gn=0.76 lr=3.00e-04 tps=22725 mem=5.5G lam=None Gm=0.3695271809895833 +[g_only s4] step 6540/12000 loss=4.5911 gn=0.82 lr=2.98e-04 tps=22726 mem=5.5G lam=None Gm=0.3845011393229167 +[g_only s4] step 6560/12000 loss=4.3726 gn=0.78 lr=2.97e-04 tps=22725 mem=5.5G lam=None Gm=0.3796742757161458 +[g_only s4] step 6580/12000 loss=4.6925 gn=0.84 lr=2.96e-04 tps=22725 mem=5.5G lam=None Gm=0.3787384033203125 +[g_only s4] step 6600/12000 loss=4.4967 gn=0.81 lr=2.94e-04 tps=22725 mem=5.5G lam=None Gm=0.382049560546875 +[g_only s4] step 6620/12000 loss=4.7582 gn=0.95 lr=2.93e-04 tps=22726 mem=5.5G lam=None Gm=0.3834431966145833 +[g_only s4] step 6640/12000 loss=4.4467 gn=0.84 lr=2.91e-04 tps=22726 mem=5.5G lam=None Gm=0.3771718343098958 +[g_only s4] step 6660/12000 loss=4.4484 gn=0.80 lr=2.90e-04 tps=22726 mem=5.5G lam=None Gm=0.3725941975911458 +[g_only s4] step 6680/12000 loss=4.4440 gn=0.82 lr=2.88e-04 tps=22726 mem=5.5G lam=None Gm=0.3738759358723958 +[g_only s4] step 6700/12000 loss=4.5993 gn=0.85 lr=2.87e-04 tps=22726 mem=5.5G lam=None Gm=0.3763224283854167 +[g_only s4] step 6720/12000 loss=4.4910 gn=0.80 lr=2.86e-04 tps=22726 mem=5.5G lam=None Gm=0.3796234130859375 +[g_only s4] step 6740/12000 loss=4.4923 gn=0.78 lr=2.84e-04 tps=22726 mem=5.5G lam=None Gm=0.3810475667317708 +[g_only s4] step 6760/12000 loss=4.5841 gn=0.81 lr=2.83e-04 tps=22726 mem=5.5G lam=None Gm=0.3834635416666667 +[g_only s4] step 6780/12000 loss=4.4513 gn=0.80 lr=2.81e-04 tps=22726 mem=5.5G lam=None Gm=0.3809407552083333 +[g_only s4] step 6800/12000 loss=4.3797 gn=0.80 lr=2.80e-04 tps=22726 mem=5.5G lam=None Gm=0.373016357421875 +[g_only s4] step 6820/12000 loss=4.6042 gn=0.83 lr=2.79e-04 tps=22726 mem=5.5G lam=None Gm=0.377349853515625 +[g_only s4] step 6840/12000 loss=4.4083 gn=0.81 lr=2.77e-04 tps=22726 mem=5.5G lam=None Gm=0.37164306640625 +[g_only s4] step 6860/12000 loss=4.6228 gn=0.80 lr=2.76e-04 tps=22726 mem=5.5G lam=None Gm=0.3792724609375 +[g_only s4] step 6880/12000 loss=4.5163 gn=0.79 lr=2.74e-04 tps=22726 mem=5.5G lam=None Gm=0.3789469401041667 +[g_only s4] step 6900/12000 loss=4.4550 gn=0.83 lr=2.73e-04 tps=22726 mem=5.5G lam=None Gm=0.3814748128255208 +[g_only s4] step 6920/12000 loss=4.4823 gn=0.83 lr=2.72e-04 tps=22726 mem=5.5G lam=None Gm=0.3781534830729167 +[g_only s4] step 6940/12000 loss=4.5475 gn=0.84 lr=2.70e-04 tps=22726 mem=5.5G lam=None Gm=0.3810068766276042 +[g_only s4] step 6960/12000 loss=4.5235 gn=0.85 lr=2.69e-04 tps=22726 mem=5.5G lam=None Gm=0.376129150390625 +[g_only s4] step 6980/12000 loss=4.4743 gn=0.78 lr=2.67e-04 tps=22727 mem=5.5G lam=None Gm=0.3729960123697917 +[g_only s4] step 7000/12000 loss=4.3363 gn=0.82 lr=2.66e-04 tps=22726 mem=5.5G lam=None Gm=0.3721466064453125 +[g_only s4] EVAL step 7000: val_loss=4.6130 val_ppl=100.79 +[g_only s4] step 7020/12000 loss=4.6842 gn=0.86 lr=2.65e-04 tps=22715 mem=5.5G lam=None Gm=0.3775533040364583 +[g_only s4] step 7040/12000 loss=4.6723 gn=0.86 lr=2.63e-04 tps=22715 mem=5.5G lam=None Gm=0.3763275146484375 +[g_only s4] step 7060/12000 loss=4.5545 gn=0.79 lr=2.62e-04 tps=22715 mem=5.5G lam=None Gm=0.3770802815755208 +[g_only s4] step 7080/12000 loss=4.5176 gn=0.84 lr=2.60e-04 tps=22715 mem=5.5G lam=None Gm=0.3772481282552083 +[g_only s4] step 7100/12000 loss=4.5343 gn=0.81 lr=2.59e-04 tps=22715 mem=5.5G lam=None Gm=0.374725341796875 +[g_only s4] step 7120/12000 loss=4.7388 gn=0.89 lr=2.58e-04 tps=22715 mem=5.5G lam=None Gm=0.3842417399088542 +[g_only s4] step 7140/12000 loss=4.4424 gn=0.84 lr=2.56e-04 tps=22715 mem=5.5G lam=None Gm=0.3679555257161458 +[g_only s4] step 7160/12000 loss=4.4809 gn=0.83 lr=2.55e-04 tps=22715 mem=5.5G lam=None Gm=0.376617431640625 +[g_only s4] step 7180/12000 loss=4.5312 gn=0.88 lr=2.53e-04 tps=22715 mem=5.5G lam=None Gm=0.3787027994791667 +[g_only s4] step 7200/12000 loss=4.5354 gn=0.86 lr=2.52e-04 tps=22715 mem=5.5G lam=None Gm=0.3795878092447917 +[g_only s4] step 7220/12000 loss=4.3417 gn=0.82 lr=2.51e-04 tps=22715 mem=5.5G lam=None Gm=0.3737436930338542 +[g_only s4] step 7240/12000 loss=4.4168 gn=0.81 lr=2.49e-04 tps=22715 mem=5.5G lam=None Gm=0.3740030924479167 +[g_only s4] step 7260/12000 loss=4.5364 gn=0.79 lr=2.48e-04 tps=22715 mem=5.5G lam=None Gm=0.3680521647135417 +[g_only s4] step 7280/12000 loss=4.4253 gn=0.83 lr=2.47e-04 tps=22715 mem=5.5G lam=None Gm=0.3690032958984375 +[g_only s4] step 7300/12000 loss=4.5672 gn=0.80 lr=2.45e-04 tps=22715 mem=5.5G lam=None Gm=0.3738911946614583 +[g_only s4] step 7320/12000 loss=4.4027 gn=0.78 lr=2.44e-04 tps=22716 mem=5.5G lam=None Gm=0.3745574951171875 +[g_only s4] step 7340/12000 loss=4.4892 gn=0.85 lr=2.42e-04 tps=22716 mem=5.5G lam=None Gm=0.3709869384765625 +[g_only s4] step 7360/12000 loss=4.5559 gn=0.81 lr=2.41e-04 tps=22716 mem=5.5G lam=None Gm=0.3720804850260417 +[g_only s4] step 7380/12000 loss=4.5357 gn=0.86 lr=2.40e-04 tps=22716 mem=5.5G lam=None Gm=0.3714650472005208 +[g_only s4] step 7400/12000 loss=4.3961 gn=0.80 lr=2.38e-04 tps=22716 mem=5.5G lam=None Gm=0.3733469645182292 +[g_only s4] step 7420/12000 loss=4.4749 gn=0.86 lr=2.37e-04 tps=22716 mem=5.5G lam=None Gm=0.373870849609375 +[g_only s4] step 7440/12000 loss=4.3922 gn=0.82 lr=2.36e-04 tps=22716 mem=5.5G lam=None Gm=0.3675486246744792 +[g_only s4] step 7460/12000 loss=4.5590 gn=0.83 lr=2.34e-04 tps=22716 mem=5.5G lam=None Gm=0.3675181070963542 +[g_only s4] step 7480/12000 loss=4.4061 gn=0.86 lr=2.33e-04 tps=22717 mem=5.5G lam=None Gm=0.3682454427083333 +[g_only s4] step 7500/12000 loss=4.4532 gn=0.86 lr=2.32e-04 tps=22717 mem=5.5G lam=None Gm=0.3680165608723958 +[g_only s4] step 7520/12000 loss=4.5027 gn=0.83 lr=2.30e-04 tps=22717 mem=5.5G lam=None Gm=0.3729960123697917 +[g_only s4] step 7540/12000 loss=4.3862 gn=0.83 lr=2.29e-04 tps=22717 mem=5.5G lam=None Gm=0.3739725748697917 +[g_only s4] step 7560/12000 loss=4.3321 gn=0.91 lr=2.28e-04 tps=22717 mem=5.5G lam=None Gm=0.3660481770833333 +[g_only s4] step 7580/12000 loss=4.5999 gn=0.89 lr=2.26e-04 tps=22717 mem=5.5G lam=None Gm=0.3724873860677083 +[g_only s4] step 7600/12000 loss=4.3382 gn=0.82 lr=2.25e-04 tps=22716 mem=5.5G lam=None Gm=0.3663787841796875 +[g_only s4] step 7620/12000 loss=4.4205 gn=0.83 lr=2.24e-04 tps=22716 mem=5.5G lam=None Gm=0.3703053792317708 +[g_only s4] step 7640/12000 loss=4.4126 gn=0.84 lr=2.22e-04 tps=22716 mem=5.5G lam=None Gm=0.3750864664713542 +[g_only s4] step 7660/12000 loss=4.4344 gn=0.83 lr=2.21e-04 tps=22716 mem=5.5G lam=None Gm=0.3707784016927083 +[g_only s4] step 7680/12000 loss=4.6170 gn=0.97 lr=2.20e-04 tps=22716 mem=5.5G lam=None Gm=0.3702341715494792 +[g_only s4] step 7700/12000 loss=4.5645 gn=0.88 lr=2.18e-04 tps=22716 mem=5.5G lam=None Gm=0.368896484375 +[g_only s4] step 7720/12000 loss=4.5593 gn=0.87 lr=2.17e-04 tps=22716 mem=5.5G lam=None Gm=0.3693084716796875 +[g_only s4] step 7740/12000 loss=4.4744 gn=0.88 lr=2.16e-04 tps=22716 mem=5.5G lam=None Gm=0.3727823893229167 +[g_only s4] step 7760/12000 loss=4.4770 gn=0.83 lr=2.15e-04 tps=22716 mem=5.5G lam=None Gm=0.3690897623697917 +[g_only s4] step 7780/12000 loss=4.5574 gn=0.87 lr=2.13e-04 tps=22716 mem=5.5G lam=None Gm=0.3710276285807292 +[g_only s4] step 7800/12000 loss=4.4730 gn=0.86 lr=2.12e-04 tps=22716 mem=5.5G lam=None Gm=0.3678639729817708 +[g_only s4] step 7820/12000 loss=4.3400 gn=0.84 lr=2.11e-04 tps=22716 mem=5.5G lam=None Gm=0.3628743489583333 +[g_only s4] step 7840/12000 loss=4.3663 gn=0.86 lr=2.09e-04 tps=22716 mem=5.5G lam=None Gm=0.3733774820963542 +[g_only s4] step 7860/12000 loss=4.3879 gn=0.88 lr=2.08e-04 tps=22716 mem=5.5G lam=None Gm=0.372894287109375 +[g_only s4] step 7880/12000 loss=4.3820 gn=0.84 lr=2.07e-04 tps=22716 mem=5.5G lam=None Gm=0.3690541585286458 +[g_only s4] step 7900/12000 loss=4.3587 gn=0.85 lr=2.06e-04 tps=22716 mem=5.5G lam=None Gm=0.3694610595703125 +[g_only s4] step 7920/12000 loss=4.5008 gn=0.91 lr=2.04e-04 tps=22717 mem=5.5G lam=None Gm=0.366485595703125 +[g_only s4] step 7940/12000 loss=4.4970 gn=0.83 lr=2.03e-04 tps=22717 mem=5.5G lam=None Gm=0.3647308349609375 +[g_only s4] step 7960/12000 loss=4.4906 gn=0.84 lr=2.02e-04 tps=22717 mem=5.5G lam=None Gm=0.3662261962890625 +[g_only s4] step 7980/12000 loss=4.4932 gn=0.85 lr=2.00e-04 tps=22717 mem=5.5G lam=None Gm=0.3687845865885417 +[g_only s4] step 8000/12000 loss=4.5444 gn=0.93 lr=1.99e-04 tps=22717 mem=5.5G lam=None Gm=0.3592071533203125 +[g_only s4] EVAL step 8000: val_loss=4.5402 val_ppl=93.71 +[g_only s4] step 8020/12000 loss=4.5221 gn=0.86 lr=1.98e-04 tps=22707 mem=5.5G lam=None Gm=0.3657633463541667 +[g_only s4] step 8040/12000 loss=4.4074 gn=0.87 lr=1.97e-04 tps=22707 mem=5.5G lam=None Gm=0.3673909505208333 +[g_only s4] step 8060/12000 loss=4.4436 gn=0.82 lr=1.95e-04 tps=22707 mem=5.5G lam=None Gm=0.3671213785807292 +[g_only s4] step 8080/12000 loss=4.5320 gn=0.86 lr=1.94e-04 tps=22707 mem=5.5G lam=None Gm=0.36712646484375 +[g_only s4] step 8100/12000 loss=4.5166 gn=0.87 lr=1.93e-04 tps=22707 mem=5.5G lam=None Gm=0.3729502360026042 +[g_only s4] step 8120/12000 loss=4.6500 gn=0.87 lr=1.92e-04 tps=22707 mem=5.5G lam=None Gm=0.3721415201822917 +[g_only s4] step 8140/12000 loss=4.3144 gn=0.83 lr=1.90e-04 tps=22707 mem=5.5G lam=None Gm=0.3609517415364583 +[g_only s4] step 8160/12000 loss=4.3910 gn=0.85 lr=1.89e-04 tps=22707 mem=5.5G lam=None Gm=0.3732961018880208 +[g_only s4] step 8180/12000 loss=4.4975 gn=0.90 lr=1.88e-04 tps=22707 mem=5.5G lam=None Gm=0.3723398844401042 +[g_only s4] step 8200/12000 loss=4.3530 gn=0.90 lr=1.87e-04 tps=22707 mem=5.5G lam=None Gm=0.3676300048828125 +[g_only s4] step 8220/12000 loss=4.4345 gn=0.85 lr=1.86e-04 tps=22707 mem=5.5G lam=None Gm=0.3614603678385417 +[g_only s4] step 8240/12000 loss=4.3618 gn=0.88 lr=1.84e-04 tps=22707 mem=5.5G lam=None Gm=0.3629506429036458 +[g_only s4] step 8260/12000 loss=4.3374 gn=0.89 lr=1.83e-04 tps=22707 mem=5.5G lam=None Gm=0.3649495442708333 +[g_only s4] step 8280/12000 loss=4.3172 gn=0.82 lr=1.82e-04 tps=22707 mem=5.5G lam=None Gm=0.3662567138671875 +[g_only s4] step 8300/12000 loss=4.5233 gn=0.86 lr=1.81e-04 tps=22707 mem=5.5G lam=None Gm=0.3710683186848958 +[g_only s4] step 8320/12000 loss=4.3652 gn=0.85 lr=1.80e-04 tps=22706 mem=5.5G lam=None Gm=0.3672587076822917 +[g_only s4] step 8340/12000 loss=4.4364 gn=0.85 lr=1.78e-04 tps=22706 mem=5.5G lam=None Gm=0.3680318196614583 +[g_only s4] step 8360/12000 loss=4.5312 gn=0.91 lr=1.77e-04 tps=22706 mem=5.5G lam=None Gm=0.3692118326822917 +[g_only s4] step 8380/12000 loss=4.3684 gn=0.87 lr=1.76e-04 tps=22706 mem=5.5G lam=None Gm=0.3664957682291667 +[g_only s4] step 8400/12000 loss=4.4556 gn=0.86 lr=1.75e-04 tps=22706 mem=5.5G lam=None Gm=0.3697713216145833 +[g_only s4] step 8420/12000 loss=4.3323 gn=0.86 lr=1.74e-04 tps=22706 mem=5.5G lam=None Gm=0.3669230143229167 +[g_only s4] step 8440/12000 loss=4.2946 gn=0.82 lr=1.72e-04 tps=22706 mem=5.5G lam=None Gm=0.3648529052734375 +[g_only s4] step 8460/12000 loss=4.3534 gn=0.95 lr=1.71e-04 tps=22707 mem=5.5G lam=None Gm=0.3707427978515625 +[g_only s4] step 8480/12000 loss=4.3630 gn=0.84 lr=1.70e-04 tps=22707 mem=5.5G lam=None Gm=0.358734130859375 +[g_only s4] step 8500/12000 loss=4.3804 gn=0.88 lr=1.69e-04 tps=22707 mem=5.5G lam=None Gm=0.3653361002604167 +[g_only s4] step 8520/12000 loss=4.4826 gn=0.88 lr=1.68e-04 tps=22707 mem=5.5G lam=None Gm=0.3706715901692708 +[g_only s4] step 8540/12000 loss=4.5579 gn=0.88 lr=1.67e-04 tps=22707 mem=5.5G lam=None Gm=0.367431640625 +[g_only s4] step 8560/12000 loss=4.3485 gn=0.85 lr=1.66e-04 tps=22707 mem=5.5G lam=None Gm=0.360992431640625 +[g_only s4] step 8580/12000 loss=4.4704 gn=0.89 lr=1.64e-04 tps=22707 mem=5.5G lam=None Gm=0.3608245849609375 +[g_only s4] step 8600/12000 loss=4.2830 gn=0.91 lr=1.63e-04 tps=22707 mem=5.5G lam=None Gm=0.3662923177083333 +[g_only s4] step 8620/12000 loss=4.4192 gn=0.88 lr=1.62e-04 tps=22707 mem=5.5G lam=None Gm=0.364959716796875 +[g_only s4] step 8640/12000 loss=4.4820 gn=0.90 lr=1.61e-04 tps=22708 mem=5.5G lam=None Gm=0.3656565348307292 +[g_only s4] step 8660/12000 loss=4.2785 gn=0.95 lr=1.60e-04 tps=22708 mem=5.5G lam=None Gm=0.3629252115885417 +[g_only s4] step 8680/12000 loss=4.3859 gn=0.84 lr=1.59e-04 tps=22708 mem=5.5G lam=None Gm=0.3680369059244792 +[g_only s4] step 8700/12000 loss=4.2621 gn=0.90 lr=1.58e-04 tps=22708 mem=5.5G lam=None Gm=0.3664499918619792 +[g_only s4] step 8720/12000 loss=4.4997 gn=0.90 lr=1.57e-04 tps=22708 mem=5.5G lam=None Gm=0.3633677164713542 +[g_only s4] step 8740/12000 loss=4.4034 gn=0.90 lr=1.55e-04 tps=22708 mem=5.5G lam=None Gm=0.3659820556640625 +[g_only s4] step 8760/12000 loss=4.3298 gn=0.88 lr=1.54e-04 tps=22708 mem=5.5G lam=None Gm=0.3653615315755208 +[g_only s4] step 8780/12000 loss=4.3991 gn=0.90 lr=1.53e-04 tps=22708 mem=5.5G lam=None Gm=0.3725077311197917 +[g_only s4] step 8800/12000 loss=4.3344 gn=0.91 lr=1.52e-04 tps=22708 mem=5.5G lam=None Gm=0.3666534423828125 +[g_only s4] step 8820/12000 loss=4.2747 gn=0.88 lr=1.51e-04 tps=22709 mem=5.5G lam=None Gm=0.3689931233723958 +[g_only s4] step 8840/12000 loss=4.4922 gn=0.94 lr=1.50e-04 tps=22709 mem=5.5G lam=None Gm=0.3676096598307292 +[g_only s4] step 8860/12000 loss=4.3519 gn=0.89 lr=1.49e-04 tps=22709 mem=5.5G lam=None Gm=0.3635152180989583 +[g_only s4] step 8880/12000 loss=4.3695 gn=0.84 lr=1.48e-04 tps=22709 mem=5.5G lam=None Gm=0.3646392822265625 +[g_only s4] step 8900/12000 loss=4.1784 gn=0.92 lr=1.47e-04 tps=22709 mem=5.5G lam=None Gm=0.3657633463541667 +[g_only s4] step 8920/12000 loss=4.3764 gn=0.89 lr=1.46e-04 tps=22709 mem=5.5G lam=None Gm=0.3710072835286458 +[g_only s4] step 8940/12000 loss=4.3861 gn=0.86 lr=1.45e-04 tps=22710 mem=5.5G lam=None Gm=0.3586985270182292 +[g_only s4] step 8960/12000 loss=4.3783 gn=0.88 lr=1.44e-04 tps=22710 mem=5.5G lam=None Gm=0.3609975179036458 +[g_only s4] step 8980/12000 loss=4.4659 gn=0.89 lr=1.43e-04 tps=22710 mem=5.5G lam=None Gm=0.3641357421875 +[g_only s4] step 9000/12000 loss=4.3748 gn=0.87 lr=1.42e-04 tps=22710 mem=5.5G lam=None Gm=0.3645121256510417 +[g_only s4] EVAL step 9000: val_loss=4.4490 val_ppl=85.54 +[g_only s4] step 9020/12000 loss=4.4694 gn=0.93 lr=1.41e-04 tps=22701 mem=5.5G lam=None Gm=0.3705190022786458 +[g_only s4] step 9040/12000 loss=4.3942 gn=0.90 lr=1.40e-04 tps=22701 mem=5.5G lam=None Gm=0.3659922281901042 +[g_only s4] step 9060/12000 loss=4.1351 gn=0.88 lr=1.39e-04 tps=22701 mem=5.5G lam=None Gm=0.3676961263020833 +[g_only s4] step 9080/12000 loss=4.4891 gn=0.91 lr=1.38e-04 tps=22701 mem=5.5G lam=None Gm=0.3704884847005208 +[g_only s4] step 9100/12000 loss=4.4182 gn=0.87 lr=1.37e-04 tps=22701 mem=5.5G lam=None Gm=0.3602396647135417 +[g_only s4] step 9120/12000 loss=4.2963 gn=0.94 lr=1.36e-04 tps=22701 mem=5.5G lam=None Gm=0.3657582600911458 +[g_only s4] step 9140/12000 loss=4.2115 gn=0.84 lr=1.35e-04 tps=22701 mem=5.5G lam=None Gm=0.3647308349609375 +[g_only s4] step 9160/12000 loss=4.4603 gn=0.92 lr=1.34e-04 tps=22701 mem=5.5G lam=None Gm=0.3637034098307292 +[g_only s4] step 9180/12000 loss=4.4518 gn=0.87 lr=1.33e-04 tps=22701 mem=5.5G lam=None Gm=0.3634847005208333 +[g_only s4] step 9200/12000 loss=4.2495 gn=0.90 lr=1.32e-04 tps=22701 mem=5.5G lam=None Gm=0.3669077555338542 +[g_only s4] step 9220/12000 loss=4.4398 gn=0.90 lr=1.31e-04 tps=22701 mem=5.5G lam=None Gm=0.3649749755859375 +[g_only s4] step 9240/12000 loss=4.4811 gn=0.92 lr=1.30e-04 tps=22701 mem=5.5G lam=None Gm=0.35992431640625 +[g_only s4] step 9260/12000 loss=4.3547 gn=0.90 lr=1.29e-04 tps=22701 mem=5.5G lam=None Gm=0.3672078450520833 +[g_only s4] step 9280/12000 loss=4.3516 gn=0.98 lr=1.28e-04 tps=22701 mem=5.5G lam=None Gm=0.3675435384114583 +[g_only s4] step 9300/12000 loss=4.4972 gn=0.98 lr=1.27e-04 tps=22701 mem=5.5G lam=None Gm=0.3683624267578125 +[g_only s4] step 9320/12000 loss=4.2876 gn=0.95 lr=1.26e-04 tps=22701 mem=5.5G lam=None Gm=0.3663889567057292 +[g_only s4] step 9340/12000 loss=4.4869 gn=0.92 lr=1.25e-04 tps=22701 mem=5.5G lam=None Gm=0.3638254801432292 +[g_only s4] step 9360/12000 loss=4.2993 gn=0.92 lr=1.24e-04 tps=22701 mem=5.5G lam=None Gm=0.3569997151692708 +[g_only s4] step 9380/12000 loss=4.4379 gn=0.87 lr=1.23e-04 tps=22701 mem=5.5G lam=None Gm=0.3606313069661458 +[g_only s4] step 9400/12000 loss=4.2098 gn=0.92 lr=1.22e-04 tps=22701 mem=5.5G lam=None Gm=0.3646748860677083 +[g_only s4] step 9420/12000 loss=4.2534 gn=0.90 lr=1.21e-04 tps=22701 mem=5.5G lam=None Gm=0.360198974609375 +[g_only s4] step 9440/12000 loss=4.2976 gn=1.02 lr=1.20e-04 tps=22702 mem=5.5G lam=None Gm=0.3594716389973958 +[g_only s4] step 9460/12000 loss=4.3765 gn=0.90 lr=1.19e-04 tps=22701 mem=5.5G lam=None Gm=0.3649495442708333 +[g_only s4] step 9480/12000 loss=4.3786 gn=0.92 lr=1.19e-04 tps=22699 mem=5.5G lam=None Gm=0.3634389241536458 +[g_only s4] step 9500/12000 loss=4.5306 gn=1.04 lr=1.18e-04 tps=22699 mem=5.5G lam=None Gm=0.36468505859375 +[g_only s4] step 9520/12000 loss=4.2146 gn=0.95 lr=1.17e-04 tps=22699 mem=5.5G lam=None Gm=0.3654683430989583 +[g_only s4] step 9540/12000 loss=4.3732 gn=0.86 lr=1.16e-04 tps=22699 mem=5.5G lam=None Gm=0.3653411865234375 +[g_only s4] step 9560/12000 loss=4.5485 gn=0.93 lr=1.15e-04 tps=22699 mem=5.5G lam=None Gm=0.3615163167317708 +[g_only s4] step 9580/12000 loss=4.2803 gn=0.94 lr=1.14e-04 tps=22700 mem=5.5G lam=None Gm=0.3634389241536458 +[g_only s4] step 9600/12000 loss=4.4170 gn=0.89 lr=1.13e-04 tps=22700 mem=5.5G lam=None Gm=0.3656056722005208 +[g_only s4] step 9620/12000 loss=4.3527 gn=0.98 lr=1.12e-04 tps=22700 mem=5.5G lam=None Gm=0.3614501953125 +[g_only s4] step 9640/12000 loss=4.5219 gn=0.97 lr=1.12e-04 tps=22700 mem=5.5G lam=None Gm=0.3778279622395833 +[g_only s4] step 9660/12000 loss=4.3250 gn=0.90 lr=1.11e-04 tps=22700 mem=5.5G lam=None Gm=0.3622792561848958 +[g_only s4] step 9680/12000 loss=4.5246 gn=0.92 lr=1.10e-04 tps=22700 mem=5.5G lam=None Gm=0.36639404296875 +[g_only s4] step 9700/12000 loss=4.2764 gn=0.90 lr=1.09e-04 tps=22700 mem=5.5G lam=None Gm=0.3625132242838542 +[g_only s4] step 9720/12000 loss=4.3171 gn=0.93 lr=1.08e-04 tps=22700 mem=5.5G lam=None Gm=0.3661651611328125 +[g_only s4] step 9740/12000 loss=4.3597 gn=0.89 lr=1.07e-04 tps=22700 mem=5.5G lam=None Gm=0.3650665283203125 +[g_only s4] step 9760/12000 loss=4.3979 gn=0.91 lr=1.07e-04 tps=22700 mem=5.5G lam=None Gm=0.3622894287109375 +[g_only s4] step 9780/12000 loss=4.1119 gn=0.90 lr=1.06e-04 tps=22700 mem=5.5G lam=None Gm=0.3589731852213542 +[g_only s4] step 9800/12000 loss=4.2748 gn=0.94 lr=1.05e-04 tps=22701 mem=5.5G lam=None Gm=0.3613942464192708 +[g_only s4] step 9820/12000 loss=4.1728 gn=0.94 lr=1.04e-04 tps=22700 mem=5.5G lam=None Gm=0.355865478515625 +[g_only s4] step 9840/12000 loss=4.4951 gn=0.90 lr=1.03e-04 tps=22701 mem=5.5G lam=None Gm=0.3721364339192708 +[g_only s4] step 9860/12000 loss=4.3068 gn=0.92 lr=1.03e-04 tps=22700 mem=5.5G lam=None Gm=0.3672231038411458 +[g_only s4] step 9880/12000 loss=4.3216 gn=0.92 lr=1.02e-04 tps=22700 mem=5.5G lam=None Gm=0.3610585530598958 +[g_only s4] step 9900/12000 loss=4.2673 gn=0.96 lr=1.01e-04 tps=22701 mem=5.5G lam=None Gm=0.3667093912760417 +[g_only s4] step 9920/12000 loss=4.3075 gn=0.93 lr=1.00e-04 tps=22700 mem=5.5G lam=None Gm=0.3636372884114583 +[g_only s4] step 9940/12000 loss=4.5051 gn=0.92 lr=9.96e-05 tps=22700 mem=5.5G lam=None Gm=0.362945556640625 +[g_only s4] step 9960/12000 loss=4.4027 gn=0.93 lr=9.89e-05 tps=22700 mem=5.5G lam=None Gm=0.3688201904296875 +[g_only s4] step 9980/12000 loss=4.3596 gn=0.89 lr=9.81e-05 tps=22700 mem=5.5G lam=None Gm=0.360443115234375 +[g_only s4] step 10000/12000 loss=4.3394 gn=0.90 lr=9.74e-05 tps=22701 mem=5.5G lam=None Gm=0.3642171223958333 +[g_only s4] EVAL step 10000: val_loss=4.4182 val_ppl=82.94 +[g_only s4] step 10020/12000 loss=4.4952 gn=0.93 lr=9.67e-05 tps=22693 mem=5.5G lam=None Gm=0.3707834879557292 +[g_only s4] step 10040/12000 loss=4.3115 gn=0.91 lr=9.59e-05 tps=22693 mem=5.5G lam=None Gm=0.3625895182291667 +[g_only s4] step 10060/12000 loss=4.3050 gn=0.92 lr=9.52e-05 tps=22693 mem=5.5G lam=None Gm=0.3580474853515625 +[g_only s4] step 10080/12000 loss=4.3422 gn=0.96 lr=9.45e-05 tps=22693 mem=5.5G lam=None Gm=0.3640238444010417 +[g_only s4] step 10100/12000 loss=4.3711 gn=0.91 lr=9.38e-05 tps=22693 mem=5.5G lam=None Gm=0.3601531982421875 +[g_only s4] step 10120/12000 loss=4.3660 gn=0.92 lr=9.31e-05 tps=22692 mem=5.5G lam=None Gm=0.3637237548828125 +[g_only s4] step 10140/12000 loss=4.1491 gn=0.92 lr=9.24e-05 tps=22692 mem=5.5G lam=None Gm=0.3648325602213542 +[g_only s4] step 10160/12000 loss=4.2775 gn=0.97 lr=9.18e-05 tps=22692 mem=5.5G lam=None Gm=0.3674112955729167 +[g_only s4] step 10180/12000 loss=4.4495 gn=0.91 lr=9.11e-05 tps=22692 mem=5.5G lam=None Gm=0.3628743489583333 +[g_only s4] step 10200/12000 loss=4.3210 gn=0.93 lr=9.04e-05 tps=22692 mem=5.5G lam=None Gm=0.37005615234375 +[g_only s4] step 10220/12000 loss=4.3996 gn=1.03 lr=8.98e-05 tps=22692 mem=5.5G lam=None Gm=0.358154296875 +[g_only s4] step 10240/12000 loss=4.2256 gn=0.98 lr=8.91e-05 tps=22692 mem=5.5G lam=None Gm=0.36346435546875 +[g_only s4] step 10260/12000 loss=4.3971 gn=0.98 lr=8.85e-05 tps=22692 mem=5.5G lam=None Gm=0.3657277425130208 +[g_only s4] step 10280/12000 loss=4.3995 gn=0.93 lr=8.78e-05 tps=22692 mem=5.5G lam=None Gm=0.3659871419270833 +[g_only s4] step 10300/12000 loss=4.2778 gn=0.92 lr=8.72e-05 tps=22692 mem=5.5G lam=None Gm=0.3607126871744792 +[g_only s4] step 10320/12000 loss=4.4408 gn=0.93 lr=8.66e-05 tps=22692 mem=5.5G lam=None Gm=0.3635711669921875 +[g_only s4] step 10340/12000 loss=4.3360 gn=0.97 lr=8.59e-05 tps=22692 mem=5.5G lam=None Gm=0.3646138509114583 +[g_only s4] step 10360/12000 loss=4.4249 gn=0.91 lr=8.53e-05 tps=22692 mem=5.5G lam=None Gm=0.362640380859375 +[g_only s4] step 10380/12000 loss=4.4061 gn=0.97 lr=8.47e-05 tps=22692 mem=5.5G lam=None Gm=0.3676350911458333 +[g_only s4] step 10400/12000 loss=4.4398 gn=0.96 lr=8.41e-05 tps=22692 mem=5.5G lam=None Gm=0.3673960367838542 +[g_only s4] step 10420/12000 loss=4.4094 gn=0.94 lr=8.35e-05 tps=22692 mem=5.5G lam=None Gm=0.3663533528645833 +[g_only s4] step 10440/12000 loss=4.2715 gn=0.91 lr=8.30e-05 tps=22692 mem=5.5G lam=None Gm=0.3657073974609375 +[g_only s4] step 10460/12000 loss=4.4034 gn=1.00 lr=8.24e-05 tps=22692 mem=5.5G lam=None Gm=0.3590799967447917 +[g_only s4] step 10480/12000 loss=4.2552 gn=0.90 lr=8.18e-05 tps=22693 mem=5.5G lam=None Gm=0.363922119140625 +[g_only s4] step 10500/12000 loss=4.3558 gn=0.97 lr=8.12e-05 tps=22693 mem=5.5G lam=None Gm=0.3645172119140625 +[g_only s4] step 10520/12000 loss=4.4177 gn=0.99 lr=8.07e-05 tps=22693 mem=5.5G lam=None Gm=0.3625081380208333 +[g_only s4] step 10540/12000 loss=4.3254 gn=1.07 lr=8.01e-05 tps=22693 mem=5.5G lam=None Gm=0.3564707438151042 +[g_only s4] step 10560/12000 loss=4.3244 gn=0.93 lr=7.96e-05 tps=22693 mem=5.5G lam=None Gm=0.3623199462890625 +[g_only s4] step 10580/12000 loss=4.3742 gn=0.94 lr=7.91e-05 tps=22693 mem=5.5G lam=None Gm=0.3582356770833333 +[g_only s4] step 10600/12000 loss=4.2349 gn=0.95 lr=7.85e-05 tps=22693 mem=5.5G lam=None Gm=0.3626963297526042 +[g_only s4] step 10620/12000 loss=4.2624 gn=0.91 lr=7.80e-05 tps=22693 mem=5.5G lam=None Gm=0.3582000732421875 +[g_only s4] step 10640/12000 loss=4.2560 gn=0.91 lr=7.75e-05 tps=22694 mem=5.5G lam=None Gm=0.3604838053385417 +[g_only s4] step 10660/12000 loss=4.4218 gn=0.92 lr=7.70e-05 tps=22693 mem=5.5G lam=None Gm=0.364501953125 +[g_only s4] step 10680/12000 loss=4.3238 gn=0.94 lr=7.65e-05 tps=22694 mem=5.5G lam=None Gm=0.3589019775390625 +[g_only s4] step 10700/12000 loss=4.3084 gn=0.97 lr=7.60e-05 tps=22694 mem=5.5G lam=None Gm=0.3656005859375 +[g_only s4] step 10720/12000 loss=4.4226 gn=1.02 lr=7.55e-05 tps=22694 mem=5.5G lam=None Gm=0.3690439860026042 +[g_only s4] step 10740/12000 loss=4.4149 gn=0.98 lr=7.50e-05 tps=22694 mem=5.5G lam=None Gm=0.3612721761067708 +[g_only s4] step 10760/12000 loss=4.1906 gn=0.99 lr=7.46e-05 tps=22694 mem=5.5G lam=None Gm=0.3662261962890625 +[g_only s4] step 10780/12000 loss=4.2581 gn=1.19 lr=7.41e-05 tps=22694 mem=5.5G lam=None Gm=0.3622233072916667 +[g_only s4] step 10800/12000 loss=4.3300 gn=0.99 lr=7.37e-05 tps=22694 mem=5.5G lam=None Gm=0.3630421956380208 +[g_only s4] step 10820/12000 loss=4.3010 gn=0.94 lr=7.32e-05 tps=22694 mem=5.5G lam=None Gm=0.3620452880859375 +[g_only s4] step 10840/12000 loss=4.2888 gn=1.04 lr=7.28e-05 tps=22695 mem=5.5G lam=None Gm=0.3580576578776042 +[g_only s4] step 10860/12000 loss=4.2732 gn=0.98 lr=7.23e-05 tps=22694 mem=5.5G lam=None Gm=0.3591969807942708 +[g_only s4] step 10880/12000 loss=4.2456 gn=1.01 lr=7.19e-05 tps=22693 mem=5.5G lam=None Gm=0.3580474853515625 +[g_only s4] step 10900/12000 loss=4.1757 gn=1.01 lr=7.15e-05 tps=22693 mem=5.5G lam=None Gm=0.35662841796875 +[g_only s4] step 10920/12000 loss=4.3380 gn=0.94 lr=7.11e-05 tps=22692 mem=5.5G lam=None Gm=0.3638356526692708 +[g_only s4] step 10940/12000 loss=4.3197 gn=1.00 lr=7.07e-05 tps=22691 mem=5.5G lam=None Gm=0.3725331624348958 +[g_only s4] step 10960/12000 loss=4.3115 gn=0.98 lr=7.03e-05 tps=22690 mem=5.5G lam=None Gm=0.3660736083984375 +[g_only s4] step 10980/12000 loss=4.4702 gn=1.05 lr=6.99e-05 tps=22689 mem=5.5G lam=None Gm=0.3626861572265625 +[g_only s4] step 11000/12000 loss=4.3979 gn=0.93 lr=6.95e-05 tps=22688 mem=5.5G lam=None Gm=0.3596293131510417 +[g_only s4] EVAL step 11000: val_loss=4.3692 val_ppl=78.98 +[g_only s4] step 11020/12000 loss=4.1841 gn=0.96 lr=6.91e-05 tps=22680 mem=5.5G lam=None Gm=0.3645273844401042 +[g_only s4] step 11040/12000 loss=4.2861 gn=0.99 lr=6.88e-05 tps=22678 mem=5.5G lam=None Gm=0.3636016845703125 +[g_only s4] step 11060/12000 loss=4.4671 gn=0.99 lr=6.84e-05 tps=22676 mem=5.5G lam=None Gm=0.3629557291666667 +[g_only s4] step 11080/12000 loss=4.3124 gn=0.96 lr=6.81e-05 tps=22674 mem=5.5G lam=None Gm=0.3620808919270833 +[g_only s4] step 11100/12000 loss=4.3399 gn=1.05 lr=6.77e-05 tps=22673 mem=5.5G lam=None Gm=0.3698221842447917 +[g_only s4] step 11120/12000 loss=4.4014 gn=0.95 lr=6.74e-05 tps=22672 mem=5.5G lam=None Gm=0.3639068603515625 +[g_only s4] step 11140/12000 loss=4.3615 gn=1.06 lr=6.70e-05 tps=22670 mem=5.5G lam=None Gm=0.3565622965494792 +[g_only s4] step 11160/12000 loss=4.2399 gn=0.94 lr=6.67e-05 tps=22668 mem=5.5G lam=None Gm=0.3595937093098958 +[g_only s4] step 11180/12000 loss=4.3751 gn=1.04 lr=6.64e-05 tps=22667 mem=5.5G lam=None Gm=0.3611195882161458 +[g_only s4] step 11200/12000 loss=4.3496 gn=0.94 lr=6.61e-05 tps=22665 mem=5.5G lam=None Gm=0.3609771728515625 +[g_only s4] step 11220/12000 loss=4.2102 gn=1.02 lr=6.58e-05 tps=22665 mem=5.5G lam=None Gm=0.3597920735677083 +[g_only s4] step 11240/12000 loss=4.4157 gn=1.07 lr=6.55e-05 tps=22665 mem=5.5G lam=None Gm=0.3590342203776042 +[g_only s4] step 11260/12000 loss=4.3925 gn=0.95 lr=6.52e-05 tps=22665 mem=5.5G lam=None Gm=0.35809326171875 +[g_only s4] step 11280/12000 loss=4.4121 gn=1.05 lr=6.49e-05 tps=22664 mem=5.5G lam=None Gm=0.3606516520182292 +[g_only s4] step 11300/12000 loss=4.2696 gn=1.06 lr=6.47e-05 tps=22665 mem=5.5G lam=None Gm=0.358917236328125 +[g_only s4] step 11320/12000 loss=4.3521 gn=0.90 lr=6.44e-05 tps=22665 mem=5.5G lam=None Gm=0.3580830891927083 +[g_only s4] step 11340/12000 loss=4.5863 gn=1.01 lr=6.42e-05 tps=22665 mem=5.5G lam=None Gm=0.3603515625 +[g_only s4] step 11360/12000 loss=4.2712 gn=0.94 lr=6.39e-05 tps=22665 mem=5.5G lam=None Gm=0.3560078938802083 +[g_only s4] step 11380/12000 loss=4.3543 gn=1.01 lr=6.37e-05 tps=22665 mem=5.5G lam=None Gm=0.356658935546875 +[g_only s4] step 11400/12000 loss=4.3124 gn=0.95 lr=6.34e-05 tps=22664 mem=5.5G lam=None Gm=0.3566335042317708 +[g_only s4] step 11420/12000 loss=4.2701 gn=0.99 lr=6.32e-05 tps=22664 mem=5.5G lam=None Gm=0.36224365234375 +[g_only s4] step 11440/12000 loss=4.3455 gn=1.00 lr=6.30e-05 tps=22664 mem=5.5G lam=None Gm=0.3650614420572917 +[g_only s4] step 11460/12000 loss=4.2770 gn=0.96 lr=6.28e-05 tps=22664 mem=5.5G lam=None Gm=0.3593597412109375 +[g_only s4] step 11480/12000 loss=4.5089 gn=0.99 lr=6.26e-05 tps=22664 mem=5.5G lam=None Gm=0.3605550130208333 +[g_only s4] step 11500/12000 loss=4.2553 gn=1.01 lr=6.24e-05 tps=22664 mem=5.5G lam=None Gm=0.3587239583333333 +[g_only s4] step 11520/12000 loss=4.4467 gn=1.01 lr=6.22e-05 tps=22664 mem=5.5G lam=None Gm=0.363128662109375 +[g_only s4] step 11540/12000 loss=4.3691 gn=0.98 lr=6.20e-05 tps=22664 mem=5.5G lam=None Gm=0.3604583740234375 +[g_only s4] step 11560/12000 loss=4.3374 gn=0.98 lr=6.19e-05 tps=22664 mem=5.5G lam=None Gm=0.3594207763671875 +[g_only s4] step 11580/12000 loss=4.4443 gn=0.97 lr=6.17e-05 tps=22664 mem=5.5G lam=None Gm=0.356658935546875 +[g_only s4] step 11600/12000 loss=4.3051 gn=0.99 lr=6.15e-05 tps=22664 mem=5.5G lam=None Gm=0.3613942464192708 +[g_only s4] step 11620/12000 loss=4.1992 gn=1.02 lr=6.14e-05 tps=22663 mem=5.5G lam=None Gm=0.3608194986979167 +[g_only s4] step 11640/12000 loss=4.3098 gn=0.95 lr=6.12e-05 tps=22663 mem=5.5G lam=None Gm=0.3599955240885417 +[g_only s4] step 11660/12000 loss=4.1996 gn=0.99 lr=6.11e-05 tps=22663 mem=5.5G lam=None Gm=0.3600107828776042 +[g_only s4] step 11680/12000 loss=4.3447 gn=1.06 lr=6.10e-05 tps=22662 mem=5.5G lam=None Gm=0.3603363037109375 +[g_only s4] step 11700/12000 loss=4.2321 gn=0.97 lr=6.09e-05 tps=22662 mem=5.5G lam=None Gm=0.358123779296875 +[g_only s4] step 11720/12000 loss=4.2964 gn=1.01 lr=6.07e-05 tps=22662 mem=5.5G lam=None Gm=0.3606821695963542 +[g_only s4] step 11740/12000 loss=4.2577 gn=0.99 lr=6.06e-05 tps=22662 mem=5.5G lam=None Gm=0.3583221435546875 +[g_only s4] step 11760/12000 loss=4.3203 gn=0.96 lr=6.06e-05 tps=22661 mem=5.5G lam=None Gm=0.3598836263020833 +[g_only s4] step 11780/12000 loss=4.5707 gn=1.10 lr=6.05e-05 tps=22661 mem=5.5G lam=None Gm=0.358154296875 +[g_only s4] step 11800/12000 loss=4.4254 gn=1.05 lr=6.04e-05 tps=22660 mem=5.5G lam=None Gm=0.3610127766927083 +[g_only s4] step 11820/12000 loss=4.4357 gn=1.02 lr=6.03e-05 tps=22660 mem=5.5G lam=None Gm=0.3675587972005208 +[g_only s4] step 11840/12000 loss=4.4260 gn=0.99 lr=6.02e-05 tps=22660 mem=5.5G lam=None Gm=0.3652852376302083 +[g_only s4] step 11860/12000 loss=4.2720 gn=1.02 lr=6.02e-05 tps=22659 mem=5.5G lam=None Gm=0.359405517578125 +[g_only s4] step 11880/12000 loss=4.3054 gn=1.02 lr=6.01e-05 tps=22659 mem=5.5G lam=None Gm=0.3626556396484375 +[g_only s4] step 11900/12000 loss=4.3132 gn=1.01 lr=6.01e-05 tps=22659 mem=5.5G lam=None Gm=0.3600616455078125 +[g_only s4] step 11920/12000 loss=4.2942 gn=0.98 lr=6.01e-05 tps=22660 mem=5.5G lam=None Gm=0.3632761637369792 +[g_only s4] step 11940/12000 loss=4.4756 gn=1.05 lr=6.00e-05 tps=22660 mem=5.5G lam=None Gm=0.3619028727213542 +[g_only s4] step 11960/12000 loss=4.3684 gn=1.06 lr=6.00e-05 tps=22659 mem=5.5G lam=None Gm=0.3579050699869792 +[g_only s4] step 11980/12000 loss=4.2970 gn=1.06 lr=6.00e-05 tps=22659 mem=5.5G lam=None Gm=0.3562774658203125 +[g_only s4] EVAL step 11999: val_loss=4.3439 val_ppl=77.01 +[init] variant=full seed=4 params=63.98M +[init] use_S=True use_G=True +[full s4] step 0/12000 loss=10.9019 gn=11.08 lr=3.00e-06 tps=6617 mem=5.2G lam=0.12692801654338837 Gm=0.49705679963032406 +[full s4] step 20/12000 loss=9.7866 gn=1.92 lr=6.30e-05 tps=15502 mem=5.7G lam=0.12692486184338728 Gm=0.4553276238342126 +[full s4] step 40/12000 loss=9.0380 gn=1.73 lr=1.23e-04 tps=15970 mem=5.7G lam=0.12685753032565117 Gm=0.4317767421404521 +[full s4] step 60/12000 loss=8.1768 gn=1.46 lr=1.83e-04 tps=16108 mem=5.7G lam=0.12675989419221878 Gm=0.43582314004500705 +[full s4] step 80/12000 loss=7.5336 gn=0.81 lr=2.43e-04 tps=16108 mem=5.7G lam=0.12662202616532645 Gm=0.4294958549241225 +[full s4] step 100/12000 loss=6.9941 gn=0.73 lr=3.03e-04 tps=16018 mem=5.7G lam=0.12649238420029482 Gm=0.4353996366262436 +[full s4] step 120/12000 loss=7.1771 gn=0.58 lr=3.63e-04 tps=16003 mem=5.7G lam=0.126394705226024 Gm=0.4306877888739109 +[full s4] step 140/12000 loss=6.8105 gn=0.59 lr=4.23e-04 tps=16134 mem=5.7G lam=0.1263536624610424 Gm=0.40672072644035023 +[full s4] step 160/12000 loss=6.7268 gn=0.61 lr=4.83e-04 tps=16337 mem=5.7G lam=0.12635492533445358 Gm=0.4088233100871245 +[full s4] step 180/12000 loss=6.7435 gn=0.60 lr=5.43e-04 tps=16530 mem=5.7G lam=0.12638081051409245 Gm=0.38508129368225735 +[full s4] step 200/12000 loss=6.6919 gn=0.51 lr=6.00e-04 tps=16692 mem=5.7G lam=0.12631004924575487 Gm=0.400378526498874 +[full s4] step 220/12000 loss=6.7361 gn=0.56 lr=6.00e-04 tps=16833 mem=5.7G lam=0.12634136465688547 Gm=0.3930956597129504 +[full s4] step 240/12000 loss=6.5209 gn=0.54 lr=6.00e-04 tps=16959 mem=5.7G lam=0.12635038855175176 Gm=0.3756640963256359 +[full s4] step 260/12000 loss=6.6180 gn=0.47 lr=6.00e-04 tps=17048 mem=5.7G lam=0.12634874321520329 Gm=0.39126606409748393 +[full s4] step 280/12000 loss=6.6694 gn=0.51 lr=6.00e-04 tps=17133 mem=5.7G lam=0.12637338073303303 Gm=0.40059853717684746 +[full s4] step 300/12000 loss=6.3403 gn=0.47 lr=6.00e-04 tps=17209 mem=5.7G lam=0.12637322520216307 Gm=0.37721312915285427 +[full s4] step 320/12000 loss=6.3943 gn=0.65 lr=6.00e-04 tps=17265 mem=5.7G lam=0.12644734481970468 Gm=0.383746216694514 +[full s4] step 340/12000 loss=6.4338 gn=0.45 lr=6.00e-04 tps=17323 mem=5.7G lam=0.12653786626954874 Gm=0.40952223042647046 +[full s4] step 360/12000 loss=6.4659 gn=0.47 lr=6.00e-04 tps=17364 mem=5.7G lam=0.12659325419614711 Gm=0.40441223109761876 +[full s4] step 380/12000 loss=6.3966 gn=0.51 lr=6.00e-04 tps=17410 mem=5.7G lam=0.12670292984694242 Gm=0.4013722079495589 +[full s4] step 400/12000 loss=6.2862 gn=0.47 lr=6.00e-04 tps=17444 mem=5.7G lam=0.1268038945272565 Gm=0.4099446510275205 +[full s4] step 420/12000 loss=6.1591 gn=0.48 lr=6.00e-04 tps=17479 mem=5.7G lam=0.1268520106871923 Gm=0.39448800310492516 +[full s4] step 440/12000 loss=6.2629 gn=0.51 lr=5.99e-04 tps=17515 mem=5.7G lam=0.1269506150856614 Gm=0.4188994790116946 +[full s4] step 460/12000 loss=6.2867 gn=0.48 lr=5.99e-04 tps=17533 mem=5.7G lam=0.12700073514133692 Gm=0.42387475073337555 +[full s4] step 480/12000 loss=6.2814 gn=0.52 lr=5.99e-04 tps=17566 mem=5.7G lam=0.12710468067477146 Gm=0.43032630036274594 +[full s4] step 500/12000 loss=6.2032 gn=0.49 lr=5.99e-04 tps=17593 mem=5.7G lam=0.12728733321030936 Gm=0.4179424283405145 +[full s4] step 520/12000 loss=6.2593 gn=0.47 lr=5.99e-04 tps=17624 mem=5.7G lam=0.1274381559342146 Gm=0.4382787458598614 +[full s4] step 540/12000 loss=6.0762 gn=0.50 lr=5.99e-04 tps=17647 mem=5.7G lam=0.12755477366348109 Gm=0.4541743149360021 +[full s4] step 560/12000 loss=6.2183 gn=0.55 lr=5.99e-04 tps=17669 mem=5.7G lam=0.12771027100582918 Gm=0.48742624123891193 +[full s4] step 580/12000 loss=6.0627 gn=0.46 lr=5.99e-04 tps=17688 mem=5.7G lam=0.12789143652965626 Gm=0.4936484396457672 +[full s4] step 600/12000 loss=6.0133 gn=0.52 lr=5.98e-04 tps=17704 mem=5.7G lam=0.12807843772073588 Gm=0.49055100108186406 +[full s4] step 620/12000 loss=6.1297 gn=0.48 lr=5.98e-04 tps=17728 mem=5.7G lam=0.12824134156107903 Gm=0.49828387548526126 +[full s4] step 640/12000 loss=6.2306 gn=0.52 lr=5.98e-04 tps=17748 mem=5.7G lam=0.1284517695506414 Gm=0.5151649701098601 +[full s4] step 660/12000 loss=6.0532 gn=0.50 lr=5.98e-04 tps=17766 mem=5.7G lam=0.12867949033776918 Gm=0.5473651749392351 +[full s4] step 680/12000 loss=6.0660 gn=0.52 lr=5.98e-04 tps=17784 mem=5.7G lam=0.1289023533463478 Gm=0.5441190103689829 +[full s4] step 700/12000 loss=5.9675 gn=0.50 lr=5.98e-04 tps=17789 mem=5.7G lam=0.1291481200605631 Gm=0.5307757084568342 +[full s4] step 720/12000 loss=5.9354 gn=0.51 lr=5.97e-04 tps=17799 mem=5.7G lam=0.1293968198200067 Gm=0.5471401537458102 +[full s4] step 740/12000 loss=6.0348 gn=0.48 lr=5.97e-04 tps=17813 mem=5.7G lam=0.1297471054519216 Gm=0.5491782960792383 +[full s4] step 760/12000 loss=6.0021 gn=0.52 lr=5.97e-04 tps=17825 mem=5.7G lam=0.1301203661908706 Gm=0.5811063498258591 +[full s4] step 780/12000 loss=6.1282 gn=0.52 lr=5.97e-04 tps=17836 mem=5.7G lam=0.1304818003748854 Gm=0.5913497967024645 +[full s4] step 800/12000 loss=5.8504 gn=0.47 lr=5.97e-04 tps=17842 mem=5.7G lam=0.13081089903910956 Gm=0.5748367843528589 +[full s4] step 820/12000 loss=5.9003 gn=0.47 lr=5.96e-04 tps=17846 mem=5.7G lam=0.13111076193551222 Gm=0.5593334771692753 +[full s4] step 840/12000 loss=5.8500 gn=0.53 lr=5.96e-04 tps=17859 mem=5.7G lam=0.13142910258223614 Gm=0.5577694786091646 +[full s4] step 860/12000 loss=5.9353 gn=0.57 lr=5.96e-04 tps=17871 mem=5.7G lam=0.1318246591836214 Gm=0.5480199865996838 +[full s4] step 880/12000 loss=5.7672 gn=0.47 lr=5.96e-04 tps=17883 mem=5.7G lam=0.1322418594111999 Gm=0.5581615579624971 +[full s4] step 900/12000 loss=5.7738 gn=0.53 lr=5.95e-04 tps=17894 mem=5.7G lam=0.13264422677457333 Gm=0.5846103429794312 +[full s4] step 920/12000 loss=5.8391 gn=0.50 lr=5.95e-04 tps=17908 mem=5.7G lam=0.13303990351657072 Gm=0.5821945940454801 +[full s4] step 940/12000 loss=5.7680 gn=0.56 lr=5.95e-04 tps=17915 mem=5.7G lam=0.133400014291207 Gm=0.5585003029555082 +[full s4] step 960/12000 loss=5.8020 gn=0.52 lr=5.94e-04 tps=17925 mem=5.7G lam=0.13383774862935147 Gm=0.5636651776731014 +[full s4] step 980/12000 loss=5.7962 gn=0.54 lr=5.94e-04 tps=17935 mem=5.7G lam=0.13432009921719631 Gm=0.5589304659515619 +[full s4] step 1000/12000 loss=5.7175 gn=0.58 lr=5.94e-04 tps=17945 mem=5.7G lam=0.1346960850059986 Gm=0.5498163489003977 +[full s4] EVAL step 1000: val_loss=5.7764 val_ppl=322.59 +[full s4] step 1020/12000 loss=5.7660 gn=0.51 lr=5.94e-04 tps=17895 mem=5.7G lam=0.13511213505019745 Gm=0.5540812549491724 +[full s4] step 1040/12000 loss=5.7096 gn=0.51 lr=5.93e-04 tps=17904 mem=5.7G lam=0.13551606175800165 Gm=0.5549819283187389 +[full s4] step 1060/12000 loss=5.6119 gn=0.54 lr=5.93e-04 tps=17913 mem=5.7G lam=0.13597818495084843 Gm=0.5292880274355412 +[full s4] step 1080/12000 loss=5.6602 gn=0.52 lr=5.93e-04 tps=17918 mem=5.7G lam=0.13638119989385208 Gm=0.5479242789248625 +[full s4] step 1100/12000 loss=5.8142 gn=0.52 lr=5.92e-04 tps=17925 mem=5.7G lam=0.1368334861472249 Gm=0.5517748631536961 +[full s4] step 1120/12000 loss=5.7619 gn=0.49 lr=5.92e-04 tps=17931 mem=5.7G lam=0.137236377534767 Gm=0.554235452786088 +[full s4] step 1140/12000 loss=5.5726 gn=0.53 lr=5.92e-04 tps=17937 mem=5.7G lam=0.1376870290065805 Gm=0.5445777500669161 +[full s4] step 1160/12000 loss=5.6135 gn=0.51 lr=5.91e-04 tps=17941 mem=5.7G lam=0.1381812443335851 Gm=0.5404178481549025 +[full s4] step 1180/12000 loss=5.6977 gn=0.53 lr=5.91e-04 tps=17947 mem=5.7G lam=0.13866149572034678 Gm=0.528130225216349 +[full s4] step 1200/12000 loss=5.6702 gn=0.52 lr=5.90e-04 tps=17951 mem=5.7G lam=0.13909149138877788 Gm=0.5411007522294918 +[full s4] step 1220/12000 loss=5.8802 gn=0.52 lr=5.90e-04 tps=17957 mem=5.7G lam=0.13950619287788868 Gm=0.5300985214610895 +[full s4] step 1240/12000 loss=5.5353 gn=0.51 lr=5.90e-04 tps=17964 mem=5.7G lam=0.13996233884245157 Gm=0.5317762636890014 +[full s4] step 1260/12000 loss=5.5595 gn=0.56 lr=5.89e-04 tps=17967 mem=5.7G lam=0.14037496565530697 Gm=0.5282444568971792 +[full s4] step 1280/12000 loss=5.4400 gn=0.53 lr=5.89e-04 tps=17976 mem=5.7G lam=0.1407824136937658 Gm=0.5381634713461002 +[full s4] step 1300/12000 loss=5.7547 gn=0.58 lr=5.89e-04 tps=17983 mem=5.7G lam=0.14121810408929983 Gm=0.5288880628844103 +[full s4] step 1320/12000 loss=5.5190 gn=0.55 lr=5.88e-04 tps=17989 mem=5.7G lam=0.1417385016878446 Gm=0.536684749647975 +[full s4] step 1340/12000 loss=5.4913 gn=0.53 lr=5.88e-04 tps=17996 mem=5.7G lam=0.14229163092871508 Gm=0.5238935432086388 +[full s4] step 1360/12000 loss=5.5927 gn=0.54 lr=5.87e-04 tps=18003 mem=5.7G lam=0.1427511948471268 Gm=0.5289315891762575 +[full s4] step 1380/12000 loss=5.5481 gn=0.58 lr=5.87e-04 tps=18011 mem=5.7G lam=0.14321863558143377 Gm=0.5182925295084715 +[full s4] step 1400/12000 loss=5.4310 gn=0.58 lr=5.86e-04 tps=18018 mem=5.7G lam=0.14365649906297526 Gm=0.5173999114582936 +[full s4] step 1420/12000 loss=5.5415 gn=0.55 lr=5.86e-04 tps=18025 mem=5.7G lam=0.1440195751686891 Gm=0.5070090169707934 +[full s4] step 1440/12000 loss=5.4904 gn=0.63 lr=5.85e-04 tps=18033 mem=5.7G lam=0.14447169036914906 Gm=0.5108577590435743 +[full s4] step 1460/12000 loss=5.4062 gn=0.54 lr=5.85e-04 tps=18039 mem=5.7G lam=0.14492431903878847 Gm=0.500774477298061 +[full s4] step 1480/12000 loss=5.4484 gn=0.56 lr=5.84e-04 tps=18047 mem=5.7G lam=0.1454170017192761 Gm=0.4957709365213911 +[full s4] step 1500/12000 loss=5.4825 gn=0.56 lr=5.84e-04 tps=18054 mem=5.7G lam=0.14591137195626894 Gm=0.5027264775708318 +[full s4] step 1520/12000 loss=5.3369 gn=0.58 lr=5.83e-04 tps=18061 mem=5.7G lam=0.14632880128920078 Gm=0.4997792715827624 +[full s4] step 1540/12000 loss=5.4203 gn=0.58 lr=5.83e-04 tps=18068 mem=5.7G lam=0.1468188784395655 Gm=0.4947134479880333 +[full s4] step 1560/12000 loss=5.4802 gn=0.56 lr=5.82e-04 tps=18076 mem=5.7G lam=0.14729215142627558 Gm=0.5041752470036348 +[full s4] step 1580/12000 loss=5.5544 gn=0.56 lr=5.82e-04 tps=18081 mem=5.7G lam=0.14775694658358893 Gm=0.48459414951503277 +[full s4] step 1600/12000 loss=5.2612 gn=0.61 lr=5.81e-04 tps=18086 mem=5.7G lam=0.14829887946446738 Gm=0.4895890870441993 +[full s4] step 1620/12000 loss=5.3872 gn=0.60 lr=5.81e-04 tps=18092 mem=5.7G lam=0.14880152978003025 Gm=0.49519906224062044 +[full s4] step 1640/12000 loss=5.4171 gn=0.61 lr=5.80e-04 tps=18098 mem=5.7G lam=0.14926041848957539 Gm=0.48630159720778465 +[full s4] step 1660/12000 loss=5.3567 gn=0.54 lr=5.80e-04 tps=18104 mem=5.7G lam=0.14977348254372677 Gm=0.48405859308938187 +[full s4] step 1680/12000 loss=5.2418 gn=0.54 lr=5.79e-04 tps=18108 mem=5.7G lam=0.15024863028277954 Gm=0.48323536819467944 +[full s4] step 1700/12000 loss=5.2985 gn=0.57 lr=5.79e-04 tps=18114 mem=5.7G lam=0.1507082600146532 Gm=0.47795741943021613 +[full s4] step 1720/12000 loss=5.4111 gn=0.62 lr=5.78e-04 tps=18119 mem=5.7G lam=0.151184001006186 Gm=0.4829820667703946 +[full s4] step 1740/12000 loss=5.4629 gn=0.60 lr=5.78e-04 tps=18123 mem=5.7G lam=0.15166255397101244 Gm=0.4837846638013919 +[full s4] step 1760/12000 loss=5.2999 gn=0.62 lr=5.77e-04 tps=18127 mem=5.7G lam=0.15222225400308767 Gm=0.4825120819732547 +[full s4] step 1780/12000 loss=5.4571 gn=0.58 lr=5.76e-04 tps=18132 mem=5.7G lam=0.1527447026843826 Gm=0.4672636315226555 +[full s4] step 1800/12000 loss=5.2077 gn=0.62 lr=5.76e-04 tps=18136 mem=5.7G lam=0.1532875563328465 Gm=0.4682622719556093 +[full s4] step 1820/12000 loss=5.2891 gn=0.56 lr=5.75e-04 tps=18139 mem=5.7G lam=0.1537786809106668 Gm=0.4795135458310445 +[full s4] step 1840/12000 loss=5.3389 gn=0.61 lr=5.75e-04 tps=18144 mem=5.7G lam=0.15427333333839974 Gm=0.46935582409302395 +[full s4] step 1860/12000 loss=5.5364 gn=0.59 lr=5.74e-04 tps=18148 mem=5.7G lam=0.15482969333728155 Gm=0.4691696744412184 +[full s4] step 1880/12000 loss=5.3982 gn=0.58 lr=5.73e-04 tps=18154 mem=5.7G lam=0.15534744380662838 Gm=0.47755700380851823 +[full s4] step 1900/12000 loss=5.2994 gn=0.69 lr=5.73e-04 tps=18158 mem=5.7G lam=0.15589065880825123 Gm=0.469314473370711 +[full s4] step 1920/12000 loss=5.3205 gn=0.60 lr=5.72e-04 tps=18158 mem=5.7G lam=0.1564192163447539 Gm=0.45790130520860356 +[full s4] step 1940/12000 loss=5.3166 gn=0.56 lr=5.72e-04 tps=18153 mem=5.7G lam=0.15701258275657892 Gm=0.4565594106291731 +[full s4] step 1960/12000 loss=5.4291 gn=0.58 lr=5.71e-04 tps=18157 mem=5.7G lam=0.157519092472891 Gm=0.46012505516409874 +[full s4] step 1980/12000 loss=5.2032 gn=0.55 lr=5.70e-04 tps=18159 mem=5.7G lam=0.15799559094011784 Gm=0.45588354859501123 +[full s4] step 2000/12000 loss=5.2721 gn=0.62 lr=5.70e-04 tps=18164 mem=5.7G lam=0.15849375197043022 Gm=0.45250152113536996 +[full s4] EVAL step 2000: val_loss=5.3038 val_ppl=201.10 +[full s4] step 2020/12000 loss=5.1998 gn=0.56 lr=5.69e-04 tps=18133 mem=5.7G lam=0.1590616898611188 Gm=0.4552545587842663 +[full s4] step 2040/12000 loss=5.3875 gn=0.63 lr=5.68e-04 tps=18138 mem=5.7G lam=0.15955673903226852 Gm=0.4415124723066886 +[full s4] step 2060/12000 loss=5.2971 gn=0.58 lr=5.68e-04 tps=18141 mem=5.7G lam=0.16005013200143972 Gm=0.45267399804045755 +[full s4] step 2080/12000 loss=5.2043 gn=0.62 lr=5.67e-04 tps=18146 mem=5.7G lam=0.16062999547769627 Gm=0.4425801398853461 +[full s4] step 2100/12000 loss=5.1318 gn=0.78 lr=5.66e-04 tps=18151 mem=5.7G lam=0.16112989280372858 Gm=0.45081614376977086 +[full s4] step 2120/12000 loss=5.2396 gn=0.63 lr=5.65e-04 tps=18155 mem=5.7G lam=0.16166726561884084 Gm=0.434432848667105 +[full s4] step 2140/12000 loss=5.1872 gn=0.60 lr=5.65e-04 tps=18158 mem=5.7G lam=0.1622170489281416 Gm=0.43607635827114183 +[full s4] step 2160/12000 loss=5.1599 gn=0.57 lr=5.64e-04 tps=18162 mem=5.7G lam=0.16275733895599842 Gm=0.45487274198482436 +[full s4] step 2180/12000 loss=5.2969 gn=0.62 lr=5.63e-04 tps=18165 mem=5.7G lam=0.1632342105731368 Gm=0.44051054616769153 +[full s4] step 2200/12000 loss=5.1926 gn=0.71 lr=5.63e-04 tps=18167 mem=5.7G lam=0.1638197060674429 Gm=0.45547968801110983 +[full s4] step 2220/12000 loss=5.1365 gn=0.64 lr=5.62e-04 tps=18170 mem=5.7G lam=0.16435561360170445 Gm=0.4370485738230248 +[full s4] step 2240/12000 loss=5.0734 gn=0.65 lr=5.61e-04 tps=18174 mem=5.7G lam=0.1648531397804618 Gm=0.44154948415234685 +[full s4] step 2260/12000 loss=5.2456 gn=0.64 lr=5.60e-04 tps=18176 mem=5.7G lam=0.16537275413672128 Gm=0.4294685274362564 +[full s4] step 2280/12000 loss=5.1219 gn=0.63 lr=5.60e-04 tps=18179 mem=5.7G lam=0.16597962907205024 Gm=0.43983693902070325 +[full s4] step 2300/12000 loss=5.1185 gn=0.67 lr=5.59e-04 tps=18182 mem=5.7G lam=0.16647502624740204 Gm=0.4488981816296776 +[full s4] step 2320/12000 loss=4.9937 gn=0.63 lr=5.58e-04 tps=18185 mem=5.7G lam=0.16704025057454905 Gm=0.431168085274597 +[full s4] step 2340/12000 loss=5.1730 gn=0.65 lr=5.57e-04 tps=18187 mem=5.7G lam=0.1675521352638801 Gm=0.432249682645003 +[full s4] step 2360/12000 loss=4.9069 gn=0.86 lr=5.57e-04 tps=18189 mem=5.7G lam=0.1681030976275603 Gm=0.4428117916298409 +[full s4] step 2380/12000 loss=5.0800 gn=0.64 lr=5.56e-04 tps=18192 mem=5.7G lam=0.16861153487116098 Gm=0.43597085820510983 +[full s4] step 2400/12000 loss=5.0732 gn=0.70 lr=5.55e-04 tps=18195 mem=5.7G lam=0.16914716052512327 Gm=0.43587293770785135 +[full s4] step 2420/12000 loss=5.0118 gn=0.64 lr=5.54e-04 tps=18197 mem=5.7G lam=0.16977064808209738 Gm=0.42831080096463364 +[full s4] step 2440/12000 loss=5.0621 gn=0.66 lr=5.53e-04 tps=18200 mem=5.7G lam=0.17031096884359917 Gm=0.4288631229040523 +[full s4] step 2460/12000 loss=5.0446 gn=0.70 lr=5.53e-04 tps=18202 mem=5.7G lam=0.1709296703338623 Gm=0.42226751257355016 +[full s4] step 2480/12000 loss=5.1667 gn=0.84 lr=5.52e-04 tps=18204 mem=5.7G lam=0.17144799449791512 Gm=0.42335283011198044 +[full s4] step 2500/12000 loss=5.0979 gn=0.66 lr=5.51e-04 tps=18206 mem=5.7G lam=0.17200684019674858 Gm=0.41905749527116615 +[full s4] step 2520/12000 loss=4.9820 gn=0.65 lr=5.50e-04 tps=18209 mem=5.7G lam=0.17253927420824766 Gm=0.422932642356803 +[full s4] step 2540/12000 loss=5.0173 gn=0.67 lr=5.49e-04 tps=18212 mem=5.7G lam=0.17308621325840554 Gm=0.42393536074087024 +[full s4] step 2560/12000 loss=5.1691 gn=0.63 lr=5.48e-04 tps=18215 mem=5.7G lam=0.17360110425700745 Gm=0.42018990172073245 +[full s4] step 2580/12000 loss=5.0192 gn=0.68 lr=5.48e-04 tps=18218 mem=5.7G lam=0.17409977378944555 Gm=0.414114067175736 +[full s4] step 2600/12000 loss=5.0040 gn=0.65 lr=5.47e-04 tps=18221 mem=5.7G lam=0.17465302037696043 Gm=0.4249946547982593 +[full s4] step 2620/12000 loss=5.0535 gn=0.63 lr=5.46e-04 tps=18224 mem=5.7G lam=0.17518282247086367 Gm=0.41459306205312413 +[full s4] step 2640/12000 loss=5.0280 gn=0.71 lr=5.45e-04 tps=18226 mem=5.7G lam=0.17561589553952217 Gm=0.40045891515910625 +[full s4] step 2660/12000 loss=5.1734 gn=0.61 lr=5.44e-04 tps=18229 mem=5.7G lam=0.17612195728967586 Gm=0.42116983653977513 +[full s4] step 2680/12000 loss=4.9011 gn=0.66 lr=5.43e-04 tps=18230 mem=5.7G lam=0.1767290337011218 Gm=0.41566596971824765 +[full s4] step 2700/12000 loss=5.1291 gn=0.61 lr=5.42e-04 tps=18233 mem=5.7G lam=0.17726234843333563 Gm=0.4082351458879809 +[full s4] step 2720/12000 loss=5.1121 gn=0.62 lr=5.41e-04 tps=18236 mem=5.7G lam=0.17775640667726597 Gm=0.4193593015273412 +[full s4] step 2740/12000 loss=5.0769 gn=0.66 lr=5.41e-04 tps=18238 mem=5.7G lam=0.1782360104843974 Gm=0.4178630290552974 +[full s4] step 2760/12000 loss=5.0109 gn=0.68 lr=5.40e-04 tps=18238 mem=5.7G lam=0.17878720288475355 Gm=0.40693980253612 +[full s4] step 2780/12000 loss=5.0611 gn=0.76 lr=5.39e-04 tps=18239 mem=5.7G lam=0.17930879164487123 Gm=0.41738420833523077 +[full s4] step 2800/12000 loss=4.9860 gn=0.66 lr=5.38e-04 tps=18241 mem=5.7G lam=0.17976458619038263 Gm=0.40641565434634686 +[full s4] step 2820/12000 loss=4.9136 gn=0.67 lr=5.37e-04 tps=18243 mem=5.7G lam=0.18034270188460746 Gm=0.4083407069556415 +[full s4] step 2840/12000 loss=4.9057 gn=0.64 lr=5.36e-04 tps=18246 mem=5.7G lam=0.18086891124645868 Gm=0.4038863863485555 +[full s4] step 2860/12000 loss=5.0142 gn=0.61 lr=5.35e-04 tps=18248 mem=5.7G lam=0.18138409405946732 Gm=0.40856135729700327 +[full s4] step 2880/12000 loss=4.9951 gn=0.65 lr=5.34e-04 tps=18250 mem=5.7G lam=0.18189018654326597 Gm=0.4082016162574291 +[full s4] step 2900/12000 loss=4.9989 gn=0.69 lr=5.33e-04 tps=18252 mem=5.7G lam=0.18243986709664264 Gm=0.4087769103546937 +[full s4] step 2920/12000 loss=5.0316 gn=0.64 lr=5.32e-04 tps=18254 mem=5.7G lam=0.18294594685236612 Gm=0.40686397192378837 +[full s4] step 2940/12000 loss=4.9719 gn=0.67 lr=5.31e-04 tps=18256 mem=5.7G lam=0.18348023730019727 Gm=0.3998545231297612 +[full s4] step 2960/12000 loss=4.9779 gn=0.67 lr=5.30e-04 tps=18258 mem=5.7G lam=0.1839823660751184 Gm=0.40471926362564165 +[full s4] step 2980/12000 loss=4.9714 gn=0.73 lr=5.29e-04 tps=18261 mem=5.7G lam=0.18449570704251528 Gm=0.4136940537331005 +[full s4] step 3000/12000 loss=4.9547 gn=0.74 lr=5.28e-04 tps=18262 mem=5.7G lam=0.18499127123504877 Gm=0.4013473060913384 +[full s4] EVAL step 3000: val_loss=4.9694 val_ppl=143.95 +[full s4] step 3020/12000 loss=4.8610 gn=0.71 lr=5.27e-04 tps=18243 mem=5.7G lam=0.18554292681316534 Gm=0.4113841637348135 +[full s4] step 3040/12000 loss=4.8491 gn=0.79 lr=5.26e-04 tps=18243 mem=5.7G lam=0.18602137981603542 Gm=0.40322845925887424 +[full s4] step 3060/12000 loss=5.1203 gn=0.76 lr=5.25e-04 tps=18245 mem=5.7G lam=0.18650012680639824 Gm=0.4081446124861638 +[full s4] step 3080/12000 loss=4.9841 gn=0.67 lr=5.24e-04 tps=18248 mem=5.7G lam=0.1870460631325841 Gm=0.40182723058387637 +[full s4] step 3100/12000 loss=5.0287 gn=0.76 lr=5.23e-04 tps=18249 mem=5.7G lam=0.18771144033720097 Gm=0.40480426761011284 +[full s4] step 3120/12000 loss=4.8807 gn=0.67 lr=5.22e-04 tps=18251 mem=5.7G lam=0.188196227264901 Gm=0.40665279763440293 +[full s4] step 3140/12000 loss=4.8805 gn=0.72 lr=5.21e-04 tps=18253 mem=5.7G lam=0.1886526901895801 Gm=0.40034463442862034 +[full s4] step 3160/12000 loss=4.9939 gn=0.68 lr=5.20e-04 tps=18255 mem=5.7G lam=0.18926715602477392 Gm=0.4086717843698959 +[full s4] step 3180/12000 loss=4.8731 gn=0.69 lr=5.19e-04 tps=18258 mem=5.7G lam=0.18983030350257954 Gm=0.394584809585164 +[full s4] step 3200/12000 loss=4.9171 gn=0.68 lr=5.18e-04 tps=18259 mem=5.7G lam=0.19045191258192062 Gm=0.39557831765462953 +[full s4] step 3220/12000 loss=5.0601 gn=0.70 lr=5.17e-04 tps=18261 mem=5.7G lam=0.19100599270313978 Gm=0.4029040946625173 +[full s4] step 3240/12000 loss=4.9347 gn=0.64 lr=5.16e-04 tps=18263 mem=5.7G lam=0.19167656047890583 Gm=0.39778519809866947 +[full s4] step 3260/12000 loss=4.9405 gn=0.71 lr=5.15e-04 tps=18265 mem=5.7G lam=0.19222778206070265 Gm=0.39729892648756504 +[full s4] step 3280/12000 loss=4.8380 gn=0.72 lr=5.14e-04 tps=18267 mem=5.7G lam=0.192804009343187 Gm=0.4013814479112625 +[full s4] step 3300/12000 loss=4.9390 gn=0.70 lr=5.13e-04 tps=18268 mem=5.7G lam=0.19334296075006327 Gm=0.39750910953929025 +[full s4] step 3320/12000 loss=4.8049 gn=0.68 lr=5.12e-04 tps=18271 mem=5.7G lam=0.19394026417285204 Gm=0.39434102084487677 +[full s4] step 3340/12000 loss=4.9297 gn=0.78 lr=5.11e-04 tps=18272 mem=5.7G lam=0.19449275576819977 Gm=0.3952434839059909 +[full s4] step 3360/12000 loss=4.8828 gn=0.70 lr=5.10e-04 tps=18274 mem=5.7G lam=0.19503929683317742 Gm=0.4020528110365073 +[full s4] step 3380/12000 loss=4.7971 gn=0.69 lr=5.09e-04 tps=18276 mem=5.7G lam=0.19556872298320135 Gm=0.39762444406126934 +[full s4] step 3400/12000 loss=4.8106 gn=0.74 lr=5.08e-04 tps=18277 mem=5.7G lam=0.19608369811127582 Gm=0.39049244082222384 +[full s4] step 3420/12000 loss=4.8245 gn=0.76 lr=5.07e-04 tps=18279 mem=5.7G lam=0.196524474148949 Gm=0.3920802442977826 +[full s4] step 3440/12000 loss=4.7646 gn=0.76 lr=5.06e-04 tps=18280 mem=5.7G lam=0.19707401748746634 Gm=0.3907691747881472 +[full s4] step 3460/12000 loss=4.7653 gn=0.72 lr=5.05e-04 tps=18281 mem=5.7G lam=0.19763365108519793 Gm=0.4012765483930707 +[full s4] step 3480/12000 loss=5.0040 gn=0.69 lr=5.03e-04 tps=18283 mem=5.7G lam=0.19814186884711185 Gm=0.3972883661432813 +[full s4] step 3500/12000 loss=4.9257 gn=0.69 lr=5.02e-04 tps=18284 mem=5.7G lam=0.19863510535409054 Gm=0.39292741008102894 +[full s4] step 3520/12000 loss=4.7473 gn=0.73 lr=5.01e-04 tps=18285 mem=5.7G lam=0.19910092682888111 Gm=0.38777242104212445 +[full s4] step 3540/12000 loss=5.0462 gn=0.74 lr=5.00e-04 tps=18287 mem=5.7G lam=0.1996177090331912 Gm=0.3928560757388671 +[full s4] step 3560/12000 loss=4.8974 gn=0.68 lr=4.99e-04 tps=18289 mem=5.7G lam=0.2001770973826448 Gm=0.39261298812925816 +[full s4] step 3580/12000 loss=5.1191 gn=0.72 lr=4.98e-04 tps=18290 mem=5.7G lam=0.20078064315021038 Gm=0.39639657037332654 +[full s4] step 3600/12000 loss=4.8797 gn=0.70 lr=4.97e-04 tps=18292 mem=5.7G lam=0.20130669015149275 Gm=0.3940795712793867 +[full s4] step 3620/12000 loss=4.7344 gn=0.70 lr=4.96e-04 tps=18293 mem=5.7G lam=0.20188738591969013 Gm=0.38934176430727047 +[full s4] step 3640/12000 loss=4.8665 gn=0.74 lr=4.94e-04 tps=18295 mem=5.7G lam=0.20240317502369484 Gm=0.38745234347879887 +[full s4] step 3660/12000 loss=4.7917 gn=0.98 lr=4.93e-04 tps=18297 mem=5.7G lam=0.20301236057033142 Gm=0.3943103381122152 +[full s4] step 3680/12000 loss=4.9072 gn=0.73 lr=4.92e-04 tps=18299 mem=5.7G lam=0.20357152509192625 Gm=0.38405395167258877 +[full s4] step 3700/12000 loss=4.8712 gn=0.76 lr=4.91e-04 tps=18301 mem=5.7G lam=0.20402023692925772 Gm=0.39341605245135725 +[full s4] step 3720/12000 loss=4.7934 gn=0.74 lr=4.90e-04 tps=18302 mem=5.7G lam=0.2044753565763434 Gm=0.39481373100231093 +[full s4] step 3740/12000 loss=4.6164 gn=0.72 lr=4.89e-04 tps=18303 mem=5.7G lam=0.20487616521616778 Gm=0.3878398030065 +[full s4] step 3760/12000 loss=4.7477 gn=0.77 lr=4.88e-04 tps=18305 mem=5.7G lam=0.20537519361823797 Gm=0.38938867424925167 +[full s4] step 3780/12000 loss=4.7581 gn=0.77 lr=4.86e-04 tps=18305 mem=5.7G lam=0.20594936050474644 Gm=0.39192002212318283 +[full s4] step 3800/12000 loss=4.8244 gn=0.76 lr=4.85e-04 tps=18307 mem=5.7G lam=0.20652295059214035 Gm=0.3849789120722562 +[full s4] step 3820/12000 loss=4.7969 gn=0.76 lr=4.84e-04 tps=18309 mem=5.7G lam=0.207056429858009 Gm=0.38775424457465607 +[full s4] step 3840/12000 loss=4.5979 gn=0.69 lr=4.83e-04 tps=18310 mem=5.7G lam=0.20758097246289253 Gm=0.3837149573955685 +[full s4] step 3860/12000 loss=4.8910 gn=0.76 lr=4.82e-04 tps=18312 mem=5.7G lam=0.2081136191263795 Gm=0.38992374188577134 +[full s4] step 3880/12000 loss=4.8147 gn=0.69 lr=4.80e-04 tps=18313 mem=5.7G lam=0.20866297433773676 Gm=0.38782511341075104 +[full s4] step 3900/12000 loss=4.8528 gn=0.72 lr=4.79e-04 tps=18314 mem=5.7G lam=0.2091416676218311 Gm=0.3915074047787736 +[full s4] step 3920/12000 loss=4.8479 gn=0.72 lr=4.78e-04 tps=18315 mem=5.7G lam=0.20965079497545958 Gm=0.3919199933297932 +[full s4] step 3940/12000 loss=4.7553 gn=0.77 lr=4.77e-04 tps=18316 mem=5.7G lam=0.21021234430372715 Gm=0.38708692028497654 +[full s4] step 3960/12000 loss=4.9313 gn=0.93 lr=4.76e-04 tps=18318 mem=5.7G lam=0.2106619564195474 Gm=0.3804688314752032 +[full s4] step 3980/12000 loss=4.7710 gn=0.75 lr=4.74e-04 tps=18319 mem=5.7G lam=0.21116971597075462 Gm=0.3778089371044189 +[full s4] step 4000/12000 loss=4.7106 gn=0.68 lr=4.73e-04 tps=18320 mem=5.7G lam=0.21163225763787827 Gm=0.3721036963785688 +[full s4] EVAL step 4000: val_loss=4.8825 val_ppl=131.96 +[full s4] step 4020/12000 loss=4.6544 gn=0.76 lr=4.72e-04 tps=18304 mem=5.7G lam=0.21211693404863277 Gm=0.38415986206382513 +[full s4] step 4040/12000 loss=4.7784 gn=0.78 lr=4.71e-04 tps=18305 mem=5.7G lam=0.21262797713279724 Gm=0.37267336063086987 +[full s4] step 4060/12000 loss=4.7658 gn=0.75 lr=4.70e-04 tps=18306 mem=5.7G lam=0.2131420054162542 Gm=0.3757492514171948 +[full s4] step 4080/12000 loss=4.7826 gn=0.77 lr=4.68e-04 tps=18306 mem=5.7G lam=0.2136875381693244 Gm=0.38047011600186426 +[full s4] step 4100/12000 loss=4.8304 gn=0.72 lr=4.67e-04 tps=18307 mem=5.7G lam=0.21415663696825504 Gm=0.3848730355966836 +[full s4] step 4120/12000 loss=4.7914 gn=0.74 lr=4.66e-04 tps=18309 mem=5.7G lam=0.2147201191013058 Gm=0.3816737122057627 +[full s4] step 4140/12000 loss=4.8105 gn=0.80 lr=4.65e-04 tps=18310 mem=5.7G lam=0.21524732622007528 Gm=0.37305716052651405 +[full s4] step 4160/12000 loss=4.8900 gn=0.84 lr=4.63e-04 tps=18313 mem=5.7G lam=0.21570021690179905 Gm=0.3762646787023793 +[full s4] step 4180/12000 loss=4.7950 gn=0.71 lr=4.62e-04 tps=18314 mem=5.7G lam=0.21615707750121751 Gm=0.3863213734390835 +[full s4] step 4200/12000 loss=4.8023 gn=0.77 lr=4.61e-04 tps=18316 mem=5.7G lam=0.21662112108121315 Gm=0.3795070277216534 +[full s4] step 4220/12000 loss=4.7425 gn=0.71 lr=4.60e-04 tps=18317 mem=5.7G lam=0.21706727550675473 Gm=0.3678625469716887 +[full s4] step 4240/12000 loss=4.6535 gn=0.74 lr=4.58e-04 tps=18319 mem=5.7G lam=0.21753226779401302 Gm=0.37695939869930345 +[full s4] step 4260/12000 loss=4.7385 gn=0.77 lr=4.57e-04 tps=18320 mem=5.7G lam=0.21805682281653085 Gm=0.3748184246942401 +[full s4] step 4280/12000 loss=4.8638 gn=0.86 lr=4.56e-04 tps=18322 mem=5.7G lam=0.21858093328773975 Gm=0.3801777984481305 +[full s4] step 4300/12000 loss=4.8819 gn=0.74 lr=4.54e-04 tps=18323 mem=5.7G lam=0.21913841273635626 Gm=0.37636969742986065 +[full s4] step 4320/12000 loss=4.6934 gn=0.88 lr=4.53e-04 tps=18324 mem=5.7G lam=0.21963754141082367 Gm=0.37204661468664807 +[full s4] step 4340/12000 loss=4.9264 gn=0.72 lr=4.52e-04 tps=18325 mem=5.7G lam=0.22017129758993784 Gm=0.37679766638514894 +[full s4] step 4360/12000 loss=4.7204 gn=0.83 lr=4.51e-04 tps=18326 mem=5.7G lam=0.2207070253789425 Gm=0.3793068192123125 +[full s4] step 4380/12000 loss=4.4376 gn=0.76 lr=4.49e-04 tps=18327 mem=5.7G lam=0.22126330342143774 Gm=0.3744656940301259 +[full s4] step 4400/12000 loss=4.9046 gn=0.77 lr=4.48e-04 tps=18329 mem=5.7G lam=0.22184408673395714 Gm=0.3763333691749722 +[full s4] step 4420/12000 loss=4.7163 gn=0.69 lr=4.47e-04 tps=18330 mem=5.7G lam=0.22239872658004364 Gm=0.3712809159575651 +[full s4] step 4440/12000 loss=4.9093 gn=0.80 lr=4.45e-04 tps=18331 mem=5.7G lam=0.2228445258612434 Gm=0.38024752323205274 +[full s4] step 4460/12000 loss=4.6145 gn=0.81 lr=4.44e-04 tps=18331 mem=5.7G lam=0.22330477833747864 Gm=0.3779793406526248 +[full s4] step 4480/12000 loss=4.7401 gn=0.75 lr=4.43e-04 tps=18332 mem=5.7G lam=0.22387578307340542 Gm=0.3764781868085265 +[full s4] step 4500/12000 loss=4.8269 gn=0.71 lr=4.42e-04 tps=18333 mem=5.7G lam=0.2243939656764269 Gm=0.368300042891254 +[full s4] step 4520/12000 loss=4.5052 gn=0.79 lr=4.40e-04 tps=18334 mem=5.7G lam=0.22492891177535057 Gm=0.3792603921610862 +[full s4] step 4540/12000 loss=4.6472 gn=0.69 lr=4.39e-04 tps=18335 mem=5.7G lam=0.22541071816037098 Gm=0.37404131004586816 +[full s4] step 4560/12000 loss=4.6427 gn=0.75 lr=4.38e-04 tps=18334 mem=5.7G lam=0.2258819549654921 Gm=0.3724429303159316 +[full s4] step 4580/12000 loss=4.6935 gn=0.76 lr=4.36e-04 tps=18335 mem=5.7G lam=0.2263390471537908 Gm=0.370774118307357 +[full s4] step 4600/12000 loss=4.5795 gn=0.85 lr=4.35e-04 tps=18336 mem=5.7G lam=0.22683018011351427 Gm=0.3693339053230981 +[full s4] step 4620/12000 loss=4.6842 gn=0.79 lr=4.34e-04 tps=18337 mem=5.7G lam=0.22729485481977463 Gm=0.3688609595410526 +[full s4] step 4640/12000 loss=4.7405 gn=0.82 lr=4.32e-04 tps=18338 mem=5.7G lam=0.22790196320662895 Gm=0.3723896581990023 +[full s4] step 4660/12000 loss=4.6589 gn=0.81 lr=4.31e-04 tps=18339 mem=5.7G lam=0.22828997081766525 Gm=0.36876893835142255 +[full s4] step 4680/12000 loss=4.6213 gn=0.78 lr=4.30e-04 tps=18340 mem=5.7G lam=0.22888455602029958 Gm=0.3721026332738499 +[full s4] step 4700/12000 loss=4.6593 gn=0.77 lr=4.28e-04 tps=18341 mem=5.7G lam=0.2294286210089922 Gm=0.36960707247878116 +[full s4] step 4720/12000 loss=4.6658 gn=0.83 lr=4.27e-04 tps=18342 mem=5.7G lam=0.22990209857622781 Gm=0.37146521809821326 +[full s4] step 4740/12000 loss=4.6991 gn=0.78 lr=4.26e-04 tps=18344 mem=5.7G lam=0.23036170502503714 Gm=0.3650000742636621 +[full s4] step 4760/12000 loss=4.7599 gn=0.77 lr=4.24e-04 tps=18345 mem=5.7G lam=0.23082886760433516 Gm=0.36394425372903544 +[full s4] step 4780/12000 loss=4.6524 gn=0.85 lr=4.23e-04 tps=18345 mem=5.7G lam=0.23133567906916142 Gm=0.3761297492310405 +[full s4] step 4800/12000 loss=4.8262 gn=0.75 lr=4.22e-04 tps=18347 mem=5.7G lam=0.2318769305323561 Gm=0.37248026533052325 +[full s4] step 4820/12000 loss=4.6316 gn=0.80 lr=4.20e-04 tps=18348 mem=5.7G lam=0.23229312368979058 Gm=0.3666700889977316 +[full s4] step 4840/12000 loss=4.5453 gn=0.77 lr=4.19e-04 tps=18349 mem=5.7G lam=0.23275776424755654 Gm=0.36984084694025415 +[full s4] step 4860/12000 loss=4.6248 gn=0.77 lr=4.18e-04 tps=18350 mem=5.7G lam=0.23325791272024313 Gm=0.36500615378220874 +[full s4] step 4880/12000 loss=4.6120 gn=0.75 lr=4.16e-04 tps=18351 mem=5.7G lam=0.23370311129838228 Gm=0.3674835685330133 +[full s4] step 4900/12000 loss=4.6888 gn=0.79 lr=4.15e-04 tps=18352 mem=5.7G lam=0.23415747315933308 Gm=0.3680456392467022 +[full s4] step 4920/12000 loss=4.6679 gn=0.83 lr=4.13e-04 tps=18353 mem=5.7G lam=0.23462593959023556 Gm=0.37244097671161097 +[full s4] step 4940/12000 loss=4.6209 gn=0.74 lr=4.12e-04 tps=18353 mem=5.7G lam=0.2350979701926311 Gm=0.369708958780393 +[full s4] step 4960/12000 loss=4.6881 gn=0.75 lr=4.11e-04 tps=18354 mem=5.7G lam=0.2356088658173879 Gm=0.36593357799574733 +[full s4] step 4980/12000 loss=4.6285 gn=0.78 lr=4.09e-04 tps=18354 mem=5.7G lam=0.23611569063117108 Gm=0.37179499957710505 +[full s4] step 5000/12000 loss=4.7936 gn=0.81 lr=4.08e-04 tps=18355 mem=5.7G lam=0.2365231616422534 Gm=0.3711463672419389 +[full s4] EVAL step 5000: val_loss=4.7514 val_ppl=115.74 +[full s4] step 5020/12000 loss=4.7408 gn=0.79 lr=4.07e-04 tps=18341 mem=5.7G lam=0.23692470074941716 Gm=0.36803771601989865 +[full s4] step 5040/12000 loss=4.8025 gn=0.88 lr=4.05e-04 tps=18342 mem=5.7G lam=0.23732075281441212 Gm=0.36967655814563233 +[full s4] step 5060/12000 loss=4.6371 gn=0.83 lr=4.04e-04 tps=18342 mem=5.7G lam=0.23775403232624134 Gm=0.3784525617181013 +[full s4] step 5080/12000 loss=4.7363 gn=0.80 lr=4.02e-04 tps=18342 mem=5.7G lam=0.23817113259186348 Gm=0.3594268919744839 +[full s4] step 5100/12000 loss=4.6796 gn=0.81 lr=4.01e-04 tps=18343 mem=5.7G lam=0.2386739533394575 Gm=0.3685165634378791 +[full s4] step 5120/12000 loss=4.7067 gn=0.75 lr=4.00e-04 tps=18344 mem=5.7G lam=0.2392581362898151 Gm=0.3697026485266785 +[full s4] step 5140/12000 loss=4.6325 gn=0.76 lr=3.98e-04 tps=18345 mem=5.7G lam=0.23977564089000225 Gm=0.3653328693471849 +[full s4] step 5160/12000 loss=4.7286 gn=0.86 lr=3.97e-04 tps=18345 mem=5.7G lam=0.2402218865851561 Gm=0.3707390239772697 +[full s4] step 5180/12000 loss=4.5450 gn=0.78 lr=3.95e-04 tps=18346 mem=5.7G lam=0.24065693467855453 Gm=0.36386202306797105 +[full s4] step 5200/12000 loss=4.5144 gn=0.80 lr=3.94e-04 tps=18346 mem=5.7G lam=0.24107904639095068 Gm=0.35834514047019184 +[full s4] step 5220/12000 loss=4.6839 gn=0.76 lr=3.93e-04 tps=18347 mem=5.7G lam=0.24153699912130833 Gm=0.36557072944318253 +[full s4] step 5240/12000 loss=4.7452 gn=0.89 lr=3.91e-04 tps=18349 mem=5.7G lam=0.24198819821079573 Gm=0.3731331959522019 +[full s4] step 5260/12000 loss=4.6174 gn=0.79 lr=3.90e-04 tps=18349 mem=5.7G lam=0.24252067878842354 Gm=0.3642084232221047 +[full s4] step 5280/12000 loss=4.5959 gn=0.86 lr=3.88e-04 tps=18350 mem=5.7G lam=0.24299585850288472 Gm=0.358302519734328 +[full s4] step 5300/12000 loss=4.6156 gn=0.79 lr=3.87e-04 tps=18351 mem=5.7G lam=0.24344167144348225 Gm=0.36473490910915035 +[full s4] step 5320/12000 loss=4.6283 gn=0.79 lr=3.86e-04 tps=18352 mem=5.7G lam=0.24390633000681797 Gm=0.3726271258977552 +[full s4] step 5340/12000 loss=4.7139 gn=0.85 lr=3.84e-04 tps=18353 mem=5.7G lam=0.24426641842971245 Gm=0.36805650948857266 +[full s4] step 5360/12000 loss=4.6431 gn=0.85 lr=3.83e-04 tps=18355 mem=5.7G lam=0.24474171486993632 Gm=0.3668080548911045 +[full s4] step 5380/12000 loss=4.5955 gn=0.80 lr=3.81e-04 tps=18355 mem=5.7G lam=0.24516871416320404 Gm=0.3570839643167953 +[full s4] step 5400/12000 loss=4.6985 gn=0.79 lr=3.80e-04 tps=18356 mem=5.7G lam=0.24562156479805708 Gm=0.3670561285689473 +[full s4] step 5420/12000 loss=4.5861 gn=0.87 lr=3.79e-04 tps=18357 mem=5.7G lam=0.2460547067845861 Gm=0.3717958456836641 +[full s4] step 5440/12000 loss=4.7438 gn=0.78 lr=3.77e-04 tps=18358 mem=5.7G lam=0.2465185613060991 Gm=0.3657076076293985 +[full s4] step 5460/12000 loss=4.5615 gn=0.79 lr=3.76e-04 tps=18359 mem=5.7G lam=0.24690743070095778 Gm=0.36334628658369184 +[full s4] step 5480/12000 loss=4.6772 gn=0.88 lr=3.74e-04 tps=18360 mem=5.7G lam=0.24736797933777174 Gm=0.3658526196765403 +[full s4] step 5500/12000 loss=4.7152 gn=0.85 lr=3.73e-04 tps=18360 mem=5.7G lam=0.24782517738640308 Gm=0.35793535714037716 +[full s4] step 5520/12000 loss=4.8157 gn=0.82 lr=3.72e-04 tps=18361 mem=5.7G lam=0.24830062159647545 Gm=0.3657913162993888 +[full s4] step 5540/12000 loss=4.7204 gn=0.77 lr=3.70e-04 tps=18361 mem=5.7G lam=0.24877672332028547 Gm=0.3674296786387761 +[full s4] step 5560/12000 loss=4.5071 gn=0.81 lr=3.69e-04 tps=18362 mem=5.7G lam=0.24915645737200975 Gm=0.36000380308056873 +[full s4] step 5580/12000 loss=4.5422 gn=0.80 lr=3.67e-04 tps=18363 mem=5.7G lam=0.24966059966633716 Gm=0.36728547769598663 +[full s4] step 5600/12000 loss=4.6742 gn=0.87 lr=3.66e-04 tps=18363 mem=5.7G lam=0.2501507631192605 Gm=0.3631510976701975 +[full s4] step 5620/12000 loss=4.4130 gn=0.77 lr=3.64e-04 tps=18364 mem=5.7G lam=0.25062733454008895 Gm=0.3629524919670075 +[full s4] step 5640/12000 loss=4.6791 gn=0.84 lr=3.63e-04 tps=18364 mem=5.7G lam=0.25102281974007684 Gm=0.35773366658637923 +[full s4] step 5660/12000 loss=4.7107 gn=0.82 lr=3.62e-04 tps=18365 mem=5.7G lam=0.25146817105511826 Gm=0.3644516132771969 +[full s4] step 5680/12000 loss=4.5013 gn=0.82 lr=3.60e-04 tps=18366 mem=5.7G lam=0.2518983759606878 Gm=0.3574371750000864 +[full s4] step 5700/12000 loss=4.4751 gn=0.78 lr=3.59e-04 tps=18366 mem=5.7G lam=0.252339162863791 Gm=0.3581862701879193 +[full s4] step 5720/12000 loss=4.6207 gn=0.79 lr=3.57e-04 tps=18367 mem=5.7G lam=0.25282227247953415 Gm=0.35826594072083634 +[full s4] step 5740/12000 loss=4.6614 gn=0.86 lr=3.56e-04 tps=18367 mem=5.7G lam=0.2532856147736311 Gm=0.36944881174713373 +[full s4] step 5760/12000 loss=4.5549 gn=0.82 lr=3.54e-04 tps=18368 mem=5.7G lam=0.2536843453223507 Gm=0.3682170473815252 +[full s4] step 5780/12000 loss=4.5239 gn=0.82 lr=3.53e-04 tps=18369 mem=5.7G lam=0.2540863190467159 Gm=0.35625974445914227 +[full s4] step 5800/12000 loss=4.4864 gn=0.81 lr=3.52e-04 tps=18369 mem=5.7G lam=0.2545256847515702 Gm=0.3682216592909147 +[full s4] step 5820/12000 loss=4.5393 gn=0.94 lr=3.50e-04 tps=18369 mem=5.7G lam=0.25500068968782824 Gm=0.36250115232542157 +[full s4] step 5840/12000 loss=4.7282 gn=0.81 lr=3.49e-04 tps=18370 mem=5.7G lam=0.25546469787756604 Gm=0.35975503098840517 +[full s4] step 5860/12000 loss=4.5453 gn=0.79 lr=3.47e-04 tps=18371 mem=5.7G lam=0.2559572284420331 Gm=0.3585992289396624 +[full s4] step 5880/12000 loss=4.6636 gn=0.86 lr=3.46e-04 tps=18372 mem=5.7G lam=0.2564028777802984 Gm=0.36334170781386393 +[full s4] step 5900/12000 loss=4.4828 gn=0.83 lr=3.44e-04 tps=18373 mem=5.7G lam=0.2567604922999938 Gm=0.3558735668193549 +[full s4] step 5920/12000 loss=4.4308 gn=0.86 lr=3.43e-04 tps=18373 mem=5.7G lam=0.25714112104227144 Gm=0.358245681039989 +[full s4] step 5940/12000 loss=4.6194 gn=0.83 lr=3.41e-04 tps=18374 mem=5.7G lam=0.25753561966121197 Gm=0.3626527128120263 +[full s4] step 5960/12000 loss=4.5184 gn=0.85 lr=3.40e-04 tps=18375 mem=5.7G lam=0.2579338162516554 Gm=0.36487312777899206 +[full s4] step 5980/12000 loss=4.6434 gn=0.84 lr=3.39e-04 tps=18374 mem=5.7G lam=0.2584414106483261 Gm=0.3603122190882762 +[full s4] step 6000/12000 loss=4.6823 gn=0.84 lr=3.37e-04 tps=18371 mem=5.7G lam=0.25883820187300444 Gm=0.35807150617862743 +[full s4] EVAL step 6000: val_loss=4.6434 val_ppl=103.89 +[full s4] step 6020/12000 loss=4.6617 gn=0.83 lr=3.36e-04 tps=18359 mem=5.7G lam=0.2592731745292743 Gm=0.35848265420645475 +[full s4] step 6040/12000 loss=4.4713 gn=0.86 lr=3.34e-04 tps=18359 mem=5.7G lam=0.2596320478866498 Gm=0.3596945912577212 +[full s4] step 6060/12000 loss=4.5453 gn=0.80 lr=3.33e-04 tps=18358 mem=5.7G lam=0.2600602243716518 Gm=0.3565436414598177 +[full s4] step 6080/12000 loss=4.7327 gn=0.78 lr=3.31e-04 tps=18359 mem=5.7G lam=0.26047793837885064 Gm=0.356764108253022 +[full s4] step 6100/12000 loss=4.3901 gn=0.81 lr=3.30e-04 tps=18356 mem=5.7G lam=0.2608625286569198 Gm=0.3546972152156134 +[full s4] step 6120/12000 loss=4.6539 gn=0.82 lr=3.29e-04 tps=18356 mem=5.7G lam=0.26122099968294304 Gm=0.3615856012329459 +[full s4] step 6140/12000 loss=4.4892 gn=0.80 lr=3.27e-04 tps=18356 mem=5.7G lam=0.2615624973550439 Gm=0.35908133854779106 +[full s4] step 6160/12000 loss=4.5623 gn=0.86 lr=3.26e-04 tps=18354 mem=5.7G lam=0.2620219448581338 Gm=0.36358975906235474 +[full s4] step 6180/12000 loss=4.4750 gn=0.81 lr=3.24e-04 tps=18351 mem=5.7G lam=0.2624577156578501 Gm=0.3516097213917722 +[full s4] step 6200/12000 loss=4.6110 gn=0.86 lr=3.23e-04 tps=18348 mem=5.7G lam=0.26288323880483705 Gm=0.3624000658746809 +[full s4] step 6220/12000 loss=4.6722 gn=0.77 lr=3.21e-04 tps=18345 mem=5.7G lam=0.26325015164911747 Gm=0.35825196661365527 +[full s4] step 6240/12000 loss=4.6025 gn=0.80 lr=3.20e-04 tps=18343 mem=5.7G lam=0.2636394447957476 Gm=0.3582523966518541 +[full s4] step 6260/12000 loss=4.4721 gn=0.77 lr=3.19e-04 tps=18340 mem=5.7G lam=0.26397693529725075 Gm=0.3529267829532425 +[full s4] step 6280/12000 loss=4.4605 gn=0.88 lr=3.17e-04 tps=18340 mem=5.7G lam=0.2643559059749047 Gm=0.35116600967012346 +[full s4] step 6300/12000 loss=4.5812 gn=1.04 lr=3.16e-04 tps=18341 mem=5.7G lam=0.2647238438948989 Gm=0.360798515068988 +[full s4] step 6320/12000 loss=4.3449 gn=0.78 lr=3.14e-04 tps=18340 mem=5.7G lam=0.26511879296352464 Gm=0.3554890203134467 +[full s4] step 6340/12000 loss=4.5888 gn=0.79 lr=3.13e-04 tps=18339 mem=5.7G lam=0.26546994006882113 Gm=0.35363700628901523 +[full s4] step 6360/12000 loss=4.4373 gn=0.82 lr=3.11e-04 tps=18337 mem=5.7G lam=0.26587214848647517 Gm=0.3576130832079798 +[full s4] step 6380/12000 loss=4.5944 gn=0.80 lr=3.10e-04 tps=18335 mem=5.7G lam=0.26626183092594147 Gm=0.3553391710544626 +[full s4] step 6400/12000 loss=4.5608 gn=0.85 lr=3.08e-04 tps=18335 mem=5.7G lam=0.2666119408483307 Gm=0.3579733884738137 +[full s4] step 6420/12000 loss=4.6126 gn=0.84 lr=3.07e-04 tps=18335 mem=5.7G lam=0.2670399636651079 Gm=0.35876511114959914 +[full s4] step 6440/12000 loss=4.6480 gn=0.84 lr=3.06e-04 tps=18334 mem=5.7G lam=0.26741636854906875 Gm=0.35659385030157864 +[full s4] step 6460/12000 loss=4.5247 gn=0.87 lr=3.04e-04 tps=18332 mem=5.7G lam=0.26777064707130194 Gm=0.3613991611637175 +[full s4] step 6480/12000 loss=4.7104 gn=0.90 lr=3.03e-04 tps=18331 mem=5.7G lam=0.2681756444896261 Gm=0.3612810665120681 +[full s4] step 6500/12000 loss=4.4532 gn=0.88 lr=3.01e-04 tps=18331 mem=5.7G lam=0.2684893946473797 Gm=0.35259960498660803 +[full s4] step 6520/12000 loss=4.5282 gn=0.82 lr=3.00e-04 tps=18331 mem=5.7G lam=0.26887457600484294 Gm=0.350173053642114 +[full s4] step 6540/12000 loss=4.5998 gn=0.83 lr=2.98e-04 tps=18330 mem=5.7G lam=0.2692048115034898 Gm=0.3562228417334457 +[full s4] step 6560/12000 loss=4.3811 gn=0.84 lr=2.97e-04 tps=18329 mem=5.7G lam=0.2695869154607256 Gm=0.35485830266649526 +[full s4] step 6580/12000 loss=4.6746 gn=0.94 lr=2.96e-04 tps=18328 mem=5.7G lam=0.26994226935009163 Gm=0.3564275673901041 +[full s4] step 6600/12000 loss=4.4935 gn=0.91 lr=2.94e-04 tps=18327 mem=5.7G lam=0.2703241373722752 Gm=0.35579354758374393 +[full s4] step 6620/12000 loss=4.7663 gn=0.95 lr=2.93e-04 tps=18327 mem=5.7G lam=0.2706438070163131 Gm=0.3596278147306293 +[full s4] step 6640/12000 loss=4.4355 gn=0.88 lr=2.91e-04 tps=18326 mem=5.7G lam=0.2710222726066907 Gm=0.3555768684794505 +[full s4] step 6660/12000 loss=4.4527 gn=0.84 lr=2.90e-04 tps=18326 mem=5.7G lam=0.27137570021053153 Gm=0.353838828081886 +[full s4] step 6680/12000 loss=4.4289 gn=0.86 lr=2.88e-04 tps=18326 mem=5.7G lam=0.27170228337248165 Gm=0.35187217003355425 +[full s4] step 6700/12000 loss=4.6021 gn=0.92 lr=2.87e-04 tps=18326 mem=5.7G lam=0.27201268666734296 Gm=0.3551136131864041 +[full s4] step 6720/12000 loss=4.4838 gn=0.87 lr=2.86e-04 tps=18327 mem=5.7G lam=0.2723744288086891 Gm=0.35324171899507445 +[full s4] step 6740/12000 loss=4.4782 gn=0.83 lr=2.84e-04 tps=18326 mem=5.7G lam=0.2727000055213769 Gm=0.3596903880437215 +[full s4] step 6760/12000 loss=4.5785 gn=0.87 lr=2.83e-04 tps=18326 mem=5.7G lam=0.2730521271005273 Gm=0.3546933950856328 +[full s4] step 6780/12000 loss=4.4450 gn=0.84 lr=2.81e-04 tps=18327 mem=5.7G lam=0.27343341584006947 Gm=0.3533131289295852 +[full s4] step 6800/12000 loss=4.3783 gn=0.83 lr=2.80e-04 tps=18327 mem=5.7G lam=0.2737687186648448 Gm=0.35417149293546873 +[full s4] step 6820/12000 loss=4.5956 gn=0.85 lr=2.79e-04 tps=18327 mem=5.7G lam=0.27411294883737963 Gm=0.3540860185554872 +[full s4] step 6840/12000 loss=4.4119 gn=0.83 lr=2.77e-04 tps=18328 mem=5.7G lam=0.27452017987767857 Gm=0.3474682103842497 +[full s4] step 6860/12000 loss=4.6088 gn=0.82 lr=2.76e-04 tps=18328 mem=5.7G lam=0.27485172760983306 Gm=0.35698158650969464 +[full s4] step 6880/12000 loss=4.5152 gn=0.86 lr=2.74e-04 tps=18328 mem=5.7G lam=0.2751132156699896 Gm=0.35282760408396524 +[full s4] step 6900/12000 loss=4.4543 gn=0.85 lr=2.73e-04 tps=18329 mem=5.7G lam=0.2753921998664737 Gm=0.3543674040120095 +[full s4] step 6920/12000 loss=4.4778 gn=0.86 lr=2.72e-04 tps=18329 mem=5.7G lam=0.2757540304834644 Gm=0.3563378950736175 +[full s4] step 6940/12000 loss=4.5417 gn=0.89 lr=2.70e-04 tps=18330 mem=5.7G lam=0.27611275172481936 Gm=0.3531963441055268 +[full s4] step 6960/12000 loss=4.5062 gn=0.88 lr=2.69e-04 tps=18330 mem=5.7G lam=0.27656918826202553 Gm=0.3506823836360127 +[full s4] step 6980/12000 loss=4.4806 gn=0.80 lr=2.67e-04 tps=18331 mem=5.7G lam=0.2768572609250744 Gm=0.35123895900323987 +[full s4] step 7000/12000 loss=4.3348 gn=0.87 lr=2.66e-04 tps=18331 mem=5.7G lam=0.27720694181819755 Gm=0.35148461527811986 +[full s4] EVAL step 7000: val_loss=4.6052 val_ppl=100.01 +[full s4] step 7020/12000 loss=4.6740 gn=0.88 lr=2.65e-04 tps=18321 mem=5.7G lam=0.27754557970911264 Gm=0.35182596595647436 +[full s4] step 7040/12000 loss=4.6632 gn=0.90 lr=2.63e-04 tps=18321 mem=5.7G lam=0.27790394456436235 Gm=0.3506757887856414 +[full s4] step 7060/12000 loss=4.5630 gn=0.80 lr=2.62e-04 tps=18321 mem=5.7G lam=0.27821611799299717 Gm=0.35237662587314844 +[full s4] step 7080/12000 loss=4.5377 gn=0.88 lr=2.60e-04 tps=18321 mem=5.7G lam=0.27848599168161553 Gm=0.35679119220003486 +[full s4] step 7100/12000 loss=4.5555 gn=0.87 lr=2.59e-04 tps=18321 mem=5.7G lam=0.2788785866772135 Gm=0.35246510120729607 +[full s4] step 7120/12000 loss=4.7347 gn=0.92 lr=2.58e-04 tps=18321 mem=5.7G lam=0.2792750112712383 Gm=0.35737204101557535 +[full s4] step 7140/12000 loss=4.4304 gn=0.87 lr=2.56e-04 tps=18321 mem=5.7G lam=0.27957310372342664 Gm=0.34507559371801716 +[full s4] step 7160/12000 loss=4.4829 gn=0.92 lr=2.55e-04 tps=18321 mem=5.7G lam=0.27983555756509304 Gm=0.3532575760812809 +[full s4] step 7180/12000 loss=4.5438 gn=1.02 lr=2.53e-04 tps=18321 mem=5.7G lam=0.28015957586467266 Gm=0.35498759523034096 +[full s4] step 7200/12000 loss=4.5388 gn=0.91 lr=2.52e-04 tps=18321 mem=5.7G lam=0.2805316358183821 Gm=0.3545356549633046 +[full s4] step 7220/12000 loss=4.3322 gn=0.89 lr=2.51e-04 tps=18322 mem=5.7G lam=0.2808396841088931 Gm=0.3546012109145522 +[full s4] step 7240/12000 loss=4.4079 gn=0.81 lr=2.49e-04 tps=18322 mem=5.7G lam=0.28116820845752954 Gm=0.3467804682441056 +[full s4] step 7260/12000 loss=4.5220 gn=0.86 lr=2.48e-04 tps=18322 mem=5.7G lam=0.2814496175075571 Gm=0.3451229579125841 +[full s4] step 7280/12000 loss=4.4217 gn=0.84 lr=2.47e-04 tps=18322 mem=5.7G lam=0.28178582650919753 Gm=0.35021370959778625 +[full s4] step 7300/12000 loss=4.5759 gn=0.84 lr=2.45e-04 tps=18323 mem=5.7G lam=0.2821484698603551 Gm=0.35073534756277996 +[full s4] step 7320/12000 loss=4.3964 gn=0.82 lr=2.44e-04 tps=18323 mem=5.7G lam=0.28248908339689177 Gm=0.3511496167629957 +[full s4] step 7340/12000 loss=4.4779 gn=0.90 lr=2.42e-04 tps=18324 mem=5.7G lam=0.2827464962999026 Gm=0.35127765328312915 +[full s4] step 7360/12000 loss=4.5437 gn=0.86 lr=2.41e-04 tps=18323 mem=5.7G lam=0.28300652373582125 Gm=0.34860042622312903 +[full s4] step 7380/12000 loss=4.5224 gn=0.88 lr=2.40e-04 tps=18324 mem=5.7G lam=0.2832796877870957 Gm=0.35020428981321555 +[full s4] step 7400/12000 loss=4.3947 gn=0.84 lr=2.38e-04 tps=18325 mem=5.7G lam=0.2835684133072694 Gm=0.3501892539982994 +[full s4] step 7420/12000 loss=4.4664 gn=0.89 lr=2.37e-04 tps=18325 mem=5.7G lam=0.28387489076703787 Gm=0.3570984903878222 +[full s4] step 7440/12000 loss=4.3752 gn=0.86 lr=2.36e-04 tps=18326 mem=5.7G lam=0.28414782260855037 Gm=0.3515256590520342 +[full s4] step 7460/12000 loss=4.5417 gn=0.93 lr=2.34e-04 tps=18325 mem=5.7G lam=0.28445135491589707 Gm=0.3482652798605462 +[full s4] step 7480/12000 loss=4.4055 gn=0.91 lr=2.33e-04 tps=18326 mem=5.7G lam=0.2847857366626461 Gm=0.3437471923728784 +[full s4] step 7500/12000 loss=4.4657 gn=0.86 lr=2.32e-04 tps=18327 mem=5.7G lam=0.28506167605519295 Gm=0.3495621548499912 +[full s4] step 7520/12000 loss=4.4984 gn=0.83 lr=2.30e-04 tps=18327 mem=5.7G lam=0.2853330882887046 Gm=0.3481289000871281 +[full s4] step 7540/12000 loss=4.3874 gn=0.92 lr=2.29e-04 tps=18327 mem=5.7G lam=0.28560480816910666 Gm=0.35537044362475473 +[full s4] step 7560/12000 loss=4.3095 gn=0.90 lr=2.28e-04 tps=18327 mem=5.7G lam=0.28587047072748345 Gm=0.34432211029343307 +[full s4] step 7580/12000 loss=4.5788 gn=0.90 lr=2.26e-04 tps=18327 mem=5.7G lam=0.2860917557651798 Gm=0.35248891978214186 +[full s4] step 7600/12000 loss=4.3381 gn=0.86 lr=2.25e-04 tps=18328 mem=5.7G lam=0.2863617753610015 Gm=0.3448391465159754 +[full s4] step 7620/12000 loss=4.4280 gn=0.89 lr=2.24e-04 tps=18328 mem=5.7G lam=0.286680169403553 Gm=0.3443421679548919 +[full s4] step 7640/12000 loss=4.3887 gn=0.89 lr=2.22e-04 tps=18328 mem=5.7G lam=0.28696960924814147 Gm=0.3516507416497916 +[full s4] step 7660/12000 loss=4.4222 gn=0.87 lr=2.21e-04 tps=18328 mem=5.7G lam=0.28722730465233326 Gm=0.350923239486292 +[full s4] step 7680/12000 loss=4.6173 gn=0.92 lr=2.20e-04 tps=18329 mem=5.7G lam=0.28749151310573023 Gm=0.3470952081649254 +[full s4] step 7700/12000 loss=4.5634 gn=0.93 lr=2.18e-04 tps=18329 mem=5.7G lam=0.28774813562631607 Gm=0.35302536911331117 +[full s4] step 7720/12000 loss=4.5499 gn=0.88 lr=2.17e-04 tps=18328 mem=5.7G lam=0.2880437957743804 Gm=0.3478489266708493 +[full s4] step 7740/12000 loss=4.4742 gn=0.91 lr=2.16e-04 tps=18329 mem=5.7G lam=0.2883159515137474 Gm=0.3537479277389745 +[full s4] step 7760/12000 loss=4.4665 gn=0.84 lr=2.15e-04 tps=18329 mem=5.7G lam=0.28854901747157174 Gm=0.350671809942772 +[full s4] step 7780/12000 loss=4.5571 gn=0.91 lr=2.13e-04 tps=18329 mem=5.7G lam=0.288831552490592 Gm=0.3543751132674515 +[full s4] step 7800/12000 loss=4.4671 gn=0.91 lr=2.12e-04 tps=18329 mem=5.7G lam=0.28906052745878696 Gm=0.34827418019995093 +[full s4] step 7820/12000 loss=4.3439 gn=0.85 lr=2.11e-04 tps=18330 mem=5.7G lam=0.2892943893869718 Gm=0.34121584706008434 +[full s4] step 7840/12000 loss=4.3816 gn=0.90 lr=2.09e-04 tps=18330 mem=5.7G lam=0.2895163893699646 Gm=0.34896242277075845 +[full s4] step 7860/12000 loss=4.3700 gn=0.92 lr=2.08e-04 tps=18330 mem=5.7G lam=0.28979839974393445 Gm=0.34943079265455407 +[full s4] step 7880/12000 loss=4.3723 gn=0.86 lr=2.07e-04 tps=18331 mem=5.7G lam=0.29002845752984285 Gm=0.3485886383180817 +[full s4] step 7900/12000 loss=4.3526 gn=0.93 lr=2.06e-04 tps=18331 mem=5.7G lam=0.2903066373740633 Gm=0.35036820895038545 +[full s4] step 7920/12000 loss=4.4947 gn=0.91 lr=2.04e-04 tps=18331 mem=5.7G lam=0.2905640384803216 Gm=0.34873160867330927 +[full s4] step 7940/12000 loss=4.4895 gn=0.88 lr=2.03e-04 tps=18331 mem=5.7G lam=0.2908319712926944 Gm=0.34590139767775935 +[full s4] step 7960/12000 loss=4.4780 gn=0.85 lr=2.02e-04 tps=18331 mem=5.7G lam=0.2910601558784644 Gm=0.3462152830325067 +[full s4] step 7980/12000 loss=4.4893 gn=0.88 lr=2.00e-04 tps=18332 mem=5.7G lam=0.2913198660438259 Gm=0.34892640309408307 +[full s4] step 8000/12000 loss=4.5500 gn=0.96 lr=1.99e-04 tps=18332 mem=5.7G lam=0.2915290417149663 Gm=0.3398551610298455 +[full s4] EVAL step 8000: val_loss=4.5334 val_ppl=93.07 +[full s4] step 8020/12000 loss=4.5092 gn=0.92 lr=1.98e-04 tps=18324 mem=5.7G lam=0.291743409819901 Gm=0.34269097819924355 +[full s4] step 8040/12000 loss=4.4275 gn=0.89 lr=1.97e-04 tps=18325 mem=5.7G lam=0.2920057000592351 Gm=0.3484524735249579 +[full s4] step 8060/12000 loss=4.4446 gn=0.89 lr=1.95e-04 tps=18325 mem=5.7G lam=0.29225503249714774 Gm=0.34696300451954204 +[full s4] step 8080/12000 loss=4.5461 gn=0.85 lr=1.94e-04 tps=18326 mem=5.7G lam=0.29250295273959637 Gm=0.3471501438568036 +[full s4] step 8100/12000 loss=4.5090 gn=0.91 lr=1.93e-04 tps=18326 mem=5.7G lam=0.2927549481391907 Gm=0.3518143813901891 +[full s4] step 8120/12000 loss=4.6401 gn=0.90 lr=1.92e-04 tps=18326 mem=5.7G lam=0.29296088094512623 Gm=0.3529688556833814 +[full s4] step 8140/12000 loss=4.3136 gn=0.88 lr=1.90e-04 tps=18327 mem=5.7G lam=0.2931942070523898 Gm=0.34224673577894765 +[full s4] step 8160/12000 loss=4.3729 gn=0.93 lr=1.89e-04 tps=18327 mem=5.7G lam=0.2934224000200629 Gm=0.35625831759534776 +[full s4] step 8180/12000 loss=4.4777 gn=0.97 lr=1.88e-04 tps=18328 mem=5.7G lam=0.29367736044029397 Gm=0.3520342997896175 +[full s4] step 8200/12000 loss=4.3416 gn=0.95 lr=1.87e-04 tps=18328 mem=5.7G lam=0.2938813157379627 Gm=0.3471233159458886 +[full s4] step 8220/12000 loss=4.4210 gn=0.91 lr=1.86e-04 tps=18328 mem=5.7G lam=0.2940479554235935 Gm=0.3441978127229959 +[full s4] step 8240/12000 loss=4.3717 gn=0.91 lr=1.84e-04 tps=18328 mem=5.7G lam=0.29426657905181247 Gm=0.3442156333476305 +[full s4] step 8260/12000 loss=4.3172 gn=0.92 lr=1.83e-04 tps=18328 mem=5.7G lam=0.2945227064192295 Gm=0.34589773214732605 +[full s4] step 8280/12000 loss=4.3101 gn=0.87 lr=1.82e-04 tps=18329 mem=5.7G lam=0.2947280090302229 Gm=0.34697897468383115 +[full s4] step 8300/12000 loss=4.5185 gn=0.89 lr=1.81e-04 tps=18329 mem=5.7G lam=0.29491738229990005 Gm=0.35269507641593617 +[full s4] step 8320/12000 loss=4.3783 gn=0.89 lr=1.80e-04 tps=18329 mem=5.7G lam=0.2951261727139354 Gm=0.3524777806984882 +[full s4] step 8340/12000 loss=4.4418 gn=0.91 lr=1.78e-04 tps=18329 mem=5.7G lam=0.29532100539654493 Gm=0.3465559324249625 +[full s4] step 8360/12000 loss=4.5386 gn=0.93 lr=1.77e-04 tps=18330 mem=5.7G lam=0.2955704800163706 Gm=0.35293608081216615 +[full s4] step 8380/12000 loss=4.3652 gn=0.93 lr=1.76e-04 tps=18330 mem=5.7G lam=0.2958135822166999 Gm=0.3480631695905079 +[full s4] step 8400/12000 loss=4.4663 gn=0.91 lr=1.75e-04 tps=18330 mem=5.7G lam=0.2960570255915324 Gm=0.34780398616567254 +[full s4] step 8420/12000 loss=4.3331 gn=0.90 lr=1.74e-04 tps=18330 mem=5.7G lam=0.29628593878199655 Gm=0.3485206082308044 +[full s4] step 8440/12000 loss=4.2976 gn=0.88 lr=1.72e-04 tps=18330 mem=5.7G lam=0.2965079980591933 Gm=0.3410729637059073 +[full s4] step 8460/12000 loss=4.3475 gn=1.00 lr=1.71e-04 tps=18330 mem=5.7G lam=0.2967175059020519 Gm=0.34998001468678314 +[full s4] step 8480/12000 loss=4.3627 gn=0.88 lr=1.70e-04 tps=18330 mem=5.7G lam=0.29693707699577016 Gm=0.34131693560630083 +[full s4] step 8500/12000 loss=4.3818 gn=1.02 lr=1.69e-04 tps=18330 mem=5.7G lam=0.2971234715854128 Gm=0.3461358911978702 +[full s4] step 8520/12000 loss=4.4836 gn=0.95 lr=1.68e-04 tps=18330 mem=5.7G lam=0.2973256617163618 Gm=0.350596243981272 +[full s4] step 8540/12000 loss=4.5549 gn=0.94 lr=1.67e-04 tps=18331 mem=5.7G lam=0.29755385778844357 Gm=0.35058673610910773 +[full s4] step 8560/12000 loss=4.3450 gn=0.89 lr=1.66e-04 tps=18331 mem=5.7G lam=0.29772287607192993 Gm=0.3433020083854596 +[full s4] step 8580/12000 loss=4.4581 gn=0.90 lr=1.64e-04 tps=18331 mem=5.7G lam=0.29789695298920077 Gm=0.34225006867200136 +[full s4] step 8600/12000 loss=4.2780 gn=0.94 lr=1.63e-04 tps=18331 mem=5.7G lam=0.2980712124456962 Gm=0.34428219497203827 +[full s4] step 8620/12000 loss=4.4094 gn=0.95 lr=1.62e-04 tps=18331 mem=5.7G lam=0.2982683734347423 Gm=0.3452728179593881 +[full s4] step 8640/12000 loss=4.4782 gn=0.94 lr=1.61e-04 tps=18330 mem=5.7G lam=0.29846715088933706 Gm=0.3466002275235951 +[full s4] step 8660/12000 loss=4.2779 gn=0.98 lr=1.60e-04 tps=18330 mem=5.7G lam=0.2986529854436715 Gm=0.3484489099743466 +[full s4] step 8680/12000 loss=4.3742 gn=0.87 lr=1.59e-04 tps=18330 mem=5.7G lam=0.29891638333598775 Gm=0.3482598649182667 +[full s4] step 8700/12000 loss=4.2446 gn=0.93 lr=1.58e-04 tps=18329 mem=5.7G lam=0.29907733233024675 Gm=0.34462310932576656 +[full s4] step 8720/12000 loss=4.4968 gn=0.98 lr=1.57e-04 tps=18330 mem=5.7G lam=0.29926438132921857 Gm=0.3491075331500421 +[full s4] step 8740/12000 loss=4.4028 gn=0.99 lr=1.55e-04 tps=18330 mem=5.7G lam=0.2994595052053531 Gm=0.34809320551964146 +[full s4] step 8760/12000 loss=4.3280 gn=0.96 lr=1.54e-04 tps=18331 mem=5.7G lam=0.2996217406665285 Gm=0.34545919850158197 +[full s4] step 8780/12000 loss=4.3867 gn=0.88 lr=1.53e-04 tps=18331 mem=5.7G lam=0.2998264009753863 Gm=0.35151597174505395 +[full s4] step 8800/12000 loss=4.3480 gn=0.99 lr=1.52e-04 tps=18331 mem=5.7G lam=0.3000129001835982 Gm=0.3494852335813145 +[full s4] step 8820/12000 loss=4.2800 gn=0.93 lr=1.51e-04 tps=18331 mem=5.7G lam=0.3002150778969129 Gm=0.34914830008832115 +[full s4] step 8840/12000 loss=4.4764 gn=0.90 lr=1.50e-04 tps=18331 mem=5.7G lam=0.30038159899413586 Gm=0.34759856822590035 +[full s4] step 8860/12000 loss=4.3415 gn=0.91 lr=1.49e-04 tps=18331 mem=5.7G lam=0.30054401761541766 Gm=0.3435556300294896 +[full s4] step 8880/12000 loss=4.3693 gn=0.92 lr=1.48e-04 tps=18331 mem=5.7G lam=0.3007311848923564 Gm=0.34510134455437463 +[full s4] step 8900/12000 loss=4.1728 gn=0.92 lr=1.47e-04 tps=18330 mem=5.7G lam=0.3009409497802456 Gm=0.34943100227974355 +[full s4] step 8920/12000 loss=4.3741 gn=0.92 lr=1.46e-04 tps=18330 mem=5.7G lam=0.30108196288347244 Gm=0.3474421286955476 +[full s4] step 8940/12000 loss=4.3872 gn=0.89 lr=1.45e-04 tps=18331 mem=5.7G lam=0.3012527246028185 Gm=0.3392605792420606 +[full s4] step 8960/12000 loss=4.3893 gn=0.95 lr=1.44e-04 tps=18331 mem=5.7G lam=0.30139413351813954 Gm=0.3409686650459965 +[full s4] step 8980/12000 loss=4.4423 gn=0.95 lr=1.43e-04 tps=18331 mem=5.7G lam=0.3016029406959812 Gm=0.34582372987642884 +[full s4] step 9000/12000 loss=4.3830 gn=0.97 lr=1.42e-04 tps=18331 mem=5.7G lam=0.30180559245248634 Gm=0.34311852324754 +[full s4] EVAL step 9000: val_loss=4.4459 val_ppl=85.28 +[full s4] step 9020/12000 loss=4.4483 gn=0.97 lr=1.41e-04 tps=18323 mem=5.7G lam=0.30196432396769524 Gm=0.3493578031969567 +[full s4] step 9040/12000 loss=4.3945 gn=0.92 lr=1.40e-04 tps=18323 mem=5.7G lam=0.30213461723178625 Gm=0.3451639513174693 +[full s4] step 9060/12000 loss=4.1268 gn=0.95 lr=1.39e-04 tps=18323 mem=5.7G lam=0.30233525422712165 Gm=0.3462749949345986 +[full s4] step 9080/12000 loss=4.4717 gn=0.99 lr=1.38e-04 tps=18323 mem=5.7G lam=0.3025022568181157 Gm=0.3491213978268206 +[full s4] step 9100/12000 loss=4.4249 gn=0.94 lr=1.37e-04 tps=18324 mem=5.7G lam=0.30264529772102833 Gm=0.3452247752963255 +[full s4] step 9120/12000 loss=4.2641 gn=0.91 lr=1.36e-04 tps=18324 mem=5.7G lam=0.30278461674849194 Gm=0.34526406282869476 +[full s4] step 9140/12000 loss=4.2067 gn=0.87 lr=1.35e-04 tps=18324 mem=5.7G lam=0.3029381713519494 Gm=0.345937874323378 +[full s4] step 9160/12000 loss=4.4532 gn=0.98 lr=1.34e-04 tps=18324 mem=5.7G lam=0.3031132124985258 Gm=0.34319375238070887 +[full s4] step 9180/12000 loss=4.4547 gn=0.88 lr=1.33e-04 tps=18324 mem=5.7G lam=0.3032781636963288 Gm=0.3460802521246175 +[full s4] step 9200/12000 loss=4.2483 gn=0.93 lr=1.32e-04 tps=18324 mem=5.7G lam=0.30346440865347785 Gm=0.3488367274403572 +[full s4] step 9220/12000 loss=4.4479 gn=0.96 lr=1.31e-04 tps=18324 mem=5.7G lam=0.3036352414637804 Gm=0.3469193447381258 +[full s4] step 9240/12000 loss=4.4684 gn=0.98 lr=1.30e-04 tps=18324 mem=5.7G lam=0.30379374977201223 Gm=0.3420507526025176 +[full s4] step 9260/12000 loss=4.3263 gn=0.91 lr=1.29e-04 tps=18324 mem=5.7G lam=0.30395191473265487 Gm=0.344086202327162 +[full s4] step 9280/12000 loss=4.3426 gn=0.94 lr=1.28e-04 tps=18324 mem=5.7G lam=0.30414252324650687 Gm=0.3478749340089659 +[full s4] step 9300/12000 loss=4.4943 gn=0.96 lr=1.27e-04 tps=18325 mem=5.7G lam=0.30428841927399236 Gm=0.3504452692189564 +[full s4] step 9320/12000 loss=4.2760 gn=0.89 lr=1.26e-04 tps=18324 mem=5.7G lam=0.304451921954751 Gm=0.34756031710033614 +[full s4] step 9340/12000 loss=4.4840 gn=0.97 lr=1.25e-04 tps=18324 mem=5.7G lam=0.30461473297327757 Gm=0.3437775594647974 +[full s4] step 9360/12000 loss=4.2950 gn=0.98 lr=1.24e-04 tps=18325 mem=5.7G lam=0.30475818552076817 Gm=0.33928323568155366 +[full s4] step 9380/12000 loss=4.4390 gn=0.89 lr=1.23e-04 tps=18325 mem=5.7G lam=0.30490000763287145 Gm=0.34434664839257795 +[full s4] step 9400/12000 loss=4.2142 gn=0.98 lr=1.22e-04 tps=18325 mem=5.7G lam=0.3050534601012866 Gm=0.34464448085054755 +[full s4] step 9420/12000 loss=4.2418 gn=0.98 lr=1.21e-04 tps=18325 mem=5.7G lam=0.30519053371002275 Gm=0.33988005854189396 +[full s4] step 9440/12000 loss=4.2890 gn=1.09 lr=1.20e-04 tps=18325 mem=5.7G lam=0.30533194666107494 Gm=0.34042999140607816 +[full s4] step 9460/12000 loss=4.3836 gn=0.94 lr=1.19e-04 tps=18325 mem=5.7G lam=0.30548406516512233 Gm=0.34283583890646696 +[full s4] step 9480/12000 loss=4.3770 gn=0.95 lr=1.19e-04 tps=18326 mem=5.7G lam=0.3055927200863759 Gm=0.34548574173823 +[full s4] step 9500/12000 loss=4.5208 gn=1.01 lr=1.18e-04 tps=18326 mem=5.7G lam=0.30577461887151003 Gm=0.34325661789625883 +[full s4] step 9520/12000 loss=4.2123 gn=0.97 lr=1.17e-04 tps=18326 mem=5.7G lam=0.30591389785210293 Gm=0.34661122985805076 +[full s4] step 9540/12000 loss=4.3607 gn=0.93 lr=1.16e-04 tps=18326 mem=5.7G lam=0.30605667643249035 Gm=0.3461743829927097 +[full s4] step 9560/12000 loss=4.5483 gn=0.96 lr=1.15e-04 tps=18327 mem=5.7G lam=0.3062013577048977 Gm=0.34228607142965 +[full s4] step 9580/12000 loss=4.2734 gn=0.97 lr=1.14e-04 tps=18327 mem=5.7G lam=0.3063488354285558 Gm=0.34344879499015707 +[full s4] step 9600/12000 loss=4.4097 gn=0.97 lr=1.13e-04 tps=18327 mem=5.7G lam=0.30649709173788625 Gm=0.34274767820412916 +[full s4] step 9620/12000 loss=4.3324 gn=0.98 lr=1.12e-04 tps=18328 mem=5.7G lam=0.3066350327183803 Gm=0.34378385093684 +[full s4] step 9640/12000 loss=4.5110 gn=1.03 lr=1.12e-04 tps=18328 mem=5.7G lam=0.3067915004988511 Gm=0.353617789534231 +[full s4] step 9660/12000 loss=4.3297 gn=0.96 lr=1.11e-04 tps=18328 mem=5.7G lam=0.30691477687408525 Gm=0.3446069451359411 +[full s4] step 9680/12000 loss=4.5123 gn=1.00 lr=1.10e-04 tps=18328 mem=5.7G lam=0.30705568070213 Gm=0.3446501987054944 +[full s4] step 9700/12000 loss=4.2800 gn=0.93 lr=1.09e-04 tps=18328 mem=5.7G lam=0.3071918698648612 Gm=0.3455254330765456 +[full s4] step 9720/12000 loss=4.3231 gn=1.07 lr=1.08e-04 tps=18329 mem=5.7G lam=0.3073451512803634 Gm=0.3464735358332594 +[full s4] step 9740/12000 loss=4.3650 gn=0.94 lr=1.07e-04 tps=18329 mem=5.7G lam=0.30748864977310103 Gm=0.34357699705287814 +[full s4] step 9760/12000 loss=4.3846 gn=0.96 lr=1.07e-04 tps=18329 mem=5.7G lam=0.30758994600425166 Gm=0.3405146359000355 +[full s4] step 9780/12000 loss=4.1121 gn=0.90 lr=1.06e-04 tps=18329 mem=5.7G lam=0.3077298669765393 Gm=0.3379040729875366 +[full s4] step 9800/12000 loss=4.2744 gn=0.96 lr=1.05e-04 tps=18329 mem=5.7G lam=0.307878908701241 Gm=0.33987252491836745 +[full s4] step 9820/12000 loss=4.1653 gn=0.91 lr=1.04e-04 tps=18329 mem=5.7G lam=0.3080027125154932 Gm=0.33754552823180956 +[full s4] step 9840/12000 loss=4.4862 gn=0.95 lr=1.03e-04 tps=18329 mem=5.7G lam=0.3081222552185257 Gm=0.35026859018641215 +[full s4] step 9860/12000 loss=4.3049 gn=1.06 lr=1.03e-04 tps=18329 mem=5.7G lam=0.30823838245123625 Gm=0.350689245465522 +[full s4] step 9880/12000 loss=4.3331 gn=0.95 lr=1.02e-04 tps=18330 mem=5.7G lam=0.3083571546400587 Gm=0.3399057885011037 +[full s4] step 9900/12000 loss=4.2503 gn=0.97 lr=1.01e-04 tps=18330 mem=5.7G lam=0.30845462065190077 Gm=0.34824877333206433 +[full s4] step 9920/12000 loss=4.2965 gn=0.95 lr=1.00e-04 tps=18330 mem=5.7G lam=0.3085564511517684 Gm=0.3420624987532695 +[full s4] step 9940/12000 loss=4.5010 gn=0.95 lr=9.96e-05 tps=18331 mem=5.7G lam=0.3086963541184862 Gm=0.34542154070610803 +[full s4] step 9960/12000 loss=4.3883 gn=0.99 lr=9.89e-05 tps=18331 mem=5.7G lam=0.3088552625849843 Gm=0.34844274586066604 +[full s4] step 9980/12000 loss=4.3562 gn=0.93 lr=9.81e-05 tps=18331 mem=5.7G lam=0.3089638880143563 Gm=0.34624715537453693 +[full s4] step 10000/12000 loss=4.3272 gn=0.91 lr=9.74e-05 tps=18329 mem=5.7G lam=0.30906392354518175 Gm=0.34430481151988107 +[full s4] EVAL step 10000: val_loss=4.4146 val_ppl=82.65 +[full s4] step 10020/12000 loss=4.4920 gn=1.07 lr=9.67e-05 tps=18322 mem=5.7G lam=0.3091837642714381 Gm=0.3492072701919824 +[full s4] step 10040/12000 loss=4.3160 gn=0.94 lr=9.59e-05 tps=18322 mem=5.7G lam=0.30929168468962115 Gm=0.3435382464279731 +[full s4] step 10060/12000 loss=4.3029 gn=0.96 lr=9.52e-05 tps=18323 mem=5.7G lam=0.3094137602796157 Gm=0.34232755719373625 +[full s4] step 10080/12000 loss=4.3370 gn=1.05 lr=9.45e-05 tps=18323 mem=5.7G lam=0.3095495185504357 Gm=0.3441431905763845 +[full s4] step 10100/12000 loss=4.3845 gn=0.97 lr=9.38e-05 tps=18323 mem=5.7G lam=0.3096608966588974 Gm=0.3388237490629156 +[full s4] step 10120/12000 loss=4.3521 gn=0.95 lr=9.31e-05 tps=18324 mem=5.7G lam=0.3097817956780394 Gm=0.3417983838977913 +[full s4] step 10140/12000 loss=4.1463 gn=0.93 lr=9.24e-05 tps=18324 mem=5.7G lam=0.30990101117640734 Gm=0.34474145000179607 +[full s4] step 10160/12000 loss=4.2730 gn=0.99 lr=9.18e-05 tps=18324 mem=5.7G lam=0.31000628632803756 Gm=0.3488178439438343 +[full s4] step 10180/12000 loss=4.4452 gn=1.03 lr=9.11e-05 tps=18325 mem=5.7G lam=0.3101037899032235 Gm=0.3443381539545953 +[full s4] step 10200/12000 loss=4.3286 gn=0.99 lr=9.04e-05 tps=18324 mem=5.7G lam=0.31020110721389454 Gm=0.3475480879812191 +[full s4] step 10220/12000 loss=4.4042 gn=1.07 lr=8.98e-05 tps=18325 mem=5.7G lam=0.3103204909712076 Gm=0.3401738195680082 +[full s4] step 10240/12000 loss=4.2338 gn=1.05 lr=8.91e-05 tps=18325 mem=5.7G lam=0.3104028108840187 Gm=0.3470736830495298 +[full s4] step 10260/12000 loss=4.3857 gn=1.01 lr=8.85e-05 tps=18325 mem=5.7G lam=0.3105178310846289 Gm=0.34524595194185775 +[full s4] step 10280/12000 loss=4.3882 gn=1.00 lr=8.78e-05 tps=18325 mem=5.7G lam=0.31062601196269196 Gm=0.3456831614021212 +[full s4] step 10300/12000 loss=4.2717 gn=0.96 lr=8.72e-05 tps=18325 mem=5.7G lam=0.3107094143827756 Gm=0.3422182679641992 +[full s4] step 10320/12000 loss=4.4326 gn=0.99 lr=8.66e-05 tps=18325 mem=5.7G lam=0.31081789173185825 Gm=0.34578449713687104 +[full s4] step 10340/12000 loss=4.3312 gn=1.01 lr=8.59e-05 tps=18325 mem=5.7G lam=0.3109361423800389 Gm=0.342937253570805 +[full s4] step 10360/12000 loss=4.4233 gn=0.97 lr=8.53e-05 tps=18325 mem=5.7G lam=0.31103919198115665 Gm=0.3393843368006249 +[full s4] step 10380/12000 loss=4.3770 gn=1.00 lr=8.47e-05 tps=18326 mem=5.7G lam=0.3111378081763784 Gm=0.3428345713764429 +[full s4] step 10400/12000 loss=4.4456 gn=1.02 lr=8.41e-05 tps=18326 mem=5.7G lam=0.311218217946589 Gm=0.3456390833792587 +[full s4] step 10420/12000 loss=4.3907 gn=0.97 lr=8.35e-05 tps=18326 mem=5.7G lam=0.31130138598382473 Gm=0.3446885173519452 +[full s4] step 10440/12000 loss=4.2572 gn=0.95 lr=8.30e-05 tps=18326 mem=5.7G lam=0.3113922458142042 Gm=0.3449616488069296 +[full s4] step 10460/12000 loss=4.3924 gn=0.95 lr=8.24e-05 tps=18326 mem=5.7G lam=0.3115001205975811 Gm=0.34174589610969025 +[full s4] step 10480/12000 loss=4.2530 gn=0.96 lr=8.18e-05 tps=18326 mem=5.7G lam=0.311597784049809 Gm=0.34365716700752574 +[full s4] step 10500/12000 loss=4.3480 gn=1.02 lr=8.12e-05 tps=18326 mem=5.7G lam=0.31169732473790646 Gm=0.3431276755097012 +[full s4] step 10520/12000 loss=4.4200 gn=1.00 lr=8.07e-05 tps=18326 mem=5.7G lam=0.31178807312001783 Gm=0.3437375579960644 +[full s4] step 10540/12000 loss=4.3243 gn=0.99 lr=8.01e-05 tps=18326 mem=5.7G lam=0.3118666124840577 Gm=0.3414178915942709 +[full s4] step 10560/12000 loss=4.3207 gn=0.97 lr=7.96e-05 tps=18326 mem=5.7G lam=0.3119821383928259 Gm=0.3433294215550025 +[full s4] step 10580/12000 loss=4.3709 gn=1.03 lr=7.91e-05 tps=18327 mem=5.7G lam=0.3120898411919673 Gm=0.3374267515415947 +[full s4] step 10600/12000 loss=4.2280 gn=0.96 lr=7.85e-05 tps=18327 mem=5.7G lam=0.31219553605963785 Gm=0.34344355249777436 +[full s4] step 10620/12000 loss=4.2515 gn=1.04 lr=7.80e-05 tps=18327 mem=5.7G lam=0.31229190311084193 Gm=0.33835393050685525 +[full s4] step 10640/12000 loss=4.2508 gn=0.99 lr=7.75e-05 tps=18327 mem=5.7G lam=0.31238084472715855 Gm=0.33978710882365704 +[full s4] step 10660/12000 loss=4.4239 gn=1.01 lr=7.70e-05 tps=18327 mem=5.7G lam=0.3124683502440651 Gm=0.3451198427937925 +[full s4] step 10680/12000 loss=4.3163 gn=1.02 lr=7.65e-05 tps=18327 mem=5.7G lam=0.3125674460704128 Gm=0.3408746011555195 +[full s4] step 10700/12000 loss=4.2977 gn=1.03 lr=7.60e-05 tps=18327 mem=5.7G lam=0.3126651793718338 Gm=0.34602868432799977 +[full s4] step 10720/12000 loss=4.4277 gn=1.02 lr=7.55e-05 tps=18327 mem=5.7G lam=0.3127626981586218 Gm=0.3488659185046951 +[full s4] step 10740/12000 loss=4.4049 gn=1.00 lr=7.50e-05 tps=18327 mem=5.7G lam=0.3128064988801877 Gm=0.34118162755233544 +[full s4] step 10760/12000 loss=4.1838 gn=1.04 lr=7.46e-05 tps=18328 mem=5.7G lam=0.3128965475286047 Gm=0.34672831821565825 +[full s4] step 10780/12000 loss=4.2389 gn=1.04 lr=7.41e-05 tps=18328 mem=5.7G lam=0.3129932287459572 Gm=0.34127421335627633 +[full s4] step 10800/12000 loss=4.3178 gn=0.96 lr=7.37e-05 tps=18328 mem=5.7G lam=0.31309217121452093 Gm=0.34093608140634996 +[full s4] step 10820/12000 loss=4.2785 gn=0.97 lr=7.32e-05 tps=18328 mem=5.7G lam=0.3131910466278593 Gm=0.3434507339261472 +[full s4] step 10840/12000 loss=4.2855 gn=1.01 lr=7.28e-05 tps=18328 mem=5.7G lam=0.31325112500538427 Gm=0.3398363692685962 +[full s4] step 10860/12000 loss=4.2601 gn=1.02 lr=7.23e-05 tps=18328 mem=5.7G lam=0.3133311967055003 Gm=0.3400708402817448 +[full s4] step 10880/12000 loss=4.2343 gn=1.06 lr=7.19e-05 tps=18328 mem=5.7G lam=0.3134084139019251 Gm=0.3391596705963214 +[full s4] step 10900/12000 loss=4.1759 gn=0.97 lr=7.15e-05 tps=18328 mem=5.7G lam=0.31347896965841454 Gm=0.33873204157377285 +[full s4] step 10920/12000 loss=4.3479 gn=1.01 lr=7.11e-05 tps=18328 mem=5.7G lam=0.31355453344682854 Gm=0.34670761227607727 +[full s4] step 10940/12000 loss=4.3210 gn=1.02 lr=7.07e-05 tps=18329 mem=5.7G lam=0.3136347591256102 Gm=0.3475120921308796 +[full s4] step 10960/12000 loss=4.2969 gn=1.03 lr=7.03e-05 tps=18328 mem=5.7G lam=0.3137175490458806 Gm=0.34767933391655487 +[full s4] step 10980/12000 loss=4.4643 gn=1.13 lr=6.99e-05 tps=18328 mem=5.7G lam=0.3137996320923169 Gm=0.34267903631553054 +[full s4] step 11000/12000 loss=4.3858 gn=0.94 lr=6.95e-05 tps=18329 mem=5.7G lam=0.31387865729629993 Gm=0.3399026563080649 +[full s4] EVAL step 11000: val_loss=4.3664 val_ppl=78.76 +[full s4] step 11020/12000 loss=4.1887 gn=0.99 lr=6.91e-05 tps=18323 mem=5.7G lam=0.3139682011678815 Gm=0.3439560338544349 +[full s4] step 11040/12000 loss=4.2853 gn=1.00 lr=6.88e-05 tps=18322 mem=5.7G lam=0.3140761209651828 Gm=0.34518624384266633 +[full s4] step 11060/12000 loss=4.4580 gn=1.09 lr=6.84e-05 tps=18322 mem=5.7G lam=0.31416218914091587 Gm=0.34230645226004225 +[full s4] step 11080/12000 loss=4.3071 gn=1.02 lr=6.81e-05 tps=18323 mem=5.7G lam=0.3142455769702792 Gm=0.3450187637936324 +[full s4] step 11100/12000 loss=4.3307 gn=1.10 lr=6.77e-05 tps=18323 mem=5.7G lam=0.3143402102092902 Gm=0.3488812033707897 +[full s4] step 11120/12000 loss=4.3909 gn=1.00 lr=6.74e-05 tps=18322 mem=5.7G lam=0.31443373517443735 Gm=0.34203995174417895 +[full s4] step 11140/12000 loss=4.3478 gn=1.08 lr=6.70e-05 tps=18321 mem=5.7G lam=0.31451515449831885 Gm=0.34011839842423797 +[full s4] step 11160/12000 loss=4.2340 gn=0.98 lr=6.67e-05 tps=18319 mem=5.7G lam=0.31458022352308035 Gm=0.3415016852474461 +[full s4] step 11180/12000 loss=4.3555 gn=1.02 lr=6.64e-05 tps=18318 mem=5.7G lam=0.31467237168302137 Gm=0.3431042606631915 +[full s4] step 11200/12000 loss=4.3272 gn=0.99 lr=6.61e-05 tps=18316 mem=5.7G lam=0.314742907260855 Gm=0.34321274670461815 +[full s4] step 11220/12000 loss=4.1960 gn=1.05 lr=6.58e-05 tps=18316 mem=5.7G lam=0.3148076043774684 Gm=0.3416780672656993 +[full s4] step 11240/12000 loss=4.3998 gn=1.04 lr=6.55e-05 tps=18313 mem=5.7G lam=0.31487283514191705 Gm=0.3391622257574151 +[full s4] step 11260/12000 loss=4.3801 gn=1.06 lr=6.52e-05 tps=18312 mem=5.7G lam=0.3149531055241823 Gm=0.3387857341828446 +[full s4] step 11280/12000 loss=4.3966 gn=1.08 lr=6.49e-05 tps=18310 mem=5.7G lam=0.3150419980908434 Gm=0.3416655626303206 +[full s4] step 11300/12000 loss=4.2811 gn=1.07 lr=6.47e-05 tps=18309 mem=5.7G lam=0.3151340528080861 Gm=0.33846419599528116 +[full s4] step 11320/12000 loss=4.3468 gn=0.95 lr=6.44e-05 tps=18308 mem=5.7G lam=0.3152175440142552 Gm=0.3387984541089584 +[full s4] step 11340/12000 loss=4.5653 gn=1.10 lr=6.42e-05 tps=18307 mem=5.7G lam=0.3152738371863961 Gm=0.34136111894622445 +[full s4] step 11360/12000 loss=4.2633 gn=1.02 lr=6.39e-05 tps=18307 mem=5.7G lam=0.3153664267932375 Gm=0.33609875307107967 +[full s4] step 11380/12000 loss=4.3479 gn=1.02 lr=6.37e-05 tps=18307 mem=5.7G lam=0.3154604847853382 Gm=0.3378335024851064 +[full s4] step 11400/12000 loss=4.2996 gn=1.02 lr=6.34e-05 tps=18307 mem=5.7G lam=0.3155368436127901 Gm=0.33700776876260835 +[full s4] step 11420/12000 loss=4.2692 gn=0.98 lr=6.32e-05 tps=18307 mem=5.7G lam=0.31560262230535346 Gm=0.34021452922994894 +[full s4] step 11440/12000 loss=4.3467 gn=0.99 lr=6.30e-05 tps=18307 mem=5.7G lam=0.3156785052269697 Gm=0.34232314753656584 +[full s4] step 11460/12000 loss=4.2651 gn=0.96 lr=6.28e-05 tps=18307 mem=5.7G lam=0.31572730621943873 Gm=0.3392535323897998 +[full s4] step 11480/12000 loss=4.4858 gn=1.02 lr=6.26e-05 tps=18307 mem=5.7G lam=0.3157919067889452 Gm=0.341283911994348 +[full s4] step 11500/12000 loss=4.2447 gn=1.01 lr=6.24e-05 tps=18307 mem=5.7G lam=0.31585169211030006 Gm=0.34187414256545406 +[full s4] step 11520/12000 loss=4.4303 gn=1.03 lr=6.22e-05 tps=18307 mem=5.7G lam=0.3159155234073599 Gm=0.34371421734491986 +[full s4] step 11540/12000 loss=4.3747 gn=1.01 lr=6.20e-05 tps=18308 mem=5.7G lam=0.31598049992074567 Gm=0.3412140649743378 +[full s4] step 11560/12000 loss=4.3309 gn=1.02 lr=6.19e-05 tps=18308 mem=5.7G lam=0.31603652487198514 Gm=0.33828476110162836 +[full s4] step 11580/12000 loss=4.4435 gn=1.03 lr=6.17e-05 tps=18308 mem=5.7G lam=0.31612058356404305 Gm=0.33860448251167935 +[full s4] step 11600/12000 loss=4.2933 gn=1.03 lr=6.15e-05 tps=18308 mem=5.7G lam=0.31620523519814014 Gm=0.34238786886756617 +[full s4] step 11620/12000 loss=4.1993 gn=1.05 lr=6.14e-05 tps=18308 mem=5.7G lam=0.31628548229734105 Gm=0.33999405421006185 +[full s4] step 11640/12000 loss=4.2956 gn=1.06 lr=6.12e-05 tps=18308 mem=5.7G lam=0.3163602016866207 Gm=0.33763236133381724 +[full s4] step 11660/12000 loss=4.1862 gn=1.01 lr=6.11e-05 tps=18308 mem=5.7G lam=0.3164376424004634 Gm=0.3383478192457308 +[full s4] step 11680/12000 loss=4.3389 gn=1.14 lr=6.10e-05 tps=18308 mem=5.7G lam=0.3165389262139797 Gm=0.34029906437111396 +[full s4] step 11700/12000 loss=4.2368 gn=1.05 lr=6.09e-05 tps=18309 mem=5.7G lam=0.3166080095494787 Gm=0.33812305765847367 +[full s4] step 11720/12000 loss=4.2768 gn=1.17 lr=6.07e-05 tps=18309 mem=5.7G lam=0.3166909177477161 Gm=0.3399431367094318 +[full s4] step 11740/12000 loss=4.2603 gn=1.01 lr=6.06e-05 tps=18309 mem=5.7G lam=0.3167643683652083 Gm=0.34166207428400713 +[full s4] step 11760/12000 loss=4.3097 gn=1.07 lr=6.06e-05 tps=18309 mem=5.7G lam=0.3168382979929447 Gm=0.33923713870656985 +[full s4] step 11780/12000 loss=4.5756 gn=1.20 lr=6.05e-05 tps=18309 mem=5.7G lam=0.3169066058471799 Gm=0.3403239132215579 +[full s4] step 11800/12000 loss=4.4296 gn=1.03 lr=6.04e-05 tps=18309 mem=5.7G lam=0.316969799498717 Gm=0.3423341928670804 +[full s4] step 11820/12000 loss=4.4439 gn=1.06 lr=6.03e-05 tps=18309 mem=5.7G lam=0.3170322844137748 Gm=0.34544164842615527 +[full s4] step 11840/12000 loss=4.4207 gn=1.10 lr=6.02e-05 tps=18309 mem=5.7G lam=0.31711076758801937 Gm=0.34383001178503036 +[full s4] step 11860/12000 loss=4.2571 gn=1.08 lr=6.02e-05 tps=18309 mem=5.7G lam=0.31718006016065675 Gm=0.3408999626214306 +[full s4] step 11880/12000 loss=4.2917 gn=1.09 lr=6.01e-05 tps=18309 mem=5.7G lam=0.3172741200154026 Gm=0.34025574033148587 +[full s4] step 11900/12000 loss=4.3057 gn=1.05 lr=6.01e-05 tps=18309 mem=5.7G lam=0.31733305379748344 Gm=0.3419168271745245 +[full s4] step 11920/12000 loss=4.2874 gn=1.02 lr=6.01e-05 tps=18309 mem=5.7G lam=0.3173962002620101 Gm=0.34694111820620793 +[full s4] step 11940/12000 loss=4.4623 gn=1.16 lr=6.00e-05 tps=18309 mem=5.7G lam=0.3174550735081236 Gm=0.3404680302677055 +[full s4] step 11960/12000 loss=4.3601 gn=1.02 lr=6.00e-05 tps=18309 mem=5.7G lam=0.31750936961422366 Gm=0.34163496022423107 +[full s4] step 11980/12000 loss=4.2871 gn=0.96 lr=6.00e-05 tps=18310 mem=5.7G lam=0.3175949805105726 Gm=0.3396819991370042 +[full s4] EVAL step 11999: val_loss=4.3359 val_ppl=76.40 + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant baseline --seed 2 --out C:\Users\Bhorn\Desktop\SMATest\runs\baseline_s2 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 2001343 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant s_only --seed 2 --out C:\Users\Bhorn\Desktop\SMATest\runs\s_only_s2 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 2001343 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant g_only --seed 2 --out C:\Users\Bhorn\Desktop\SMATest\runs\g_only_s2 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 2001343 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant full --seed 2 --out C:\Users\Bhorn\Desktop\SMATest\runs\full_s2 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 2001343 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant baseline --seed 3 --out C:\Users\Bhorn\Desktop\SMATest\runs\baseline_s3 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 3001346 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant s_only --seed 3 --out C:\Users\Bhorn\Desktop\SMATest\runs\s_only_s3 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 3001346 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant g_only --seed 3 --out C:\Users\Bhorn\Desktop\SMATest\runs\g_only_s3 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 3001346 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant full --seed 3 --out C:\Users\Bhorn\Desktop\SMATest\runs\full_s3 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 3001346 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant baseline --seed 4 --out C:\Users\Bhorn\Desktop\SMATest\runs\baseline_s4 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 4001349 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant s_only --seed 4 --out C:\Users\Bhorn\Desktop\SMATest\runs\s_only_s4 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 4001349 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant g_only --seed 4 --out C:\Users\Bhorn\Desktop\SMATest\runs\g_only_s4 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 4001349 === + + +=== C:\Users\Bhorn\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.13_qbz5n2kfra8p0\python.exe C:\Users\Bhorn\Desktop\SMATest\train.py --variant full --seed 4 --out C:\Users\Bhorn\Desktop\SMATest\runs\full_s4 --max_steps 12000 --block_size 256 --batch_size 16 --grad_accum 2 --eval_interval 1000 --data_seed 4001349 === + diff --git a/s_only_s0/config.json b/s_only_s0/config.json new file mode 100644 index 0000000000000000000000000000000000000000..cdcd15a83e7dc8d8e541e83e4d2b73aa98a518fd --- /dev/null +++ b/s_only_s0/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 0, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s0", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1337, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/s_only_s0/final.pt b/s_only_s0/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..410474c187af75373f970cc9f53de38e4916352d --- /dev/null +++ b/s_only_s0/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8433e0d3eed1e842691e7a83c4af45791eb6776982e125f84ce9fafc632a3844 +size 255945415 diff --git a/s_only_s0/metrics.jsonl b/s_only_s0/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..901d9f752e83597db30b6b1bb8a233707b682c80 --- /dev/null +++ b/s_only_s0/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 0, "config": {"variant": "s_only", "seed": 0, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s0", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1337, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.925628662109375, "grad_norm": 14.688569068908691, "tokens_seen": 8192, "tokens_per_sec": 7702.543779520852, "vram_gb": 5.63817984, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.3523610432942708, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.760961532592773, "grad_norm": 2.199491262435913, "tokens_seen": 172032, "tokens_per_sec": 15735.015611881663, "vram_gb": 6.154519552, "lambda": 0.1269291490316391, "lambda_min": 0.12689170241355896, "lambda_max": 0.12697453796863556, "S_mean": 0.4612884521484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.036649703979492, "grad_norm": 1.7531449794769287, "tokens_seen": 335872, "tokens_per_sec": 16148.581019447292, "vram_gb": 6.154519552, "lambda": 0.1268723625689745, "lambda_min": 0.12678594887256622, "lambda_max": 0.12697356939315796, "S_mean": 0.49810791015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.086606979370117, "grad_norm": 1.5451058149337769, "tokens_seen": 499712, "tokens_per_sec": 16270.152291433842, "vram_gb": 6.154519552, "lambda": 0.12678918428719044, "lambda_min": 0.12667027115821838, "lambda_max": 0.12693430483341217, "S_mean": 0.4738871256510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.560234069824219, "grad_norm": 0.7172354459762573, "tokens_seen": 663552, "tokens_per_sec": 16351.255580479223, "vram_gb": 6.154519552, "lambda": 0.12669831266005835, "lambda_min": 0.1265718638896942, "lambda_max": 0.12694358825683594, "S_mean": 0.3958892822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.3254008293151855, "grad_norm": 1.046948790550232, "tokens_seen": 827392, "tokens_per_sec": 16413.3686077174, "vram_gb": 6.154519552, "lambda": 0.12659604599078497, "lambda_min": 0.12621843814849854, "lambda_max": 0.12721045315265656, "S_mean": 0.26605224609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.091524839401245, "grad_norm": 0.6135762929916382, "tokens_seen": 991232, "tokens_per_sec": 16447.769255309784, "vram_gb": 6.154519552, "lambda": 0.12653159288068613, "lambda_min": 0.1258685290813446, "lambda_max": 0.12731485068798065, "S_mean": 0.27545166015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.978118181228638, "grad_norm": 0.6487322449684143, "tokens_seen": 1155072, "tokens_per_sec": 16471.754699394154, "vram_gb": 6.154519552, "lambda": 0.1264833795527617, "lambda_min": 0.12574267387390137, "lambda_max": 0.12734214961528778, "S_mean": 0.2970377604166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.790403604507446, "grad_norm": 0.7322251796722412, "tokens_seen": 1318912, "tokens_per_sec": 16494.179948014593, "vram_gb": 6.154519552, "lambda": 0.12647119164466858, "lambda_min": 0.12559328973293304, "lambda_max": 0.12754444777965546, "S_mean": 0.245361328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.848598003387451, "grad_norm": 0.5474071502685547, "tokens_seen": 1482752, "tokens_per_sec": 16504.46196538139, "vram_gb": 6.154519552, "lambda": 0.12654991447925568, "lambda_min": 0.12551629543304443, "lambda_max": 0.1278074085712433, "S_mean": 0.2691853841145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.692680835723877, "grad_norm": 0.6192702651023865, "tokens_seen": 1646592, "tokens_per_sec": 16518.364566150092, "vram_gb": 6.154519552, "lambda": 0.12651852642496428, "lambda_min": 0.12537355720996857, "lambda_max": 0.1278560906648636, "S_mean": 0.2548319498697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.843076229095459, "grad_norm": 0.5660887956619263, "tokens_seen": 1810432, "tokens_per_sec": 16519.885661338805, "vram_gb": 6.154519552, "lambda": 0.12652307624618211, "lambda_min": 0.1252964586019516, "lambda_max": 0.1280301958322525, "S_mean": 0.22515869140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.556800603866577, "grad_norm": 0.5147239565849304, "tokens_seen": 1974272, "tokens_per_sec": 16522.240578121844, "vram_gb": 6.154519552, "lambda": 0.12653816243012747, "lambda_min": 0.12510161101818085, "lambda_max": 0.1281237006187439, "S_mean": 0.21885172526041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.66349458694458, "grad_norm": 0.6148719191551208, "tokens_seen": 2138112, "tokens_per_sec": 16526.57104452146, "vram_gb": 6.154519552, "lambda": 0.126507175154984, "lambda_min": 0.12492793053388596, "lambda_max": 0.12828396260738373, "S_mean": 0.223602294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.4217729568481445, "grad_norm": 0.6227661967277527, "tokens_seen": 2301952, "tokens_per_sec": 16523.056207998794, "vram_gb": 6.154519552, "lambda": 0.1264537163078785, "lambda_min": 0.12473845481872559, "lambda_max": 0.1285243183374405, "S_mean": 0.2692667643229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.554183721542358, "grad_norm": 0.5609625577926636, "tokens_seen": 2465792, "tokens_per_sec": 16530.215592867175, "vram_gb": 6.154519552, "lambda": 0.1264992943033576, "lambda_min": 0.12465939670801163, "lambda_max": 0.12874744832515717, "S_mean": 0.24840291341145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.360423564910889, "grad_norm": 0.6116106510162354, "tokens_seen": 2629632, "tokens_per_sec": 16532.928085361378, "vram_gb": 6.154519552, "lambda": 0.12649505771696568, "lambda_min": 0.12450471520423889, "lambda_max": 0.12890000641345978, "S_mean": 0.2699381510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.423640727996826, "grad_norm": 0.5763270258903503, "tokens_seen": 2793472, "tokens_per_sec": 16531.562951178177, "vram_gb": 6.154519552, "lambda": 0.1265031366298596, "lambda_min": 0.12426996976137161, "lambda_max": 0.1290610283613205, "S_mean": 0.2696736653645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.549881935119629, "grad_norm": 0.4863588213920593, "tokens_seen": 2957312, "tokens_per_sec": 16536.184395705754, "vram_gb": 6.154519552, "lambda": 0.12650990641365448, "lambda_min": 0.12416188418865204, "lambda_max": 0.12939277291297913, "S_mean": 0.2777913411458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.405775547027588, "grad_norm": 0.6294438242912292, "tokens_seen": 3121152, "tokens_per_sec": 16539.46577909142, "vram_gb": 6.154519552, "lambda": 0.12655804896106324, "lambda_min": 0.12411973625421524, "lambda_max": 0.12973865866661072, "S_mean": 0.2658793131510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.308966159820557, "grad_norm": 0.5131077766418457, "tokens_seen": 3284992, "tokens_per_sec": 16540.518479289236, "vram_gb": 6.154519552, "lambda": 0.1266178029278914, "lambda_min": 0.1240268275141716, "lambda_max": 0.13002583384513855, "S_mean": 0.2515767415364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.423472166061401, "grad_norm": 0.535566508769989, "tokens_seen": 3448832, "tokens_per_sec": 16541.89514390518, "vram_gb": 6.154519552, "lambda": 0.12665767719348273, "lambda_min": 0.12393064051866531, "lambda_max": 0.13025334477424622, "S_mean": 0.2764180501302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.380058765411377, "grad_norm": 0.562008798122406, "tokens_seen": 3612672, "tokens_per_sec": 16543.13810930044, "vram_gb": 6.154519552, "lambda": 0.1267284316321214, "lambda_min": 0.12394464761018753, "lambda_max": 0.13070300221443176, "S_mean": 0.3001505533854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.433675765991211, "grad_norm": 0.5677646994590759, "tokens_seen": 3776512, "tokens_per_sec": 16544.977268739094, "vram_gb": 6.154519552, "lambda": 0.12682047113776207, "lambda_min": 0.12374703586101532, "lambda_max": 0.13123652338981628, "S_mean": 0.3092447916666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.190523624420166, "grad_norm": 0.4817119836807251, "tokens_seen": 3940352, "tokens_per_sec": 16546.8386193869, "vram_gb": 6.154519552, "lambda": 0.12688480038195848, "lambda_min": 0.12351170182228088, "lambda_max": 0.1317373365163803, "S_mean": 0.3211263020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.088219881057739, "grad_norm": 0.5543108582496643, "tokens_seen": 4104192, "tokens_per_sec": 16547.25497635638, "vram_gb": 6.154519552, "lambda": 0.12700997106730938, "lambda_min": 0.12343407422304153, "lambda_max": 0.13207335770130157, "S_mean": 0.3135579427083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.301767098903656, "val_ppl": 545.5350733029372, "lambda": 0.12701023649424314, "lambda_min": 0.1234312579035759, "lambda_max": 0.13207708299160004, "lambda_per_layer": [0.13207708299160004, 0.12766626477241516, 0.12987782061100006, 0.12860751152038574, 0.12794417142868042, 0.12760312855243683, 0.12602831423282623, 0.1271485537290573, 0.1271422952413559, 0.12840500473976135, 0.12652865052223206, 0.12680268287658691, 0.1273096203804016, 0.12616917490959167, 0.1280900239944458, 0.12836585938930511, 0.1269630342721939, 0.12680268287658691, 0.12470059841871262, 0.1249258890748024, 0.123964324593544, 0.1234312579035759, 0.12540511786937714, 0.1262866109609604], "S_mean": 0.3033650716145833, "G_mean": null, "G_std": null, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.273839473724365, "grad_norm": 0.554238498210907, "tokens_seen": 4268032, "tokens_per_sec": 16310.801548411951, "vram_gb": 6.154519552, "lambda": 0.12714012960592905, "lambda_min": 0.12348220497369766, "lambda_max": 0.1323927491903305, "S_mean": 0.285552978515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.198536396026611, "grad_norm": 0.5492505431175232, "tokens_seen": 4431872, "tokens_per_sec": 16317.121015005625, "vram_gb": 6.154519552, "lambda": 0.12725414087375006, "lambda_min": 0.12355977296829224, "lambda_max": 0.13273081183433533, "S_mean": 0.3133443196614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.176833629608154, "grad_norm": 0.5152229070663452, "tokens_seen": 4595712, "tokens_per_sec": 16321.93274160171, "vram_gb": 6.154519552, "lambda": 0.12743254657834768, "lambda_min": 0.1236424446105957, "lambda_max": 0.13299690186977386, "S_mean": 0.304534912109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.226116418838501, "grad_norm": 0.5399757623672485, "tokens_seen": 4759552, "tokens_per_sec": 16331.44723925319, "vram_gb": 6.154519552, "lambda": 0.12754579167813063, "lambda_min": 0.1235351487994194, "lambda_max": 0.13347215950489044, "S_mean": 0.3068440755208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.170430660247803, "grad_norm": 0.5182417035102844, "tokens_seen": 4923392, "tokens_per_sec": 16340.692421684698, "vram_gb": 6.154519552, "lambda": 0.1276875389739871, "lambda_min": 0.12362095713615417, "lambda_max": 0.13383165001869202, "S_mean": 0.3199462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.058792591094971, "grad_norm": 0.48541226983070374, "tokens_seen": 5087232, "tokens_per_sec": 16348.5556592003, "vram_gb": 6.154519552, "lambda": 0.12788229497770467, "lambda_min": 0.12377098202705383, "lambda_max": 0.13410797715187073, "S_mean": 0.3073069254557292, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.157637357711792, "grad_norm": 0.5322015285491943, "tokens_seen": 5251072, "tokens_per_sec": 16354.76101701099, "vram_gb": 6.154519552, "lambda": 0.12811957268665233, "lambda_min": 0.12396907806396484, "lambda_max": 0.1344953030347824, "S_mean": 0.330963134765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.054185152053833, "grad_norm": 0.5900415182113647, "tokens_seen": 5414912, "tokens_per_sec": 16365.55984024882, "vram_gb": 6.154519552, "lambda": 0.12820511466513076, "lambda_min": 0.12361082434654236, "lambda_max": 0.13476316630840302, "S_mean": 0.34295654296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.038771152496338, "grad_norm": 0.5264078974723816, "tokens_seen": 5578752, "tokens_per_sec": 16370.267221860686, "vram_gb": 6.154519552, "lambda": 0.12838880127916732, "lambda_min": 0.12382928282022476, "lambda_max": 0.1352706104516983, "S_mean": 0.3626200358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 6.0081751346588135, "grad_norm": 0.5836257934570312, "tokens_seen": 5742592, "tokens_per_sec": 16367.677008634566, "vram_gb": 6.154519552, "lambda": 0.12858952147265276, "lambda_min": 0.12402649968862534, "lambda_max": 0.13563284277915955, "S_mean": 0.3387247721354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.112834215164185, "grad_norm": 0.6155970692634583, "tokens_seen": 5906432, "tokens_per_sec": 16365.583387223684, "vram_gb": 6.154519552, "lambda": 0.12882865810145935, "lambda_min": 0.12441467493772507, "lambda_max": 0.13605955243110657, "S_mean": 0.307769775390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.949610471725464, "grad_norm": 0.5209303498268127, "tokens_seen": 6070272, "tokens_per_sec": 16363.777721280796, "vram_gb": 6.154519552, "lambda": 0.12914348921428123, "lambda_min": 0.12428810447454453, "lambda_max": 0.13660649955272675, "S_mean": 0.3181050618489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.935041666030884, "grad_norm": 0.541314959526062, "tokens_seen": 6234112, "tokens_per_sec": 16362.844748204121, "vram_gb": 6.154519552, "lambda": 0.12943589904656014, "lambda_min": 0.12416676431894302, "lambda_max": 0.13713733851909637, "S_mean": 0.333221435546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.946949243545532, "grad_norm": 0.5159759521484375, "tokens_seen": 6397952, "tokens_per_sec": 16361.011354346028, "vram_gb": 6.154519552, "lambda": 0.12967557397981486, "lambda_min": 0.12381567060947418, "lambda_max": 0.13736915588378906, "S_mean": 0.3218485514322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.861619472503662, "grad_norm": 0.606601357460022, "tokens_seen": 6561792, "tokens_per_sec": 16359.27662530161, "vram_gb": 6.154519552, "lambda": 0.12991350578765073, "lambda_min": 0.12362536787986755, "lambda_max": 0.13767099380493164, "S_mean": 0.3233846028645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.830610752105713, "grad_norm": 0.5699958205223083, "tokens_seen": 6725632, "tokens_per_sec": 16357.877909106926, "vram_gb": 6.154519552, "lambda": 0.13019642885774374, "lambda_min": 0.12346949428319931, "lambda_max": 0.137894406914711, "S_mean": 0.3323872884114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.816241025924683, "grad_norm": 0.5522083640098572, "tokens_seen": 6889472, "tokens_per_sec": 16356.557264683724, "vram_gb": 6.154519552, "lambda": 0.13047212238113085, "lambda_min": 0.1232302263379097, "lambda_max": 0.13884688913822174, "S_mean": 0.3267415364583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.762941598892212, "grad_norm": 0.5572078227996826, "tokens_seen": 7053312, "tokens_per_sec": 16357.161088812558, "vram_gb": 6.154519552, "lambda": 0.1307267645994822, "lambda_min": 0.12307729572057724, "lambda_max": 0.1399075984954834, "S_mean": 0.31201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.942544221878052, "grad_norm": 0.5627756714820862, "tokens_seen": 7217152, "tokens_per_sec": 16357.176435950318, "vram_gb": 6.154519552, "lambda": 0.13104286634673676, "lambda_min": 0.12299304455518723, "lambda_max": 0.14095719158649445, "S_mean": 0.3304443359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.767329216003418, "grad_norm": 0.530323326587677, "tokens_seen": 7380992, "tokens_per_sec": 16358.147112118631, "vram_gb": 6.154519552, "lambda": 0.13141013123095036, "lambda_min": 0.12284353375434875, "lambda_max": 0.14214305579662323, "S_mean": 0.28118896484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.799370050430298, "grad_norm": 0.535502016544342, "tokens_seen": 7544832, "tokens_per_sec": 16358.776101392128, "vram_gb": 6.154519552, "lambda": 0.1317999962096413, "lambda_min": 0.12271346151828766, "lambda_max": 0.1431686282157898, "S_mean": 0.32647705078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.758881330490112, "grad_norm": 0.5248954892158508, "tokens_seen": 7708672, "tokens_per_sec": 16358.389664916245, "vram_gb": 6.154519552, "lambda": 0.13219300843775272, "lambda_min": 0.12280253320932388, "lambda_max": 0.14435744285583496, "S_mean": 0.3160807291666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.79512882232666, "grad_norm": 0.5584592223167419, "tokens_seen": 7872512, "tokens_per_sec": 16356.927717276654, "vram_gb": 6.154519552, "lambda": 0.13255574026455483, "lambda_min": 0.12263508886098862, "lambda_max": 0.14565157890319824, "S_mean": 0.290771484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.909126043319702, "grad_norm": 0.5668708682060242, "tokens_seen": 8036352, "tokens_per_sec": 16356.762545191952, "vram_gb": 6.154519552, "lambda": 0.13292946728567281, "lambda_min": 0.12195872515439987, "lambda_max": 0.1469794064760208, "S_mean": 0.3133646647135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.83882737159729, "grad_norm": 0.5923149585723877, "tokens_seen": 8200192, "tokens_per_sec": 16352.756209220077, "vram_gb": 6.154519552, "lambda": 0.13331958558410406, "lambda_min": 0.12155242264270782, "lambda_max": 0.14813101291656494, "S_mean": 0.3035990397135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.834398317337036, "val_ppl": 341.85898153525244, "lambda": 0.1333371583993236, "lambda_min": 0.1215352788567543, "lambda_max": 0.14818662405014038, "lambda_per_layer": [0.14158402383327484, 0.1289932131767273, 0.13365478813648224, 0.13174989819526672, 0.12996448576450348, 0.12763626873493195, 0.12205778062343597, 0.1215352788567543, 0.12408959865570068, 0.1328001171350479, 0.1304013729095459, 0.14072272181510925, 0.13978248834609985, 0.13806596398353577, 0.14818662405014038, 0.1425640881061554, 0.14073091745376587, 0.1434503197669983, 0.13401579856872559, 0.13407807052135468, 0.12846256792545319, 0.12900389730930328, 0.12762458622455597, 0.12893693149089813], "S_mean": 0.3284403483072917, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.6499924659729, "grad_norm": 0.5494115352630615, "tokens_seen": 8364032, "tokens_per_sec": 16232.873646874827, "vram_gb": 6.154519552, "lambda": 0.13371922448277473, "lambda_min": 0.12123194336891174, "lambda_max": 0.14941202104091644, "S_mean": 0.2906087239583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.721419811248779, "grad_norm": 0.5575782060623169, "tokens_seen": 8527872, "tokens_per_sec": 16233.383343173136, "vram_gb": 6.154519552, "lambda": 0.13419483602046967, "lambda_min": 0.12101933360099792, "lambda_max": 0.1507621854543686, "S_mean": 0.2989044189453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.762885332107544, "grad_norm": 0.537432849407196, "tokens_seen": 8691712, "tokens_per_sec": 16235.222097665794, "vram_gb": 6.154519552, "lambda": 0.13460853043943644, "lambda_min": 0.12059590220451355, "lambda_max": 0.15210220217704773, "S_mean": 0.3155924479166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.750052213668823, "grad_norm": 0.6416492462158203, "tokens_seen": 8855552, "tokens_per_sec": 16236.962976594848, "vram_gb": 6.154519552, "lambda": 0.13512087613344193, "lambda_min": 0.12023721635341644, "lambda_max": 0.1534857600927353, "S_mean": 0.3123677571614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.636714458465576, "grad_norm": 0.5308393239974976, "tokens_seen": 9019392, "tokens_per_sec": 16238.329282937257, "vram_gb": 6.154519552, "lambda": 0.135609638877213, "lambda_min": 0.1200370341539383, "lambda_max": 0.15471406280994415, "S_mean": 0.3047587076822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.741352081298828, "grad_norm": 0.5577899813652039, "tokens_seen": 9183232, "tokens_per_sec": 16238.108790749291, "vram_gb": 6.154519552, "lambda": 0.13602933008223772, "lambda_min": 0.11986948549747467, "lambda_max": 0.15601399540901184, "S_mean": 0.303924560546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.5556371212005615, "grad_norm": 0.5518603324890137, "tokens_seen": 9347072, "tokens_per_sec": 16241.025597959444, "vram_gb": 6.154519552, "lambda": 0.13641619806488356, "lambda_min": 0.11986977607011795, "lambda_max": 0.15737184882164001, "S_mean": 0.2980753580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.6024158000946045, "grad_norm": 0.5939144492149353, "tokens_seen": 9510912, "tokens_per_sec": 16242.541612715593, "vram_gb": 6.154519552, "lambda": 0.13685524898270765, "lambda_min": 0.11953702569007874, "lambda_max": 0.15864421427249908, "S_mean": 0.2840118408203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.668419361114502, "grad_norm": 0.5077447891235352, "tokens_seen": 9674752, "tokens_per_sec": 16244.444419885915, "vram_gb": 6.154519552, "lambda": 0.137272611570855, "lambda_min": 0.11896297335624695, "lambda_max": 0.15992851555347443, "S_mean": 0.294342041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.535736083984375, "grad_norm": 0.6504412889480591, "tokens_seen": 9838592, "tokens_per_sec": 16244.862247658762, "vram_gb": 6.154519552, "lambda": 0.13768705042699972, "lambda_min": 0.11858074367046356, "lambda_max": 0.16125763952732086, "S_mean": 0.2762349446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.73411750793457, "grad_norm": 0.5654910802841187, "tokens_seen": 10002432, "tokens_per_sec": 16245.338176956806, "vram_gb": 6.154519552, "lambda": 0.13817448013772568, "lambda_min": 0.11823441088199615, "lambda_max": 0.16263996064662933, "S_mean": 0.3185323079427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.689031362533569, "grad_norm": 0.5388892889022827, "tokens_seen": 10166272, "tokens_per_sec": 16246.516857858087, "vram_gb": 6.154519552, "lambda": 0.13863906636834145, "lambda_min": 0.11792685836553574, "lambda_max": 0.16408860683441162, "S_mean": 0.30255126953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.6540608406066895, "grad_norm": 0.7863057851791382, "tokens_seen": 10330112, "tokens_per_sec": 16248.325288255339, "vram_gb": 6.154519552, "lambda": 0.13909072522073984, "lambda_min": 0.11790654063224792, "lambda_max": 0.16527871787548065, "S_mean": 0.3190714518229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.490113735198975, "grad_norm": 0.5377177596092224, "tokens_seen": 10493952, "tokens_per_sec": 16248.071692720889, "vram_gb": 6.154519552, "lambda": 0.13960564974695444, "lambda_min": 0.11777402460575104, "lambda_max": 0.16655953228473663, "S_mean": 0.2781219482421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.587963819503784, "grad_norm": 0.560196042060852, "tokens_seen": 10657792, "tokens_per_sec": 16249.708148894846, "vram_gb": 6.154519552, "lambda": 0.14002052570382753, "lambda_min": 0.1177426129579544, "lambda_max": 0.16787226498126984, "S_mean": 0.3093058268229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.5450663566589355, "grad_norm": 0.5587137937545776, "tokens_seen": 10821632, "tokens_per_sec": 16251.220051433034, "vram_gb": 6.154519552, "lambda": 0.14044411045809588, "lambda_min": 0.11727344244718552, "lambda_max": 0.16913875937461853, "S_mean": 0.3102925618489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.6545867919921875, "grad_norm": 0.6014097929000854, "tokens_seen": 10985472, "tokens_per_sec": 16253.05234548823, "vram_gb": 6.154519552, "lambda": 0.14091022374729315, "lambda_min": 0.11698122322559357, "lambda_max": 0.17054076492786407, "S_mean": 0.2988789876302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.473868131637573, "grad_norm": 0.5577253103256226, "tokens_seen": 11149312, "tokens_per_sec": 16254.502489129767, "vram_gb": 6.154519552, "lambda": 0.1413696048160394, "lambda_min": 0.11655095964670181, "lambda_max": 0.1718759834766388, "S_mean": 0.2975870768229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.300252914428711, "grad_norm": 0.5759209990501404, "tokens_seen": 11313152, "tokens_per_sec": 16255.966785342705, "vram_gb": 6.154519552, "lambda": 0.14189015049487352, "lambda_min": 0.1158924400806427, "lambda_max": 0.1732180416584015, "S_mean": 0.275146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.588867902755737, "grad_norm": 0.5483385920524597, "tokens_seen": 11476992, "tokens_per_sec": 16257.627555634817, "vram_gb": 6.154519552, "lambda": 0.1423905324190855, "lambda_min": 0.11577700078487396, "lambda_max": 0.17426453530788422, "S_mean": 0.2728271484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.536706447601318, "grad_norm": 0.5535503625869751, "tokens_seen": 11640832, "tokens_per_sec": 16259.216145131793, "vram_gb": 6.154519552, "lambda": 0.14287546804795662, "lambda_min": 0.11547210067510605, "lambda_max": 0.17559607326984406, "S_mean": 0.2706197102864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.41114068031311, "grad_norm": 0.5306376814842224, "tokens_seen": 11804672, "tokens_per_sec": 16259.826997825905, "vram_gb": 6.154519552, "lambda": 0.143426896383365, "lambda_min": 0.11487135291099548, "lambda_max": 0.17697837948799133, "S_mean": 0.3018391927083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.530771017074585, "grad_norm": 0.5668029189109802, "tokens_seen": 11968512, "tokens_per_sec": 16261.076827946183, "vram_gb": 6.154519552, "lambda": 0.14388229014972845, "lambda_min": 0.11444707959890366, "lambda_max": 0.17823833227157593, "S_mean": 0.2944742838541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.43363094329834, "grad_norm": 0.562773585319519, "tokens_seen": 12132352, "tokens_per_sec": 16261.694836892868, "vram_gb": 6.154519552, "lambda": 0.14435840335985026, "lambda_min": 0.11444797366857529, "lambda_max": 0.17953695356845856, "S_mean": 0.2917683919270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.6808202266693115, "grad_norm": 0.5982413291931152, "tokens_seen": 12296192, "tokens_per_sec": 16261.399573523637, "vram_gb": 6.154519552, "lambda": 0.1448592577750484, "lambda_min": 0.11438814550638199, "lambda_max": 0.18088789284229279, "S_mean": 0.3269449869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.5502973079681395, "val_ppl": 257.3140460508517, "lambda": 0.14488916068027416, "lambda_min": 0.11437331140041351, "lambda_max": 0.1809546947479248, "lambda_per_layer": [0.14975300431251526, 0.12921102344989777, 0.13573764264583588, 0.1326431781053543, 0.1306908279657364, 0.1282563954591751, 0.12125341594219208, 0.11437331140041351, 0.11895065754652023, 0.14040203392505646, 0.14648602902889252, 0.16518375277519226, 0.16766807436943054, 0.16349732875823975, 0.1809546947479248, 0.16373524069786072, 0.16717945039272308, 0.16929848492145538, 0.15738046169281006, 0.1512286365032196, 0.1365227848291397, 0.13623501360416412, 0.1369630992412567, 0.1337353140115738], "S_mean": 0.2934773763020833, "G_mean": null, "G_std": null, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.4572179317474365, "grad_norm": 0.6381241679191589, "tokens_seen": 12460032, "tokens_per_sec": 16183.441196717127, "vram_gb": 6.154519552, "lambda": 0.14535083683828512, "lambda_min": 0.11406568437814713, "lambda_max": 0.18205195665359497, "S_mean": 0.278350830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.308258533477783, "grad_norm": 0.5368949770927429, "tokens_seen": 12623872, "tokens_per_sec": 16185.24051333753, "vram_gb": 6.154519552, "lambda": 0.14577612529198328, "lambda_min": 0.11366383731365204, "lambda_max": 0.18303744494915009, "S_mean": 0.266510009765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.556448459625244, "grad_norm": 0.6932092905044556, "tokens_seen": 12787712, "tokens_per_sec": 16187.714131100343, "vram_gb": 6.154519552, "lambda": 0.14625812321901321, "lambda_min": 0.11324971914291382, "lambda_max": 0.18422037363052368, "S_mean": 0.3162841796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.448531150817871, "grad_norm": 0.6101506352424622, "tokens_seen": 12951552, "tokens_per_sec": 16190.15447774905, "vram_gb": 6.154519552, "lambda": 0.14671009592711926, "lambda_min": 0.1128857359290123, "lambda_max": 0.18544748425483704, "S_mean": 0.28289794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.57084846496582, "grad_norm": 0.5963761210441589, "tokens_seen": 13115392, "tokens_per_sec": 16192.237689196805, "vram_gb": 6.154519552, "lambda": 0.14719284946719804, "lambda_min": 0.11234772950410843, "lambda_max": 0.18663448095321655, "S_mean": 0.2792460123697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.3608880043029785, "grad_norm": 0.5718636512756348, "tokens_seen": 13279232, "tokens_per_sec": 16195.003165820119, "vram_gb": 6.154519552, "lambda": 0.1477067389835914, "lambda_min": 0.11197847127914429, "lambda_max": 0.18780668079853058, "S_mean": 0.2768656412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.410727024078369, "grad_norm": 0.5744139552116394, "tokens_seen": 13443072, "tokens_per_sec": 16196.698499462904, "vram_gb": 6.154519552, "lambda": 0.1481638333449761, "lambda_min": 0.11154165118932724, "lambda_max": 0.18914203345775604, "S_mean": 0.277313232421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.402293920516968, "grad_norm": 0.6190707087516785, "tokens_seen": 13606912, "tokens_per_sec": 16198.502221740322, "vram_gb": 6.154519552, "lambda": 0.14868756011128426, "lambda_min": 0.11129354685544968, "lambda_max": 0.19052095711231232, "S_mean": 0.28472900390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.493335723876953, "grad_norm": 0.5947735905647278, "tokens_seen": 13770752, "tokens_per_sec": 16199.46485065894, "vram_gb": 6.154519552, "lambda": 0.14921100592861572, "lambda_min": 0.11130726337432861, "lambda_max": 0.19193144142627716, "S_mean": 0.2649434407552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.432307004928589, "grad_norm": 0.6556434631347656, "tokens_seen": 13934592, "tokens_per_sec": 16200.877788366153, "vram_gb": 6.154519552, "lambda": 0.14970508000502983, "lambda_min": 0.11121158301830292, "lambda_max": 0.19316060841083527, "S_mean": 0.2661031087239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.389029026031494, "grad_norm": 0.8077160120010376, "tokens_seen": 14098432, "tokens_per_sec": 16202.453706929908, "vram_gb": 6.154519552, "lambda": 0.1502062166109681, "lambda_min": 0.11077576875686646, "lambda_max": 0.194486603140831, "S_mean": 0.2851765950520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.442358016967773, "grad_norm": 0.7641776204109192, "tokens_seen": 14262272, "tokens_per_sec": 16203.24384118331, "vram_gb": 6.154519552, "lambda": 0.15072913623104492, "lambda_min": 0.11044943332672119, "lambda_max": 0.19591538608074188, "S_mean": 0.27630615234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.400934219360352, "grad_norm": 0.5686379671096802, "tokens_seen": 14426112, "tokens_per_sec": 16204.406212380694, "vram_gb": 6.154519552, "lambda": 0.1511542359367013, "lambda_min": 0.1101161316037178, "lambda_max": 0.19700519740581512, "S_mean": 0.277923583984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.350971698760986, "grad_norm": 0.5952969193458557, "tokens_seen": 14589952, "tokens_per_sec": 16204.918891389038, "vram_gb": 6.154519552, "lambda": 0.1516593744357427, "lambda_min": 0.10984304547309875, "lambda_max": 0.1982942372560501, "S_mean": 0.2892252604166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.381747722625732, "grad_norm": 0.6235257387161255, "tokens_seen": 14753792, "tokens_per_sec": 16205.88260966548, "vram_gb": 6.154519552, "lambda": 0.15221729936699072, "lambda_min": 0.10976938158273697, "lambda_max": 0.19978363811969757, "S_mean": 0.2636210123697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.303675889968872, "grad_norm": 0.5741193294525146, "tokens_seen": 14917632, "tokens_per_sec": 16207.140087778918, "vram_gb": 6.154519552, "lambda": 0.152794297474126, "lambda_min": 0.1095517948269844, "lambda_max": 0.2012704312801361, "S_mean": 0.2780863444010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.445024013519287, "grad_norm": 0.6352857947349548, "tokens_seen": 15081472, "tokens_per_sec": 16208.88981887711, "vram_gb": 6.154519552, "lambda": 0.15329882968217134, "lambda_min": 0.10911458730697632, "lambda_max": 0.20268304646015167, "S_mean": 0.2738240559895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.4056878089904785, "grad_norm": 0.6440662145614624, "tokens_seen": 15245312, "tokens_per_sec": 16210.694696300165, "vram_gb": 6.154519552, "lambda": 0.15386295039206743, "lambda_min": 0.10889874398708344, "lambda_max": 0.20423667132854462, "S_mean": 0.2823486328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.337356805801392, "grad_norm": 0.5976075530052185, "tokens_seen": 15409152, "tokens_per_sec": 16212.12461276287, "vram_gb": 6.154519552, "lambda": 0.15437764674425125, "lambda_min": 0.10857870429754257, "lambda_max": 0.2058199793100357, "S_mean": 0.291473388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.276200294494629, "grad_norm": 0.6352754235267639, "tokens_seen": 15572992, "tokens_per_sec": 16214.109610451762, "vram_gb": 6.154519552, "lambda": 0.15489212019989887, "lambda_min": 0.10832621157169342, "lambda_max": 0.2070039063692093, "S_mean": 0.3021443684895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.236283302307129, "grad_norm": 0.5937898755073547, "tokens_seen": 15736832, "tokens_per_sec": 16215.297764892965, "vram_gb": 6.154519552, "lambda": 0.155486432214578, "lambda_min": 0.1080838069319725, "lambda_max": 0.20852091908454895, "S_mean": 0.2828267415364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.209023714065552, "grad_norm": 0.5850639939308167, "tokens_seen": 15900672, "tokens_per_sec": 16216.574758401, "vram_gb": 6.154519552, "lambda": 0.1559885237365961, "lambda_min": 0.10810219496488571, "lambda_max": 0.20984266698360443, "S_mean": 0.2773946126302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.1999192237854, "grad_norm": 0.648499071598053, "tokens_seen": 16064512, "tokens_per_sec": 16217.534201194545, "vram_gb": 6.154519552, "lambda": 0.15653540287166834, "lambda_min": 0.10760340839624405, "lambda_max": 0.21085572242736816, "S_mean": 0.28594970703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.0822813510894775, "grad_norm": 0.6417776346206665, "tokens_seen": 16228352, "tokens_per_sec": 16218.708758536197, "vram_gb": 6.154519552, "lambda": 0.15712275231877962, "lambda_min": 0.10722942650318146, "lambda_max": 0.21227875351905823, "S_mean": 0.3030904134114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.255295038223267, "grad_norm": 0.6254792809486389, "tokens_seen": 16392192, "tokens_per_sec": 16220.351594211661, "vram_gb": 6.154519552, "lambda": 0.1575803036491076, "lambda_min": 0.10691603273153305, "lambda_max": 0.21358367800712585, "S_mean": 0.3012593587239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.360101664066315, "val_ppl": 212.74657403196483, "lambda": 0.15760540558646122, "lambda_min": 0.10690365731716156, "lambda_max": 0.21364153921604156, "lambda_per_layer": [0.1550525426864624, 0.13075223565101624, 0.13677892088890076, 0.13330568373203278, 0.13156664371490479, 0.12833185493946075, 0.11882799863815308, 0.10690365731716156, 0.11233232170343399, 0.14674541354179382, 0.16487324237823486, 0.1872970461845398, 0.20047204196453094, 0.18876078724861145, 0.21364153921604156, 0.18572714924812317, 0.19962778687477112, 0.20294912159442902, 0.18670307099819183, 0.17184928059577942, 0.14866042137145996, 0.14445041120052338, 0.14766085147857666, 0.1392597109079361], "S_mean": 0.2898661295572917, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.1400487422943115, "grad_norm": 0.6762721538543701, "tokens_seen": 16556032, "tokens_per_sec": 16161.719229627148, "vram_gb": 6.154519552, "lambda": 0.1581548529987534, "lambda_min": 0.10677851736545563, "lambda_max": 0.21487554907798767, "S_mean": 0.2802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.3224406242370605, "grad_norm": 0.5971077084541321, "tokens_seen": 16719872, "tokens_per_sec": 16163.762745364453, "vram_gb": 6.154519552, "lambda": 0.15864485315978527, "lambda_min": 0.10668017715215683, "lambda_max": 0.2161921113729477, "S_mean": 0.2870992024739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.324337482452393, "grad_norm": 0.614822268486023, "tokens_seen": 16883712, "tokens_per_sec": 16165.286451584809, "vram_gb": 6.154519552, "lambda": 0.15910970916350684, "lambda_min": 0.10629883408546448, "lambda_max": 0.21744360029697418, "S_mean": 0.2728983561197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.262296915054321, "grad_norm": 0.7054792046546936, "tokens_seen": 17047552, "tokens_per_sec": 16167.635339555984, "vram_gb": 6.154519552, "lambda": 0.15963038491706052, "lambda_min": 0.10604335367679596, "lambda_max": 0.2187192738056183, "S_mean": 0.2685597737630208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.266428709030151, "grad_norm": 0.6284931302070618, "tokens_seen": 17211392, "tokens_per_sec": 16169.030442678297, "vram_gb": 6.154519552, "lambda": 0.16016417586555085, "lambda_min": 0.10556048154830933, "lambda_max": 0.22010454535484314, "S_mean": 0.287017822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.225936412811279, "grad_norm": 0.7028906941413879, "tokens_seen": 17375232, "tokens_per_sec": 16171.215444286714, "vram_gb": 6.154519552, "lambda": 0.16073347721248865, "lambda_min": 0.10543413460254669, "lambda_max": 0.22164784371852875, "S_mean": 0.2777506510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.300853729248047, "grad_norm": 0.6309285759925842, "tokens_seen": 17539072, "tokens_per_sec": 16173.272968241736, "vram_gb": 6.154519552, "lambda": 0.1612242196376125, "lambda_min": 0.10518274456262589, "lambda_max": 0.22315658628940582, "S_mean": 0.2686055501302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.340150356292725, "grad_norm": 0.710176944732666, "tokens_seen": 17702912, "tokens_per_sec": 16174.926875356445, "vram_gb": 6.154519552, "lambda": 0.16176896231869856, "lambda_min": 0.10514882951974869, "lambda_max": 0.22461576759815216, "S_mean": 0.2710978190104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.297566175460815, "grad_norm": 0.6350734829902649, "tokens_seen": 17866752, "tokens_per_sec": 16176.293664331652, "vram_gb": 6.154519552, "lambda": 0.1622620935862263, "lambda_min": 0.10491596162319183, "lambda_max": 0.22609415650367737, "S_mean": 0.2820536295572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.243932008743286, "grad_norm": 0.6145501732826233, "tokens_seen": 18030592, "tokens_per_sec": 16177.56059709483, "vram_gb": 6.154519552, "lambda": 0.16275164764374495, "lambda_min": 0.10441883653402328, "lambda_max": 0.22763113677501678, "S_mean": 0.261199951171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.2438929080963135, "grad_norm": 0.6603432297706604, "tokens_seen": 18194432, "tokens_per_sec": 16178.517085526726, "vram_gb": 6.154519552, "lambda": 0.16329598737259707, "lambda_min": 0.10422676801681519, "lambda_max": 0.22872743010520935, "S_mean": 0.2756551106770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.143738269805908, "grad_norm": 0.6552905440330505, "tokens_seen": 18358272, "tokens_per_sec": 16180.28620978698, "vram_gb": 6.154519552, "lambda": 0.16387491455922523, "lambda_min": 0.10384377092123032, "lambda_max": 0.23014813661575317, "S_mean": 0.2608744303385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.2503602504730225, "grad_norm": 0.6460416913032532, "tokens_seen": 18522112, "tokens_per_sec": 16181.384012138202, "vram_gb": 6.154519552, "lambda": 0.16447194665670395, "lambda_min": 0.1033526137471199, "lambda_max": 0.2315320521593094, "S_mean": 0.2617085774739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.176873683929443, "grad_norm": 0.6996647119522095, "tokens_seen": 18685952, "tokens_per_sec": 16181.784211106045, "vram_gb": 6.154519552, "lambda": 0.16499119562407336, "lambda_min": 0.10280215740203857, "lambda_max": 0.23258249461650848, "S_mean": 0.26055908203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.203664302825928, "grad_norm": 0.6545577645301819, "tokens_seen": 18849792, "tokens_per_sec": 16183.121700000598, "vram_gb": 6.154519552, "lambda": 0.1655031287421783, "lambda_min": 0.10248968005180359, "lambda_max": 0.23389632999897003, "S_mean": 0.2746988932291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.1658923625946045, "grad_norm": 0.648708164691925, "tokens_seen": 19013632, "tokens_per_sec": 16184.030901752403, "vram_gb": 6.154519552, "lambda": 0.16607369109988213, "lambda_min": 0.1022682711482048, "lambda_max": 0.23529405891895294, "S_mean": 0.2608439127604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.126562833786011, "grad_norm": 0.6481983661651611, "tokens_seen": 19177472, "tokens_per_sec": 16185.41823355059, "vram_gb": 6.154519552, "lambda": 0.16658391834547123, "lambda_min": 0.10208617895841599, "lambda_max": 0.23672553896903992, "S_mean": 0.2892557779947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.204838275909424, "grad_norm": 0.6551932096481323, "tokens_seen": 19341312, "tokens_per_sec": 16186.792638115669, "vram_gb": 6.154519552, "lambda": 0.16710987128317356, "lambda_min": 0.10166148841381073, "lambda_max": 0.23816893994808197, "S_mean": 0.2782185872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.2975664138793945, "grad_norm": 0.7882553339004517, "tokens_seen": 19505152, "tokens_per_sec": 16187.85638739152, "vram_gb": 6.154519552, "lambda": 0.1676124402632316, "lambda_min": 0.1013035699725151, "lambda_max": 0.23932451009750366, "S_mean": 0.2548624674479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.176799774169922, "grad_norm": 0.6522775888442993, "tokens_seen": 19668992, "tokens_per_sec": 16188.439701589334, "vram_gb": 6.154519552, "lambda": 0.16816102030376592, "lambda_min": 0.10101649910211563, "lambda_max": 0.2404613196849823, "S_mean": 0.2685343424479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.278141021728516, "grad_norm": 0.6511480808258057, "tokens_seen": 19832832, "tokens_per_sec": 16189.537926346138, "vram_gb": 6.154519552, "lambda": 0.1687153841679295, "lambda_min": 0.10092904418706894, "lambda_max": 0.24167411029338837, "S_mean": 0.2838236490885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.069621801376343, "grad_norm": 0.6863592267036438, "tokens_seen": 19996672, "tokens_per_sec": 16190.553585353215, "vram_gb": 6.154519552, "lambda": 0.16925037807474533, "lambda_min": 0.1005568876862526, "lambda_max": 0.24295710027217865, "S_mean": 0.28106689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.111523151397705, "grad_norm": 0.7249911427497864, "tokens_seen": 20160512, "tokens_per_sec": 16191.65718925072, "vram_gb": 6.154519552, "lambda": 0.16977348178625107, "lambda_min": 0.10023926198482513, "lambda_max": 0.2441737800836563, "S_mean": 0.2727254231770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.089618444442749, "grad_norm": 0.6413151621818542, "tokens_seen": 20324352, "tokens_per_sec": 16192.932341373102, "vram_gb": 6.154519552, "lambda": 0.17023571642736593, "lambda_min": 0.0997491255402565, "lambda_max": 0.24514666199684143, "S_mean": 0.27166748046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.2228357791900635, "grad_norm": 0.6540395021438599, "tokens_seen": 20488192, "tokens_per_sec": 16194.433673168145, "vram_gb": 6.154519552, "lambda": 0.17075345230599245, "lambda_min": 0.09926433116197586, "lambda_max": 0.24620454013347626, "S_mean": 0.2810262044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.213802897930146, "val_ppl": 183.7916718446572, "lambda": 0.17078115791082382, "lambda_min": 0.09922786056995392, "lambda_max": 0.2462700456380844, "lambda_per_layer": [0.1573614627122879, 0.13182970881462097, 0.13815583288669586, 0.13396258652210236, 0.1322489231824875, 0.12894710898399353, 0.11629972606897354, 0.09922786056995392, 0.10490449517965317, 0.15181215107440948, 0.18092723190784454, 0.20686084032058716, 0.23309838771820068, 0.21212387084960938, 0.2462700456380844, 0.20898154377937317, 0.2386000156402588, 0.24328012764453888, 0.2209152728319168, 0.19213517010211945, 0.1622866690158844, 0.15167251229286194, 0.16009721159934998, 0.14674903452396393], "S_mean": 0.2727559407552083, "G_mean": null, "G_std": null, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.129201412200928, "grad_norm": 0.6390261650085449, "tokens_seen": 20652032, "tokens_per_sec": 16148.735361463481, "vram_gb": 6.154519552, "lambda": 0.1712930944437782, "lambda_min": 0.09864762425422668, "lambda_max": 0.2475050985813141, "S_mean": 0.2780354817708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.013932228088379, "grad_norm": 0.6341016292572021, "tokens_seen": 20815872, "tokens_per_sec": 16149.714799116298, "vram_gb": 6.154519552, "lambda": 0.17185424516598383, "lambda_min": 0.09864171594381332, "lambda_max": 0.24888524413108826, "S_mean": 0.2978617350260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.253535270690918, "grad_norm": 0.7633364796638489, "tokens_seen": 20979712, "tokens_per_sec": 16151.25114373186, "vram_gb": 6.154519552, "lambda": 0.17237655725330114, "lambda_min": 0.09862274676561356, "lambda_max": 0.2501905560493469, "S_mean": 0.2794698079427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.144296169281006, "grad_norm": 0.6668788194656372, "tokens_seen": 21143552, "tokens_per_sec": 16152.641476783223, "vram_gb": 6.154519552, "lambda": 0.17285159106055895, "lambda_min": 0.09832947701215744, "lambda_max": 0.2514011263847351, "S_mean": 0.2878926595052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 4.9839091300964355, "grad_norm": 0.6088351607322693, "tokens_seen": 21307392, "tokens_per_sec": 16153.335751558845, "vram_gb": 6.154519552, "lambda": 0.1733904123927156, "lambda_min": 0.0979728028178215, "lambda_max": 0.2525513768196106, "S_mean": 0.2702128092447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.009662866592407, "grad_norm": 0.6878920793533325, "tokens_seen": 21471232, "tokens_per_sec": 16154.476091726057, "vram_gb": 6.154519552, "lambda": 0.1739516311014692, "lambda_min": 0.09764248877763748, "lambda_max": 0.25412866473197937, "S_mean": 0.297637939453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.149697780609131, "grad_norm": 0.660448431968689, "tokens_seen": 21635072, "tokens_per_sec": 16155.859983051418, "vram_gb": 6.154519552, "lambda": 0.17445672551790872, "lambda_min": 0.09741053730249405, "lambda_max": 0.2554875612258911, "S_mean": 0.2764790852864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.090920448303223, "grad_norm": 0.7026114463806152, "tokens_seen": 21798912, "tokens_per_sec": 16156.9015387679, "vram_gb": 6.154519552, "lambda": 0.17500938288867474, "lambda_min": 0.09707964211702347, "lambda_max": 0.256974995136261, "S_mean": 0.2706807454427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.0821311473846436, "grad_norm": 0.65497225522995, "tokens_seen": 21962752, "tokens_per_sec": 16158.003253602054, "vram_gb": 6.154519552, "lambda": 0.17556125267098346, "lambda_min": 0.09684291481971741, "lambda_max": 0.2584756016731262, "S_mean": 0.2888895670572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.091814041137695, "grad_norm": 0.7021520733833313, "tokens_seen": 22126592, "tokens_per_sec": 16159.419796072423, "vram_gb": 6.154519552, "lambda": 0.1761414504920443, "lambda_min": 0.09663832932710648, "lambda_max": 0.26020124554634094, "S_mean": 0.2609965006510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.055244207382202, "grad_norm": 0.7241472005844116, "tokens_seen": 22290432, "tokens_per_sec": 16160.649241802954, "vram_gb": 6.154519552, "lambda": 0.1767077585682273, "lambda_min": 0.09637591242790222, "lambda_max": 0.26223379373550415, "S_mean": 0.2670694986979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.121567249298096, "grad_norm": 0.7141526341438293, "tokens_seen": 22454272, "tokens_per_sec": 16161.782922677317, "vram_gb": 6.154519552, "lambda": 0.1772888901953896, "lambda_min": 0.09572072327136993, "lambda_max": 0.26420602202415466, "S_mean": 0.2737223307291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.270342588424683, "grad_norm": 0.660784900188446, "tokens_seen": 22618112, "tokens_per_sec": 16162.920974252074, "vram_gb": 6.154519552, "lambda": 0.1778625544781486, "lambda_min": 0.0953914225101471, "lambda_max": 0.26606154441833496, "S_mean": 0.2561136881510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.245071887969971, "grad_norm": 0.6829572319984436, "tokens_seen": 22781952, "tokens_per_sec": 16164.466890343258, "vram_gb": 6.154519552, "lambda": 0.1784231293325623, "lambda_min": 0.09521736949682236, "lambda_max": 0.2677804231643677, "S_mean": 0.2860310872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.015129566192627, "grad_norm": 0.6441442370414734, "tokens_seen": 22945792, "tokens_per_sec": 16166.131460792982, "vram_gb": 6.154519552, "lambda": 0.17898354027420282, "lambda_min": 0.09490185976028442, "lambda_max": 0.26980018615722656, "S_mean": 0.2688802083333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.280963897705078, "grad_norm": 0.750127911567688, "tokens_seen": 23109632, "tokens_per_sec": 16167.189118653549, "vram_gb": 6.154519552, "lambda": 0.179576115993162, "lambda_min": 0.0946437269449234, "lambda_max": 0.2716578245162964, "S_mean": 0.2911478678385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.024816036224365, "grad_norm": 0.7418879866600037, "tokens_seen": 23273472, "tokens_per_sec": 16168.211220977031, "vram_gb": 6.154519552, "lambda": 0.18015736465652785, "lambda_min": 0.09424178302288055, "lambda_max": 0.27301838994026184, "S_mean": 0.280731201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.050545930862427, "grad_norm": 0.6698710322380066, "tokens_seen": 23437312, "tokens_per_sec": 16169.140022117364, "vram_gb": 6.154519552, "lambda": 0.18060852338870367, "lambda_min": 0.09399766474962234, "lambda_max": 0.2744773030281067, "S_mean": 0.279449462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.101682186126709, "grad_norm": 0.7073833346366882, "tokens_seen": 23601152, "tokens_per_sec": 16169.963391836302, "vram_gb": 6.154519552, "lambda": 0.18111553757141033, "lambda_min": 0.09368792921304703, "lambda_max": 0.27599602937698364, "S_mean": 0.2697347005208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.915022373199463, "grad_norm": 0.6731104850769043, "tokens_seen": 23764992, "tokens_per_sec": 16171.20195891831, "vram_gb": 6.154519552, "lambda": 0.18161660339683294, "lambda_min": 0.09340543299913406, "lambda_max": 0.27759432792663574, "S_mean": 0.2654012044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.0557897090911865, "grad_norm": 0.7151062488555908, "tokens_seen": 23928832, "tokens_per_sec": 16172.261202882204, "vram_gb": 6.154519552, "lambda": 0.18219109748800597, "lambda_min": 0.09324992448091507, "lambda_max": 0.27959775924682617, "S_mean": 0.259490966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.110750913619995, "grad_norm": 0.737566351890564, "tokens_seen": 24092672, "tokens_per_sec": 16173.81266591344, "vram_gb": 6.154519552, "lambda": 0.1827462576329708, "lambda_min": 0.09309280663728714, "lambda_max": 0.2814205288887024, "S_mean": 0.2943318684895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.961812973022461, "grad_norm": 0.7304709553718567, "tokens_seen": 24256512, "tokens_per_sec": 16175.579775562997, "vram_gb": 6.154519552, "lambda": 0.18327279606213173, "lambda_min": 0.09299390017986298, "lambda_max": 0.28331106901168823, "S_mean": 0.262908935546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.102810859680176, "grad_norm": 0.6636468768119812, "tokens_seen": 24420352, "tokens_per_sec": 16177.255671961017, "vram_gb": 6.154519552, "lambda": 0.1837539408976833, "lambda_min": 0.09275123476982117, "lambda_max": 0.2850445806980133, "S_mean": 0.2803853352864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.096365213394165, "grad_norm": 0.7171574831008911, "tokens_seen": 24584192, "tokens_per_sec": 16178.659910135206, "vram_gb": 6.154519552, "lambda": 0.18421449698507786, "lambda_min": 0.0925472304224968, "lambda_max": 0.28690212965011597, "S_mean": 0.2842305501302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.057959353923797, "val_ppl": 157.26925775358788, "lambda": 0.184238209699591, "lambda_min": 0.09254569560289383, "lambda_max": 0.2869974970817566, "lambda_per_layer": [0.15888704359531403, 0.13209719955921173, 0.13894441723823547, 0.1348799765110016, 0.13318681716918945, 0.13256998360157013, 0.11263614892959595, 0.09254569560289383, 0.09632357209920883, 0.15660850703716278, 0.1951884776353836, 0.22672851383686066, 0.2655499577522278, 0.2361750602722168, 0.2785744071006775, 0.22666317224502563, 0.27759844064712524, 0.2869974970817566, 0.2598666548728943, 0.21868544816970825, 0.17690226435661316, 0.15931721031665802, 0.17407730221748352, 0.1507132649421692], "S_mean": 0.2852986653645833, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.064821720123291, "grad_norm": 0.7012909054756165, "tokens_seen": 24748032, "tokens_per_sec": 16139.796184925779, "vram_gb": 6.154519552, "lambda": 0.1847746204584837, "lambda_min": 0.09226534515619278, "lambda_max": 0.2889181673526764, "S_mean": 0.2829386393229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.172181129455566, "grad_norm": 0.6829792261123657, "tokens_seen": 24911872, "tokens_per_sec": 16141.744491114077, "vram_gb": 6.154519552, "lambda": 0.18537040334194899, "lambda_min": 0.09193705767393112, "lambda_max": 0.2906920611858368, "S_mean": 0.2762858072916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.913817882537842, "grad_norm": 0.6423631906509399, "tokens_seen": 25075712, "tokens_per_sec": 16143.369489775721, "vram_gb": 6.154519552, "lambda": 0.18592176058640084, "lambda_min": 0.09177237004041672, "lambda_max": 0.29257622361183167, "S_mean": 0.2795308430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.992800712585449, "grad_norm": 0.7415454983711243, "tokens_seen": 25239552, "tokens_per_sec": 16144.07624290742, "vram_gb": 6.154519552, "lambda": 0.18644982296973467, "lambda_min": 0.09135802835226059, "lambda_max": 0.29439181089401245, "S_mean": 0.2812906901041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.918187618255615, "grad_norm": 0.7572674751281738, "tokens_seen": 25403392, "tokens_per_sec": 16145.230548042167, "vram_gb": 6.154519552, "lambda": 0.1869327879200379, "lambda_min": 0.09102455526590347, "lambda_max": 0.29598698019981384, "S_mean": 0.2808430989583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.8904807567596436, "grad_norm": 0.6750758290290833, "tokens_seen": 25567232, "tokens_per_sec": 16146.556731353925, "vram_gb": 6.154519552, "lambda": 0.1874176071335872, "lambda_min": 0.09076298028230667, "lambda_max": 0.29753828048706055, "S_mean": 0.2665201822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 5.058022499084473, "grad_norm": 0.710905134677887, "tokens_seen": 25731072, "tokens_per_sec": 16147.872671355437, "vram_gb": 6.154519552, "lambda": 0.1879212474450469, "lambda_min": 0.09080687165260315, "lambda_max": 0.29897600412368774, "S_mean": 0.2649739583333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.861875534057617, "grad_norm": 0.7398398518562317, "tokens_seen": 25894912, "tokens_per_sec": 16149.458521181017, "vram_gb": 6.154519552, "lambda": 0.18845899713536104, "lambda_min": 0.09084348380565643, "lambda_max": 0.3004058003425598, "S_mean": 0.2510477701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.885305881500244, "grad_norm": 0.7568672895431519, "tokens_seen": 26058752, "tokens_per_sec": 16150.84347437094, "vram_gb": 6.154519552, "lambda": 0.18896493346740803, "lambda_min": 0.09062013775110245, "lambda_max": 0.3020811676979065, "S_mean": 0.260833740234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.832994222640991, "grad_norm": 0.6818154454231262, "tokens_seen": 26222592, "tokens_per_sec": 16151.669927188694, "vram_gb": 6.154519552, "lambda": 0.18949866201728582, "lambda_min": 0.09022471308708191, "lambda_max": 0.3033979833126068, "S_mean": 0.2658487955729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.811842441558838, "grad_norm": 0.7447913885116577, "tokens_seen": 26386432, "tokens_per_sec": 16152.719857077507, "vram_gb": 6.154519552, "lambda": 0.19001475628465414, "lambda_min": 0.08992691338062286, "lambda_max": 0.3052094876766205, "S_mean": 0.2757364908854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.883727073669434, "grad_norm": 0.7389617562294006, "tokens_seen": 26550272, "tokens_per_sec": 16153.980038440892, "vram_gb": 6.154519552, "lambda": 0.19054187182337046, "lambda_min": 0.08962973207235336, "lambda_max": 0.30709487199783325, "S_mean": 0.2595316569010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.004491090774536, "grad_norm": 0.7807090878486633, "tokens_seen": 26714112, "tokens_per_sec": 16155.272885382294, "vram_gb": 6.154519552, "lambda": 0.19109232847889265, "lambda_min": 0.089562326669693, "lambda_max": 0.3087259829044342, "S_mean": 0.2557423909505208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.8701512813568115, "grad_norm": 0.6824589967727661, "tokens_seen": 26877952, "tokens_per_sec": 16156.420234782338, "vram_gb": 6.154519552, "lambda": 0.1916848849505186, "lambda_min": 0.08942446857690811, "lambda_max": 0.3107258379459381, "S_mean": 0.2587178548177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.048319339752197, "grad_norm": 0.7171062231063843, "tokens_seen": 27041792, "tokens_per_sec": 16157.348257159254, "vram_gb": 6.154519552, "lambda": 0.19218628481030464, "lambda_min": 0.08926256000995636, "lambda_max": 0.3121204078197479, "S_mean": 0.2588297526041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.912560224533081, "grad_norm": 0.6878513097763062, "tokens_seen": 27205632, "tokens_per_sec": 16158.516757194742, "vram_gb": 6.154519552, "lambda": 0.19261476366470257, "lambda_min": 0.08882919698953629, "lambda_max": 0.31353649497032166, "S_mean": 0.2741190592447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.991406440734863, "grad_norm": 0.7267003059387207, "tokens_seen": 27369472, "tokens_per_sec": 16159.378331499614, "vram_gb": 6.154519552, "lambda": 0.19318069579700628, "lambda_min": 0.08858160674571991, "lambda_max": 0.3153059482574463, "S_mean": 0.2640889485677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.787264347076416, "grad_norm": 0.7331607937812805, "tokens_seen": 27533312, "tokens_per_sec": 16160.375128752723, "vram_gb": 6.154519552, "lambda": 0.1936395252123475, "lambda_min": 0.08814159035682678, "lambda_max": 0.316681832075119, "S_mean": 0.2755126953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.990664720535278, "grad_norm": 0.9196872711181641, "tokens_seen": 27697152, "tokens_per_sec": 16161.267222863435, "vram_gb": 6.154519552, "lambda": 0.19413988385349512, "lambda_min": 0.08779862523078918, "lambda_max": 0.3179747760295868, "S_mean": 0.2693277994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.997473239898682, "grad_norm": 0.7498503923416138, "tokens_seen": 27860992, "tokens_per_sec": 16162.245255975431, "vram_gb": 6.154519552, "lambda": 0.19463006996860108, "lambda_min": 0.08759253472089767, "lambda_max": 0.31947097182273865, "S_mean": 0.2645467122395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.816570281982422, "grad_norm": 0.687647819519043, "tokens_seen": 28024832, "tokens_per_sec": 16163.490685121596, "vram_gb": 6.154519552, "lambda": 0.19516540008286634, "lambda_min": 0.08740583062171936, "lambda_max": 0.3209473490715027, "S_mean": 0.2746988932291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.938575983047485, "grad_norm": 0.7203260660171509, "tokens_seen": 28188672, "tokens_per_sec": 16164.823059512444, "vram_gb": 6.154519552, "lambda": 0.19572093648215136, "lambda_min": 0.08716820180416107, "lambda_max": 0.32283133268356323, "S_mean": 0.260528564453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.837875843048096, "grad_norm": 0.7684145569801331, "tokens_seen": 28352512, "tokens_per_sec": 16165.892995701686, "vram_gb": 6.154519552, "lambda": 0.19626024613777796, "lambda_min": 0.08695279806852341, "lambda_max": 0.3246602714061737, "S_mean": 0.27398681640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.808856010437012, "grad_norm": 0.7149838209152222, "tokens_seen": 28516352, "tokens_per_sec": 16167.312571262053, "vram_gb": 6.154519552, "lambda": 0.1968085989356041, "lambda_min": 0.08670589327812195, "lambda_max": 0.32662951946258545, "S_mean": 0.265472412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.975912094116211, "grad_norm": 0.8063885569572449, "tokens_seen": 28680192, "tokens_per_sec": 16169.048688638128, "vram_gb": 6.154519552, "lambda": 0.1974132585649689, "lambda_min": 0.08657220005989075, "lambda_max": 0.3286297917366028, "S_mean": 0.25823974609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.94185596704483, "val_ppl": 140.0298994132867, "lambda": 0.19744260050356388, "lambda_min": 0.08655311912298203, "lambda_max": 0.32873305678367615, "lambda_per_layer": [0.16018034517765045, 0.13290102779865265, 0.14002177119255066, 0.13597357273101807, 0.1345335841178894, 0.13679902255535126, 0.1113671362400055, 0.08655311912298203, 0.08945024758577347, 0.16021637618541718, 0.20582905411720276, 0.244812473654747, 0.29762163758277893, 0.2586228549480438, 0.30844831466674805, 0.24602848291397095, 0.31601375341415405, 0.32873305678367615, 0.2982502579689026, 0.24218180775642395, 0.19383226335048676, 0.16575102508068085, 0.18853314220905304, 0.1559680849313736], "S_mean": 0.2697957356770833, "G_mean": null, "G_std": null, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.948944568634033, "grad_norm": 0.7448777556419373, "tokens_seen": 28844032, "tokens_per_sec": 16136.270101449503, "vram_gb": 6.154519552, "lambda": 0.1979413339868188, "lambda_min": 0.08630594611167908, "lambda_max": 0.33054041862487793, "S_mean": 0.2716166178385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.818563938140869, "grad_norm": 0.690068781375885, "tokens_seen": 29007872, "tokens_per_sec": 16137.185419255109, "vram_gb": 6.154519552, "lambda": 0.1984906168654561, "lambda_min": 0.08612930774688721, "lambda_max": 0.3324569761753082, "S_mean": 0.2873433430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.855785846710205, "grad_norm": 0.7151216864585876, "tokens_seen": 29171712, "tokens_per_sec": 16138.364899540595, "vram_gb": 6.154519552, "lambda": 0.1990638313194116, "lambda_min": 0.08585638552904129, "lambda_max": 0.33443817496299744, "S_mean": 0.2587076822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.926857233047485, "grad_norm": 0.7227994799613953, "tokens_seen": 29335552, "tokens_per_sec": 16139.651512803133, "vram_gb": 6.154519552, "lambda": 0.19960614200681448, "lambda_min": 0.08559609204530716, "lambda_max": 0.3364127278327942, "S_mean": 0.2726847330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.7921717166900635, "grad_norm": 0.7140278816223145, "tokens_seen": 29499392, "tokens_per_sec": 16140.644770085202, "vram_gb": 6.154519552, "lambda": 0.20009012271960577, "lambda_min": 0.08535873144865036, "lambda_max": 0.33808761835098267, "S_mean": 0.26275634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.879737138748169, "grad_norm": 0.7317127585411072, "tokens_seen": 29663232, "tokens_per_sec": 16141.755315786724, "vram_gb": 6.154519552, "lambda": 0.2006464945152402, "lambda_min": 0.08503691852092743, "lambda_max": 0.34007367491722107, "S_mean": 0.2595113118489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.933736801147461, "grad_norm": 0.7462812066078186, "tokens_seen": 29827072, "tokens_per_sec": 16142.977815625556, "vram_gb": 6.154519552, "lambda": 0.2011732952669263, "lambda_min": 0.08485797792673111, "lambda_max": 0.3417680859565735, "S_mean": 0.2647806803385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.810486793518066, "grad_norm": 0.6899187564849854, "tokens_seen": 29990912, "tokens_per_sec": 16143.399670043038, "vram_gb": 6.154519552, "lambda": 0.2016546931117773, "lambda_min": 0.08446943759918213, "lambda_max": 0.34333986043930054, "S_mean": 0.2676493326822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.948333501815796, "grad_norm": 0.7221020460128784, "tokens_seen": 30154752, "tokens_per_sec": 16144.478802945296, "vram_gb": 6.154519552, "lambda": 0.20222297900666794, "lambda_min": 0.0841275230050087, "lambda_max": 0.3452470600605011, "S_mean": 0.277740478515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.786917209625244, "grad_norm": 0.6978982090950012, "tokens_seen": 30318592, "tokens_per_sec": 16145.35566175222, "vram_gb": 6.154519552, "lambda": 0.20278107188642025, "lambda_min": 0.08404836058616638, "lambda_max": 0.3468659818172455, "S_mean": 0.2636617024739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.9359588623046875, "grad_norm": 0.9009190201759338, "tokens_seen": 30482432, "tokens_per_sec": 16146.555349560002, "vram_gb": 6.154519552, "lambda": 0.20334272366017103, "lambda_min": 0.08374025672674179, "lambda_max": 0.34842196106910706, "S_mean": 0.25592041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.887932777404785, "grad_norm": 0.7738649249076843, "tokens_seen": 30646272, "tokens_per_sec": 16147.762029880234, "vram_gb": 6.154519552, "lambda": 0.20382423481593528, "lambda_min": 0.08358084410429001, "lambda_max": 0.34987348318099976, "S_mean": 0.249755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.945967435836792, "grad_norm": 0.7328440546989441, "tokens_seen": 30810112, "tokens_per_sec": 16148.792103903981, "vram_gb": 6.154519552, "lambda": 0.20428088183204332, "lambda_min": 0.08316630870103836, "lambda_max": 0.35158687829971313, "S_mean": 0.2730509440104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.910674810409546, "grad_norm": 0.7215652465820312, "tokens_seen": 30973952, "tokens_per_sec": 16150.018165807547, "vram_gb": 6.154519552, "lambda": 0.20476394860694805, "lambda_min": 0.08298005163669586, "lambda_max": 0.353674978017807, "S_mean": 0.2649027506510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.908976078033447, "grad_norm": 0.7627097964286804, "tokens_seen": 31137792, "tokens_per_sec": 16150.768719713797, "vram_gb": 6.154519552, "lambda": 0.20527261961251497, "lambda_min": 0.08267679810523987, "lambda_max": 0.35537397861480713, "S_mean": 0.2587381998697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.694072008132935, "grad_norm": 0.9094268083572388, "tokens_seen": 31301632, "tokens_per_sec": 16151.93708152602, "vram_gb": 6.154519552, "lambda": 0.20580366626381874, "lambda_min": 0.08241377770900726, "lambda_max": 0.35692545771598816, "S_mean": 0.2777506510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.965497732162476, "grad_norm": 0.7325593829154968, "tokens_seen": 31465472, "tokens_per_sec": 16153.056576559082, "vram_gb": 6.154519552, "lambda": 0.20623450446873903, "lambda_min": 0.08228950947523117, "lambda_max": 0.3580884039402008, "S_mean": 0.270660400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.862736701965332, "grad_norm": 0.8302729725837708, "tokens_seen": 31629312, "tokens_per_sec": 16154.191758217821, "vram_gb": 6.154519552, "lambda": 0.20679831225425005, "lambda_min": 0.08222196251153946, "lambda_max": 0.35996538400650024, "S_mean": 0.2591959635416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.774860382080078, "grad_norm": 0.7494895458221436, "tokens_seen": 31793152, "tokens_per_sec": 16155.283870419771, "vram_gb": 6.154519552, "lambda": 0.20726905297487974, "lambda_min": 0.08202186226844788, "lambda_max": 0.36173078417778015, "S_mean": 0.258941650390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.719260215759277, "grad_norm": 0.741629958152771, "tokens_seen": 31956992, "tokens_per_sec": 16156.200180934906, "vram_gb": 6.154519552, "lambda": 0.20780920734008154, "lambda_min": 0.08188636600971222, "lambda_max": 0.36357441544532776, "S_mean": 0.2539164225260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.889046669006348, "grad_norm": 1.216410756111145, "tokens_seen": 32120832, "tokens_per_sec": 16157.294293786003, "vram_gb": 6.154519552, "lambda": 0.2083284022907416, "lambda_min": 0.08172275125980377, "lambda_max": 0.36510640382766724, "S_mean": 0.2862141927083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.6700825691223145, "grad_norm": 0.7132147550582886, "tokens_seen": 32284672, "tokens_per_sec": 16158.537834106253, "vram_gb": 6.154519552, "lambda": 0.20892691984772682, "lambda_min": 0.08161801844835281, "lambda_max": 0.36697208881378174, "S_mean": 0.2543131510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.792524814605713, "grad_norm": 0.7456054091453552, "tokens_seen": 32448512, "tokens_per_sec": 16159.611877387193, "vram_gb": 6.154519552, "lambda": 0.20948684805383286, "lambda_min": 0.0813969075679779, "lambda_max": 0.36868816614151, "S_mean": 0.2678019205729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.676145792007446, "grad_norm": 0.6795365214347839, "tokens_seen": 32612352, "tokens_per_sec": 16160.692296886129, "vram_gb": 6.154519552, "lambda": 0.20997001510113478, "lambda_min": 0.08128832280635834, "lambda_max": 0.37044236063957214, "S_mean": 0.2580769856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.775681734085083, "grad_norm": 0.7498477697372437, "tokens_seen": 32776192, "tokens_per_sec": 16161.766151764725, "vram_gb": 6.154519552, "lambda": 0.210554842526714, "lambda_min": 0.08114634454250336, "lambda_max": 0.3720828890800476, "S_mean": 0.2662251790364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.89936193227768, "val_ppl": 134.20412104199482, "lambda": 0.21058974880725145, "lambda_min": 0.08114028722047806, "lambda_max": 0.3721620738506317, "lambda_per_layer": [0.16138629615306854, 0.13224057853221893, 0.14058063924312592, 0.137314572930336, 0.1357046514749527, 0.1389351785182953, 0.11154356598854065, 0.08114028722047806, 0.08414410054683685, 0.16413530707359314, 0.21573536098003387, 0.2630142271518707, 0.3301900029182434, 0.2804296612739563, 0.33703574538230896, 0.2647794485092163, 0.3542640209197998, 0.3721620738506317, 0.3352942168712616, 0.26527905464172363, 0.20858314633369446, 0.17438247799873352, 0.20440346002578735, 0.16147589683532715], "S_mean": 0.2727457682291667, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.949815273284912, "grad_norm": 0.7396031618118286, "tokens_seen": 32940032, "tokens_per_sec": 16133.165874333461, "vram_gb": 6.154519552, "lambda": 0.21111888128022352, "lambda_min": 0.081007219851017, "lambda_max": 0.3734624683856964, "S_mean": 0.2501627604166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.922213077545166, "grad_norm": 0.6934394240379333, "tokens_seen": 33103872, "tokens_per_sec": 16133.780929634322, "vram_gb": 6.154519552, "lambda": 0.2115729625026385, "lambda_min": 0.08089249581098557, "lambda_max": 0.37462109327316284, "S_mean": 0.264129638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.786732912063599, "grad_norm": 0.755628764629364, "tokens_seen": 33267712, "tokens_per_sec": 16134.75558043529, "vram_gb": 6.154519552, "lambda": 0.21207218430936337, "lambda_min": 0.08084764331579208, "lambda_max": 0.3762570023536682, "S_mean": 0.2706705729166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.801859617233276, "grad_norm": 0.7519621849060059, "tokens_seen": 33431552, "tokens_per_sec": 16135.328280610922, "vram_gb": 6.154519552, "lambda": 0.21255518527080616, "lambda_min": 0.08090732246637344, "lambda_max": 0.3778957426548004, "S_mean": 0.2584431966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.7154223918914795, "grad_norm": 0.7859010696411133, "tokens_seen": 33595392, "tokens_per_sec": 16136.232795629881, "vram_gb": 6.154519552, "lambda": 0.2130913178746899, "lambda_min": 0.08089285343885422, "lambda_max": 0.3794753849506378, "S_mean": 0.280364990234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.653739929199219, "grad_norm": 0.9589067101478577, "tokens_seen": 33759232, "tokens_per_sec": 16137.074216580048, "vram_gb": 6.154519552, "lambda": 0.21349258441478014, "lambda_min": 0.0805429220199585, "lambda_max": 0.38055771589279175, "S_mean": 0.3129679361979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.753014087677002, "grad_norm": 0.7424250245094299, "tokens_seen": 33923072, "tokens_per_sec": 16137.93607328926, "vram_gb": 6.154519552, "lambda": 0.21397934729854265, "lambda_min": 0.08032158762216568, "lambda_max": 0.3823339641094208, "S_mean": 0.2712910970052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.756777048110962, "grad_norm": 0.8227521777153015, "tokens_seen": 34086912, "tokens_per_sec": 16138.937481004466, "vram_gb": 6.154519552, "lambda": 0.21449112767974535, "lambda_min": 0.080155149102211, "lambda_max": 0.3838306665420532, "S_mean": 0.2698160807291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.933750152587891, "grad_norm": 0.8162205219268799, "tokens_seen": 34250752, "tokens_per_sec": 16140.084075282273, "vram_gb": 6.154519552, "lambda": 0.2150273428608974, "lambda_min": 0.07992444932460785, "lambda_max": 0.3855646550655365, "S_mean": 0.2655029296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.7576093673706055, "grad_norm": 0.7986825108528137, "tokens_seen": 34414592, "tokens_per_sec": 16140.576468193369, "vram_gb": 6.154519552, "lambda": 0.21556786571939787, "lambda_min": 0.07986191660165787, "lambda_max": 0.3874668478965759, "S_mean": 0.2848002115885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.822808504104614, "grad_norm": 0.7590571641921997, "tokens_seen": 34578432, "tokens_per_sec": 16141.537950298425, "vram_gb": 6.154519552, "lambda": 0.21601732540875673, "lambda_min": 0.07998120039701462, "lambda_max": 0.389042466878891, "S_mean": 0.2623087565104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.823237895965576, "grad_norm": 0.7327103614807129, "tokens_seen": 34742272, "tokens_per_sec": 16142.565845705982, "vram_gb": 6.154519552, "lambda": 0.21646190155297518, "lambda_min": 0.07978756725788116, "lambda_max": 0.39008280634880066, "S_mean": 0.2523396809895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.84969425201416, "grad_norm": 0.7275848388671875, "tokens_seen": 34906112, "tokens_per_sec": 16143.544849237149, "vram_gb": 6.154519552, "lambda": 0.21695900615304708, "lambda_min": 0.0797039046883583, "lambda_max": 0.3919016122817993, "S_mean": 0.2678426106770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.8859543800354, "grad_norm": 0.8389278650283813, "tokens_seen": 35069952, "tokens_per_sec": 16144.456865241787, "vram_gb": 6.154519552, "lambda": 0.21744878496974707, "lambda_min": 0.07974080741405487, "lambda_max": 0.39357173442840576, "S_mean": 0.2746378580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.845590114593506, "grad_norm": 0.7935529947280884, "tokens_seen": 35233792, "tokens_per_sec": 16144.772184635967, "vram_gb": 6.154519552, "lambda": 0.21787717348585525, "lambda_min": 0.07955983281135559, "lambda_max": 0.39479562640190125, "S_mean": 0.2655843098958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.704818964004517, "grad_norm": 0.7930161952972412, "tokens_seen": 35397632, "tokens_per_sec": 16145.925254075539, "vram_gb": 6.154519552, "lambda": 0.21842833422124386, "lambda_min": 0.07928388565778732, "lambda_max": 0.3967441916465759, "S_mean": 0.2756856282552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.73800253868103, "grad_norm": 0.8579527735710144, "tokens_seen": 35561472, "tokens_per_sec": 16147.03507864358, "vram_gb": 6.154519552, "lambda": 0.218914441143473, "lambda_min": 0.07910451292991638, "lambda_max": 0.39824989438056946, "S_mean": 0.2676798502604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.870057106018066, "grad_norm": 0.7456783056259155, "tokens_seen": 35725312, "tokens_per_sec": 16148.196678970424, "vram_gb": 6.154519552, "lambda": 0.21934434367964664, "lambda_min": 0.07885108143091202, "lambda_max": 0.3995625972747803, "S_mean": 0.257232666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.861536741256714, "grad_norm": 0.7675063610076904, "tokens_seen": 35889152, "tokens_per_sec": 16149.291956739089, "vram_gb": 6.154519552, "lambda": 0.21977617622663578, "lambda_min": 0.07853908836841583, "lambda_max": 0.40128394961357117, "S_mean": 0.2525634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.614075422286987, "grad_norm": 0.783237636089325, "tokens_seen": 36052992, "tokens_per_sec": 16150.136450865924, "vram_gb": 6.154519552, "lambda": 0.22030633222311735, "lambda_min": 0.07844731956720352, "lambda_max": 0.40312430262565613, "S_mean": 0.2612406412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.903042316436768, "grad_norm": 0.7910538911819458, "tokens_seen": 36216832, "tokens_per_sec": 16151.083212051171, "vram_gb": 6.154519552, "lambda": 0.22080085054039955, "lambda_min": 0.07830867171287537, "lambda_max": 0.4047452211380005, "S_mean": 0.2602132161458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.717573165893555, "grad_norm": 0.8575606346130371, "tokens_seen": 36380672, "tokens_per_sec": 16152.409766868677, "vram_gb": 6.154519552, "lambda": 0.22125315200537443, "lambda_min": 0.07804249972105026, "lambda_max": 0.40585795044898987, "S_mean": 0.24871826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.735207557678223, "grad_norm": 0.7653822302818298, "tokens_seen": 36544512, "tokens_per_sec": 16153.47787343608, "vram_gb": 6.154519552, "lambda": 0.221769320157667, "lambda_min": 0.07792937010526657, "lambda_max": 0.4068989157676697, "S_mean": 0.26702880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.715848207473755, "grad_norm": 0.7758042812347412, "tokens_seen": 36708352, "tokens_per_sec": 16154.551717633112, "vram_gb": 6.154519552, "lambda": 0.22233792239179215, "lambda_min": 0.0779564380645752, "lambda_max": 0.408701092004776, "S_mean": 0.244598388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.73714804649353, "grad_norm": 0.844622790813446, "tokens_seen": 36872192, "tokens_per_sec": 16155.484007434496, "vram_gb": 6.154519552, "lambda": 0.2228744880606731, "lambda_min": 0.07799385488033295, "lambda_max": 0.4105834364891052, "S_mean": 0.2566019694010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.830313575267792, "val_ppl": 125.25022987188886, "lambda": 0.22289786705126366, "lambda_min": 0.077983058989048, "lambda_max": 0.4106711149215698, "lambda_per_layer": [0.16203801333904266, 0.13209597766399384, 0.1415216326713562, 0.13923229277133942, 0.1371944397687912, 0.1412450224161148, 0.11151385307312012, 0.077983058989048, 0.08021046221256256, 0.1673082560300827, 0.2236029952764511, 0.2807804048061371, 0.36223241686820984, 0.302543044090271, 0.36518940329551697, 0.2821754813194275, 0.3860360383987427, 0.4106711149215698, 0.3734540045261383, 0.2860594689846039, 0.22279870510101318, 0.17945343255996704, 0.21620744466781616, 0.16800184547901154], "S_mean": 0.2777303059895833, "G_mean": null, "G_std": null, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.730362415313721, "grad_norm": 0.7742741107940674, "tokens_seen": 37036032, "tokens_per_sec": 16129.811342579216, "vram_gb": 6.154519552, "lambda": 0.2233520494773984, "lambda_min": 0.07772300392389297, "lambda_max": 0.41198378801345825, "S_mean": 0.2544453938802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.77234411239624, "grad_norm": 0.7887251377105713, "tokens_seen": 37199872, "tokens_per_sec": 16130.354295860037, "vram_gb": 6.154519552, "lambda": 0.22376834383855262, "lambda_min": 0.0777798444032669, "lambda_max": 0.41330090165138245, "S_mean": 0.267822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.7712037563323975, "grad_norm": 0.7612297534942627, "tokens_seen": 37363712, "tokens_per_sec": 16130.773657953045, "vram_gb": 6.154519552, "lambda": 0.2242610106865565, "lambda_min": 0.07758484780788422, "lambda_max": 0.41523584723472595, "S_mean": 0.2567240397135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.731543064117432, "grad_norm": 0.7906785011291504, "tokens_seen": 37527552, "tokens_per_sec": 16131.67601578629, "vram_gb": 6.154519552, "lambda": 0.2247393016392986, "lambda_min": 0.07738404721021652, "lambda_max": 0.4171222448348999, "S_mean": 0.253662109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.738653182983398, "grad_norm": 0.7928621768951416, "tokens_seen": 37691392, "tokens_per_sec": 16132.577773264804, "vram_gb": 6.154519552, "lambda": 0.2251900608340899, "lambda_min": 0.07727736234664917, "lambda_max": 0.4183058440685272, "S_mean": 0.271514892578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.7495996952056885, "grad_norm": 0.8157086968421936, "tokens_seen": 37855232, "tokens_per_sec": 16133.470081903706, "vram_gb": 6.154519552, "lambda": 0.22558404753605524, "lambda_min": 0.07725037634372711, "lambda_max": 0.41916587948799133, "S_mean": 0.2601725260416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.819446563720703, "grad_norm": 0.8193637132644653, "tokens_seen": 38019072, "tokens_per_sec": 16134.408995085314, "vram_gb": 6.154519552, "lambda": 0.22604842856526375, "lambda_min": 0.07709116488695145, "lambda_max": 0.4202849566936493, "S_mean": 0.2757670084635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.693276405334473, "grad_norm": 0.750886857509613, "tokens_seen": 38182912, "tokens_per_sec": 16135.316419151299, "vram_gb": 6.154519552, "lambda": 0.22652095276862383, "lambda_min": 0.07691529393196106, "lambda_max": 0.4212656617164612, "S_mean": 0.2618306477864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.754004240036011, "grad_norm": 0.8066550493240356, "tokens_seen": 38346752, "tokens_per_sec": 16136.243720620563, "vram_gb": 6.154519552, "lambda": 0.22703739690283933, "lambda_min": 0.07671696692705154, "lambda_max": 0.4228546619415283, "S_mean": 0.2559305826822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.699479103088379, "grad_norm": 0.8580237627029419, "tokens_seen": 38510592, "tokens_per_sec": 16136.763541716848, "vram_gb": 6.154519552, "lambda": 0.22749096093078455, "lambda_min": 0.07653689384460449, "lambda_max": 0.42435118556022644, "S_mean": 0.2608235677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.740078449249268, "grad_norm": 0.7974342107772827, "tokens_seen": 38674432, "tokens_per_sec": 16137.70269190391, "vram_gb": 6.154519552, "lambda": 0.22802438121289015, "lambda_min": 0.07646380364894867, "lambda_max": 0.42586326599121094, "S_mean": 0.2557373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.618248462677002, "grad_norm": 0.824561595916748, "tokens_seen": 38838272, "tokens_per_sec": 16138.571085430909, "vram_gb": 6.154519552, "lambda": 0.22852983387808004, "lambda_min": 0.07618657499551773, "lambda_max": 0.4275396168231964, "S_mean": 0.2686971028645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.731117010116577, "grad_norm": 0.8695266246795654, "tokens_seen": 39002112, "tokens_per_sec": 16139.460344700323, "vram_gb": 6.154519552, "lambda": 0.228978686966002, "lambda_min": 0.07604917883872986, "lambda_max": 0.42907944321632385, "S_mean": 0.267303466796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.7595953941345215, "grad_norm": 0.7519428730010986, "tokens_seen": 39165952, "tokens_per_sec": 16140.046654643485, "vram_gb": 6.154519552, "lambda": 0.2294976394623518, "lambda_min": 0.07597778737545013, "lambda_max": 0.4308943450450897, "S_mean": 0.25823974609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.819749116897583, "grad_norm": 0.7695877552032471, "tokens_seen": 39329792, "tokens_per_sec": 16140.980222158267, "vram_gb": 6.154519552, "lambda": 0.2300543443610271, "lambda_min": 0.07588771730661392, "lambda_max": 0.43266355991363525, "S_mean": 0.2627970377604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.7158122062683105, "grad_norm": 0.8488335013389587, "tokens_seen": 39493632, "tokens_per_sec": 16141.864565782796, "vram_gb": 6.154519552, "lambda": 0.23049756636222204, "lambda_min": 0.07584460824728012, "lambda_max": 0.43432435393333435, "S_mean": 0.2596537272135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.481667757034302, "grad_norm": 0.8225690126419067, "tokens_seen": 39657472, "tokens_per_sec": 16142.74005303009, "vram_gb": 6.154519552, "lambda": 0.23090325699498257, "lambda_min": 0.07573794573545456, "lambda_max": 0.4353897273540497, "S_mean": 0.287017822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.735045909881592, "grad_norm": 0.7931352853775024, "tokens_seen": 39821312, "tokens_per_sec": 16143.710113190285, "vram_gb": 6.154519552, "lambda": 0.23133533913642168, "lambda_min": 0.0757148414850235, "lambda_max": 0.43673175573349, "S_mean": 0.2747294108072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.72134256362915, "grad_norm": 0.7691394686698914, "tokens_seen": 39985152, "tokens_per_sec": 16144.736531161217, "vram_gb": 6.154519552, "lambda": 0.23179953824728727, "lambda_min": 0.07576446980237961, "lambda_max": 0.43837741017341614, "S_mean": 0.2658894856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.5494725704193115, "grad_norm": 0.7441647052764893, "tokens_seen": 40148992, "tokens_per_sec": 16145.68131790386, "vram_gb": 6.154519552, "lambda": 0.23226681165397167, "lambda_min": 0.07577837258577347, "lambda_max": 0.4398637115955353, "S_mean": 0.285736083984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.787657976150513, "grad_norm": 0.8310348987579346, "tokens_seen": 40312832, "tokens_per_sec": 16146.448288774473, "vram_gb": 6.154519552, "lambda": 0.23271866360058388, "lambda_min": 0.07560330629348755, "lambda_max": 0.44149884581565857, "S_mean": 0.3013814290364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.656689643859863, "grad_norm": 0.754972517490387, "tokens_seen": 40476672, "tokens_per_sec": 16147.467963235626, "vram_gb": 6.154519552, "lambda": 0.23308005277067423, "lambda_min": 0.07554257661104202, "lambda_max": 0.4429471790790558, "S_mean": 0.267547607421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.710378170013428, "grad_norm": 0.81003338098526, "tokens_seen": 40640512, "tokens_per_sec": 16148.222084172123, "vram_gb": 6.154519552, "lambda": 0.2335965841387709, "lambda_min": 0.0755479633808136, "lambda_max": 0.44434261322021484, "S_mean": 0.26446533203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.575535774230957, "grad_norm": 0.7658308744430542, "tokens_seen": 40804352, "tokens_per_sec": 16148.687356121178, "vram_gb": 6.154519552, "lambda": 0.23406649194657803, "lambda_min": 0.07567696273326874, "lambda_max": 0.4454198479652405, "S_mean": 0.2725626627604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.784950494766235, "grad_norm": 1.0222115516662598, "tokens_seen": 40968192, "tokens_per_sec": 16149.253705278863, "vram_gb": 6.154519552, "lambda": 0.23458139908810458, "lambda_min": 0.07561706006526947, "lambda_max": 0.44698524475097656, "S_mean": 0.284088134765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.790410625934601, "val_ppl": 120.35077766877183, "lambda": 0.2346075220654408, "lambda_min": 0.07561549544334412, "lambda_max": 0.4470770061016083, "lambda_per_layer": [0.1620422750711441, 0.13187676668167114, 0.1410694569349289, 0.1413075029850006, 0.13865207135677338, 0.14364545047283173, 0.11112666875123978, 0.07561549544334412, 0.07843907922506332, 0.17021436989307404, 0.23114317655563354, 0.29345589876174927, 0.3898339867591858, 0.3216530680656433, 0.39358633756637573, 0.2960750162601471, 0.4202362596988678, 0.4470770061016083, 0.4053018391132355, 0.3084527850151062, 0.23653872311115265, 0.18620125949382782, 0.23254673182964325, 0.17448930442333221], "S_mean": 0.2722981770833333, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.663856506347656, "grad_norm": 0.8876217603683472, "tokens_seen": 41132032, "tokens_per_sec": 16126.42223484696, "vram_gb": 6.154519552, "lambda": 0.23510567812869945, "lambda_min": 0.0755014717578888, "lambda_max": 0.44868189096450806, "S_mean": 0.2798563639322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.671810626983643, "grad_norm": 0.8334974646568298, "tokens_seen": 41295872, "tokens_per_sec": 16127.136965387275, "vram_gb": 6.154519552, "lambda": 0.23558136355131865, "lambda_min": 0.07536017149686813, "lambda_max": 0.4505334496498108, "S_mean": 0.2645365397135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.566517353057861, "grad_norm": 0.8573307394981384, "tokens_seen": 41459712, "tokens_per_sec": 16128.06927948158, "vram_gb": 6.154519552, "lambda": 0.23603251234938702, "lambda_min": 0.075189508497715, "lambda_max": 0.4523617625236511, "S_mean": 0.2700297037760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.674891471862793, "grad_norm": 0.8000473380088806, "tokens_seen": 41623552, "tokens_per_sec": 16128.89751796113, "vram_gb": 6.154519552, "lambda": 0.2364618411908547, "lambda_min": 0.07516905665397644, "lambda_max": 0.4538061022758484, "S_mean": 0.2845052083333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.6959991455078125, "grad_norm": 0.7724489569664001, "tokens_seen": 41787392, "tokens_per_sec": 16129.550686550527, "vram_gb": 6.154519552, "lambda": 0.23687253861377636, "lambda_min": 0.07514607906341553, "lambda_max": 0.45512688159942627, "S_mean": 0.2594095865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.617343425750732, "grad_norm": 0.7987869381904602, "tokens_seen": 41951232, "tokens_per_sec": 16130.560944398765, "vram_gb": 6.154519552, "lambda": 0.2373960828408599, "lambda_min": 0.07496906071901321, "lambda_max": 0.45665210485458374, "S_mean": 0.2778727213541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.716531991958618, "grad_norm": 0.7818719148635864, "tokens_seen": 42115072, "tokens_per_sec": 16131.540643007665, "vram_gb": 6.154519552, "lambda": 0.23792181257158518, "lambda_min": 0.0748855322599411, "lambda_max": 0.4583905339241028, "S_mean": 0.2785237630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.566024541854858, "grad_norm": 0.7995215058326721, "tokens_seen": 42278912, "tokens_per_sec": 16132.296245006572, "vram_gb": 6.154519552, "lambda": 0.23831176726768413, "lambda_min": 0.0746532753109932, "lambda_max": 0.4599807858467102, "S_mean": 0.2924296061197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.594595193862915, "grad_norm": 0.8185252547264099, "tokens_seen": 42442752, "tokens_per_sec": 16133.285948890683, "vram_gb": 6.154519552, "lambda": 0.23875507029394308, "lambda_min": 0.0745624303817749, "lambda_max": 0.4614599347114563, "S_mean": 0.2659810384114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.533696889877319, "grad_norm": 0.8312360644340515, "tokens_seen": 42606592, "tokens_per_sec": 16134.271961635635, "vram_gb": 6.154519552, "lambda": 0.23933975491672754, "lambda_min": 0.07439897954463959, "lambda_max": 0.463637113571167, "S_mean": 0.2744852701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.63237738609314, "grad_norm": 0.7629782557487488, "tokens_seen": 42770432, "tokens_per_sec": 16135.352773157183, "vram_gb": 6.154519552, "lambda": 0.23975173570215702, "lambda_min": 0.07422392070293427, "lambda_max": 0.4656020998954773, "S_mean": 0.2666117350260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.615908861160278, "grad_norm": 0.8918854594230652, "tokens_seen": 42934272, "tokens_per_sec": 16136.137565355053, "vram_gb": 6.154519552, "lambda": 0.24012123855451742, "lambda_min": 0.07427928596735, "lambda_max": 0.4669969975948334, "S_mean": 0.28826904296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.571869373321533, "grad_norm": 0.7494592070579529, "tokens_seen": 43098112, "tokens_per_sec": 16136.8565099972, "vram_gb": 6.154519552, "lambda": 0.24047718166063228, "lambda_min": 0.07419299334287643, "lambda_max": 0.46862953901290894, "S_mean": 0.2682291666666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.606633901596069, "grad_norm": 0.83780837059021, "tokens_seen": 43261952, "tokens_per_sec": 16137.944996604556, "vram_gb": 6.154519552, "lambda": 0.24095913426329693, "lambda_min": 0.07407114654779434, "lambda_max": 0.4702097773551941, "S_mean": 0.2702534993489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.728524684906006, "grad_norm": 0.8239432573318481, "tokens_seen": 43425792, "tokens_per_sec": 16139.169822717939, "vram_gb": 6.154519552, "lambda": 0.24136390319714943, "lambda_min": 0.07392862439155579, "lambda_max": 0.47151076793670654, "S_mean": 0.283721923828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.601941823959351, "grad_norm": 0.7981506586074829, "tokens_seen": 43589632, "tokens_per_sec": 16140.387903604826, "vram_gb": 6.154519552, "lambda": 0.2418892669181029, "lambda_min": 0.07375895231962204, "lambda_max": 0.47269755601882935, "S_mean": 0.2698974609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.671489238739014, "grad_norm": 0.8317182064056396, "tokens_seen": 43753472, "tokens_per_sec": 16141.400145666501, "vram_gb": 6.154519552, "lambda": 0.24226428413142762, "lambda_min": 0.07367534190416336, "lambda_max": 0.4737789034843445, "S_mean": 0.2733357747395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.525099992752075, "grad_norm": 0.8443008065223694, "tokens_seen": 43917312, "tokens_per_sec": 16142.42730748352, "vram_gb": 6.154519552, "lambda": 0.24271236546337605, "lambda_min": 0.07359492033720016, "lambda_max": 0.47516363859176636, "S_mean": 0.276641845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.7984137535095215, "grad_norm": 0.8132748603820801, "tokens_seen": 44081152, "tokens_per_sec": 16143.388210567184, "vram_gb": 6.154519552, "lambda": 0.24320416369785866, "lambda_min": 0.07360036671161652, "lambda_max": 0.47678232192993164, "S_mean": 0.2822062174479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.495278358459473, "grad_norm": 0.8016942739486694, "tokens_seen": 44244992, "tokens_per_sec": 16144.597493006397, "vram_gb": 6.154519552, "lambda": 0.24369302547226349, "lambda_min": 0.07354077696800232, "lambda_max": 0.4785846173763275, "S_mean": 0.264678955078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.675218343734741, "grad_norm": 0.8943013548851013, "tokens_seen": 44408832, "tokens_per_sec": 16145.599771845644, "vram_gb": 6.154519552, "lambda": 0.2441459676871697, "lambda_min": 0.0734705999493599, "lambda_max": 0.47978663444519043, "S_mean": 0.265289306640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.529200553894043, "grad_norm": 0.8564379215240479, "tokens_seen": 44572672, "tokens_per_sec": 16146.436299921425, "vram_gb": 6.154519552, "lambda": 0.24451540472606817, "lambda_min": 0.07334686815738678, "lambda_max": 0.4806678593158722, "S_mean": 0.2642008463541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.675407886505127, "grad_norm": 0.8823602795600891, "tokens_seen": 44736512, "tokens_per_sec": 16147.521026372697, "vram_gb": 6.154519552, "lambda": 0.24500690152247748, "lambda_min": 0.0733223482966423, "lambda_max": 0.4821926951408386, "S_mean": 0.2988789876302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.560598134994507, "grad_norm": 0.789590060710907, "tokens_seen": 44900352, "tokens_per_sec": 16148.464866518178, "vram_gb": 6.154519552, "lambda": 0.2454573983947436, "lambda_min": 0.07321452349424362, "lambda_max": 0.48352670669555664, "S_mean": 0.259063720703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.62157940864563, "grad_norm": 0.7736595869064331, "tokens_seen": 45064192, "tokens_per_sec": 16149.502002510764, "vram_gb": 6.154519552, "lambda": 0.2459756676107645, "lambda_min": 0.07301504164934158, "lambda_max": 0.48507171869277954, "S_mean": 0.2853495279947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.709350299835205, "val_ppl": 110.98003273216304, "lambda": 0.24599563299367824, "lambda_min": 0.07301358878612518, "lambda_max": 0.48510411381721497, "lambda_per_layer": [0.16324755549430847, 0.13193024694919586, 0.14130543172359467, 0.1438041776418686, 0.1402895599603653, 0.1462203413248062, 0.11146658658981323, 0.07301358878612518, 0.07717248052358627, 0.1727278083562851, 0.237404927611351, 0.3056188225746155, 0.41690656542778015, 0.3400381803512573, 0.4203229248523712, 0.3112213909626007, 0.45125579833984375, 0.48510411381721497, 0.43680134415626526, 0.3288883566856384, 0.25022658705711365, 0.19182471930980682, 0.24525749683380127, 0.18184618651866913], "S_mean": 0.2701009114583333, "G_mean": null, "G_std": null, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.447880744934082, "grad_norm": 0.805586040019989, "tokens_seen": 45228032, "tokens_per_sec": 16128.95784176252, "vram_gb": 6.154519552, "lambda": 0.24639331921935081, "lambda_min": 0.07295199483633041, "lambda_max": 0.48615968227386475, "S_mean": 0.2595011393229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.666791915893555, "grad_norm": 0.888394832611084, "tokens_seen": 45391872, "tokens_per_sec": 16129.77355645473, "vram_gb": 6.154519552, "lambda": 0.24690922566999993, "lambda_min": 0.07288474589586258, "lambda_max": 0.4881412982940674, "S_mean": 0.269927978515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.695976257324219, "grad_norm": 0.8023860454559326, "tokens_seen": 45555712, "tokens_per_sec": 16130.414464002995, "vram_gb": 6.154519552, "lambda": 0.24734488502144814, "lambda_min": 0.07279888540506363, "lambda_max": 0.48941731452941895, "S_mean": 0.262481689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.73979377746582, "grad_norm": 0.8281108736991882, "tokens_seen": 45719552, "tokens_per_sec": 16131.419560558144, "vram_gb": 6.154519552, "lambda": 0.24773935228586197, "lambda_min": 0.07278602570295334, "lambda_max": 0.490691602230072, "S_mean": 0.2736307779947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.553887367248535, "grad_norm": 0.7822831869125366, "tokens_seen": 45883392, "tokens_per_sec": 16132.468757903736, "vram_gb": 6.154519552, "lambda": 0.2482747072353959, "lambda_min": 0.07265307754278183, "lambda_max": 0.49226367473602295, "S_mean": 0.257720947265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.637993335723877, "grad_norm": 0.9113191366195679, "tokens_seen": 46047232, "tokens_per_sec": 16133.152366586188, "vram_gb": 6.154519552, "lambda": 0.24876787109921375, "lambda_min": 0.07253614068031311, "lambda_max": 0.4938889741897583, "S_mean": 0.2656453450520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.725446701049805, "grad_norm": 0.8308588862419128, "tokens_seen": 46211072, "tokens_per_sec": 16133.950861552925, "vram_gb": 6.154519552, "lambda": 0.2493281296143929, "lambda_min": 0.07258036732673645, "lambda_max": 0.4955183267593384, "S_mean": 0.2576904296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.516924619674683, "grad_norm": 0.7992744445800781, "tokens_seen": 46374912, "tokens_per_sec": 16134.628464818852, "vram_gb": 6.154519552, "lambda": 0.2497758191699783, "lambda_min": 0.07269211113452911, "lambda_max": 0.4971734881401062, "S_mean": 0.265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.586075305938721, "grad_norm": 0.797340452671051, "tokens_seen": 46538752, "tokens_per_sec": 16135.43732260638, "vram_gb": 6.154519552, "lambda": 0.2501976250981291, "lambda_min": 0.07260088622570038, "lambda_max": 0.49885761737823486, "S_mean": 0.26739501953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.719087839126587, "grad_norm": 0.9498261213302612, "tokens_seen": 46702592, "tokens_per_sec": 16136.141468517648, "vram_gb": 6.154519552, "lambda": 0.25061630364507437, "lambda_min": 0.07233450561761856, "lambda_max": 0.5003744959831238, "S_mean": 0.270233154296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.592263698577881, "grad_norm": 0.8496268391609192, "tokens_seen": 46866432, "tokens_per_sec": 16136.988485770851, "vram_gb": 6.154519552, "lambda": 0.25099703048666316, "lambda_min": 0.0723334401845932, "lambda_max": 0.5021905899047852, "S_mean": 0.2610575358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.726396560668945, "grad_norm": 0.8677143454551697, "tokens_seen": 47030272, "tokens_per_sec": 16137.507352754137, "vram_gb": 6.154519552, "lambda": 0.2514810236170888, "lambda_min": 0.07224833220243454, "lambda_max": 0.5036274194717407, "S_mean": 0.2560933430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.591373920440674, "grad_norm": 0.8524949550628662, "tokens_seen": 47194112, "tokens_per_sec": 16137.989176476478, "vram_gb": 6.154519552, "lambda": 0.2518500291431944, "lambda_min": 0.07217532396316528, "lambda_max": 0.5048904418945312, "S_mean": 0.2584940592447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.636737823486328, "grad_norm": 0.8444114923477173, "tokens_seen": 47357952, "tokens_per_sec": 16139.035218965308, "vram_gb": 6.154519552, "lambda": 0.25223508073637885, "lambda_min": 0.07217755168676376, "lambda_max": 0.5062940120697021, "S_mean": 0.265106201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.496199607849121, "grad_norm": 0.8154577612876892, "tokens_seen": 47521792, "tokens_per_sec": 16139.818788299852, "vram_gb": 6.154519552, "lambda": 0.25256541060904664, "lambda_min": 0.07216966152191162, "lambda_max": 0.5073041319847107, "S_mean": 0.2667338053385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.742865800857544, "grad_norm": 0.8450354933738708, "tokens_seen": 47685632, "tokens_per_sec": 16140.91876977761, "vram_gb": 6.154519552, "lambda": 0.2530466482664148, "lambda_min": 0.07207201421260834, "lambda_max": 0.5088163614273071, "S_mean": 0.2650655110677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.546119213104248, "grad_norm": 0.7981614470481873, "tokens_seen": 47849472, "tokens_per_sec": 16141.918923829942, "vram_gb": 6.154519552, "lambda": 0.25343132484704256, "lambda_min": 0.07194807380437851, "lambda_max": 0.5101974606513977, "S_mean": 0.266754150390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.634289503097534, "grad_norm": 0.836075484752655, "tokens_seen": 48013312, "tokens_per_sec": 16142.51196717939, "vram_gb": 6.154519552, "lambda": 0.2538413793469469, "lambda_min": 0.07188715785741806, "lambda_max": 0.5115576386451721, "S_mean": 0.2663167317708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.6912806034088135, "grad_norm": 0.9119663238525391, "tokens_seen": 48177152, "tokens_per_sec": 16143.11879792772, "vram_gb": 6.154519552, "lambda": 0.25414797974129516, "lambda_min": 0.07175840437412262, "lambda_max": 0.5124454498291016, "S_mean": 0.2777608235677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.593770980834961, "grad_norm": 0.8598976731300354, "tokens_seen": 48340992, "tokens_per_sec": 16143.701212874832, "vram_gb": 6.154519552, "lambda": 0.25460825363794964, "lambda_min": 0.07162223756313324, "lambda_max": 0.5139439702033997, "S_mean": 0.2541707356770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.582424879074097, "grad_norm": 0.8251415491104126, "tokens_seen": 48504832, "tokens_per_sec": 16144.544637983725, "vram_gb": 6.154519552, "lambda": 0.2550163685033719, "lambda_min": 0.07161496579647064, "lambda_max": 0.5152735114097595, "S_mean": 0.2534688313802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.496814250946045, "grad_norm": 0.8219190835952759, "tokens_seen": 48668672, "tokens_per_sec": 16145.335058465707, "vram_gb": 6.154519552, "lambda": 0.2553661344572902, "lambda_min": 0.07144913077354431, "lambda_max": 0.5165131092071533, "S_mean": 0.2577921549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.572046279907227, "grad_norm": 0.8547254800796509, "tokens_seen": 48832512, "tokens_per_sec": 16146.261607292048, "vram_gb": 6.154519552, "lambda": 0.25568556661407155, "lambda_min": 0.07137002795934677, "lambda_max": 0.517899751663208, "S_mean": 0.262725830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.808257818222046, "grad_norm": 0.8298336863517761, "tokens_seen": 48996352, "tokens_per_sec": 16146.984054974422, "vram_gb": 6.154519552, "lambda": 0.25609944729755324, "lambda_min": 0.07137127220630646, "lambda_max": 0.5191361904144287, "S_mean": 0.2662455240885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.57867956161499, "grad_norm": 0.8041784167289734, "tokens_seen": 49160192, "tokens_per_sec": 16147.655029934203, "vram_gb": 6.154519552, "lambda": 0.25658431835472584, "lambda_min": 0.07132229954004288, "lambda_max": 0.5207424759864807, "S_mean": 0.2570393880208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.665593421459198, "val_ppl": 106.22860485522615, "lambda": 0.2566099436953664, "lambda_min": 0.07131382822990417, "lambda_max": 0.5208040475845337, "lambda_per_layer": [0.16414563357830048, 0.13205932080745697, 0.1416650265455246, 0.14628343284130096, 0.1417883336544037, 0.1482180505990982, 0.111664317548275, 0.07131382822990417, 0.0759052187204361, 0.17545919120311737, 0.24217577278614044, 0.31645911931991577, 0.44009724259376526, 0.3543747365474701, 0.4437043070793152, 0.32572272419929504, 0.4821692705154419, 0.5208040475845337, 0.46787160634994507, 0.34769561886787415, 0.2655530571937561, 0.19819918274879456, 0.2582302391529083, 0.18707937002182007], "S_mean": 0.2547607421875, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.514281272888184, "grad_norm": 0.783341646194458, "tokens_seen": 49324032, "tokens_per_sec": 16128.82917552015, "vram_gb": 6.154519552, "lambda": 0.2569734662150343, "lambda_min": 0.07120722532272339, "lambda_max": 0.5219476819038391, "S_mean": 0.24906412760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.505578517913818, "grad_norm": 0.8146607279777527, "tokens_seen": 49487872, "tokens_per_sec": 16129.65493049318, "vram_gb": 6.154519552, "lambda": 0.25745457131415606, "lambda_min": 0.07124602794647217, "lambda_max": 0.5234377384185791, "S_mean": 0.25164794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.681875467300415, "grad_norm": 0.8390154838562012, "tokens_seen": 49651712, "tokens_per_sec": 16130.589166598918, "vram_gb": 6.154519552, "lambda": 0.25780809484422207, "lambda_min": 0.07119667530059814, "lambda_max": 0.5247904658317566, "S_mean": 0.2686564127604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.617986679077148, "grad_norm": 0.8314343094825745, "tokens_seen": 49815552, "tokens_per_sec": 16131.455178365837, "vram_gb": 6.154519552, "lambda": 0.2581539597983162, "lambda_min": 0.07128774374723434, "lambda_max": 0.525658130645752, "S_mean": 0.2671000162760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.5484349727630615, "grad_norm": 0.8797746300697327, "tokens_seen": 49979392, "tokens_per_sec": 16132.330095040006, "vram_gb": 6.154519552, "lambda": 0.258528473166128, "lambda_min": 0.07117467373609543, "lambda_max": 0.5269787311553955, "S_mean": 0.2727254231770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.688624382019043, "grad_norm": 0.8886569738388062, "tokens_seen": 50143232, "tokens_per_sec": 16133.16265607363, "vram_gb": 6.154519552, "lambda": 0.25887297187000513, "lambda_min": 0.07102706283330917, "lambda_max": 0.5281155705451965, "S_mean": 0.25567626953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.590478420257568, "grad_norm": 0.8964084386825562, "tokens_seen": 50307072, "tokens_per_sec": 16133.772606701115, "vram_gb": 6.154519552, "lambda": 0.259189138499399, "lambda_min": 0.07081206887960434, "lambda_max": 0.529107928276062, "S_mean": 0.2612508138020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.484646320343018, "grad_norm": 0.8642573356628418, "tokens_seen": 50470912, "tokens_per_sec": 16134.608064961165, "vram_gb": 6.154519552, "lambda": 0.25953735255946714, "lambda_min": 0.07076229155063629, "lambda_max": 0.5304405689239502, "S_mean": 0.2690938313802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.642908573150635, "grad_norm": 0.8647579550743103, "tokens_seen": 50634752, "tokens_per_sec": 16135.555152204995, "vram_gb": 6.154519552, "lambda": 0.2600186796238025, "lambda_min": 0.07072384655475616, "lambda_max": 0.5320475101470947, "S_mean": 0.245391845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.566427707672119, "grad_norm": 0.9246394634246826, "tokens_seen": 50798592, "tokens_per_sec": 16136.164064362716, "vram_gb": 6.154519552, "lambda": 0.26035670408358175, "lambda_min": 0.0705336406826973, "lambda_max": 0.5331193208694458, "S_mean": 0.2622578938802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.545607805252075, "grad_norm": 0.8680334687232971, "tokens_seen": 50962432, "tokens_per_sec": 16136.907983056866, "vram_gb": 6.154519552, "lambda": 0.2608103146776557, "lambda_min": 0.07050422579050064, "lambda_max": 0.5342945456504822, "S_mean": 0.263092041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.4753382205963135, "grad_norm": 0.7974619269371033, "tokens_seen": 51126272, "tokens_per_sec": 16137.730494606174, "vram_gb": 6.154519552, "lambda": 0.26122718894233304, "lambda_min": 0.07056154310703278, "lambda_max": 0.5350884795188904, "S_mean": 0.26025390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.613256931304932, "grad_norm": 0.9293235540390015, "tokens_seen": 51290112, "tokens_per_sec": 16138.569955050207, "vram_gb": 6.154519552, "lambda": 0.2615890757491191, "lambda_min": 0.07041455805301666, "lambda_max": 0.5358747243881226, "S_mean": 0.2694600423177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.574558734893799, "grad_norm": 0.8472437858581543, "tokens_seen": 51453952, "tokens_per_sec": 16139.37632209132, "vram_gb": 6.154519552, "lambda": 0.2619721883287032, "lambda_min": 0.07041788101196289, "lambda_max": 0.5370217561721802, "S_mean": 0.2508951822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.519729137420654, "grad_norm": 0.9027578830718994, "tokens_seen": 51617792, "tokens_per_sec": 16140.418791682128, "vram_gb": 6.154519552, "lambda": 0.2622793447226286, "lambda_min": 0.0704173594713211, "lambda_max": 0.5381608009338379, "S_mean": 0.2646382649739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.4728147983551025, "grad_norm": 0.9379920959472656, "tokens_seen": 51781632, "tokens_per_sec": 16141.12944012787, "vram_gb": 6.154519552, "lambda": 0.26265330457439023, "lambda_min": 0.0704125314950943, "lambda_max": 0.5395851731300354, "S_mean": 0.2728983561197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.564558982849121, "grad_norm": 0.8423553109169006, "tokens_seen": 51945472, "tokens_per_sec": 16141.92055975242, "vram_gb": 6.154519552, "lambda": 0.262972737972935, "lambda_min": 0.07046515494585037, "lambda_max": 0.5407214760780334, "S_mean": 0.2652384440104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.567298173904419, "grad_norm": 0.8109541535377502, "tokens_seen": 52109312, "tokens_per_sec": 16142.775729917374, "vram_gb": 6.154519552, "lambda": 0.26328661230703193, "lambda_min": 0.07053642719984055, "lambda_max": 0.541938841342926, "S_mean": 0.2613525390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.593758821487427, "grad_norm": 0.8479346632957458, "tokens_seen": 52273152, "tokens_per_sec": 16143.619795098799, "vram_gb": 6.154519552, "lambda": 0.2636836841702461, "lambda_min": 0.07067321985960007, "lambda_max": 0.5431305170059204, "S_mean": 0.2557576497395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.4459733963012695, "grad_norm": 0.891511857509613, "tokens_seen": 52436992, "tokens_per_sec": 16144.286025962083, "vram_gb": 6.154519552, "lambda": 0.26405732488880557, "lambda_min": 0.07064477354288101, "lambda_max": 0.5440178513526917, "S_mean": 0.2613016764322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.622771263122559, "grad_norm": 0.9032310247421265, "tokens_seen": 52600832, "tokens_per_sec": 16144.869471080687, "vram_gb": 6.154519552, "lambda": 0.2644826766724388, "lambda_min": 0.07072483748197556, "lambda_max": 0.545748770236969, "S_mean": 0.2528584798177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.655794620513916, "grad_norm": 0.8727472424507141, "tokens_seen": 52764672, "tokens_per_sec": 16145.570663182518, "vram_gb": 6.154519552, "lambda": 0.2648964145531257, "lambda_min": 0.07081296294927597, "lambda_max": 0.5472002029418945, "S_mean": 0.260589599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.495954513549805, "grad_norm": 0.8871104121208191, "tokens_seen": 52928512, "tokens_per_sec": 16146.341183839795, "vram_gb": 6.154519552, "lambda": 0.26526647216329974, "lambda_min": 0.07072290778160095, "lambda_max": 0.5485268235206604, "S_mean": 0.270416259765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.52588963508606, "grad_norm": 0.8387609720230103, "tokens_seen": 53092352, "tokens_per_sec": 16147.023597532445, "vram_gb": 6.154519552, "lambda": 0.26567857588330906, "lambda_min": 0.0707511156797409, "lambda_max": 0.5498859286308289, "S_mean": 0.2738444010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.526022672653198, "grad_norm": 0.8669865727424622, "tokens_seen": 53256192, "tokens_per_sec": 16147.689500600674, "vram_gb": 6.154519552, "lambda": 0.2660436149065693, "lambda_min": 0.07072071731090546, "lambda_max": 0.5511029958724976, "S_mean": 0.2707112630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.638076627254486, "val_ppl": 103.34538460850526, "lambda": 0.26605841517448425, "lambda_min": 0.07071880251169205, "lambda_max": 0.551127552986145, "lambda_per_layer": [0.1654149740934372, 0.13243737816810608, 0.14135269820690155, 0.14823488891124725, 0.1429414302110672, 0.15046612918376923, 0.1123981848359108, 0.07071880251169205, 0.07423599064350128, 0.17782029509544373, 0.2458362579345703, 0.32642653584480286, 0.45843467116355896, 0.36778804659843445, 0.466600626707077, 0.33552294969558716, 0.5092415809631348, 0.551127552986145, 0.49620532989501953, 0.36498913168907166, 0.2772308588027954, 0.20370933413505554, 0.27129992842674255, 0.19496838748455048], "S_mean": 0.2753092447916667, "G_mean": null, "G_std": null, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.688273906707764, "grad_norm": 0.8622307777404785, "tokens_seen": 53420032, "tokens_per_sec": 16130.477358822463, "vram_gb": 6.154519552, "lambda": 0.26638095329205197, "lambda_min": 0.07070722430944443, "lambda_max": 0.5515029430389404, "S_mean": 0.2682393391927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.494651794433594, "grad_norm": 0.843377411365509, "tokens_seen": 53583872, "tokens_per_sec": 16131.495234105489, "vram_gb": 6.154519552, "lambda": 0.26664580839375657, "lambda_min": 0.07072800397872925, "lambda_max": 0.5519024729728699, "S_mean": 0.275634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.695894718170166, "grad_norm": 0.9025454521179199, "tokens_seen": 53747712, "tokens_per_sec": 16132.617252843616, "vram_gb": 6.154519552, "lambda": 0.2670180688922604, "lambda_min": 0.07063797861337662, "lambda_max": 0.553178071975708, "S_mean": 0.2717692057291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.6390931606292725, "grad_norm": 0.9292586445808411, "tokens_seen": 53911552, "tokens_per_sec": 16133.175530614479, "vram_gb": 6.154519552, "lambda": 0.26743728170792264, "lambda_min": 0.07062919437885284, "lambda_max": 0.5542726516723633, "S_mean": 0.258575439453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.371925592422485, "grad_norm": 0.9570267796516418, "tokens_seen": 54075392, "tokens_per_sec": 16133.920506677465, "vram_gb": 6.154519552, "lambda": 0.2677856758236885, "lambda_min": 0.07070942223072052, "lambda_max": 0.555137574672699, "S_mean": 0.2686360677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.392845630645752, "grad_norm": 0.8970481157302856, "tokens_seen": 54239232, "tokens_per_sec": 16134.656532475003, "vram_gb": 6.154519552, "lambda": 0.268091787584126, "lambda_min": 0.07083546370267868, "lambda_max": 0.556563138961792, "S_mean": 0.2782491048177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.690078020095825, "grad_norm": 0.855381429195404, "tokens_seen": 54403072, "tokens_per_sec": 16135.431305677937, "vram_gb": 6.154519552, "lambda": 0.26836703221003216, "lambda_min": 0.07068894058465958, "lambda_max": 0.5577637553215027, "S_mean": 0.2601216634114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.662856817245483, "grad_norm": 0.8264659643173218, "tokens_seen": 54566912, "tokens_per_sec": 16136.148469217227, "vram_gb": 6.154519552, "lambda": 0.268718325222532, "lambda_min": 0.07068130373954773, "lambda_max": 0.5593035817146301, "S_mean": 0.261474609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.531438112258911, "grad_norm": 0.8705062866210938, "tokens_seen": 54730752, "tokens_per_sec": 16136.883845321732, "vram_gb": 6.154519552, "lambda": 0.2690673191100359, "lambda_min": 0.07074723392724991, "lambda_max": 0.5603967905044556, "S_mean": 0.2606709798177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.5002264976501465, "grad_norm": 0.9253311157226562, "tokens_seen": 54894592, "tokens_per_sec": 16136.840623387849, "vram_gb": 6.154519552, "lambda": 0.26939949424316484, "lambda_min": 0.0707884430885315, "lambda_max": 0.5613402128219604, "S_mean": 0.24552408854166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.58938455581665, "grad_norm": 0.8853017687797546, "tokens_seen": 55058432, "tokens_per_sec": 16136.723515749503, "vram_gb": 6.154519552, "lambda": 0.26969714121272165, "lambda_min": 0.07071292400360107, "lambda_max": 0.5624027252197266, "S_mean": 0.249359130859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.483956813812256, "grad_norm": 0.8629613518714905, "tokens_seen": 55222272, "tokens_per_sec": 16136.465593954299, "vram_gb": 6.154519552, "lambda": 0.270036765995125, "lambda_min": 0.07074597477912903, "lambda_max": 0.563556432723999, "S_mean": 0.2723897298177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.502446174621582, "grad_norm": 0.9102721214294434, "tokens_seen": 55386112, "tokens_per_sec": 16135.39510536931, "vram_gb": 6.154519552, "lambda": 0.2703805410613616, "lambda_min": 0.0706339105963707, "lambda_max": 0.5645874738693237, "S_mean": 0.25970458984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.501631259918213, "grad_norm": 0.9705734848976135, "tokens_seen": 55549952, "tokens_per_sec": 16133.696476326437, "vram_gb": 6.154519552, "lambda": 0.27067977966119844, "lambda_min": 0.07064025104045868, "lambda_max": 0.5654911994934082, "S_mean": 0.2525736490885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.435414552688599, "grad_norm": 0.84778892993927, "tokens_seen": 55713792, "tokens_per_sec": 16132.346172972442, "vram_gb": 6.154519552, "lambda": 0.2710682429994146, "lambda_min": 0.07050630450248718, "lambda_max": 0.5668593645095825, "S_mean": 0.24742635091145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.4435811042785645, "grad_norm": 0.8699102997779846, "tokens_seen": 55877632, "tokens_per_sec": 16130.910201230514, "vram_gb": 6.154519552, "lambda": 0.2714441018179059, "lambda_min": 0.07049622386693954, "lambda_max": 0.5683602094650269, "S_mean": 0.2624918619791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.36960506439209, "grad_norm": 0.8514255881309509, "tokens_seen": 56041472, "tokens_per_sec": 16129.927358051164, "vram_gb": 6.154519552, "lambda": 0.27181279721359414, "lambda_min": 0.07039029151201248, "lambda_max": 0.5694301724433899, "S_mean": 0.2599690755208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.570538282394409, "grad_norm": 0.8989320993423462, "tokens_seen": 56205312, "tokens_per_sec": 16127.967430858556, "vram_gb": 6.154519552, "lambda": 0.27209067686150473, "lambda_min": 0.07039951533079147, "lambda_max": 0.569965660572052, "S_mean": 0.2600809733072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.501352548599243, "grad_norm": 0.8474579453468323, "tokens_seen": 56369152, "tokens_per_sec": 16127.174524279662, "vram_gb": 6.154519552, "lambda": 0.27237822860479355, "lambda_min": 0.0702972263097763, "lambda_max": 0.5706485509872437, "S_mean": 0.2711690266927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.643942356109619, "grad_norm": 0.9120400547981262, "tokens_seen": 56532992, "tokens_per_sec": 16126.617026500755, "vram_gb": 6.154519552, "lambda": 0.2726485515013337, "lambda_min": 0.07023025304079056, "lambda_max": 0.5717151165008545, "S_mean": 0.2523396809895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.594601154327393, "grad_norm": 0.9607181549072266, "tokens_seen": 56696832, "tokens_per_sec": 16125.590322145441, "vram_gb": 6.154519552, "lambda": 0.2729827808216214, "lambda_min": 0.07015572488307953, "lambda_max": 0.5729711055755615, "S_mean": 0.24930826822916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.416042804718018, "grad_norm": 0.8204804062843323, "tokens_seen": 56860672, "tokens_per_sec": 16125.67783895177, "vram_gb": 6.154519552, "lambda": 0.2733424073085189, "lambda_min": 0.07011539489030838, "lambda_max": 0.5742692351341248, "S_mean": 0.25439453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.5359485149383545, "grad_norm": 0.9616599678993225, "tokens_seen": 57024512, "tokens_per_sec": 16125.400444011104, "vram_gb": 6.154519552, "lambda": 0.2736334915583332, "lambda_min": 0.06994501501321793, "lambda_max": 0.5755179524421692, "S_mean": 0.2523396809895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.358806610107422, "grad_norm": 0.8638235926628113, "tokens_seen": 57188352, "tokens_per_sec": 16124.631489829198, "vram_gb": 6.154519552, "lambda": 0.27394513972103596, "lambda_min": 0.06986093521118164, "lambda_max": 0.5766833424568176, "S_mean": 0.2577006022135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.672555923461914, "grad_norm": 0.9567858576774597, "tokens_seen": 57352192, "tokens_per_sec": 16123.915750570395, "vram_gb": 6.154519552, "lambda": 0.27427069656550884, "lambda_min": 0.06984599679708481, "lambda_max": 0.5778303146362305, "S_mean": 0.2545166015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.584732127189636, "val_ppl": 97.97693826878107, "lambda": 0.27429071130851906, "lambda_min": 0.06984609365463257, "lambda_max": 0.5779255628585815, "lambda_per_layer": [0.16674761474132538, 0.13255654275417328, 0.1414952576160431, 0.15012481808662415, 0.14416496455669403, 0.15280194580554962, 0.11256694048643112, 0.06984609365463257, 0.07334447652101517, 0.17867138981819153, 0.2492368519306183, 0.3353866934776306, 0.47392046451568604, 0.3785848319530487, 0.486013799905777, 0.346597284078598, 0.5324484705924988, 0.5779255628585815, 0.5201611518859863, 0.3799307346343994, 0.28857511281967163, 0.20849758386611938, 0.2827109098434448, 0.2006675750017166], "S_mean": 0.2562357584635417, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.450207471847534, "grad_norm": 0.8767351508140564, "tokens_seen": 57516032, "tokens_per_sec": 16107.475846702773, "vram_gb": 6.154519552, "lambda": 0.2746367147192359, "lambda_min": 0.06990346312522888, "lambda_max": 0.5790207386016846, "S_mean": 0.2683003743489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.489661693572998, "grad_norm": 0.8776440024375916, "tokens_seen": 57679872, "tokens_per_sec": 16108.170878281107, "vram_gb": 6.154519552, "lambda": 0.2749103515719374, "lambda_min": 0.06987321376800537, "lambda_max": 0.5799442529678345, "S_mean": 0.2588399251302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.5623040199279785, "grad_norm": 0.9616817235946655, "tokens_seen": 57843712, "tokens_per_sec": 16108.886824960566, "vram_gb": 6.154519552, "lambda": 0.2752482444047928, "lambda_min": 0.06986447423696518, "lambda_max": 0.5809059143066406, "S_mean": 0.2637125651041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.589480400085449, "grad_norm": 0.9416043162345886, "tokens_seen": 58007552, "tokens_per_sec": 16109.722079909752, "vram_gb": 6.154519552, "lambda": 0.27555989536146325, "lambda_min": 0.06971142441034317, "lambda_max": 0.5819215178489685, "S_mean": 0.245147705078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.536134719848633, "grad_norm": 0.9057261943817139, "tokens_seen": 58171392, "tokens_per_sec": 16110.406267654755, "vram_gb": 6.154519552, "lambda": 0.27586640138179064, "lambda_min": 0.069611094892025, "lambda_max": 0.5826612710952759, "S_mean": 0.26385498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.74319863319397, "grad_norm": 0.919207751750946, "tokens_seen": 58335232, "tokens_per_sec": 16111.327425404435, "vram_gb": 6.154519552, "lambda": 0.276191510570546, "lambda_min": 0.06972955167293549, "lambda_max": 0.583493709564209, "S_mean": 0.260986328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.518805027008057, "grad_norm": 0.9907713532447815, "tokens_seen": 58499072, "tokens_per_sec": 16112.200465572569, "vram_gb": 6.154519552, "lambda": 0.276406048797071, "lambda_min": 0.06981061398983002, "lambda_max": 0.5840065479278564, "S_mean": 0.2596944173177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.462024688720703, "grad_norm": 0.8859684467315674, "tokens_seen": 58662912, "tokens_per_sec": 16113.154281816847, "vram_gb": 6.154519552, "lambda": 0.2767351033786933, "lambda_min": 0.06982254236936569, "lambda_max": 0.5847185850143433, "S_mean": 0.2518107096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.591569900512695, "grad_norm": 0.8731820583343506, "tokens_seen": 58826752, "tokens_per_sec": 16114.175224064358, "vram_gb": 6.154519552, "lambda": 0.2769890542452534, "lambda_min": 0.0699029415845871, "lambda_max": 0.5856376886367798, "S_mean": 0.2675374348958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.634812593460083, "grad_norm": 0.9481473565101624, "tokens_seen": 58990592, "tokens_per_sec": 16115.043536607998, "vram_gb": 6.154519552, "lambda": 0.27736977611978847, "lambda_min": 0.06988312304019928, "lambda_max": 0.5866988897323608, "S_mean": 0.2524210611979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.589492559432983, "grad_norm": 0.928129255771637, "tokens_seen": 59154432, "tokens_per_sec": 16115.927520512852, "vram_gb": 6.154519552, "lambda": 0.27765374816954136, "lambda_min": 0.069839708507061, "lambda_max": 0.587847888469696, "S_mean": 0.24660237630208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.392970561981201, "grad_norm": 0.9135310053825378, "tokens_seen": 59318272, "tokens_per_sec": 16116.849192485475, "vram_gb": 6.154519552, "lambda": 0.27793166041374207, "lambda_min": 0.06975796818733215, "lambda_max": 0.5889254808425903, "S_mean": 0.2557474772135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.440944194793701, "grad_norm": 0.9370925426483154, "tokens_seen": 59482112, "tokens_per_sec": 16117.560768196505, "vram_gb": 6.154519552, "lambda": 0.2782314857468009, "lambda_min": 0.06968145817518234, "lambda_max": 0.5900896787643433, "S_mean": 0.28155517578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.51357364654541, "grad_norm": 0.9213422536849976, "tokens_seen": 59645952, "tokens_per_sec": 16118.485330312355, "vram_gb": 6.154519552, "lambda": 0.2785340202972293, "lambda_min": 0.06955992430448532, "lambda_max": 0.591475784778595, "S_mean": 0.24969482421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.362667798995972, "grad_norm": 0.9288856387138367, "tokens_seen": 59809792, "tokens_per_sec": 16119.258786613991, "vram_gb": 6.154519552, "lambda": 0.2788186551382144, "lambda_min": 0.06954201310873032, "lambda_max": 0.5927560329437256, "S_mean": 0.2618204752604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.618250846862793, "grad_norm": 0.8723403215408325, "tokens_seen": 59973632, "tokens_per_sec": 16120.03292617874, "vram_gb": 6.154519552, "lambda": 0.2791850545133154, "lambda_min": 0.06948035955429077, "lambda_max": 0.5943001508712769, "S_mean": 0.2559916178385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.432756423950195, "grad_norm": 0.9140462279319763, "tokens_seen": 60137472, "tokens_per_sec": 16120.907376977548, "vram_gb": 6.154519552, "lambda": 0.2795635983347893, "lambda_min": 0.06951615959405899, "lambda_max": 0.5957950353622437, "S_mean": 0.2763875325520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.388393402099609, "grad_norm": 0.9024583697319031, "tokens_seen": 60301312, "tokens_per_sec": 16121.731554486625, "vram_gb": 6.154519552, "lambda": 0.27983981433014077, "lambda_min": 0.06945732235908508, "lambda_max": 0.5966772437095642, "S_mean": 0.2633260091145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.359056234359741, "grad_norm": 0.9704884886741638, "tokens_seen": 60465152, "tokens_per_sec": 16122.54975428502, "vram_gb": 6.154519552, "lambda": 0.2800653586164117, "lambda_min": 0.0694054365158081, "lambda_max": 0.5973005890846252, "S_mean": 0.24951171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.477058410644531, "grad_norm": 0.8812622427940369, "tokens_seen": 60628992, "tokens_per_sec": 16123.3671432819, "vram_gb": 6.154519552, "lambda": 0.280342352266113, "lambda_min": 0.06943965703248978, "lambda_max": 0.5977798104286194, "S_mean": 0.24993896484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.5912463665008545, "grad_norm": 0.9979318976402283, "tokens_seen": 60792832, "tokens_per_sec": 16124.07990616768, "vram_gb": 6.154519552, "lambda": 0.28066197130829096, "lambda_min": 0.06945085525512695, "lambda_max": 0.5990818738937378, "S_mean": 0.2588094075520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.62716817855835, "grad_norm": 0.9290640950202942, "tokens_seen": 60956672, "tokens_per_sec": 16124.748678056178, "vram_gb": 6.154519552, "lambda": 0.2809149483218789, "lambda_min": 0.06947963684797287, "lambda_max": 0.5998488068580627, "S_mean": 0.2537740071614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.4216225147247314, "grad_norm": 0.9053225517272949, "tokens_seen": 61120512, "tokens_per_sec": 16125.564161982405, "vram_gb": 6.154519552, "lambda": 0.28117808420211077, "lambda_min": 0.0695088654756546, "lambda_max": 0.6010158061981201, "S_mean": 0.2633056640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.47690224647522, "grad_norm": 0.9076870083808899, "tokens_seen": 61284352, "tokens_per_sec": 16126.429657993394, "vram_gb": 6.154519552, "lambda": 0.2814760835220416, "lambda_min": 0.06944520771503448, "lambda_max": 0.6023529767990112, "S_mean": 0.2579345703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.533656358718872, "grad_norm": 0.8980927467346191, "tokens_seen": 61448192, "tokens_per_sec": 16127.393895317915, "vram_gb": 6.154519552, "lambda": 0.28182004981984693, "lambda_min": 0.06940124183893204, "lambda_max": 0.6033719182014465, "S_mean": 0.244049072265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.5993412733078, "val_ppl": 99.41880424718633, "lambda": 0.2818329818546772, "lambda_min": 0.06940525025129318, "lambda_max": 0.6034249067306519, "lambda_per_layer": [0.168025940656662, 0.13265296816825867, 0.14160414040088654, 0.15177887678146362, 0.14523036777973175, 0.15479561686515808, 0.11277716606855392, 0.06940525025129318, 0.07277853786945343, 0.18028682470321655, 0.25145265460014343, 0.3413330018520355, 0.4876658320426941, 0.38900303840637207, 0.5016632676124573, 0.35775187611579895, 0.5528472065925598, 0.6034249067306519, 0.5422776937484741, 0.3930739462375641, 0.29927876591682434, 0.21383118629455566, 0.29434552788734436, 0.2067069709300995], "S_mean": 0.2754007975260417, "G_mean": null, "G_std": null, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.522007465362549, "grad_norm": 0.9198376536369324, "tokens_seen": 61612032, "tokens_per_sec": 16112.585976530952, "vram_gb": 6.154519552, "lambda": 0.28211297436306876, "lambda_min": 0.06951098889112473, "lambda_max": 0.6044970750808716, "S_mean": 0.2532755533854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.583690166473389, "grad_norm": 0.8654879331588745, "tokens_seen": 61775872, "tokens_per_sec": 16113.298600195381, "vram_gb": 6.154519552, "lambda": 0.28235764894634485, "lambda_min": 0.06958192586898804, "lambda_max": 0.6051550507545471, "S_mean": 0.2617899576822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.288999080657959, "grad_norm": 0.8599641919136047, "tokens_seen": 61939712, "tokens_per_sec": 16113.906653596396, "vram_gb": 6.154519552, "lambda": 0.28262589623530704, "lambda_min": 0.0696357935667038, "lambda_max": 0.6062316298484802, "S_mean": 0.24454752604166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.50140118598938, "grad_norm": 0.8741991519927979, "tokens_seen": 62103552, "tokens_per_sec": 16114.615366569727, "vram_gb": 6.154519552, "lambda": 0.2829236580679814, "lambda_min": 0.06959357112646103, "lambda_max": 0.6071894764900208, "S_mean": 0.2589823404947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.369568824768066, "grad_norm": 0.940298318862915, "tokens_seen": 62267392, "tokens_per_sec": 16115.283788469693, "vram_gb": 6.154519552, "lambda": 0.28318110077331465, "lambda_min": 0.06955000013113022, "lambda_max": 0.6078588962554932, "S_mean": 0.2586263020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.552292346954346, "grad_norm": 0.9725144505500793, "tokens_seen": 62431232, "tokens_per_sec": 16115.991592600627, "vram_gb": 6.154519552, "lambda": 0.28345720563083887, "lambda_min": 0.0695495754480362, "lambda_max": 0.6089694499969482, "S_mean": 0.2539774576822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.370127439498901, "grad_norm": 0.9301874041557312, "tokens_seen": 62595072, "tokens_per_sec": 16116.724163482528, "vram_gb": 6.154519552, "lambda": 0.28370180757095415, "lambda_min": 0.0695401057600975, "lambda_max": 0.6100153923034668, "S_mean": 0.2536722819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.5280702114105225, "grad_norm": 0.9255432486534119, "tokens_seen": 62758912, "tokens_per_sec": 16117.486496405487, "vram_gb": 6.154519552, "lambda": 0.28389049352457124, "lambda_min": 0.06958657503128052, "lambda_max": 0.6104608774185181, "S_mean": 0.25982666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.486393690109253, "grad_norm": 0.9555727243423462, "tokens_seen": 62922752, "tokens_per_sec": 16118.145201842302, "vram_gb": 6.154519552, "lambda": 0.28412259959926206, "lambda_min": 0.06957286596298218, "lambda_max": 0.6112620830535889, "S_mean": 0.239501953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.540814161300659, "grad_norm": 0.920619785785675, "tokens_seen": 63086592, "tokens_per_sec": 16119.041066839658, "vram_gb": 6.154519552, "lambda": 0.2843953262393673, "lambda_min": 0.06959953159093857, "lambda_max": 0.6121300458908081, "S_mean": 0.24361165364583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.514208078384399, "grad_norm": 1.0354639291763306, "tokens_seen": 63250432, "tokens_per_sec": 16120.056185106003, "vram_gb": 6.154519552, "lambda": 0.2846067106972138, "lambda_min": 0.06956588476896286, "lambda_max": 0.6127507090568542, "S_mean": 0.24302164713541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.354156494140625, "grad_norm": 0.9113000631332397, "tokens_seen": 63414272, "tokens_per_sec": 16121.407830482016, "vram_gb": 6.154519552, "lambda": 0.28483075990031165, "lambda_min": 0.06944245845079422, "lambda_max": 0.6133849620819092, "S_mean": 0.24625651041666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.474567890167236, "grad_norm": 1.025619626045227, "tokens_seen": 63578112, "tokens_per_sec": 16122.553798931567, "vram_gb": 6.154519552, "lambda": 0.2850479284922282, "lambda_min": 0.0694161206483841, "lambda_max": 0.6141388416290283, "S_mean": 0.24565633138020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.421447038650513, "grad_norm": 1.0408776998519897, "tokens_seen": 63741952, "tokens_per_sec": 16123.150725262307, "vram_gb": 6.154519552, "lambda": 0.2853077305480838, "lambda_min": 0.06932936608791351, "lambda_max": 0.6150099635124207, "S_mean": 0.2705586751302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.500120639801025, "grad_norm": 0.9024571180343628, "tokens_seen": 63905792, "tokens_per_sec": 16123.809054466798, "vram_gb": 6.154519552, "lambda": 0.28561234287917614, "lambda_min": 0.06928503513336182, "lambda_max": 0.6158750057220459, "S_mean": 0.24485270182291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.561095714569092, "grad_norm": 0.9994178414344788, "tokens_seen": 64069632, "tokens_per_sec": 16124.469985439264, "vram_gb": 6.154519552, "lambda": 0.28590454751004774, "lambda_min": 0.06921118497848511, "lambda_max": 0.6166386604309082, "S_mean": 0.2543131510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.322396516799927, "grad_norm": 0.9256152510643005, "tokens_seen": 64233472, "tokens_per_sec": 16125.16032994284, "vram_gb": 6.154519552, "lambda": 0.28618164360523224, "lambda_min": 0.06910048425197601, "lambda_max": 0.6174619793891907, "S_mean": 0.2674763997395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.472527265548706, "grad_norm": 1.0914229154586792, "tokens_seen": 64397312, "tokens_per_sec": 16125.841372559897, "vram_gb": 6.154519552, "lambda": 0.28646007645875216, "lambda_min": 0.06906894594430923, "lambda_max": 0.6181970834732056, "S_mean": 0.253631591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.530910015106201, "grad_norm": 0.9846135377883911, "tokens_seen": 64561152, "tokens_per_sec": 16126.48784990677, "vram_gb": 6.154519552, "lambda": 0.28668402569989365, "lambda_min": 0.06905291229486465, "lambda_max": 0.6190630793571472, "S_mean": 0.24939982096354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.305903911590576, "grad_norm": 0.9667578935623169, "tokens_seen": 64724992, "tokens_per_sec": 16127.271355993358, "vram_gb": 6.154519552, "lambda": 0.2869128854945302, "lambda_min": 0.06905972212553024, "lambda_max": 0.6197086572647095, "S_mean": 0.2515970865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.363046169281006, "grad_norm": 0.8970730304718018, "tokens_seen": 64888832, "tokens_per_sec": 16127.632201843253, "vram_gb": 6.154519552, "lambda": 0.2871545773620407, "lambda_min": 0.06903386861085892, "lambda_max": 0.620361328125, "S_mean": 0.2594807942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.494290113449097, "grad_norm": 0.8919264078140259, "tokens_seen": 65052672, "tokens_per_sec": 16128.276126779961, "vram_gb": 6.154519552, "lambda": 0.28738891116033, "lambda_min": 0.06896921992301941, "lambda_max": 0.6210496425628662, "S_mean": 0.2583719889322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.519160747528076, "grad_norm": 0.983947217464447, "tokens_seen": 65216512, "tokens_per_sec": 16128.882786415852, "vram_gb": 6.154519552, "lambda": 0.2876265387361248, "lambda_min": 0.06891680508852005, "lambda_max": 0.6218582987785339, "S_mean": 0.2549641927083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.425504922866821, "grad_norm": 0.9116804003715515, "tokens_seen": 65380352, "tokens_per_sec": 16129.5351340803, "vram_gb": 6.154519552, "lambda": 0.28791743361701566, "lambda_min": 0.06894982606172562, "lambda_max": 0.6230531334877014, "S_mean": 0.2489013671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.39837646484375, "grad_norm": 1.0269968509674072, "tokens_seen": 65544192, "tokens_per_sec": 16130.20498240981, "vram_gb": 6.154519552, "lambda": 0.2882212145874898, "lambda_min": 0.06895938515663147, "lambda_max": 0.6240218877792358, "S_mean": 0.2510477701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.5275097489356995, "val_ppl": 92.52785635222699, "lambda": 0.2882384977613886, "lambda_min": 0.06895685940980911, "lambda_max": 0.6240699291229248, "lambda_per_layer": [0.16976886987686157, 0.13267795741558075, 0.14206992089748383, 0.15318624675273895, 0.1463353931903839, 0.155806303024292, 0.11270073056221008, 0.06895685940980911, 0.07244041562080383, 0.18130086362361908, 0.25395312905311584, 0.3470326066017151, 0.4989030361175537, 0.39748185873031616, 0.5144902467727661, 0.3658803403377533, 0.5720478892326355, 0.6240699291229248, 0.5616084933280945, 0.40456661581993103, 0.308298796415329, 0.2192586362361908, 0.30325064063072205, 0.2116381675004959], "S_mean": 0.25628662109375, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.441315412521362, "grad_norm": 0.9063171148300171, "tokens_seen": 65708032, "tokens_per_sec": 16115.906709462697, "vram_gb": 6.154519552, "lambda": 0.28852348184833926, "lambda_min": 0.06896904855966568, "lambda_max": 0.6249656677246094, "S_mean": 0.2669169108072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.589094400405884, "grad_norm": 0.9628768563270569, "tokens_seen": 65871872, "tokens_per_sec": 16116.59368182892, "vram_gb": 6.154519552, "lambda": 0.28873354227592546, "lambda_min": 0.06897182762622833, "lambda_max": 0.6255350708961487, "S_mean": 0.269866943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.512319564819336, "grad_norm": 0.9433371424674988, "tokens_seen": 66035712, "tokens_per_sec": 16117.21206519533, "vram_gb": 6.154519552, "lambda": 0.28898813140889007, "lambda_min": 0.06900165975093842, "lambda_max": 0.6264545321464539, "S_mean": 0.2536722819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.3904993534088135, "grad_norm": 0.9548090100288391, "tokens_seen": 66199552, "tokens_per_sec": 16117.9537309105, "vram_gb": 6.154519552, "lambda": 0.2892303755506873, "lambda_min": 0.068975068628788, "lambda_max": 0.6274115443229675, "S_mean": 0.2491455078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.5345048904418945, "grad_norm": 0.9839745759963989, "tokens_seen": 66363392, "tokens_per_sec": 16118.613115519629, "vram_gb": 6.154519552, "lambda": 0.2894964615503947, "lambda_min": 0.06897100806236267, "lambda_max": 0.6282265186309814, "S_mean": 0.25555419921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.365135669708252, "grad_norm": 0.8587566614151001, "tokens_seen": 66527232, "tokens_per_sec": 16119.3090336351, "vram_gb": 6.154519552, "lambda": 0.2897577394420902, "lambda_min": 0.06896616518497467, "lambda_max": 0.6289550065994263, "S_mean": 0.24476114908854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.528045654296875, "grad_norm": 0.968193531036377, "tokens_seen": 66691072, "tokens_per_sec": 16120.137476724272, "vram_gb": 6.154519552, "lambda": 0.2899898402392864, "lambda_min": 0.0689089223742485, "lambda_max": 0.6297160983085632, "S_mean": 0.2560933430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.497956991195679, "grad_norm": 0.9992125630378723, "tokens_seen": 66854912, "tokens_per_sec": 16120.910534103308, "vram_gb": 6.154519552, "lambda": 0.29022093831251067, "lambda_min": 0.06889662891626358, "lambda_max": 0.6304150223731995, "S_mean": 0.2670796712239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.579078197479248, "grad_norm": 0.9113086462020874, "tokens_seen": 67018752, "tokens_per_sec": 16121.73272950361, "vram_gb": 6.154519552, "lambda": 0.2904351207738121, "lambda_min": 0.06894984841346741, "lambda_max": 0.6309427618980408, "S_mean": 0.25592041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.244122743606567, "grad_norm": 0.9214779734611511, "tokens_seen": 67182592, "tokens_per_sec": 16122.470038951285, "vram_gb": 6.154519552, "lambda": 0.2906233662118514, "lambda_min": 0.0690397098660469, "lambda_max": 0.6316334009170532, "S_mean": 0.2582906087239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.406296968460083, "grad_norm": 0.952558696269989, "tokens_seen": 67346432, "tokens_per_sec": 16123.254589830027, "vram_gb": 6.154519552, "lambda": 0.29084122522423667, "lambda_min": 0.06911623477935791, "lambda_max": 0.6324028372764587, "S_mean": 0.2540791829427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.259645938873291, "grad_norm": 0.95842045545578, "tokens_seen": 67510272, "tokens_per_sec": 16124.071461891526, "vram_gb": 6.154519552, "lambda": 0.29107419432451326, "lambda_min": 0.06912128627300262, "lambda_max": 0.6334199905395508, "S_mean": 0.2708943684895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.39815092086792, "grad_norm": 0.9099115133285522, "tokens_seen": 67674112, "tokens_per_sec": 16124.589199718293, "vram_gb": 6.154519552, "lambda": 0.29126286537696916, "lambda_min": 0.06913227587938309, "lambda_max": 0.6342106461524963, "S_mean": 0.24700927734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.34062385559082, "grad_norm": 0.916073739528656, "tokens_seen": 67837952, "tokens_per_sec": 16125.322963819273, "vram_gb": 6.154519552, "lambda": 0.29142970498651266, "lambda_min": 0.06915692985057831, "lambda_max": 0.6345229744911194, "S_mean": 0.24413045247395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.364148855209351, "grad_norm": 1.015223503112793, "tokens_seen": 68001792, "tokens_per_sec": 16125.990276794386, "vram_gb": 6.154519552, "lambda": 0.29166133391360444, "lambda_min": 0.06911832094192505, "lambda_max": 0.6350966691970825, "S_mean": 0.2564595540364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.411790132522583, "grad_norm": 0.9665365219116211, "tokens_seen": 68165632, "tokens_per_sec": 16126.76040259199, "vram_gb": 6.154519552, "lambda": 0.2918169867868225, "lambda_min": 0.06906624138355255, "lambda_max": 0.6357305645942688, "S_mean": 0.2571004231770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.444181442260742, "grad_norm": 0.9036914110183716, "tokens_seen": 68329472, "tokens_per_sec": 16127.381370364996, "vram_gb": 6.154519552, "lambda": 0.2920383792370558, "lambda_min": 0.06902382522821426, "lambda_max": 0.6362997889518738, "S_mean": 0.2565409342447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.490655422210693, "grad_norm": 0.9424726963043213, "tokens_seen": 68493312, "tokens_per_sec": 16127.930071278828, "vram_gb": 6.154519552, "lambda": 0.2922313576564193, "lambda_min": 0.06896743923425674, "lambda_max": 0.6367461085319519, "S_mean": 0.2630411783854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.436992406845093, "grad_norm": 0.8919740319252014, "tokens_seen": 68657152, "tokens_per_sec": 16128.607882769711, "vram_gb": 6.154519552, "lambda": 0.2924636627237002, "lambda_min": 0.06893405318260193, "lambda_max": 0.6374523639678955, "S_mean": 0.25042724609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.390095949172974, "grad_norm": 0.8732760548591614, "tokens_seen": 68820992, "tokens_per_sec": 16129.20710209407, "vram_gb": 6.154519552, "lambda": 0.2926519817362229, "lambda_min": 0.06896628439426422, "lambda_max": 0.6380151510238647, "S_mean": 0.2541300455729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.515831470489502, "grad_norm": 0.9044153094291687, "tokens_seen": 68984832, "tokens_per_sec": 16129.846740120684, "vram_gb": 6.154519552, "lambda": 0.29286807247747976, "lambda_min": 0.06896115094423294, "lambda_max": 0.6383222937583923, "S_mean": 0.265411376953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.192048072814941, "grad_norm": 0.97661954164505, "tokens_seen": 69148672, "tokens_per_sec": 16130.556113890341, "vram_gb": 6.154519552, "lambda": 0.29309562624742586, "lambda_min": 0.06897308677434921, "lambda_max": 0.6389379501342773, "S_mean": 0.234344482421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.357727527618408, "grad_norm": 0.9904423356056213, "tokens_seen": 69312512, "tokens_per_sec": 16131.12322064804, "vram_gb": 6.154519552, "lambda": 0.2932690183321635, "lambda_min": 0.06898830831050873, "lambda_max": 0.6393051743507385, "S_mean": 0.24547322591145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.317386150360107, "grad_norm": 0.9672633409500122, "tokens_seen": 69476352, "tokens_per_sec": 16131.71764522875, "vram_gb": 6.154519552, "lambda": 0.29344680936386186, "lambda_min": 0.06898947060108185, "lambda_max": 0.6398281455039978, "S_mean": 0.2598470052083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.608724594116211, "grad_norm": 0.9672340154647827, "tokens_seen": 69640192, "tokens_per_sec": 16132.352428834536, "vram_gb": 6.154519552, "lambda": 0.29361132842799026, "lambda_min": 0.069000244140625, "lambda_max": 0.6404197216033936, "S_mean": 0.2489013671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.519596755504608, "val_ppl": 91.79857324430283, "lambda": 0.2936211209744215, "lambda_min": 0.06899821013212204, "lambda_max": 0.6404475569725037, "lambda_per_layer": [0.1707434505224228, 0.1327361762523651, 0.1423237919807434, 0.1543995589017868, 0.14721053838729858, 0.15698906779289246, 0.11275006085634232, 0.06899821013212204, 0.07212486863136292, 0.182307168841362, 0.25592395663261414, 0.35082998871803284, 0.5081543326377869, 0.40503382682800293, 0.5268533229827881, 0.3728078007698059, 0.5873579382896423, 0.6404475569725037, 0.5761870741844177, 0.4149761497974396, 0.3166859745979309, 0.2235635370016098, 0.3115752935409546, 0.21592725813388824], "S_mean": 0.24727376302083334, "G_mean": null, "G_std": null, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.419729948043823, "grad_norm": 0.953633725643158, "tokens_seen": 69804032, "tokens_per_sec": 16118.824033290815, "vram_gb": 6.154519552, "lambda": 0.2938387679557006, "lambda_min": 0.0689832866191864, "lambda_max": 0.6412710547447205, "S_mean": 0.24272664388020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.476521015167236, "grad_norm": 1.0161094665527344, "tokens_seen": 69967872, "tokens_per_sec": 16119.47114469933, "vram_gb": 6.154519552, "lambda": 0.29404696480681497, "lambda_min": 0.06902873516082764, "lambda_max": 0.6418035626411438, "S_mean": 0.23597208658854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.531521320343018, "grad_norm": 0.9576185345649719, "tokens_seen": 70131712, "tokens_per_sec": 16120.057593338242, "vram_gb": 6.154519552, "lambda": 0.2942091707761089, "lambda_min": 0.06897931545972824, "lambda_max": 0.6424678564071655, "S_mean": 0.2614339192708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.385939836502075, "grad_norm": 0.9515169262886047, "tokens_seen": 70295552, "tokens_per_sec": 16120.585879436709, "vram_gb": 6.154519552, "lambda": 0.294420891130964, "lambda_min": 0.06896097958087921, "lambda_max": 0.6435471773147583, "S_mean": 0.24757893880208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.494227886199951, "grad_norm": 1.0911028385162354, "tokens_seen": 70459392, "tokens_per_sec": 16121.304817104294, "vram_gb": 6.154519552, "lambda": 0.2946446466570099, "lambda_min": 0.06891714036464691, "lambda_max": 0.6444702744483948, "S_mean": 0.24776204427083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.44945216178894, "grad_norm": 1.0198513269424438, "tokens_seen": 70623232, "tokens_per_sec": 16121.978016859262, "vram_gb": 6.154519552, "lambda": 0.2948167234038313, "lambda_min": 0.06890684366226196, "lambda_max": 0.6451445817947388, "S_mean": 0.2517598470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.4908270835876465, "grad_norm": 0.975866973400116, "tokens_seen": 70787072, "tokens_per_sec": 16122.751196802192, "vram_gb": 6.154519552, "lambda": 0.2950603763262431, "lambda_min": 0.06885984539985657, "lambda_max": 0.6458332538604736, "S_mean": 0.2501017252604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.334603309631348, "grad_norm": 1.0124017000198364, "tokens_seen": 70950912, "tokens_per_sec": 16123.593909275563, "vram_gb": 6.154519552, "lambda": 0.29521816068639356, "lambda_min": 0.06879086047410965, "lambda_max": 0.6460034847259521, "S_mean": 0.2704060872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.394267320632935, "grad_norm": 0.9210063219070435, "tokens_seen": 71114752, "tokens_per_sec": 16124.270503901555, "vram_gb": 6.154519552, "lambda": 0.2954234428082903, "lambda_min": 0.06878582388162613, "lambda_max": 0.6464794874191284, "S_mean": 0.25592041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.380805730819702, "grad_norm": 0.9221457242965698, "tokens_seen": 71278592, "tokens_per_sec": 16125.005437110685, "vram_gb": 6.154519552, "lambda": 0.29561913789560396, "lambda_min": 0.0688198134303093, "lambda_max": 0.6470376253128052, "S_mean": 0.2554829915364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.346181392669678, "grad_norm": 1.0206105709075928, "tokens_seen": 71442432, "tokens_per_sec": 16125.596898198997, "vram_gb": 6.154519552, "lambda": 0.2957868929952383, "lambda_min": 0.06877800822257996, "lambda_max": 0.647770881652832, "S_mean": 0.24907430013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.446565866470337, "grad_norm": 0.9322757720947266, "tokens_seen": 71606272, "tokens_per_sec": 16126.330396822117, "vram_gb": 6.154519552, "lambda": 0.29597753286361694, "lambda_min": 0.06871919333934784, "lambda_max": 0.6484038829803467, "S_mean": 0.27227783203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.320403814315796, "grad_norm": 1.1327009201049805, "tokens_seen": 71770112, "tokens_per_sec": 16126.904581815348, "vram_gb": 6.154519552, "lambda": 0.29616829566657543, "lambda_min": 0.06867248564958572, "lambda_max": 0.6491577625274658, "S_mean": 0.25421142578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.436129808425903, "grad_norm": 1.0714771747589111, "tokens_seen": 71933952, "tokens_per_sec": 16127.403537499878, "vram_gb": 6.154519552, "lambda": 0.2963369864349564, "lambda_min": 0.06866824626922607, "lambda_max": 0.649832010269165, "S_mean": 0.25848388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.435243606567383, "grad_norm": 0.9862684011459351, "tokens_seen": 72097792, "tokens_per_sec": 16128.138727416766, "vram_gb": 6.154519552, "lambda": 0.29647776670753956, "lambda_min": 0.06869255751371384, "lambda_max": 0.6503829956054688, "S_mean": 0.2804768880208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.44499659538269, "grad_norm": 0.9759925007820129, "tokens_seen": 72261632, "tokens_per_sec": 16128.625874592119, "vram_gb": 6.154519552, "lambda": 0.2966680508106947, "lambda_min": 0.06866399198770523, "lambda_max": 0.6510356664657593, "S_mean": 0.2584940592447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.4266133308410645, "grad_norm": 1.012241244316101, "tokens_seen": 72425472, "tokens_per_sec": 16129.34782703761, "vram_gb": 6.154519552, "lambda": 0.2968801188593109, "lambda_min": 0.06866764277219772, "lambda_max": 0.6517839431762695, "S_mean": 0.2504475911458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.4712042808532715, "grad_norm": 0.9753625988960266, "tokens_seen": 72589312, "tokens_per_sec": 16129.971349412768, "vram_gb": 6.154519552, "lambda": 0.2971161361783743, "lambda_min": 0.06867045909166336, "lambda_max": 0.6526935696601868, "S_mean": 0.2577921549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.282861709594727, "grad_norm": 0.9538633227348328, "tokens_seen": 72753152, "tokens_per_sec": 16130.524559305248, "vram_gb": 6.154519552, "lambda": 0.29728054410467547, "lambda_min": 0.06870035082101822, "lambda_max": 0.6532351970672607, "S_mean": 0.2598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.3924174308776855, "grad_norm": 0.9816431403160095, "tokens_seen": 72916992, "tokens_per_sec": 16131.18124528834, "vram_gb": 6.154519552, "lambda": 0.29742020554840565, "lambda_min": 0.06869396567344666, "lambda_max": 0.6538012027740479, "S_mean": 0.25714111328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.472063064575195, "grad_norm": 1.1260414123535156, "tokens_seen": 73080832, "tokens_per_sec": 16131.9132478577, "vram_gb": 6.154519552, "lambda": 0.29759298140803975, "lambda_min": 0.06870106607675552, "lambda_max": 0.6545262336730957, "S_mean": 0.246307373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.292881488800049, "grad_norm": 1.0250717401504517, "tokens_seen": 73244672, "tokens_per_sec": 16132.4924540999, "vram_gb": 6.154519552, "lambda": 0.29777022016545135, "lambda_min": 0.06873203814029694, "lambda_max": 0.655164897441864, "S_mean": 0.24663289388020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.513611316680908, "grad_norm": 0.9907810091972351, "tokens_seen": 73408512, "tokens_per_sec": 16133.044823915088, "vram_gb": 6.154519552, "lambda": 0.29792562561730546, "lambda_min": 0.06879846006631851, "lambda_max": 0.6557823419570923, "S_mean": 0.2532145182291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.385931491851807, "grad_norm": 1.026242971420288, "tokens_seen": 73572352, "tokens_per_sec": 16133.30916787318, "vram_gb": 6.154519552, "lambda": 0.2980987538273136, "lambda_min": 0.06877561658620834, "lambda_max": 0.6565760374069214, "S_mean": 0.2663675944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.327686071395874, "grad_norm": 0.9465431571006775, "tokens_seen": 73736192, "tokens_per_sec": 16133.8996667478, "vram_gb": 6.154519552, "lambda": 0.29826742441703874, "lambda_min": 0.06873493641614914, "lambda_max": 0.6570757031440735, "S_mean": 0.251068115234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.483867585659027, "val_ppl": 88.5765885986767, "lambda": 0.29827946486572426, "lambda_min": 0.06873293966054916, "lambda_max": 0.6571081876754761, "lambda_per_layer": [0.17198032140731812, 0.13281206786632538, 0.14263278245925903, 0.15544091165065765, 0.14783063530921936, 0.1583063155412674, 0.11283174157142639, 0.06873293966054916, 0.07189115136861801, 0.18315403163433075, 0.25647255778312683, 0.3535928428173065, 0.5152904987335205, 0.4099476933479309, 0.5367560386657715, 0.3781549036502838, 0.6012012362480164, 0.6571081876754761, 0.5903856754302979, 0.42393073439598083, 0.3241010010242462, 0.22737623751163483, 0.31841710209846497, 0.22035954892635345], "S_mean": 0.240203857421875, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.3696465492248535, "grad_norm": 0.9506285786628723, "tokens_seen": 73900032, "tokens_per_sec": 16121.443793812652, "vram_gb": 6.154519552, "lambda": 0.2984508974477649, "lambda_min": 0.06872342526912689, "lambda_max": 0.6576130986213684, "S_mean": 0.2646280924479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.3183228969573975, "grad_norm": 0.93751060962677, "tokens_seen": 74063872, "tokens_per_sec": 16121.854160135981, "vram_gb": 6.154519552, "lambda": 0.2985867454359929, "lambda_min": 0.06874274462461472, "lambda_max": 0.6581296324729919, "S_mean": 0.2524007161458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.338008642196655, "grad_norm": 1.007438063621521, "tokens_seen": 74227712, "tokens_per_sec": 16122.401288295561, "vram_gb": 6.154519552, "lambda": 0.29872787930071354, "lambda_min": 0.06872733682394028, "lambda_max": 0.6585438251495361, "S_mean": 0.2675069173177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.319023847579956, "grad_norm": 0.9756544828414917, "tokens_seen": 74391552, "tokens_per_sec": 16123.190078278863, "vram_gb": 6.154519552, "lambda": 0.29888784047216177, "lambda_min": 0.06870833784341812, "lambda_max": 0.6589941382408142, "S_mean": 0.2578938802083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.275925397872925, "grad_norm": 0.9092803001403809, "tokens_seen": 74555392, "tokens_per_sec": 16123.712574963882, "vram_gb": 6.154519552, "lambda": 0.2990593835711479, "lambda_min": 0.06868456304073334, "lambda_max": 0.6595327854156494, "S_mean": 0.256622314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.360936403274536, "grad_norm": 0.9871377944946289, "tokens_seen": 74719232, "tokens_per_sec": 16124.461266688364, "vram_gb": 6.154519552, "lambda": 0.2992296653489272, "lambda_min": 0.06866437941789627, "lambda_max": 0.6600750684738159, "S_mean": 0.2535502115885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.4854607582092285, "grad_norm": 0.9527536630630493, "tokens_seen": 74883072, "tokens_per_sec": 16125.081755738343, "vram_gb": 6.154519552, "lambda": 0.2993747644747297, "lambda_min": 0.06865204125642776, "lambda_max": 0.660574734210968, "S_mean": 0.24430338541666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.317211627960205, "grad_norm": 0.9997917413711548, "tokens_seen": 75046912, "tokens_per_sec": 16125.835090486407, "vram_gb": 6.154519552, "lambda": 0.2994982100402315, "lambda_min": 0.06863094121217728, "lambda_max": 0.6609048247337341, "S_mean": 0.25262451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.475472688674927, "grad_norm": 1.01362943649292, "tokens_seen": 75210752, "tokens_per_sec": 16126.385273012042, "vram_gb": 6.154519552, "lambda": 0.29968910571187735, "lambda_min": 0.0686049684882164, "lambda_max": 0.6616970896720886, "S_mean": 0.2724202473958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.415571689605713, "grad_norm": 1.0531015396118164, "tokens_seen": 75374592, "tokens_per_sec": 16126.899405046906, "vram_gb": 6.154519552, "lambda": 0.29986171517521143, "lambda_min": 0.06859545409679413, "lambda_max": 0.6621713638305664, "S_mean": 0.2576904296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.473625898361206, "grad_norm": 1.1337445974349976, "tokens_seen": 75538432, "tokens_per_sec": 16127.488226408952, "vram_gb": 6.154519552, "lambda": 0.30000419324884814, "lambda_min": 0.06861747801303864, "lambda_max": 0.6627402901649475, "S_mean": 0.2532552083333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.481706619262695, "grad_norm": 1.0523370504379272, "tokens_seen": 75702272, "tokens_per_sec": 16128.06592774919, "vram_gb": 6.154519552, "lambda": 0.3001790726557374, "lambda_min": 0.0686107724905014, "lambda_max": 0.6633819341659546, "S_mean": 0.24347941080729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.423892498016357, "grad_norm": 0.9509623050689697, "tokens_seen": 75866112, "tokens_per_sec": 16128.525139059662, "vram_gb": 6.154519552, "lambda": 0.30033059728642303, "lambda_min": 0.06860357522964478, "lambda_max": 0.6640595197677612, "S_mean": 0.2562764485677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.352764129638672, "grad_norm": 0.9597234129905701, "tokens_seen": 76029952, "tokens_per_sec": 16129.197394592931, "vram_gb": 6.154519552, "lambda": 0.3004874673982461, "lambda_min": 0.06862760335206985, "lambda_max": 0.6645761728286743, "S_mean": 0.24857584635416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.4547131061553955, "grad_norm": 0.9320910573005676, "tokens_seen": 76193792, "tokens_per_sec": 16129.736937009235, "vram_gb": 6.154519552, "lambda": 0.30062550865113735, "lambda_min": 0.06860224902629852, "lambda_max": 0.6651225090026855, "S_mean": 0.252593994140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.318549156188965, "grad_norm": 0.9458197355270386, "tokens_seen": 76357632, "tokens_per_sec": 16130.305946214357, "vram_gb": 6.154519552, "lambda": 0.3007676613827546, "lambda_min": 0.06856538355350494, "lambda_max": 0.6658376455307007, "S_mean": 0.24498494466145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.481395959854126, "grad_norm": 1.0148330926895142, "tokens_seen": 76521472, "tokens_per_sec": 16130.86079025327, "vram_gb": 6.154519552, "lambda": 0.30092645188172656, "lambda_min": 0.06856847554445267, "lambda_max": 0.6664119362831116, "S_mean": 0.2541605631510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.541472434997559, "grad_norm": 1.1796917915344238, "tokens_seen": 76685312, "tokens_per_sec": 16131.26592703187, "vram_gb": 6.154519552, "lambda": 0.3010989126438896, "lambda_min": 0.0685555711388588, "lambda_max": 0.6669493913650513, "S_mean": 0.2543436686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.437610626220703, "grad_norm": 1.02358078956604, "tokens_seen": 76849152, "tokens_per_sec": 16131.896571915442, "vram_gb": 6.154519552, "lambda": 0.3012465611100197, "lambda_min": 0.06856538355350494, "lambda_max": 0.667624294757843, "S_mean": 0.24692789713541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.304456472396851, "grad_norm": 0.9382671117782593, "tokens_seen": 77012992, "tokens_per_sec": 16132.483552424268, "vram_gb": 6.154519552, "lambda": 0.3014300012340148, "lambda_min": 0.06857533007860184, "lambda_max": 0.6683506965637207, "S_mean": 0.2551778157552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.542890548706055, "grad_norm": 1.1605157852172852, "tokens_seen": 77176832, "tokens_per_sec": 16133.091311606597, "vram_gb": 6.154519552, "lambda": 0.30156367427359027, "lambda_min": 0.06856465339660645, "lambda_max": 0.6686227917671204, "S_mean": 0.2707621256510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.420127868652344, "grad_norm": 1.0364723205566406, "tokens_seen": 77340672, "tokens_per_sec": 16133.496755687827, "vram_gb": 6.154519552, "lambda": 0.3016795888543129, "lambda_min": 0.06856866925954819, "lambda_max": 0.6689150929450989, "S_mean": 0.24904378255208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.353049039840698, "grad_norm": 0.9687014222145081, "tokens_seen": 77504512, "tokens_per_sec": 16134.064482378293, "vram_gb": 6.154519552, "lambda": 0.3018177195141713, "lambda_min": 0.06859081983566284, "lambda_max": 0.6691923141479492, "S_mean": 0.2636311848958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.412281513214111, "grad_norm": 0.9526203274726868, "tokens_seen": 77668352, "tokens_per_sec": 16134.41161741874, "vram_gb": 6.154519552, "lambda": 0.30197125642249983, "lambda_min": 0.06855809688568115, "lambda_max": 0.6698500514030457, "S_mean": 0.2570597330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.352084636688232, "grad_norm": 0.9934906363487244, "tokens_seen": 77832192, "tokens_per_sec": 16134.956301004611, "vram_gb": 6.154519552, "lambda": 0.3021107396731774, "lambda_min": 0.06852178275585175, "lambda_max": 0.6704956293106079, "S_mean": 0.24535115559895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.447093760967254, "val_ppl": 85.37845289746156, "lambda": 0.30211833616097766, "lambda_min": 0.06851831078529358, "lambda_max": 0.6705130934715271, "lambda_per_layer": [0.17306219041347504, 0.1329428106546402, 0.1428917944431305, 0.1562906801700592, 0.14846111834049225, 0.15911002457141876, 0.11310966312885284, 0.06851831078529358, 0.07153254747390747, 0.18367867171764374, 0.25762808322906494, 0.35589948296546936, 0.520653247833252, 0.4142766296863556, 0.5453895330429077, 0.38310688734054565, 0.6116285920143127, 0.6705130934715271, 0.6010115742683411, 0.43156251311302185, 0.32878732681274414, 0.2308405637741089, 0.32549023628234863, 0.22445449233055115], "S_mean": 0.2516682942708333, "G_mean": null, "G_std": null, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.217607021331787, "grad_norm": 0.9529623985290527, "tokens_seen": 77996032, "tokens_per_sec": 16122.973656775157, "vram_gb": 6.154519552, "lambda": 0.30225958799322444, "lambda_min": 0.06850375980138779, "lambda_max": 0.6711177825927734, "S_mean": 0.24428304036458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.522722005844116, "grad_norm": 1.1077326536178589, "tokens_seen": 78159872, "tokens_per_sec": 16123.671139085436, "vram_gb": 6.154519552, "lambda": 0.3023770597452919, "lambda_min": 0.06851250678300858, "lambda_max": 0.6714114546775818, "S_mean": 0.2597147623697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.492985725402832, "grad_norm": 0.9993686079978943, "tokens_seen": 78323712, "tokens_per_sec": 16124.07903043935, "vram_gb": 6.154519552, "lambda": 0.3025206609939535, "lambda_min": 0.06847354769706726, "lambda_max": 0.6718335747718811, "S_mean": 0.2601826985677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.277391195297241, "grad_norm": 1.0115208625793457, "tokens_seen": 78487552, "tokens_per_sec": 16124.738518398131, "vram_gb": 6.154519552, "lambda": 0.30265992507338524, "lambda_min": 0.06845119595527649, "lambda_max": 0.6722504496574402, "S_mean": 0.2529500325520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.350672245025635, "grad_norm": 0.9921881556510925, "tokens_seen": 78651392, "tokens_per_sec": 16125.423949720736, "vram_gb": 6.154519552, "lambda": 0.30279487123092014, "lambda_min": 0.06850025057792664, "lambda_max": 0.6726838946342468, "S_mean": 0.258270263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.466640949249268, "grad_norm": 1.0016051530838013, "tokens_seen": 78815232, "tokens_per_sec": 16126.035697784318, "vram_gb": 6.154519552, "lambda": 0.302939726660649, "lambda_min": 0.06849680840969086, "lambda_max": 0.6728615164756775, "S_mean": 0.24665323893229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.318801403045654, "grad_norm": 1.1034221649169922, "tokens_seen": 78979072, "tokens_per_sec": 16126.75912629823, "vram_gb": 6.154519552, "lambda": 0.30306793935596943, "lambda_min": 0.06850001215934753, "lambda_max": 0.6732168793678284, "S_mean": 0.2573140462239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.43383002281189, "grad_norm": 0.9966452717781067, "tokens_seen": 79142912, "tokens_per_sec": 16127.333608825198, "vram_gb": 6.154519552, "lambda": 0.30318890667210024, "lambda_min": 0.06854403018951416, "lambda_max": 0.673660933971405, "S_mean": 0.240875244140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.615313291549683, "grad_norm": 1.0527979135513306, "tokens_seen": 79306752, "tokens_per_sec": 16127.83462200816, "vram_gb": 6.154519552, "lambda": 0.303310786994795, "lambda_min": 0.06853467226028442, "lambda_max": 0.6739387512207031, "S_mean": 0.262664794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.275497913360596, "grad_norm": 0.9405262470245361, "tokens_seen": 79470592, "tokens_per_sec": 16128.43674924099, "vram_gb": 6.154519552, "lambda": 0.30346473089108866, "lambda_min": 0.06852114945650101, "lambda_max": 0.6743987202644348, "S_mean": 0.24443562825520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.270977258682251, "grad_norm": 1.0116509199142456, "tokens_seen": 79634432, "tokens_per_sec": 16128.960981491633, "vram_gb": 6.154519552, "lambda": 0.30359996265421313, "lambda_min": 0.06854258477687836, "lambda_max": 0.6748754978179932, "S_mean": 0.24785359700520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.441773891448975, "grad_norm": 1.0372436046600342, "tokens_seen": 79798272, "tokens_per_sec": 16129.493986434982, "vram_gb": 6.154519552, "lambda": 0.30373777480175096, "lambda_min": 0.06855165213346481, "lambda_max": 0.6754196286201477, "S_mean": 0.255340576171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.2935099601745605, "grad_norm": 0.9970953464508057, "tokens_seen": 79962112, "tokens_per_sec": 16130.024811913849, "vram_gb": 6.154519552, "lambda": 0.3038620452086131, "lambda_min": 0.06858455389738083, "lambda_max": 0.6759086847305298, "S_mean": 0.24946085611979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.369824409484863, "grad_norm": 1.0671602487564087, "tokens_seen": 80125952, "tokens_per_sec": 16130.572492186127, "vram_gb": 6.154519552, "lambda": 0.30398966496189433, "lambda_min": 0.06860566139221191, "lambda_max": 0.6764700412750244, "S_mean": 0.2585652669270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.31647515296936, "grad_norm": 0.9950375556945801, "tokens_seen": 80289792, "tokens_per_sec": 16131.089758502701, "vram_gb": 6.154519552, "lambda": 0.30412731133401394, "lambda_min": 0.06858745217323303, "lambda_max": 0.6768428087234497, "S_mean": 0.24867757161458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.481927394866943, "grad_norm": 1.0875728130340576, "tokens_seen": 80453632, "tokens_per_sec": 16131.590784653003, "vram_gb": 6.154519552, "lambda": 0.30422808043658733, "lambda_min": 0.06857942789793015, "lambda_max": 0.6771532893180847, "S_mean": 0.24774169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.153930187225342, "grad_norm": 1.0297133922576904, "tokens_seen": 80617472, "tokens_per_sec": 16132.098197500321, "vram_gb": 6.154519552, "lambda": 0.30434557888656855, "lambda_min": 0.0685771033167839, "lambda_max": 0.6774318218231201, "S_mean": 0.2679341634114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.445535182952881, "grad_norm": 0.9827134609222412, "tokens_seen": 80781312, "tokens_per_sec": 16132.583012390507, "vram_gb": 6.154519552, "lambda": 0.30449891556054354, "lambda_min": 0.06856860220432281, "lambda_max": 0.6779524087905884, "S_mean": 0.24501546223958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.514377117156982, "grad_norm": 1.0552725791931152, "tokens_seen": 80945152, "tokens_per_sec": 16133.119680997377, "vram_gb": 6.154519552, "lambda": 0.3046369568134348, "lambda_min": 0.06858167052268982, "lambda_max": 0.6783426403999329, "S_mean": 0.2537638346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.319247484207153, "grad_norm": 1.0281541347503662, "tokens_seen": 81108992, "tokens_per_sec": 16133.49154762689, "vram_gb": 6.154519552, "lambda": 0.3047385923564434, "lambda_min": 0.06854739785194397, "lambda_max": 0.678853452205658, "S_mean": 0.2550557454427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.425465822219849, "grad_norm": 0.94708651304245, "tokens_seen": 81272832, "tokens_per_sec": 16133.97930956009, "vram_gb": 6.154519552, "lambda": 0.30487463250756264, "lambda_min": 0.06851345300674438, "lambda_max": 0.6792905330657959, "S_mean": 0.253326416015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.569298505783081, "grad_norm": 1.0093783140182495, "tokens_seen": 81436672, "tokens_per_sec": 16134.565723240881, "vram_gb": 6.154519552, "lambda": 0.3050095026070873, "lambda_min": 0.06850781291723251, "lambda_max": 0.6797189116477966, "S_mean": 0.2571512858072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.481262683868408, "grad_norm": 1.0042401552200317, "tokens_seen": 81600512, "tokens_per_sec": 16135.065675611018, "vram_gb": 6.154519552, "lambda": 0.3051137924194336, "lambda_min": 0.06851810961961746, "lambda_max": 0.6801217794418335, "S_mean": 0.245880126953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.3532798290252686, "grad_norm": 0.95843505859375, "tokens_seen": 81764352, "tokens_per_sec": 16135.516926292099, "vram_gb": 6.154519552, "lambda": 0.30521447646121186, "lambda_min": 0.06849920749664307, "lambda_max": 0.6804251670837402, "S_mean": 0.24033610026041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.477292776107788, "grad_norm": 1.0388566255569458, "tokens_seen": 81928192, "tokens_per_sec": 16136.069194154625, "vram_gb": 6.154519552, "lambda": 0.30531759777416784, "lambda_min": 0.06847500056028366, "lambda_max": 0.6804797649383545, "S_mean": 0.24782307942708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.441749477386475, "val_ppl": 84.92338332702292, "lambda": 0.30532630657156307, "lambda_min": 0.0684758797287941, "lambda_max": 0.6805042624473572, "lambda_per_layer": [0.1740533411502838, 0.13289086520671844, 0.14317382872104645, 0.15707170963287354, 0.1490345448255539, 0.1598576158285141, 0.1131870225071907, 0.0684758797287941, 0.07153460383415222, 0.1842128336429596, 0.2586840093135834, 0.3584461808204651, 0.5260736346244812, 0.41854003071784973, 0.5517153739929199, 0.3871921896934509, 0.6214432120323181, 0.6805042624473572, 0.6098172068595886, 0.43686914443969727, 0.33391687273979187, 0.23381076753139496, 0.33016660809516907, 0.22715961933135986], "S_mean": 0.24867757161458334, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.256266355514526, "grad_norm": 1.0218205451965332, "tokens_seen": 82092032, "tokens_per_sec": 16124.885594661158, "vram_gb": 6.154519552, "lambda": 0.3054550839588046, "lambda_min": 0.06851453334093094, "lambda_max": 0.6808189153671265, "S_mean": 0.24930826822916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.2388811111450195, "grad_norm": 0.9523033499717712, "tokens_seen": 82255872, "tokens_per_sec": 16125.300578612543, "vram_gb": 6.154519552, "lambda": 0.30555308672289055, "lambda_min": 0.0685216560959816, "lambda_max": 0.6811216473579407, "S_mean": 0.24676513671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.258218050003052, "grad_norm": 0.9501421451568604, "tokens_seen": 82419712, "tokens_per_sec": 16125.631620783068, "vram_gb": 6.154519552, "lambda": 0.3056671616310875, "lambda_min": 0.06853742897510529, "lambda_max": 0.6816059350967407, "S_mean": 0.24165852864583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.6049721240997314, "grad_norm": 1.1243674755096436, "tokens_seen": 82583552, "tokens_per_sec": 16126.237632449578, "vram_gb": 6.154519552, "lambda": 0.30576302856206894, "lambda_min": 0.06852497905492783, "lambda_max": 0.6821542382240295, "S_mean": 0.2537740071614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.5861639976501465, "grad_norm": 1.0808416604995728, "tokens_seen": 82747392, "tokens_per_sec": 16126.844762823144, "vram_gb": 6.154519552, "lambda": 0.30588390678167343, "lambda_min": 0.06853393465280533, "lambda_max": 0.682632565498352, "S_mean": 0.245819091796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.331860780715942, "grad_norm": 1.0315475463867188, "tokens_seen": 82911232, "tokens_per_sec": 16127.439750317477, "vram_gb": 6.154519552, "lambda": 0.30598066623012227, "lambda_min": 0.06852193176746368, "lambda_max": 0.6829540133476257, "S_mean": 0.2529296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.355335712432861, "grad_norm": 1.132855772972107, "tokens_seen": 83075072, "tokens_per_sec": 16127.969991272223, "vram_gb": 6.154519552, "lambda": 0.3060676859070857, "lambda_min": 0.06853527575731277, "lambda_max": 0.6833581328392029, "S_mean": 0.24982706705729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.313282251358032, "grad_norm": 1.0450330972671509, "tokens_seen": 83238912, "tokens_per_sec": 16128.524942766879, "vram_gb": 6.154519552, "lambda": 0.3061769533281525, "lambda_min": 0.06855503469705582, "lambda_max": 0.6837647557258606, "S_mean": 0.2562459309895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.30222749710083, "grad_norm": 1.0151721239089966, "tokens_seen": 83402752, "tokens_per_sec": 16129.111382831372, "vram_gb": 6.154519552, "lambda": 0.30623981977502507, "lambda_min": 0.06857237964868546, "lambda_max": 0.6839046478271484, "S_mean": 0.2524922688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.31463360786438, "grad_norm": 1.0632600784301758, "tokens_seen": 83566592, "tokens_per_sec": 16129.68292274177, "vram_gb": 6.154519552, "lambda": 0.3063527972747882, "lambda_min": 0.06857259571552277, "lambda_max": 0.6842268705368042, "S_mean": 0.2557881673177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.370006322860718, "grad_norm": 1.1238207817077637, "tokens_seen": 83730432, "tokens_per_sec": 16130.066185276371, "vram_gb": 6.154519552, "lambda": 0.3064659346515934, "lambda_min": 0.06854918599128723, "lambda_max": 0.6846197843551636, "S_mean": 0.24883015950520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.3410584926605225, "grad_norm": 1.0469144582748413, "tokens_seen": 83894272, "tokens_per_sec": 16130.676211694165, "vram_gb": 6.154519552, "lambda": 0.3065769240881006, "lambda_min": 0.06854413449764252, "lambda_max": 0.6849895119667053, "S_mean": 0.2457275390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.41198992729187, "grad_norm": 1.0641844272613525, "tokens_seen": 84058112, "tokens_per_sec": 16130.63829725108, "vram_gb": 6.154519552, "lambda": 0.3066984995578726, "lambda_min": 0.06856517493724823, "lambda_max": 0.6854362487792969, "S_mean": 0.25836181640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.39678955078125, "grad_norm": 1.0388894081115723, "tokens_seen": 84221952, "tokens_per_sec": 16130.05458477635, "vram_gb": 6.154519552, "lambda": 0.3068163835753997, "lambda_min": 0.06857980787754059, "lambda_max": 0.6858190298080444, "S_mean": 0.2540079752604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.293914318084717, "grad_norm": 1.0101370811462402, "tokens_seen": 84385792, "tokens_per_sec": 16130.093694914727, "vram_gb": 6.154519552, "lambda": 0.3069254057481885, "lambda_min": 0.06859337538480759, "lambda_max": 0.6863195896148682, "S_mean": 0.2559000651041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.241091728210449, "grad_norm": 0.951271116733551, "tokens_seen": 84549632, "tokens_per_sec": 16130.1856098331, "vram_gb": 6.154519552, "lambda": 0.3070480947693189, "lambda_min": 0.06860220432281494, "lambda_max": 0.6865863800048828, "S_mean": 0.24302164713541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.317607402801514, "grad_norm": 1.176809549331665, "tokens_seen": 84713472, "tokens_per_sec": 16130.065757144092, "vram_gb": 6.154519552, "lambda": 0.3071647270893057, "lambda_min": 0.06861591339111328, "lambda_max": 0.687021017074585, "S_mean": 0.2511800130208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.412954330444336, "grad_norm": 1.0272140502929688, "tokens_seen": 84877312, "tokens_per_sec": 16129.973255356184, "vram_gb": 6.154519552, "lambda": 0.3072299202904105, "lambda_min": 0.06860066950321198, "lambda_max": 0.6873031854629517, "S_mean": 0.25775146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.2657575607299805, "grad_norm": 1.1212937831878662, "tokens_seen": 85041152, "tokens_per_sec": 16129.934190583883, "vram_gb": 6.154519552, "lambda": 0.30731492551664513, "lambda_min": 0.06858716160058975, "lambda_max": 0.6874210238456726, "S_mean": 0.248992919921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.3634350299835205, "grad_norm": 1.0967015027999878, "tokens_seen": 85204992, "tokens_per_sec": 16130.305403120714, "vram_gb": 6.154519552, "lambda": 0.3074271818622947, "lambda_min": 0.06857828795909882, "lambda_max": 0.6876316070556641, "S_mean": 0.2647501627604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.41165828704834, "grad_norm": 1.1541751623153687, "tokens_seen": 85368832, "tokens_per_sec": 16130.83846780999, "vram_gb": 6.154519552, "lambda": 0.3075336180627346, "lambda_min": 0.06855585426092148, "lambda_max": 0.6879763007164001, "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.443082332611084, "grad_norm": 1.1181659698486328, "tokens_seen": 85532672, "tokens_per_sec": 16131.112746540532, "vram_gb": 6.154519552, "lambda": 0.30764831354220706, "lambda_min": 0.06854479759931564, "lambda_max": 0.688383162021637, "S_mean": 0.24918619791666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.461336374282837, "grad_norm": 1.029821515083313, "tokens_seen": 85696512, "tokens_per_sec": 16131.598185500228, "vram_gb": 6.154519552, "lambda": 0.3077319609001279, "lambda_min": 0.06856544315814972, "lambda_max": 0.6885712742805481, "S_mean": 0.24018351236979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.343791961669922, "grad_norm": 1.023840308189392, "tokens_seen": 85860352, "tokens_per_sec": 16131.95556927872, "vram_gb": 6.154519552, "lambda": 0.30780448733518523, "lambda_min": 0.06855176389217377, "lambda_max": 0.6887006759643555, "S_mean": 0.2616984049479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.451322317123413, "grad_norm": 1.0978842973709106, "tokens_seen": 86024192, "tokens_per_sec": 16132.525448916338, "vram_gb": 6.154519552, "lambda": 0.30791467397163313, "lambda_min": 0.06854825466871262, "lambda_max": 0.6890650391578674, "S_mean": 0.24757893880208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.446343183517456, "val_ppl": 85.31439379968258, "lambda": 0.307919935323298, "lambda_min": 0.0685490146279335, "lambda_max": 0.6890933513641357, "lambda_per_layer": [0.17517627775669098, 0.13288114964962006, 0.1431557536125183, 0.15761208534240723, 0.14936833083629608, 0.16043971478939056, 0.11336648464202881, 0.0685490146279335, 0.07156150043010712, 0.1848606914281845, 0.25952351093292236, 0.36008256673812866, 0.5297110080718994, 0.42176368832588196, 0.5563184022903442, 0.39045655727386475, 0.6289762854576111, 0.6890933513641357, 0.6180858612060547, 0.4413456618785858, 0.3376448154449463, 0.23625494539737701, 0.33427998423576355, 0.22957080602645874], "S_mean": 0.23673502604166666, "G_mean": null, "G_std": null, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.238342046737671, "grad_norm": 1.044836401939392, "tokens_seen": 86188032, "tokens_per_sec": 16121.852167167293, "vram_gb": 6.154519552, "lambda": 0.3080063049371044, "lambda_min": 0.06855760514736176, "lambda_max": 0.6895177960395813, "S_mean": 0.241546630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.4220359325408936, "grad_norm": 1.0584155321121216, "tokens_seen": 86351872, "tokens_per_sec": 16122.462980359967, "vram_gb": 6.154519552, "lambda": 0.30813345530380803, "lambda_min": 0.06858357042074203, "lambda_max": 0.6898829936981201, "S_mean": 0.2537027994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.3597166538238525, "grad_norm": 0.9944546222686768, "tokens_seen": 86515712, "tokens_per_sec": 16123.115596211577, "vram_gb": 6.154519552, "lambda": 0.3082221153502663, "lambda_min": 0.06861420720815659, "lambda_max": 0.6901527047157288, "S_mean": 0.24650065104166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.36875057220459, "grad_norm": 1.1516271829605103, "tokens_seen": 86679552, "tokens_per_sec": 16123.622404605703, "vram_gb": 6.154519552, "lambda": 0.3083371141304572, "lambda_min": 0.06861792504787445, "lambda_max": 0.6905871033668518, "S_mean": 0.2537740071614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.391287803649902, "grad_norm": 1.0019983053207397, "tokens_seen": 86843392, "tokens_per_sec": 16124.181312527715, "vram_gb": 6.154519552, "lambda": 0.3084687264636159, "lambda_min": 0.068621926009655, "lambda_max": 0.6910207867622375, "S_mean": 0.24974568684895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.204275369644165, "grad_norm": 1.1296100616455078, "tokens_seen": 87007232, "tokens_per_sec": 16124.649340888973, "vram_gb": 6.154519552, "lambda": 0.30855484399944544, "lambda_min": 0.06865116208791733, "lambda_max": 0.6914130449295044, "S_mean": 0.2508544921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.226434946060181, "grad_norm": 1.02627432346344, "tokens_seen": 87171072, "tokens_per_sec": 16125.107619202896, "vram_gb": 6.154519552, "lambda": 0.3086452490339677, "lambda_min": 0.0686245858669281, "lambda_max": 0.6917305588722229, "S_mean": 0.24979654947916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.362237453460693, "grad_norm": 1.0636866092681885, "tokens_seen": 87334912, "tokens_per_sec": 16125.54092221786, "vram_gb": 6.154519552, "lambda": 0.3087216891969244, "lambda_min": 0.06862127780914307, "lambda_max": 0.6919997334480286, "S_mean": 0.24027506510416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.432812452316284, "grad_norm": 1.0256123542785645, "tokens_seen": 87498752, "tokens_per_sec": 16125.942146089728, "vram_gb": 6.154519552, "lambda": 0.30878500764568645, "lambda_min": 0.06860595941543579, "lambda_max": 0.6922658085823059, "S_mean": 0.24171956380208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.257791757583618, "grad_norm": 1.0512882471084595, "tokens_seen": 87662592, "tokens_per_sec": 16126.48921353983, "vram_gb": 6.154519552, "lambda": 0.3088736794888973, "lambda_min": 0.06860291212797165, "lambda_max": 0.6925015449523926, "S_mean": 0.2602284749348958, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.478522777557373, "grad_norm": 1.1343026161193848, "tokens_seen": 87826432, "tokens_per_sec": 16126.821116095056, "vram_gb": 6.154519552, "lambda": 0.30894628446549177, "lambda_min": 0.06858228892087936, "lambda_max": 0.69267338514328, "S_mean": 0.2546793619791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.40561842918396, "grad_norm": 0.9581692814826965, "tokens_seen": 87990272, "tokens_per_sec": 16127.16520858665, "vram_gb": 6.154519552, "lambda": 0.3090308668712775, "lambda_min": 0.06859523057937622, "lambda_max": 0.6930301785469055, "S_mean": 0.24696858723958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.3063905239105225, "grad_norm": 1.0201587677001953, "tokens_seen": 88154112, "tokens_per_sec": 16127.390945746849, "vram_gb": 6.154519552, "lambda": 0.3091060410564144, "lambda_min": 0.0686100497841835, "lambda_max": 0.6933330297470093, "S_mean": 0.23981730143229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.39158296585083, "grad_norm": 1.127642273902893, "tokens_seen": 88317952, "tokens_per_sec": 16127.88063911368, "vram_gb": 6.154519552, "lambda": 0.309191457927227, "lambda_min": 0.06862474232912064, "lambda_max": 0.6936087012290955, "S_mean": 0.2536722819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.480140924453735, "grad_norm": 1.1945635080337524, "tokens_seen": 88481792, "tokens_per_sec": 16128.352915989752, "vram_gb": 6.154519552, "lambda": 0.30928928777575493, "lambda_min": 0.06859546899795532, "lambda_max": 0.6939063668251038, "S_mean": 0.24754842122395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.265834808349609, "grad_norm": 1.2063413858413696, "tokens_seen": 88645632, "tokens_per_sec": 16128.73377705715, "vram_gb": 6.154519552, "lambda": 0.3093623553092281, "lambda_min": 0.06860969215631485, "lambda_max": 0.6940971612930298, "S_mean": 0.2533060709635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.372408390045166, "grad_norm": 1.0415173768997192, "tokens_seen": 88809472, "tokens_per_sec": 16129.153822776623, "vram_gb": 6.154519552, "lambda": 0.3094145366922021, "lambda_min": 0.0686212033033371, "lambda_max": 0.6943425536155701, "S_mean": 0.263031005859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.469550371170044, "grad_norm": 1.1485567092895508, "tokens_seen": 88973312, "tokens_per_sec": 16129.378059746929, "vram_gb": 6.154519552, "lambda": 0.3094859082872669, "lambda_min": 0.0686437264084816, "lambda_max": 0.6945673227310181, "S_mean": 0.24455769856770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.293941497802734, "grad_norm": 1.0244578123092651, "tokens_seen": 89137152, "tokens_per_sec": 16129.809893200205, "vram_gb": 6.154519552, "lambda": 0.3095793429141243, "lambda_min": 0.06863920390605927, "lambda_max": 0.6947600841522217, "S_mean": 0.23945109049479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.398649454116821, "grad_norm": 1.109264850616455, "tokens_seen": 89300992, "tokens_per_sec": 16130.350590736156, "vram_gb": 6.154519552, "lambda": 0.3096870860705773, "lambda_min": 0.06863069534301758, "lambda_max": 0.6950482130050659, "S_mean": 0.2555643717447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.31108546257019, "grad_norm": 1.0934308767318726, "tokens_seen": 89464832, "tokens_per_sec": 16130.750027551954, "vram_gb": 6.154519552, "lambda": 0.30977474711835384, "lambda_min": 0.06863526254892349, "lambda_max": 0.6953715085983276, "S_mean": 0.24735514322916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.413877010345459, "grad_norm": 1.13329017162323, "tokens_seen": 89628672, "tokens_per_sec": 16131.172156861889, "vram_gb": 6.154519552, "lambda": 0.309862667384247, "lambda_min": 0.06864753365516663, "lambda_max": 0.6956354379653931, "S_mean": 0.2521158854166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.441938400268555, "grad_norm": 1.1412198543548584, "tokens_seen": 89792512, "tokens_per_sec": 16131.754196086777, "vram_gb": 6.154519552, "lambda": 0.3099531127760808, "lambda_min": 0.06866022199392319, "lambda_max": 0.6959524750709534, "S_mean": 0.24611409505208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.3519792556762695, "grad_norm": 1.1248831748962402, "tokens_seen": 89956352, "tokens_per_sec": 16132.341845758549, "vram_gb": 6.154519552, "lambda": 0.3100305811191599, "lambda_min": 0.06865495443344116, "lambda_max": 0.6961624026298523, "S_mean": 0.2525838216145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.37726902961731, "grad_norm": 1.0865681171417236, "tokens_seen": 90120192, "tokens_per_sec": 16132.952894734717, "vram_gb": 6.154519552, "lambda": 0.3100839105124275, "lambda_min": 0.06865944713354111, "lambda_max": 0.6963533759117126, "S_mean": 0.24683634440104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.402186238765717, "val_ppl": 81.62913450638747, "lambda": 0.310087944380939, "lambda_min": 0.06865853071212769, "lambda_max": 0.6963711977005005, "lambda_per_layer": [0.17628604173660278, 0.1330188363790512, 0.14338849484920502, 0.15804944932460785, 0.14970891177654266, 0.16100265085697174, 0.11331693828105927, 0.06865853071212769, 0.07157092541456223, 0.18504203855991364, 0.25972315669059753, 0.36099040508270264, 0.5329251289367676, 0.42440199851989746, 0.5600593090057373, 0.39306554198265076, 0.6351823210716248, 0.6963711977005005, 0.6242024302482605, 0.44552600383758545, 0.3410833775997162, 0.23836609721183777, 0.3380483090877533, 0.23212257027626038], "S_mean": 0.2574361165364583, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.2313220500946045, "grad_norm": 1.062788724899292, "tokens_seen": 90284032, "tokens_per_sec": 16122.725835669598, "vram_gb": 6.154519552, "lambda": 0.31015896735092, "lambda_min": 0.06866027414798737, "lambda_max": 0.696636438369751, "S_mean": 0.249755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.2449119091033936, "grad_norm": 1.0259912014007568, "tokens_seen": 90447872, "tokens_per_sec": 16123.256535984272, "vram_gb": 6.154519552, "lambda": 0.3102403013035655, "lambda_min": 0.06866924464702606, "lambda_max": 0.6968304514884949, "S_mean": 0.23798624674479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.438594341278076, "grad_norm": 1.0897722244262695, "tokens_seen": 90611712, "tokens_per_sec": 16123.812388429116, "vram_gb": 6.154519552, "lambda": 0.31032549031078815, "lambda_min": 0.06864646077156067, "lambda_max": 0.6969185471534729, "S_mean": 0.251220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.373380184173584, "grad_norm": 1.0012178421020508, "tokens_seen": 90775552, "tokens_per_sec": 16124.333378822677, "vram_gb": 6.154519552, "lambda": 0.3104364260410269, "lambda_min": 0.06866797804832458, "lambda_max": 0.6973260641098022, "S_mean": 0.24212646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.441701412200928, "grad_norm": 1.1102542877197266, "tokens_seen": 90939392, "tokens_per_sec": 16124.854211902775, "vram_gb": 6.154519552, "lambda": 0.3104982062553366, "lambda_min": 0.0686652660369873, "lambda_max": 0.6975042819976807, "S_mean": 0.256805419921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.325453042984009, "grad_norm": 1.108743667602539, "tokens_seen": 91103232, "tokens_per_sec": 16125.466752826165, "vram_gb": 6.154519552, "lambda": 0.3105800474683444, "lambda_min": 0.06866256892681122, "lambda_max": 0.6977245211601257, "S_mean": 0.2576904296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.203110218048096, "grad_norm": 1.075944423675537, "tokens_seen": 91267072, "tokens_per_sec": 16126.097842862524, "vram_gb": 6.154519552, "lambda": 0.3106411180148522, "lambda_min": 0.06868166476488113, "lambda_max": 0.6979817748069763, "S_mean": 0.2666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.290026903152466, "grad_norm": 1.260605812072754, "tokens_seen": 91430912, "tokens_per_sec": 16126.625574340438, "vram_gb": 6.154519552, "lambda": 0.3107173986112078, "lambda_min": 0.06870027631521225, "lambda_max": 0.6982738375663757, "S_mean": 0.2532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.369639873504639, "grad_norm": 1.0796693563461304, "tokens_seen": 91594752, "tokens_per_sec": 16127.210395593504, "vram_gb": 6.154519552, "lambda": 0.31083083990961313, "lambda_min": 0.0686928778886795, "lambda_max": 0.6986170411109924, "S_mean": 0.24677530924479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.376075983047485, "grad_norm": 1.0423250198364258, "tokens_seen": 91758592, "tokens_per_sec": 16127.771678249423, "vram_gb": 6.154519552, "lambda": 0.310928404952089, "lambda_min": 0.06867412477731705, "lambda_max": 0.6989372968673706, "S_mean": 0.2653299967447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.353738307952881, "grad_norm": 1.0464439392089844, "tokens_seen": 91922432, "tokens_per_sec": 16128.254938640563, "vram_gb": 6.154519552, "lambda": 0.31100600709517795, "lambda_min": 0.06865248829126358, "lambda_max": 0.6992506980895996, "S_mean": 0.24733479817708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.210780143737793, "grad_norm": 1.005739688873291, "tokens_seen": 92086272, "tokens_per_sec": 16128.680655996359, "vram_gb": 6.154519552, "lambda": 0.3110955087468028, "lambda_min": 0.06863734871149063, "lambda_max": 0.6993868350982666, "S_mean": 0.2558186848958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.385981559753418, "grad_norm": 1.1139323711395264, "tokens_seen": 92250112, "tokens_per_sec": 16129.198250930187, "vram_gb": 6.154519552, "lambda": 0.3111806216960152, "lambda_min": 0.06862469017505646, "lambda_max": 0.6996451616287231, "S_mean": 0.259918212890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.398066520690918, "grad_norm": 1.162177562713623, "tokens_seen": 92413952, "tokens_per_sec": 16129.68160564644, "vram_gb": 6.154519552, "lambda": 0.31125370413064957, "lambda_min": 0.06862558424472809, "lambda_max": 0.7000150680541992, "S_mean": 0.251556396484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.376701831817627, "grad_norm": 1.0568054914474487, "tokens_seen": 92577792, "tokens_per_sec": 16130.22282505701, "vram_gb": 6.154519552, "lambda": 0.31134983617812395, "lambda_min": 0.06861209869384766, "lambda_max": 0.7002962231636047, "S_mean": 0.24969482421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.2400665283203125, "grad_norm": 1.1323697566986084, "tokens_seen": 92741632, "tokens_per_sec": 16130.723420477632, "vram_gb": 6.154519552, "lambda": 0.31142853666096926, "lambda_min": 0.06861095875501633, "lambda_max": 0.7006446123123169, "S_mean": 0.24912516276041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.306265354156494, "grad_norm": 1.0707714557647705, "tokens_seen": 92905472, "tokens_per_sec": 16131.208293887645, "vram_gb": 6.154519552, "lambda": 0.31151006898532313, "lambda_min": 0.06860770285129547, "lambda_max": 0.7009550333023071, "S_mean": 0.2553812662760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.18933629989624, "grad_norm": 1.1272469758987427, "tokens_seen": 93069312, "tokens_per_sec": 16131.637098581035, "vram_gb": 6.154519552, "lambda": 0.3116032707815369, "lambda_min": 0.06860988587141037, "lambda_max": 0.7012550830841064, "S_mean": 0.2594401041666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.393616676330566, "grad_norm": 1.0593113899230957, "tokens_seen": 93233152, "tokens_per_sec": 16132.134961759499, "vram_gb": 6.154519552, "lambda": 0.3116645651559035, "lambda_min": 0.06860251724720001, "lambda_max": 0.7013833522796631, "S_mean": 0.2607014973958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.309041738510132, "grad_norm": 1.0777751207351685, "tokens_seen": 93396992, "tokens_per_sec": 16132.443128168743, "vram_gb": 6.154519552, "lambda": 0.31172104304035503, "lambda_min": 0.0686095654964447, "lambda_max": 0.7016515731811523, "S_mean": 0.2526652018229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.327779531478882, "grad_norm": 1.0931984186172485, "tokens_seen": 93560832, "tokens_per_sec": 16132.886648835929, "vram_gb": 6.154519552, "lambda": 0.3118023608500759, "lambda_min": 0.06858475506305695, "lambda_max": 0.7019470930099487, "S_mean": 0.24586995442708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.183219909667969, "grad_norm": 1.1414626836776733, "tokens_seen": 93724672, "tokens_per_sec": 16133.458236173283, "vram_gb": 6.154519552, "lambda": 0.31188417381296557, "lambda_min": 0.06855405122041702, "lambda_max": 0.7022721767425537, "S_mean": 0.2545979817708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.407305717468262, "grad_norm": 1.1508389711380005, "tokens_seen": 93888512, "tokens_per_sec": 16133.891269385422, "vram_gb": 6.154519552, "lambda": 0.31196018649886054, "lambda_min": 0.06855340301990509, "lambda_max": 0.7026386260986328, "S_mean": 0.24940999348958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.271397113800049, "grad_norm": 1.1229575872421265, "tokens_seen": 94052352, "tokens_per_sec": 16134.444633745605, "vram_gb": 6.154519552, "lambda": 0.3120608953759074, "lambda_min": 0.06854436546564102, "lambda_max": 0.702921450138092, "S_mean": 0.24702962239583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.307477235794067, "grad_norm": 1.0661571025848389, "tokens_seen": 94216192, "tokens_per_sec": 16135.030057905165, "vram_gb": 6.154519552, "lambda": 0.3121458950142066, "lambda_min": 0.06854936480522156, "lambda_max": 0.70305335521698, "S_mean": 0.24127197265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.409333944320679, "val_ppl": 82.21468570535708, "lambda": 0.3121494638423125, "lambda_min": 0.06855019927024841, "lambda_max": 0.7030569314956665, "lambda_per_layer": [0.17729803919792175, 0.13297151029109955, 0.14354108273983002, 0.15847456455230713, 0.15003058314323425, 0.161348357796669, 0.11325041204690933, 0.06855019927024841, 0.07155779749155045, 0.1852813959121704, 0.26009804010391235, 0.36275798082351685, 0.5357977747917175, 0.4269813299179077, 0.5648276805877686, 0.39578261971473694, 0.6401740312576294, 0.7030569314956665, 0.6295872330665588, 0.44938522577285767, 0.3445557951927185, 0.2404581606388092, 0.34162694215774536, 0.23419344425201416], "S_mean": 0.2587992350260417, "G_mean": null, "G_std": null, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.312360763549805, "grad_norm": 1.1332978010177612, "tokens_seen": 94380032, "tokens_per_sec": 16125.176351926675, "vram_gb": 6.154519552, "lambda": 0.3122201127310594, "lambda_min": 0.0685541033744812, "lambda_max": 0.7032403349876404, "S_mean": 0.2618204752604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.438021659851074, "grad_norm": 1.1458826065063477, "tokens_seen": 94543872, "tokens_per_sec": 16125.7851570012, "vram_gb": 6.154519552, "lambda": 0.31230275239795446, "lambda_min": 0.06856456398963928, "lambda_max": 0.7035407423973083, "S_mean": 0.245025634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.384324312210083, "grad_norm": 1.1188448667526245, "tokens_seen": 94707712, "tokens_per_sec": 16126.272994031548, "vram_gb": 6.154519552, "lambda": 0.3123857543493311, "lambda_min": 0.06857568025588989, "lambda_max": 0.7037842869758606, "S_mean": 0.24144490559895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.1990296840667725, "grad_norm": 1.103992223739624, "tokens_seen": 94871552, "tokens_per_sec": 16126.811480283772, "vram_gb": 6.154519552, "lambda": 0.31247828745593625, "lambda_min": 0.06858614087104797, "lambda_max": 0.7040989398956299, "S_mean": 0.23994954427083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.317140579223633, "grad_norm": 1.0976665019989014, "tokens_seen": 95035392, "tokens_per_sec": 16127.128134041768, "vram_gb": 6.154519552, "lambda": 0.31256062258034945, "lambda_min": 0.06860119104385376, "lambda_max": 0.7044049501419067, "S_mean": 0.268157958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.374170780181885, "grad_norm": 1.1262943744659424, "tokens_seen": 95199232, "tokens_per_sec": 16127.732077645598, "vram_gb": 6.154519552, "lambda": 0.3126506805419922, "lambda_min": 0.06858860701322556, "lambda_max": 0.7047432065010071, "S_mean": 0.24713134765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.1331787109375, "grad_norm": 1.075629711151123, "tokens_seen": 95363072, "tokens_per_sec": 16128.251325730791, "vram_gb": 6.154519552, "lambda": 0.31272664417823154, "lambda_min": 0.06857724487781525, "lambda_max": 0.705015242099762, "S_mean": 0.2503865559895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.299893856048584, "grad_norm": 1.0890603065490723, "tokens_seen": 95526912, "tokens_per_sec": 16128.759190964876, "vram_gb": 6.154519552, "lambda": 0.31279188642899197, "lambda_min": 0.06857862323522568, "lambda_max": 0.7052555680274963, "S_mean": 0.24139404296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.273369312286377, "grad_norm": 1.0634434223175049, "tokens_seen": 95690752, "tokens_per_sec": 16129.18695125146, "vram_gb": 6.154519552, "lambda": 0.3128799091403683, "lambda_min": 0.06859558075666428, "lambda_max": 0.7056158185005188, "S_mean": 0.258758544921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.290773868560791, "grad_norm": 1.1354520320892334, "tokens_seen": 95854592, "tokens_per_sec": 16129.618000210015, "vram_gb": 6.154519552, "lambda": 0.3129816207413872, "lambda_min": 0.06857758015394211, "lambda_max": 0.7059944868087769, "S_mean": 0.266510009765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.260226726531982, "grad_norm": 1.0693482160568237, "tokens_seen": 96018432, "tokens_per_sec": 16130.26397307896, "vram_gb": 6.154519552, "lambda": 0.3130528898909688, "lambda_min": 0.06858672946691513, "lambda_max": 0.7062214612960815, "S_mean": 0.246734619140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.4306817054748535, "grad_norm": 1.0850178003311157, "tokens_seen": 96182272, "tokens_per_sec": 16130.730837152038, "vram_gb": 6.154519552, "lambda": 0.31312065074841183, "lambda_min": 0.06860429793596268, "lambda_max": 0.7063997983932495, "S_mean": 0.248992919921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.35948371887207, "grad_norm": 1.2036221027374268, "tokens_seen": 96346112, "tokens_per_sec": 16131.28772961656, "vram_gb": 6.154519552, "lambda": 0.31317413536210853, "lambda_min": 0.06860686838626862, "lambda_max": 0.7065420150756836, "S_mean": 0.2518208821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.375071287155151, "grad_norm": 1.0785048007965088, "tokens_seen": 96509952, "tokens_per_sec": 16131.829178219461, "vram_gb": 6.154519552, "lambda": 0.3132516921808322, "lambda_min": 0.06859731674194336, "lambda_max": 0.7068039178848267, "S_mean": 0.2447509765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.400209426879883, "grad_norm": 1.0710723400115967, "tokens_seen": 96673792, "tokens_per_sec": 16132.3370438069, "vram_gb": 6.154519552, "lambda": 0.31333007508267957, "lambda_min": 0.06857927143573761, "lambda_max": 0.7071484327316284, "S_mean": 0.2573649088541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.317583799362183, "grad_norm": 1.136327862739563, "tokens_seen": 96837632, "tokens_per_sec": 16132.864493842586, "vram_gb": 6.154519552, "lambda": 0.31340450793504715, "lambda_min": 0.06858568638563156, "lambda_max": 0.7073440551757812, "S_mean": 0.259185791015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.349410057067871, "grad_norm": 1.168487548828125, "tokens_seen": 97001472, "tokens_per_sec": 16133.482848052976, "vram_gb": 6.154519552, "lambda": 0.3134999359026551, "lambda_min": 0.06859032064676285, "lambda_max": 0.7076295018196106, "S_mean": 0.2364501953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.156541347503662, "grad_norm": 1.1126254796981812, "tokens_seen": 97165312, "tokens_per_sec": 16133.925408018353, "vram_gb": 6.154519552, "lambda": 0.3135545486584306, "lambda_min": 0.0685926079750061, "lambda_max": 0.7077429890632629, "S_mean": 0.25091552734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.490251064300537, "grad_norm": 1.244337558746338, "tokens_seen": 97329152, "tokens_per_sec": 16134.42908661497, "vram_gb": 6.154519552, "lambda": 0.3136437699819605, "lambda_min": 0.06858903914690018, "lambda_max": 0.7080872058868408, "S_mean": 0.2564493815104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.2864603996276855, "grad_norm": 1.3350701332092285, "tokens_seen": 97492992, "tokens_per_sec": 16134.882462400188, "vram_gb": 6.154519552, "lambda": 0.3136974945664406, "lambda_min": 0.06857636570930481, "lambda_max": 0.7083151340484619, "S_mean": 0.24497477213541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.459423065185547, "grad_norm": 1.2039813995361328, "tokens_seen": 97656832, "tokens_per_sec": 16135.526001212673, "vram_gb": 6.154519552, "lambda": 0.3137782495468855, "lambda_min": 0.06856801360845566, "lambda_max": 0.7085457444190979, "S_mean": 0.2583821614583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.144392728805542, "grad_norm": 1.136773943901062, "tokens_seen": 97820672, "tokens_per_sec": 16136.01338472443, "vram_gb": 6.154519552, "lambda": 0.31386074889451265, "lambda_min": 0.06855782866477966, "lambda_max": 0.7087744474411011, "S_mean": 0.24281819661458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.347534894943237, "grad_norm": 1.104763388633728, "tokens_seen": 97984512, "tokens_per_sec": 16136.572196259478, "vram_gb": 6.154519552, "lambda": 0.31391672417521477, "lambda_min": 0.06855925917625427, "lambda_max": 0.7089791297912598, "S_mean": 0.2512410481770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.232576608657837, "grad_norm": 1.100122332572937, "tokens_seen": 98148352, "tokens_per_sec": 16137.166909199297, "vram_gb": 6.154519552, "lambda": 0.31397200624148053, "lambda_min": 0.06856299191713333, "lambda_max": 0.7092267870903015, "S_mean": 0.24071248372395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.366122233867645, "val_ppl": 78.7377125064686, "lambda": 0.314064453355968, "lambda_min": 0.06856122612953186, "lambda_max": 0.7095168232917786, "lambda_per_layer": [0.17817959189414978, 0.1331695318222046, 0.14379164576530457, 0.1589106172323227, 0.15034425258636475, 0.1616923213005066, 0.1132238432765007, 0.06856122612953186, 0.07156935334205627, 0.1857236921787262, 0.26066502928733826, 0.36342641711235046, 0.5381518602371216, 0.4290919601917267, 0.5682402849197388, 0.39777809381484985, 0.6462647318840027, 0.7095168232917786, 0.6352243423461914, 0.45301729440689087, 0.3474453091621399, 0.2423446923494339, 0.34502896666526794, 0.2361849993467331], "S_mean": 0.256561279296875, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 6095.362888336182, "nan_failures": 0} diff --git a/s_only_s1/config.json b/s_only_s1/config.json new file mode 100644 index 0000000000000000000000000000000000000000..d556abb396301314eec1742a316e78c05e4de3c5 --- /dev/null +++ b/s_only_s1/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 1, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s1", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 1001340, + "eval_interval": 500, + "eval_batches": 40, + "log_interval": 20 +} \ No newline at end of file diff --git a/s_only_s1/final.pt b/s_only_s1/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..422230f48a1e311d00a07553a1eab68d887cc406 --- /dev/null +++ b/s_only_s1/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5cfaf6db0ff2e7aaa36e74863a68953a2262abb9561e5ac6233b98ff6aaa8b28 +size 255945415 diff --git a/s_only_s1/metrics.jsonl b/s_only_s1/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..52b66b0d2f2d31b4349fef8c3d95a5b27fd6f6e4 --- /dev/null +++ b/s_only_s1/metrics.jsonl @@ -0,0 +1,626 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 1, "config": {"variant": "s_only", "seed": 1, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s1", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 1001340, "eval_interval": 500, "eval_batches": 40, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.913787841796875, "grad_norm": 14.32506275177002, "tokens_seen": 8192, "tokens_per_sec": 7772.831252267847, "vram_gb": 5.63817984, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.3306986490885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.808298110961914, "grad_norm": 1.9825159311294556, "tokens_seen": 172032, "tokens_per_sec": 15762.755780611335, "vram_gb": 6.154519552, "lambda": 0.1269337609410286, "lambda_min": 0.12689295411109924, "lambda_max": 0.12696589529514313, "S_mean": 0.4456939697265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.159536361694336, "grad_norm": 1.6472218036651611, "tokens_seen": 335872, "tokens_per_sec": 16173.761419749182, "vram_gb": 6.154519552, "lambda": 0.1268890624245008, "lambda_min": 0.12679970264434814, "lambda_max": 0.12699560821056366, "S_mean": 0.4812978108723958, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.263507843017578, "grad_norm": 1.4421693086624146, "tokens_seen": 499712, "tokens_per_sec": 16311.785761574145, "vram_gb": 6.154519552, "lambda": 0.12680185213685036, "lambda_min": 0.12667778134346008, "lambda_max": 0.1270368993282318, "S_mean": 0.502532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.417176246643066, "grad_norm": 0.9122244715690613, "tokens_seen": 663552, "tokens_per_sec": 16374.405891451564, "vram_gb": 6.154519552, "lambda": 0.1266710323592027, "lambda_min": 0.12648634612560272, "lambda_max": 0.12700791656970978, "S_mean": 0.3697102864583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.303841829299927, "grad_norm": 0.8113967776298523, "tokens_seen": 827392, "tokens_per_sec": 16387.577513667533, "vram_gb": 6.154519552, "lambda": 0.126587375998497, "lambda_min": 0.12630142271518707, "lambda_max": 0.1270931214094162, "S_mean": 0.2546895345052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.0750579833984375, "grad_norm": 0.6926398873329163, "tokens_seen": 991232, "tokens_per_sec": 16419.485588223062, "vram_gb": 6.154519552, "lambda": 0.1264726873487234, "lambda_min": 0.12594753503799438, "lambda_max": 0.1273777186870575, "S_mean": 0.24509684244791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.813499927520752, "grad_norm": 0.7067312002182007, "tokens_seen": 1155072, "tokens_per_sec": 16440.105590968637, "vram_gb": 6.154519552, "lambda": 0.12643363699316978, "lambda_min": 0.1257578730583191, "lambda_max": 0.12768220901489258, "S_mean": 0.22889200846354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.7891130447387695, "grad_norm": 0.6845285892486572, "tokens_seen": 1318912, "tokens_per_sec": 16453.77184613207, "vram_gb": 6.154519552, "lambda": 0.12643427463869253, "lambda_min": 0.12560011446475983, "lambda_max": 0.12782178819179535, "S_mean": 0.2658487955729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.743035554885864, "grad_norm": 0.5879111289978027, "tokens_seen": 1482752, "tokens_per_sec": 16473.514419691524, "vram_gb": 6.154519552, "lambda": 0.12639994733035564, "lambda_min": 0.12524381279945374, "lambda_max": 0.1279430389404297, "S_mean": 0.22527058919270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.600972890853882, "grad_norm": 0.6206941604614258, "tokens_seen": 1646592, "tokens_per_sec": 16483.75338962344, "vram_gb": 6.154519552, "lambda": 0.12635636826356253, "lambda_min": 0.12494225054979324, "lambda_max": 0.1280350238084793, "S_mean": 0.22017415364583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.603224754333496, "grad_norm": 0.6337317228317261, "tokens_seen": 1810432, "tokens_per_sec": 16490.217866645915, "vram_gb": 6.154519552, "lambda": 0.12631579457471767, "lambda_min": 0.1247335895895958, "lambda_max": 0.12834875285625458, "S_mean": 0.21282958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.61124587059021, "grad_norm": 0.48415008187294006, "tokens_seen": 1974272, "tokens_per_sec": 16497.324356590365, "vram_gb": 6.154519552, "lambda": 0.12623944816490015, "lambda_min": 0.1244703084230423, "lambda_max": 0.1286226063966751, "S_mean": 0.2562662760416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.63941502571106, "grad_norm": 0.6153481006622314, "tokens_seen": 2138112, "tokens_per_sec": 16503.716928436206, "vram_gb": 6.154519552, "lambda": 0.12624299277861914, "lambda_min": 0.12427704781293869, "lambda_max": 0.12877178192138672, "S_mean": 0.22539265950520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.701580047607422, "grad_norm": 0.5634744763374329, "tokens_seen": 2301952, "tokens_per_sec": 16515.999220588667, "vram_gb": 6.154519552, "lambda": 0.12625163421034813, "lambda_min": 0.12417778372764587, "lambda_max": 0.1289921998977661, "S_mean": 0.23191324869791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.540439605712891, "grad_norm": 0.5402431488037109, "tokens_seen": 2465792, "tokens_per_sec": 16520.147890194883, "vram_gb": 6.154519552, "lambda": 0.12629619799554348, "lambda_min": 0.12407085299491882, "lambda_max": 0.12918291985988617, "S_mean": 0.2191162109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.478904724121094, "grad_norm": 0.5682463645935059, "tokens_seen": 2629632, "tokens_per_sec": 16524.0262082727, "vram_gb": 6.154519552, "lambda": 0.12625167922427258, "lambda_min": 0.12381769716739655, "lambda_max": 0.12923701107501984, "S_mean": 0.248291015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.481551647186279, "grad_norm": 0.5514647364616394, "tokens_seen": 2793472, "tokens_per_sec": 16526.116561432154, "vram_gb": 6.154519552, "lambda": 0.12621581585456929, "lambda_min": 0.1236264705657959, "lambda_max": 0.12946780025959015, "S_mean": 0.24207560221354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.231963872909546, "grad_norm": 0.5101732015609741, "tokens_seen": 2957312, "tokens_per_sec": 16524.987367783848, "vram_gb": 6.154519552, "lambda": 0.12625759281218052, "lambda_min": 0.1236034631729126, "lambda_max": 0.12969587743282318, "S_mean": 0.265869140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.306389808654785, "grad_norm": 0.5585957765579224, "tokens_seen": 3121152, "tokens_per_sec": 16530.013546212343, "vram_gb": 6.154519552, "lambda": 0.12635301953802505, "lambda_min": 0.12354259938001633, "lambda_max": 0.1299976408481598, "S_mean": 0.2614237467447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.272565841674805, "grad_norm": 0.502890944480896, "tokens_seen": 3284992, "tokens_per_sec": 16531.904327127006, "vram_gb": 6.154519552, "lambda": 0.12638964938620725, "lambda_min": 0.12350483983755112, "lambda_max": 0.13016480207443237, "S_mean": 0.2603556315104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.366705417633057, "grad_norm": 0.5512903332710266, "tokens_seen": 3448832, "tokens_per_sec": 16531.623902671876, "vram_gb": 6.154519552, "lambda": 0.1264471293737491, "lambda_min": 0.12328857183456421, "lambda_max": 0.13049304485321045, "S_mean": 0.3069559733072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.25626277923584, "grad_norm": 0.5096892714500427, "tokens_seen": 3612672, "tokens_per_sec": 16529.679705206665, "vram_gb": 6.154519552, "lambda": 0.1265504832069079, "lambda_min": 0.12328192591667175, "lambda_max": 0.13076774775981903, "S_mean": 0.24689229329427084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.340661287307739, "grad_norm": 0.5179641246795654, "tokens_seen": 3776512, "tokens_per_sec": 16529.79603289451, "vram_gb": 6.154519552, "lambda": 0.12668250656376281, "lambda_min": 0.12323523312807083, "lambda_max": 0.1309831440448761, "S_mean": 0.24706522623697916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.207332372665405, "grad_norm": 0.5814391374588013, "tokens_seen": 3940352, "tokens_per_sec": 16532.718765028167, "vram_gb": 6.154519552, "lambda": 0.12679455522447824, "lambda_min": 0.12338022142648697, "lambda_max": 0.13124701380729675, "S_mean": 0.3083292643229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.337316989898682, "grad_norm": 0.5393162369728088, "tokens_seen": 4104192, "tokens_per_sec": 16534.293042088473, "vram_gb": 6.154519552, "lambda": 0.1268880981951952, "lambda_min": 0.12350647151470184, "lambda_max": 0.13156846165657043, "S_mean": 0.270538330078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 500, "val_loss": 6.259992301464081, "val_ppl": 523.2149121037016, "lambda": 0.12689607621481022, "lambda_min": 0.12352295964956284, "lambda_max": 0.1316060870885849, "lambda_per_layer": [0.1316060870885849, 0.12739430367946625, 0.12917234003543854, 0.12759354710578918, 0.12777777016162872, 0.12730003893375397, 0.12703101336956024, 0.1264016479253769, 0.1262616515159607, 0.1275402307510376, 0.12630130350589752, 0.12624625861644745, 0.12774063646793365, 0.1274903267621994, 0.12727972865104675, 0.12633320689201355, 0.1265619397163391, 0.12547634541988373, 0.12446950376033783, 0.12548238039016724, 0.1268416792154312, 0.12352295964956284, 0.12692096829414368, 0.12675996124744415], "S_mean": 0.2982177734375, "G_mean": null, "G_std": null, "tokens_seen": 4104192} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.175951957702637, "grad_norm": 0.5350412726402283, "tokens_seen": 4268032, "tokens_per_sec": 16301.719135029927, "vram_gb": 6.154519552, "lambda": 0.1270150557781259, "lambda_min": 0.12365680932998657, "lambda_max": 0.13203169405460358, "S_mean": 0.3135782877604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.257891416549683, "grad_norm": 0.5315160155296326, "tokens_seen": 4431872, "tokens_per_sec": 16310.63236653399, "vram_gb": 6.154519552, "lambda": 0.1271467193340262, "lambda_min": 0.12361135333776474, "lambda_max": 0.13236263394355774, "S_mean": 0.2825826009114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.151679992675781, "grad_norm": 0.48343637585639954, "tokens_seen": 4595712, "tokens_per_sec": 16315.360806394809, "vram_gb": 6.154519552, "lambda": 0.12724667911728224, "lambda_min": 0.12358731776475906, "lambda_max": 0.1327396184206009, "S_mean": 0.285675048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.105440139770508, "grad_norm": 0.6293532252311707, "tokens_seen": 4759552, "tokens_per_sec": 16323.566706915095, "vram_gb": 6.154519552, "lambda": 0.12735473550856113, "lambda_min": 0.12370163202285767, "lambda_max": 0.1330345869064331, "S_mean": 0.32562255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.15365743637085, "grad_norm": 0.5783795714378357, "tokens_seen": 4923392, "tokens_per_sec": 16331.199464009127, "vram_gb": 6.154519552, "lambda": 0.1275418313841025, "lambda_min": 0.1239277571439743, "lambda_max": 0.13338015973567963, "S_mean": 0.27056884765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.258706092834473, "grad_norm": 0.48946669697761536, "tokens_seen": 5087232, "tokens_per_sec": 16338.634102671309, "vram_gb": 6.154519552, "lambda": 0.1277033037816485, "lambda_min": 0.12401067465543747, "lambda_max": 0.13376642763614655, "S_mean": 0.303314208984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.252086877822876, "grad_norm": 0.5725744962692261, "tokens_seen": 5251072, "tokens_per_sec": 16344.50363056809, "vram_gb": 6.154519552, "lambda": 0.12790542840957642, "lambda_min": 0.124632328748703, "lambda_max": 0.13420259952545166, "S_mean": 0.2932840983072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.978738307952881, "grad_norm": 0.5433061718940735, "tokens_seen": 5414912, "tokens_per_sec": 16351.738610371083, "vram_gb": 6.154519552, "lambda": 0.12812327376256386, "lambda_min": 0.12496397644281387, "lambda_max": 0.13483956456184387, "S_mean": 0.2821146647135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.011162042617798, "grad_norm": 0.5319437384605408, "tokens_seen": 5578752, "tokens_per_sec": 16359.091477736318, "vram_gb": 6.154519552, "lambda": 0.12831780749062696, "lambda_min": 0.1252266764640808, "lambda_max": 0.13523262739181519, "S_mean": 0.2685343424479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.9847092628479, "grad_norm": 0.5287937521934509, "tokens_seen": 5742592, "tokens_per_sec": 16366.168434056981, "vram_gb": 6.154519552, "lambda": 0.12856514068941274, "lambda_min": 0.1254170835018158, "lambda_max": 0.13549184799194336, "S_mean": 0.2971293131510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.085318326950073, "grad_norm": 0.5286767482757568, "tokens_seen": 5906432, "tokens_per_sec": 16372.315962226581, "vram_gb": 6.154519552, "lambda": 0.12878051586449146, "lambda_min": 0.12539274990558624, "lambda_max": 0.13587182760238647, "S_mean": 0.2835489908854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 5.993716716766357, "grad_norm": 0.506059467792511, "tokens_seen": 6070272, "tokens_per_sec": 16377.772821484756, "vram_gb": 6.154519552, "lambda": 0.1290573781977097, "lambda_min": 0.12552618980407715, "lambda_max": 0.13620427250862122, "S_mean": 0.3359171549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.060713052749634, "grad_norm": 0.5165417790412903, "tokens_seen": 6234112, "tokens_per_sec": 16383.684036152583, "vram_gb": 6.154519552, "lambda": 0.12932050041854382, "lambda_min": 0.12529465556144714, "lambda_max": 0.1365220695734024, "S_mean": 0.2886759440104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.02627420425415, "grad_norm": 0.507694661617279, "tokens_seen": 6397952, "tokens_per_sec": 16388.773657577698, "vram_gb": 6.154519552, "lambda": 0.1295907891665896, "lambda_min": 0.12474570423364639, "lambda_max": 0.13694365322589874, "S_mean": 0.2860107421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.94743537902832, "grad_norm": 0.5361756682395935, "tokens_seen": 6561792, "tokens_per_sec": 16392.34798179381, "vram_gb": 6.154519552, "lambda": 0.1299926262969772, "lambda_min": 0.12448186427354813, "lambda_max": 0.13732437789440155, "S_mean": 0.2785746256510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.841953992843628, "grad_norm": 0.5208631157875061, "tokens_seen": 6725632, "tokens_per_sec": 16396.49204628221, "vram_gb": 6.154519552, "lambda": 0.13038730093588433, "lambda_min": 0.12410486489534378, "lambda_max": 0.13764691352844238, "S_mean": 0.2756449381510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.794466972351074, "grad_norm": 0.563789963722229, "tokens_seen": 6889472, "tokens_per_sec": 16399.950142204623, "vram_gb": 6.154519552, "lambda": 0.1307868113120397, "lambda_min": 0.12370482087135315, "lambda_max": 0.13793306052684784, "S_mean": 0.2942911783854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.687493324279785, "grad_norm": 0.5458532571792603, "tokens_seen": 7053312, "tokens_per_sec": 16403.485781547388, "vram_gb": 6.154519552, "lambda": 0.13115752022713423, "lambda_min": 0.123412124812603, "lambda_max": 0.13857559859752655, "S_mean": 0.2768656412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.776203155517578, "grad_norm": 0.5939895510673523, "tokens_seen": 7217152, "tokens_per_sec": 16407.401211045184, "vram_gb": 6.154519552, "lambda": 0.13149215218921503, "lambda_min": 0.123047836124897, "lambda_max": 0.1397126019001007, "S_mean": 0.2802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.695283889770508, "grad_norm": 0.5145086646080017, "tokens_seen": 7380992, "tokens_per_sec": 16409.903950976593, "vram_gb": 6.154519552, "lambda": 0.1318753423790137, "lambda_min": 0.12254880368709564, "lambda_max": 0.14101047813892365, "S_mean": 0.2904154459635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.878530979156494, "grad_norm": 0.5410987734794617, "tokens_seen": 7544832, "tokens_per_sec": 16413.120247345432, "vram_gb": 6.154519552, "lambda": 0.13227269115547338, "lambda_min": 0.1222086250782013, "lambda_max": 0.14237621426582336, "S_mean": 0.2869771321614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.800983905792236, "grad_norm": 0.5269562602043152, "tokens_seen": 7708672, "tokens_per_sec": 16416.348546470897, "vram_gb": 6.154519552, "lambda": 0.13269481404374042, "lambda_min": 0.12182575464248657, "lambda_max": 0.14354833960533142, "S_mean": 0.2987467447916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.930435419082642, "grad_norm": 0.528228223323822, "tokens_seen": 7872512, "tokens_per_sec": 16418.456947332408, "vram_gb": 6.154519552, "lambda": 0.1330983421454827, "lambda_min": 0.12149578332901001, "lambda_max": 0.14467836916446686, "S_mean": 0.2914835611979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.638961315155029, "grad_norm": 0.5077663064002991, "tokens_seen": 8036352, "tokens_per_sec": 16422.368967416056, "vram_gb": 6.154519552, "lambda": 0.1335068643093109, "lambda_min": 0.12101743370294571, "lambda_max": 0.1459568291902542, "S_mean": 0.2714894612630208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.767230033874512, "grad_norm": 0.5166292190551758, "tokens_seen": 8200192, "tokens_per_sec": 16425.1176589527, "vram_gb": 6.154519552, "lambda": 0.1339150465403994, "lambda_min": 0.12053637951612473, "lambda_max": 0.14719775319099426, "S_mean": 0.2885640462239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.7917481660842896, "val_ppl": 327.585197435357, "lambda": 0.1339356672639648, "lambda_min": 0.12051347643136978, "lambda_max": 0.1472582370042801, "lambda_per_layer": [0.14055712521076202, 0.13006405532360077, 0.13374559581279755, 0.13090474903583527, 0.13107389211654663, 0.12785513699054718, 0.12242225557565689, 0.12051347643136978, 0.12483271211385727, 0.13156577944755554, 0.13102518022060394, 0.13716056942939758, 0.14451175928115845, 0.1472582370042801, 0.14120179414749146, 0.14331838488578796, 0.14094528555870056, 0.136167973279953, 0.1283750832080841, 0.13355785608291626, 0.1368548572063446, 0.13308565318584442, 0.1343718022108078, 0.13308680057525635], "S_mean": 0.294769287109375, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.684432029724121, "grad_norm": 0.5826789140701294, "tokens_seen": 8364032, "tokens_per_sec": 16308.931228494514, "vram_gb": 6.154519552, "lambda": 0.13436406229933104, "lambda_min": 0.12025093287229538, "lambda_max": 0.14842703938484192, "S_mean": 0.3037312825520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.733770370483398, "grad_norm": 0.5785160660743713, "tokens_seen": 8527872, "tokens_per_sec": 16312.964600613324, "vram_gb": 6.154519552, "lambda": 0.1347834775224328, "lambda_min": 0.11976844072341919, "lambda_max": 0.14986634254455566, "S_mean": 0.3032938639322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.7809271812438965, "grad_norm": 0.5335716009140015, "tokens_seen": 8691712, "tokens_per_sec": 16316.798297236137, "vram_gb": 6.154519552, "lambda": 0.1352554593856136, "lambda_min": 0.11928533762693405, "lambda_max": 0.15116050839424133, "S_mean": 0.257720947265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.6063268184661865, "grad_norm": 0.5543310046195984, "tokens_seen": 8855552, "tokens_per_sec": 16319.69490202821, "vram_gb": 6.154519552, "lambda": 0.13574631170680126, "lambda_min": 0.11902175098657608, "lambda_max": 0.15254035592079163, "S_mean": 0.2936197916666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.790219068527222, "grad_norm": 0.5206776261329651, "tokens_seen": 9019392, "tokens_per_sec": 16323.869268521703, "vram_gb": 6.154519552, "lambda": 0.13624282553792, "lambda_min": 0.1186065673828125, "lambda_max": 0.1537541151046753, "S_mean": 0.2845967610677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.652514696121216, "grad_norm": 0.5757941603660583, "tokens_seen": 9183232, "tokens_per_sec": 16328.295495998922, "vram_gb": 6.154519552, "lambda": 0.13670004003991684, "lambda_min": 0.11826296150684357, "lambda_max": 0.15492698550224304, "S_mean": 0.27642822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.705215692520142, "grad_norm": 0.5237170457839966, "tokens_seen": 9347072, "tokens_per_sec": 16332.120237856792, "vram_gb": 6.154519552, "lambda": 0.1371044429639975, "lambda_min": 0.11790590733289719, "lambda_max": 0.156114399433136, "S_mean": 0.2679036458333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.683315277099609, "grad_norm": 0.5229763984680176, "tokens_seen": 9510912, "tokens_per_sec": 16335.438592076267, "vram_gb": 6.154519552, "lambda": 0.13758571663250527, "lambda_min": 0.11748187243938446, "lambda_max": 0.1574094593524933, "S_mean": 0.2559712727864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.723148822784424, "grad_norm": 0.5434519648551941, "tokens_seen": 9674752, "tokens_per_sec": 16340.0879754823, "vram_gb": 6.154519552, "lambda": 0.13802034252633652, "lambda_min": 0.11706601083278656, "lambda_max": 0.15858370065689087, "S_mean": 0.2684529622395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.651993989944458, "grad_norm": 0.5482762455940247, "tokens_seen": 9838592, "tokens_per_sec": 16344.326021459694, "vram_gb": 6.154519552, "lambda": 0.13843956217169762, "lambda_min": 0.11663976311683655, "lambda_max": 0.15957379341125488, "S_mean": 0.25714111328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.846109628677368, "grad_norm": 0.5681812167167664, "tokens_seen": 10002432, "tokens_per_sec": 16349.059700104355, "vram_gb": 6.154519552, "lambda": 0.1389069234331449, "lambda_min": 0.11609184741973877, "lambda_max": 0.16082677245140076, "S_mean": 0.2942911783854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.609116554260254, "grad_norm": 0.541037380695343, "tokens_seen": 10166272, "tokens_per_sec": 16353.694179912975, "vram_gb": 6.154519552, "lambda": 0.13948270430167517, "lambda_min": 0.11591020971536636, "lambda_max": 0.16204014420509338, "S_mean": 0.24169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.5989086627960205, "grad_norm": 0.6071934700012207, "tokens_seen": 10330112, "tokens_per_sec": 16356.703062266763, "vram_gb": 6.154519552, "lambda": 0.13999891777833304, "lambda_min": 0.11565004289150238, "lambda_max": 0.16333086788654327, "S_mean": 0.2662150065104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.495262622833252, "grad_norm": 0.5372776389122009, "tokens_seen": 10493952, "tokens_per_sec": 16358.508510022115, "vram_gb": 6.154519552, "lambda": 0.1404899936169386, "lambda_min": 0.1151719018816948, "lambda_max": 0.16458338499069214, "S_mean": 0.2622172037760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.631236791610718, "grad_norm": 0.5329630374908447, "tokens_seen": 10657792, "tokens_per_sec": 16361.1620130373, "vram_gb": 6.154519552, "lambda": 0.14096317067742348, "lambda_min": 0.1147606298327446, "lambda_max": 0.1658911257982254, "S_mean": 0.2583516438802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.484615325927734, "grad_norm": 0.5077133178710938, "tokens_seen": 10821632, "tokens_per_sec": 16363.639652395437, "vram_gb": 6.154519552, "lambda": 0.14142730024953684, "lambda_min": 0.11423999071121216, "lambda_max": 0.16720062494277954, "S_mean": 0.2508036295572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.749541759490967, "grad_norm": 0.5946022868156433, "tokens_seen": 10985472, "tokens_per_sec": 16365.58163816373, "vram_gb": 6.154519552, "lambda": 0.1419028239324689, "lambda_min": 0.1138329729437828, "lambda_max": 0.16839252412319183, "S_mean": 0.256683349609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.573653936386108, "grad_norm": 0.5963412523269653, "tokens_seen": 11149312, "tokens_per_sec": 16367.839551616296, "vram_gb": 6.154519552, "lambda": 0.14241558189193407, "lambda_min": 0.11364386230707169, "lambda_max": 0.1695716232061386, "S_mean": 0.23711140950520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.494607448577881, "grad_norm": 0.593133270740509, "tokens_seen": 11313152, "tokens_per_sec": 16370.436433698067, "vram_gb": 6.154519552, "lambda": 0.14287282464404902, "lambda_min": 0.11328566819429398, "lambda_max": 0.17074036598205566, "S_mean": 0.2671915690104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.692322731018066, "grad_norm": 0.5369494557380676, "tokens_seen": 11476992, "tokens_per_sec": 16372.478190999504, "vram_gb": 6.154519552, "lambda": 0.1434031662841638, "lambda_min": 0.1131390854716301, "lambda_max": 0.171916201710701, "S_mean": 0.24889628092447916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.464737415313721, "grad_norm": 0.5499512553215027, "tokens_seen": 11640832, "tokens_per_sec": 16374.857799221485, "vram_gb": 6.154519552, "lambda": 0.1438761449729403, "lambda_min": 0.11276084184646606, "lambda_max": 0.1731761395931244, "S_mean": 0.2672526041666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.594857215881348, "grad_norm": 0.5933676362037659, "tokens_seen": 11804672, "tokens_per_sec": 16377.658396506124, "vram_gb": 6.154519552, "lambda": 0.14437459564457336, "lambda_min": 0.11248350143432617, "lambda_max": 0.17433762550354004, "S_mean": 0.24223836263020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.626343727111816, "grad_norm": 0.6622198820114136, "tokens_seen": 11968512, "tokens_per_sec": 16379.705742737538, "vram_gb": 6.154519552, "lambda": 0.14482611448814472, "lambda_min": 0.11229690909385681, "lambda_max": 0.17522315680980682, "S_mean": 0.24605305989583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.494366407394409, "grad_norm": 0.556059718132019, "tokens_seen": 12132352, "tokens_per_sec": 16382.2131844229, "vram_gb": 6.154519552, "lambda": 0.14530750301977, "lambda_min": 0.11192052811384201, "lambda_max": 0.17651104927062988, "S_mean": 0.2531331380208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.323910713195801, "grad_norm": 0.6634514331817627, "tokens_seen": 12296192, "tokens_per_sec": 16384.015057683726, "vram_gb": 6.154519552, "lambda": 0.1457845807696382, "lambda_min": 0.1115463599562645, "lambda_max": 0.1777251660823822, "S_mean": 0.24232991536458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1500, "val_loss": 5.52355672121048, "val_ppl": 250.52450026978494, "lambda": 0.14580429873118797, "lambda_min": 0.1115170419216156, "lambda_max": 0.17777752876281738, "lambda_per_layer": [0.1485660821199417, 0.13072502613067627, 0.13495489954948425, 0.1316545456647873, 0.13262197375297546, 0.12780389189720154, 0.11660322546958923, 0.1115170419216156, 0.12394586950540543, 0.14678965508937836, 0.14029285311698914, 0.15959714353084564, 0.17688508331775665, 0.17777752876281738, 0.16034162044525146, 0.17129693925380707, 0.16964387893676758, 0.15600913763046265, 0.1375216692686081, 0.14686916768550873, 0.15303315222263336, 0.1519983559846878, 0.14936606585979462, 0.14348836243152618], "S_mean": 0.2752482096354167, "G_mean": null, "G_std": null, "tokens_seen": 12296192} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.505437612533569, "grad_norm": 0.5724397897720337, "tokens_seen": 12460032, "tokens_per_sec": 16305.788010511158, "vram_gb": 6.154519552, "lambda": 0.14629698637872934, "lambda_min": 0.11113954335451126, "lambda_max": 0.1789979338645935, "S_mean": 0.22035725911458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.5262086391448975, "grad_norm": 0.6569513082504272, "tokens_seen": 12623872, "tokens_per_sec": 16308.277318908486, "vram_gb": 6.154519552, "lambda": 0.1468135400985678, "lambda_min": 0.11080417037010193, "lambda_max": 0.18034380674362183, "S_mean": 0.24506632486979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.476892709732056, "grad_norm": 0.5794680118560791, "tokens_seen": 12787712, "tokens_per_sec": 16310.410814760062, "vram_gb": 6.154519552, "lambda": 0.1472929505010446, "lambda_min": 0.11033204197883606, "lambda_max": 0.1814710795879364, "S_mean": 0.23551432291666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.1674299240112305, "grad_norm": 0.5516547560691833, "tokens_seen": 12951552, "tokens_per_sec": 16313.238696549835, "vram_gb": 6.154519552, "lambda": 0.1477924439435204, "lambda_min": 0.10990405082702637, "lambda_max": 0.18287746608257294, "S_mean": 0.24262491861979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.25242018699646, "grad_norm": 0.6116674542427063, "tokens_seen": 13115392, "tokens_per_sec": 16315.651082119513, "vram_gb": 6.154519552, "lambda": 0.14826941831658283, "lambda_min": 0.10926783084869385, "lambda_max": 0.1842527836561203, "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.259106397628784, "grad_norm": 0.5913317203521729, "tokens_seen": 13279232, "tokens_per_sec": 16318.719412656215, "vram_gb": 6.154519552, "lambda": 0.14885611552745104, "lambda_min": 0.10886536538600922, "lambda_max": 0.1855931580066681, "S_mean": 0.24982706705729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.268409013748169, "grad_norm": 0.554283618927002, "tokens_seen": 13443072, "tokens_per_sec": 16321.85773344851, "vram_gb": 6.154519552, "lambda": 0.1493890251343449, "lambda_min": 0.10845782607793808, "lambda_max": 0.18689513206481934, "S_mean": 0.23732503255208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.405864715576172, "grad_norm": 0.6234826445579529, "tokens_seen": 13606912, "tokens_per_sec": 16325.111744597954, "vram_gb": 6.154519552, "lambda": 0.14986131712794304, "lambda_min": 0.10797346383333206, "lambda_max": 0.1883731186389923, "S_mean": 0.27557373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.200766563415527, "grad_norm": 0.5692837834358215, "tokens_seen": 13770752, "tokens_per_sec": 16328.133720110764, "vram_gb": 6.154519552, "lambda": 0.150368538685143, "lambda_min": 0.10748469084501266, "lambda_max": 0.18979796767234802, "S_mean": 0.23473103841145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.2902512550354, "grad_norm": 0.5599589347839355, "tokens_seen": 13934592, "tokens_per_sec": 16331.097508023962, "vram_gb": 6.154519552, "lambda": 0.15082499819497266, "lambda_min": 0.10693077743053436, "lambda_max": 0.19109761714935303, "S_mean": 0.23966471354166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.35822057723999, "grad_norm": 0.5988028049468994, "tokens_seen": 14098432, "tokens_per_sec": 16334.211067728633, "vram_gb": 6.154519552, "lambda": 0.1512852944433689, "lambda_min": 0.10653962194919586, "lambda_max": 0.192227303981781, "S_mean": 0.248504638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.295207977294922, "grad_norm": 0.571427583694458, "tokens_seen": 14262272, "tokens_per_sec": 16336.77694389352, "vram_gb": 6.154519552, "lambda": 0.151852834969759, "lambda_min": 0.10609398782253265, "lambda_max": 0.19357913732528687, "S_mean": 0.256622314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.3256001472473145, "grad_norm": 0.5979456305503845, "tokens_seen": 14426112, "tokens_per_sec": 16339.196441981185, "vram_gb": 6.154519552, "lambda": 0.15237282123416662, "lambda_min": 0.10559362918138504, "lambda_max": 0.19483241438865662, "S_mean": 0.24985758463541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.340120077133179, "grad_norm": 0.5731381773948669, "tokens_seen": 14589952, "tokens_per_sec": 16341.935906551904, "vram_gb": 6.154519552, "lambda": 0.1528942659497261, "lambda_min": 0.10509943962097168, "lambda_max": 0.19641128182411194, "S_mean": 0.24125162760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.455566167831421, "grad_norm": 0.5985439419746399, "tokens_seen": 14753792, "tokens_per_sec": 16344.658932168688, "vram_gb": 6.154519552, "lambda": 0.15338196170826754, "lambda_min": 0.10458152741193771, "lambda_max": 0.19748428463935852, "S_mean": 0.23382568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.4393861293792725, "grad_norm": 0.5813873410224915, "tokens_seen": 14917632, "tokens_per_sec": 16346.814203245682, "vram_gb": 6.154519552, "lambda": 0.153898353377978, "lambda_min": 0.10409309715032578, "lambda_max": 0.19892403483390808, "S_mean": 0.24436442057291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.202556133270264, "grad_norm": 0.5829033255577087, "tokens_seen": 15081472, "tokens_per_sec": 16349.034647805176, "vram_gb": 6.154519552, "lambda": 0.15443131203452745, "lambda_min": 0.1038309633731842, "lambda_max": 0.20042099058628082, "S_mean": 0.255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.304246664047241, "grad_norm": 0.6061760187149048, "tokens_seen": 15245312, "tokens_per_sec": 16350.870582855145, "vram_gb": 6.154519552, "lambda": 0.15492957613120475, "lambda_min": 0.10331687331199646, "lambda_max": 0.20151731371879578, "S_mean": 0.2687276204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.130035877227783, "grad_norm": 0.6055389046669006, "tokens_seen": 15409152, "tokens_per_sec": 16352.904941500088, "vram_gb": 6.154519552, "lambda": 0.15545474427441755, "lambda_min": 0.10268822312355042, "lambda_max": 0.20289522409439087, "S_mean": 0.25164794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.371861457824707, "grad_norm": 0.6329984664916992, "tokens_seen": 15572992, "tokens_per_sec": 16354.623641670196, "vram_gb": 6.154519552, "lambda": 0.15595518549283346, "lambda_min": 0.10221706330776215, "lambda_max": 0.20402806997299194, "S_mean": 0.241485595703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.40321946144104, "grad_norm": 0.6790639162063599, "tokens_seen": 15736832, "tokens_per_sec": 16357.00943150415, "vram_gb": 6.154519552, "lambda": 0.1564826499670744, "lambda_min": 0.10191941261291504, "lambda_max": 0.20527511835098267, "S_mean": 0.24476114908854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.491262435913086, "grad_norm": 0.6272996664047241, "tokens_seen": 15900672, "tokens_per_sec": 16359.188150351985, "vram_gb": 6.154519552, "lambda": 0.15702887500325838, "lambda_min": 0.10169763118028641, "lambda_max": 0.20660629868507385, "S_mean": 0.24516805013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.144312381744385, "grad_norm": 0.5936050415039062, "tokens_seen": 16064512, "tokens_per_sec": 16361.494102374432, "vram_gb": 6.154519552, "lambda": 0.1576035919909676, "lambda_min": 0.10140965133905411, "lambda_max": 0.20812992751598358, "S_mean": 0.24634806315104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.2560954093933105, "grad_norm": 0.619968831539154, "tokens_seen": 16228352, "tokens_per_sec": 16363.46717274905, "vram_gb": 6.154519552, "lambda": 0.15813736959050098, "lambda_min": 0.10106188803911209, "lambda_max": 0.20979522168636322, "S_mean": 0.24143473307291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.334981679916382, "grad_norm": 0.6011673212051392, "tokens_seen": 16392192, "tokens_per_sec": 16364.219546425596, "vram_gb": 6.154519552, "lambda": 0.15865843867262205, "lambda_min": 0.10080090910196304, "lambda_max": 0.21109122037887573, "S_mean": 0.2713623046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.360867166519165, "val_ppl": 212.909494406239, "lambda": 0.1586826459194223, "lambda_min": 0.10077977925539017, "lambda_max": 0.2111523449420929, "lambda_per_layer": [0.15364022552967072, 0.13173803687095642, 0.13596971333026886, 0.13238127529621124, 0.1343645453453064, 0.12784814834594727, 0.11168761551380157, 0.10077977925539017, 0.12326173484325409, 0.15965694189071655, 0.14992761611938477, 0.18004126846790314, 0.20836564898490906, 0.2111523449420929, 0.18292152881622314, 0.20190587639808655, 0.20072659850120544, 0.18625782430171967, 0.1495160311460495, 0.1622159779071808, 0.17308539152145386, 0.17307297885417938, 0.1646076887845993, 0.15325871109962463], "S_mean": 0.2610677083333333, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.225472688674927, "grad_norm": 0.5954309105873108, "tokens_seen": 16556032, "tokens_per_sec": 16305.237935244631, "vram_gb": 6.154519552, "lambda": 0.15921892349918684, "lambda_min": 0.10045313835144043, "lambda_max": 0.21241869032382965, "S_mean": 0.2535603841145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.313514709472656, "grad_norm": 0.6298387050628662, "tokens_seen": 16719872, "tokens_per_sec": 16306.988860071608, "vram_gb": 6.154519552, "lambda": 0.15979720714191595, "lambda_min": 0.10005027055740356, "lambda_max": 0.21387721598148346, "S_mean": 0.2540690104166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.264294862747192, "grad_norm": 0.6015843152999878, "tokens_seen": 16883712, "tokens_per_sec": 16308.726769062165, "vram_gb": 6.154519552, "lambda": 0.16033665463328362, "lambda_min": 0.0996793806552887, "lambda_max": 0.21525640785694122, "S_mean": 0.24505615234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.281183242797852, "grad_norm": 0.6366837620735168, "tokens_seen": 17047552, "tokens_per_sec": 16310.728051782307, "vram_gb": 6.154519552, "lambda": 0.16084065133084854, "lambda_min": 0.09927618503570557, "lambda_max": 0.21660976111888885, "S_mean": 0.2629292805989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.309812545776367, "grad_norm": 0.5953851938247681, "tokens_seen": 17211392, "tokens_per_sec": 16313.338105776063, "vram_gb": 6.154519552, "lambda": 0.16127128433436155, "lambda_min": 0.09883720427751541, "lambda_max": 0.2181922346353531, "S_mean": 0.24861653645833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.171822547912598, "grad_norm": 0.6132662892341614, "tokens_seen": 17375232, "tokens_per_sec": 16315.61647525355, "vram_gb": 6.154519552, "lambda": 0.16179857278863588, "lambda_min": 0.09852541238069534, "lambda_max": 0.21970976889133453, "S_mean": 0.2588602701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.1545116901397705, "grad_norm": 0.6625962257385254, "tokens_seen": 17539072, "tokens_per_sec": 16317.919642578214, "vram_gb": 6.154519552, "lambda": 0.16237957713504633, "lambda_min": 0.0982506200671196, "lambda_max": 0.22100450098514557, "S_mean": 0.247161865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.197911262512207, "grad_norm": 0.6269406080245972, "tokens_seen": 17702912, "tokens_per_sec": 16319.557114213785, "vram_gb": 6.154519552, "lambda": 0.16286179361244044, "lambda_min": 0.09792929142713547, "lambda_max": 0.22213628888130188, "S_mean": 0.262725830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.227501153945923, "grad_norm": 0.6278223991394043, "tokens_seen": 17866752, "tokens_per_sec": 16321.189660563283, "vram_gb": 6.154519552, "lambda": 0.16340992165108523, "lambda_min": 0.09741347283124924, "lambda_max": 0.22366955876350403, "S_mean": 0.27783203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.1612389087677, "grad_norm": 0.5967092514038086, "tokens_seen": 18030592, "tokens_per_sec": 16322.762405865253, "vram_gb": 6.154519552, "lambda": 0.1639583039407929, "lambda_min": 0.09677187353372574, "lambda_max": 0.22527644038200378, "S_mean": 0.2565205891927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.305620908737183, "grad_norm": 0.6154391765594482, "tokens_seen": 18194432, "tokens_per_sec": 16320.623437244723, "vram_gb": 6.154519552, "lambda": 0.16446124607076248, "lambda_min": 0.09631067514419556, "lambda_max": 0.2267952412366867, "S_mean": 0.259857177734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.095325708389282, "grad_norm": 0.6131608486175537, "tokens_seen": 18358272, "tokens_per_sec": 16318.452961007468, "vram_gb": 6.154519552, "lambda": 0.16502089177568754, "lambda_min": 0.09591077268123627, "lambda_max": 0.22842787206172943, "S_mean": 0.24171956380208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.076551675796509, "grad_norm": 0.6501729488372803, "tokens_seen": 18522112, "tokens_per_sec": 16319.879451016253, "vram_gb": 6.154519552, "lambda": 0.16543854027986526, "lambda_min": 0.0954742357134819, "lambda_max": 0.2299695611000061, "S_mean": 0.2631022135416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.198680877685547, "grad_norm": 0.6383368968963623, "tokens_seen": 18685952, "tokens_per_sec": 16319.539873517133, "vram_gb": 6.154519552, "lambda": 0.16603718946377435, "lambda_min": 0.09518374502658844, "lambda_max": 0.23136453330516815, "S_mean": 0.2748514811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.130303621292114, "grad_norm": 0.6305639743804932, "tokens_seen": 18849792, "tokens_per_sec": 16316.77850956361, "vram_gb": 6.154519552, "lambda": 0.1665330103908976, "lambda_min": 0.09486807137727737, "lambda_max": 0.23293747007846832, "S_mean": 0.264495849609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.014000415802002, "grad_norm": 0.6329451203346252, "tokens_seen": 19013632, "tokens_per_sec": 16315.956162442479, "vram_gb": 6.154519552, "lambda": 0.16711237064252296, "lambda_min": 0.09429759532213211, "lambda_max": 0.23434269428253174, "S_mean": 0.256256103515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.191569805145264, "grad_norm": 0.617281436920166, "tokens_seen": 19177472, "tokens_per_sec": 16315.017944099087, "vram_gb": 6.154519552, "lambda": 0.16765353952844939, "lambda_min": 0.09391122311353683, "lambda_max": 0.23571395874023438, "S_mean": 0.25030517578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.257931232452393, "grad_norm": 0.706250786781311, "tokens_seen": 19341312, "tokens_per_sec": 16314.418570177906, "vram_gb": 6.154519552, "lambda": 0.16824979707598686, "lambda_min": 0.09350291639566422, "lambda_max": 0.23712539672851562, "S_mean": 0.24921671549479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.206092357635498, "grad_norm": 0.6678099036216736, "tokens_seen": 19505152, "tokens_per_sec": 16314.864304914136, "vram_gb": 6.154519552, "lambda": 0.16881747792164484, "lambda_min": 0.09308025985956192, "lambda_max": 0.23855742812156677, "S_mean": 0.2601318359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.128675222396851, "grad_norm": 0.7022557854652405, "tokens_seen": 19668992, "tokens_per_sec": 16315.763709263862, "vram_gb": 6.154519552, "lambda": 0.16938115749508142, "lambda_min": 0.09276428073644638, "lambda_max": 0.24015267193317413, "S_mean": 0.26580810546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.158831834793091, "grad_norm": 0.586889922618866, "tokens_seen": 19832832, "tokens_per_sec": 16318.298196688374, "vram_gb": 6.154519552, "lambda": 0.16997747868299484, "lambda_min": 0.09261909872293472, "lambda_max": 0.24162374436855316, "S_mean": 0.23475138346354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.144316911697388, "grad_norm": 0.6760258674621582, "tokens_seen": 19996672, "tokens_per_sec": 16318.825684022726, "vram_gb": 6.154519552, "lambda": 0.17054113652557135, "lambda_min": 0.09220102429389954, "lambda_max": 0.2430107593536377, "S_mean": 0.2615966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.178163766860962, "grad_norm": 0.6576570272445679, "tokens_seen": 20160512, "tokens_per_sec": 16319.5528798143, "vram_gb": 6.154519552, "lambda": 0.17114587097118297, "lambda_min": 0.09199376404285431, "lambda_max": 0.2446979582309723, "S_mean": 0.24732462565104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.152257680892944, "grad_norm": 0.6571093201637268, "tokens_seen": 20324352, "tokens_per_sec": 16319.274012064321, "vram_gb": 6.154519552, "lambda": 0.17173184982190529, "lambda_min": 0.09194590151309967, "lambda_max": 0.2458716630935669, "S_mean": 0.23027547200520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.168530464172363, "grad_norm": 0.6381192803382874, "tokens_seen": 20488192, "tokens_per_sec": 16319.320371059546, "vram_gb": 6.154519552, "lambda": 0.17231567110866308, "lambda_min": 0.09183274209499359, "lambda_max": 0.24742601811885834, "S_mean": 0.249847412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2500, "val_loss": 5.200698018074036, "val_ppl": 181.39881734714533, "lambda": 0.17234101612120867, "lambda_min": 0.09182432293891907, "lambda_max": 0.24749529361724854, "lambda_per_layer": [0.1561667025089264, 0.13146644830703735, 0.13695235550403595, 0.13344551622867584, 0.13651491701602936, 0.12805381417274475, 0.10723873227834702, 0.09182432293891907, 0.12350776791572571, 0.16908851265907288, 0.15898942947387695, 0.1980939656496048, 0.2449013888835907, 0.24749529361724854, 0.2056485265493393, 0.23528960347175598, 0.23201316595077515, 0.22430121898651123, 0.16405901312828064, 0.17869484424591064, 0.19494237005710602, 0.19549775123596191, 0.17796647548675537, 0.1640322506427765], "S_mean": 0.2553609212239583, "G_mean": null, "G_std": null, "tokens_seen": 20488192} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.257410049438477, "grad_norm": 0.7222276926040649, "tokens_seen": 20652032, "tokens_per_sec": 16271.860926936131, "vram_gb": 6.154519552, "lambda": 0.172867512019972, "lambda_min": 0.09168322384357452, "lambda_max": 0.24864815175533295, "S_mean": 0.2511800130208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.212296485900879, "grad_norm": 0.6383551359176636, "tokens_seen": 20815872, "tokens_per_sec": 16272.53495292506, "vram_gb": 6.154519552, "lambda": 0.17346866025278965, "lambda_min": 0.09159000962972641, "lambda_max": 0.2502460777759552, "S_mean": 0.249267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.184267282485962, "grad_norm": 0.6562051177024841, "tokens_seen": 20979712, "tokens_per_sec": 16272.002748659977, "vram_gb": 6.154519552, "lambda": 0.17405668056259552, "lambda_min": 0.09139137715101242, "lambda_max": 0.2516377568244934, "S_mean": 0.269622802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.013412237167358, "grad_norm": 0.6267328262329102, "tokens_seen": 21143552, "tokens_per_sec": 16270.975066792018, "vram_gb": 6.154519552, "lambda": 0.17460820513466993, "lambda_min": 0.09131111949682236, "lambda_max": 0.2533833086490631, "S_mean": 0.24604288736979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.116547346115112, "grad_norm": 0.6944923400878906, "tokens_seen": 21307392, "tokens_per_sec": 16270.999988318046, "vram_gb": 6.154519552, "lambda": 0.175164177082479, "lambda_min": 0.09088628739118576, "lambda_max": 0.254563570022583, "S_mean": 0.23537190755208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.016887903213501, "grad_norm": 0.6987268924713135, "tokens_seen": 21471232, "tokens_per_sec": 16269.224791570263, "vram_gb": 6.154519552, "lambda": 0.17576510272920132, "lambda_min": 0.09064526855945587, "lambda_max": 0.2558351457118988, "S_mean": 0.2508748372395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.214071273803711, "grad_norm": 0.7049095630645752, "tokens_seen": 21635072, "tokens_per_sec": 16268.770769988487, "vram_gb": 6.154519552, "lambda": 0.1763387325530251, "lambda_min": 0.0904693529009819, "lambda_max": 0.25742998719215393, "S_mean": 0.24442545572916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.043819427490234, "grad_norm": 0.6479237079620361, "tokens_seen": 21798912, "tokens_per_sec": 16267.111382845194, "vram_gb": 6.154519552, "lambda": 0.17688615278651318, "lambda_min": 0.0901036188006401, "lambda_max": 0.25883468985557556, "S_mean": 0.24214680989583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.035176038742065, "grad_norm": 0.685257077217102, "tokens_seen": 21962752, "tokens_per_sec": 16265.635888340148, "vram_gb": 6.154519552, "lambda": 0.17741507881631455, "lambda_min": 0.08966688066720963, "lambda_max": 0.2603331208229065, "S_mean": 0.22054036458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.191617727279663, "grad_norm": 0.7315213084220886, "tokens_seen": 22126592, "tokens_per_sec": 16263.317370806153, "vram_gb": 6.154519552, "lambda": 0.1779287482301394, "lambda_min": 0.08953110873699188, "lambda_max": 0.2615303099155426, "S_mean": 0.23157755533854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.042855501174927, "grad_norm": 0.6852569580078125, "tokens_seen": 22290432, "tokens_per_sec": 16260.904120815248, "vram_gb": 6.154519552, "lambda": 0.1784988402699431, "lambda_min": 0.08907455205917358, "lambda_max": 0.2624966502189636, "S_mean": 0.249114990234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.196487665176392, "grad_norm": 0.7477536797523499, "tokens_seen": 22454272, "tokens_per_sec": 16260.047405186362, "vram_gb": 6.154519552, "lambda": 0.17905204960455498, "lambda_min": 0.08861017227172852, "lambda_max": 0.2639042139053345, "S_mean": 0.24339803059895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.031410455703735, "grad_norm": 0.6799942851066589, "tokens_seen": 22618112, "tokens_per_sec": 16258.748042193458, "vram_gb": 6.154519552, "lambda": 0.17957370976607004, "lambda_min": 0.08829452842473984, "lambda_max": 0.26561811566352844, "S_mean": 0.24747721354166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.144340753555298, "grad_norm": 0.6652689576148987, "tokens_seen": 22781952, "tokens_per_sec": 16258.960481354417, "vram_gb": 6.154519552, "lambda": 0.18005858516941467, "lambda_min": 0.08790493756532669, "lambda_max": 0.2666955292224884, "S_mean": 0.238372802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 4.914794921875, "grad_norm": 0.6756376624107361, "tokens_seen": 22945792, "tokens_per_sec": 16260.352971974753, "vram_gb": 6.154519552, "lambda": 0.18056016570578018, "lambda_min": 0.08749881386756897, "lambda_max": 0.26766547560691833, "S_mean": 0.23786417643229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.210336446762085, "grad_norm": 0.6689028739929199, "tokens_seen": 23109632, "tokens_per_sec": 16261.607219917632, "vram_gb": 6.154519552, "lambda": 0.18117344876130423, "lambda_min": 0.08729562908411026, "lambda_max": 0.26899564266204834, "S_mean": 0.23590087890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.068452835083008, "grad_norm": 0.6800563335418701, "tokens_seen": 23273472, "tokens_per_sec": 16263.221934566867, "vram_gb": 6.154519552, "lambda": 0.1817519503335158, "lambda_min": 0.08696278929710388, "lambda_max": 0.27017855644226074, "S_mean": 0.23118082682291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.9970786571502686, "grad_norm": 0.6583322286605835, "tokens_seen": 23437312, "tokens_per_sec": 16264.76390811864, "vram_gb": 6.154519552, "lambda": 0.1822829246520996, "lambda_min": 0.08668314665555954, "lambda_max": 0.2716541588306427, "S_mean": 0.266876220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.062437295913696, "grad_norm": 0.7189803123474121, "tokens_seen": 23601152, "tokens_per_sec": 16266.308336905298, "vram_gb": 6.154519552, "lambda": 0.18283796838174263, "lambda_min": 0.08644991368055344, "lambda_max": 0.27264028787612915, "S_mean": 0.24919637044270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.050966739654541, "grad_norm": 0.6541616320610046, "tokens_seen": 23764992, "tokens_per_sec": 16267.983125168801, "vram_gb": 6.154519552, "lambda": 0.18338641648491225, "lambda_min": 0.08619807660579681, "lambda_max": 0.2740110754966736, "S_mean": 0.226806640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.022448539733887, "grad_norm": 0.7296541333198547, "tokens_seen": 23928832, "tokens_per_sec": 16269.024317356158, "vram_gb": 6.154519552, "lambda": 0.18394001200795174, "lambda_min": 0.08584124594926834, "lambda_max": 0.27572551369667053, "S_mean": 0.24702962239583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.098915338516235, "grad_norm": 0.7375298738479614, "tokens_seen": 24092672, "tokens_per_sec": 16270.663741712471, "vram_gb": 6.154519552, "lambda": 0.18452770542353392, "lambda_min": 0.08543279021978378, "lambda_max": 0.277200847864151, "S_mean": 0.24790445963541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.866302728652954, "grad_norm": 0.6553515195846558, "tokens_seen": 24256512, "tokens_per_sec": 16272.185981370349, "vram_gb": 6.154519552, "lambda": 0.18501598419000706, "lambda_min": 0.08505263179540634, "lambda_max": 0.2783004641532898, "S_mean": 0.24378458658854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.988217115402222, "grad_norm": 0.6678467392921448, "tokens_seen": 24420352, "tokens_per_sec": 16273.92226773651, "vram_gb": 6.154519552, "lambda": 0.185632831727465, "lambda_min": 0.08480916917324066, "lambda_max": 0.27983301877975464, "S_mean": 0.22962443033854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.981154203414917, "grad_norm": 0.6982645988464355, "tokens_seen": 24584192, "tokens_per_sec": 16275.664757769058, "vram_gb": 6.154519552, "lambda": 0.18626184264818826, "lambda_min": 0.08452845364809036, "lambda_max": 0.2814409136772156, "S_mean": 0.254150390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.055122709274292, "val_ppl": 156.82377289488795, "lambda": 0.1863005400324861, "lambda_min": 0.08451955020427704, "lambda_max": 0.28151461482048035, "lambda_per_layer": [0.15552696585655212, 0.1322023570537567, 0.138221874833107, 0.1346832662820816, 0.13972698152065277, 0.12911738455295563, 0.10396940261125565, 0.08451955020427704, 0.12305569648742676, 0.17693132162094116, 0.16828911006450653, 0.21381044387817383, 0.28151461482048035, 0.2814996540546417, 0.2282997965812683, 0.2716527581214905, 0.26259171962738037, 0.2654431164264679, 0.17836220562458038, 0.19781950116157532, 0.21640485525131226, 0.22181886434555054, 0.19288022816181183, 0.17287129163742065], "S_mean": 0.2684733072916667, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.0229973793029785, "grad_norm": 0.6846699714660645, "tokens_seen": 24748032, "tokens_per_sec": 16237.501833457769, "vram_gb": 6.154519552, "lambda": 0.1868814124415318, "lambda_min": 0.08444088697433472, "lambda_max": 0.28342217206954956, "S_mean": 0.23680623372395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.84332013130188, "grad_norm": 0.6447768807411194, "tokens_seen": 24911872, "tokens_per_sec": 16239.545517154002, "vram_gb": 6.154519552, "lambda": 0.18740633316338062, "lambda_min": 0.08426490426063538, "lambda_max": 0.2851227819919586, "S_mean": 0.24107869466145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.005768299102783, "grad_norm": 0.7173804044723511, "tokens_seen": 25075712, "tokens_per_sec": 16241.238333952255, "vram_gb": 6.154519552, "lambda": 0.18796476597587267, "lambda_min": 0.0837898924946785, "lambda_max": 0.28654035925865173, "S_mean": 0.24949137369791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.863328456878662, "grad_norm": 0.6838859915733337, "tokens_seen": 25239552, "tokens_per_sec": 16243.447863864536, "vram_gb": 6.154519552, "lambda": 0.18859058059751987, "lambda_min": 0.08350349217653275, "lambda_max": 0.28808683156967163, "S_mean": 0.24489339192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.0337042808532715, "grad_norm": 0.7404363751411438, "tokens_seen": 25403392, "tokens_per_sec": 16245.760020179116, "vram_gb": 6.154519552, "lambda": 0.18914883149166903, "lambda_min": 0.08317561447620392, "lambda_max": 0.2895698845386505, "S_mean": 0.267791748046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.990986347198486, "grad_norm": 0.6764127612113953, "tokens_seen": 25567232, "tokens_per_sec": 16247.945141881137, "vram_gb": 6.154519552, "lambda": 0.1896604880069693, "lambda_min": 0.0827595517039299, "lambda_max": 0.2910441756248474, "S_mean": 0.24626668294270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.9561684131622314, "grad_norm": 0.7415813207626343, "tokens_seen": 25731072, "tokens_per_sec": 16250.093174725684, "vram_gb": 6.154519552, "lambda": 0.19020988481740156, "lambda_min": 0.08228500932455063, "lambda_max": 0.29218345880508423, "S_mean": 0.2632242838541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.977159023284912, "grad_norm": 0.6979233026504517, "tokens_seen": 25894912, "tokens_per_sec": 16251.700810806642, "vram_gb": 6.154519552, "lambda": 0.19080132991075516, "lambda_min": 0.08195901662111282, "lambda_max": 0.29370588064193726, "S_mean": 0.240386962890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 5.010462284088135, "grad_norm": 0.7455146312713623, "tokens_seen": 26058752, "tokens_per_sec": 16252.70397028831, "vram_gb": 6.154519552, "lambda": 0.19126997329294682, "lambda_min": 0.08166109770536423, "lambda_max": 0.2946334183216095, "S_mean": 0.2517598470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.792485952377319, "grad_norm": 0.6806946992874146, "tokens_seen": 26222592, "tokens_per_sec": 16254.187448018223, "vram_gb": 6.154519552, "lambda": 0.19178635037193695, "lambda_min": 0.08128109574317932, "lambda_max": 0.29577070474624634, "S_mean": 0.2598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.005496025085449, "grad_norm": 0.8059168457984924, "tokens_seen": 26386432, "tokens_per_sec": 16255.847798270803, "vram_gb": 6.154519552, "lambda": 0.1923125380029281, "lambda_min": 0.08092275261878967, "lambda_max": 0.29705461859703064, "S_mean": 0.26043701171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.0382115840911865, "grad_norm": 0.7131288051605225, "tokens_seen": 26550272, "tokens_per_sec": 16257.682397991477, "vram_gb": 6.154519552, "lambda": 0.1928144202878078, "lambda_min": 0.08067617565393448, "lambda_max": 0.2983163595199585, "S_mean": 0.24405924479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.9081621170043945, "grad_norm": 0.719901442527771, "tokens_seen": 26714112, "tokens_per_sec": 16259.218010499559, "vram_gb": 6.154519552, "lambda": 0.19340320397168398, "lambda_min": 0.08055315911769867, "lambda_max": 0.2998802661895752, "S_mean": 0.24247233072916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.845202922821045, "grad_norm": 0.6827150583267212, "tokens_seen": 26877952, "tokens_per_sec": 16261.072338791048, "vram_gb": 6.154519552, "lambda": 0.19391346598664919, "lambda_min": 0.0801922008395195, "lambda_max": 0.30119940638542175, "S_mean": 0.23801676432291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.032770156860352, "grad_norm": 0.793328046798706, "tokens_seen": 27041792, "tokens_per_sec": 16262.705710385739, "vram_gb": 6.154519552, "lambda": 0.19451883838822445, "lambda_min": 0.07984878122806549, "lambda_max": 0.3026784658432007, "S_mean": 0.230712890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 5.060955047607422, "grad_norm": 0.7850750088691711, "tokens_seen": 27205632, "tokens_per_sec": 16264.532095859982, "vram_gb": 6.154519552, "lambda": 0.19497454476853213, "lambda_min": 0.07952515780925751, "lambda_max": 0.3036630153656006, "S_mean": 0.25909423828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.893962144851685, "grad_norm": 0.6661237478256226, "tokens_seen": 27369472, "tokens_per_sec": 16266.341284315093, "vram_gb": 6.154519552, "lambda": 0.19549941613028446, "lambda_min": 0.07928850501775742, "lambda_max": 0.3049240708351135, "S_mean": 0.223052978515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.805403709411621, "grad_norm": 0.6941295862197876, "tokens_seen": 27533312, "tokens_per_sec": 16268.063109165641, "vram_gb": 6.154519552, "lambda": 0.19609533064067364, "lambda_min": 0.07920796424150467, "lambda_max": 0.30651038885116577, "S_mean": 0.235198974609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.081403732299805, "grad_norm": 0.6616429686546326, "tokens_seen": 27697152, "tokens_per_sec": 16269.995808211972, "vram_gb": 6.154519552, "lambda": 0.1965920760606726, "lambda_min": 0.07897625118494034, "lambda_max": 0.30789420008659363, "S_mean": 0.24788411458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 5.16620945930481, "grad_norm": 0.6968934535980225, "tokens_seen": 27860992, "tokens_per_sec": 16271.323227934268, "vram_gb": 6.154519552, "lambda": 0.19719922076910734, "lambda_min": 0.0788862407207489, "lambda_max": 0.30912598967552185, "S_mean": 0.23552449544270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.051239967346191, "grad_norm": 0.7315935492515564, "tokens_seen": 28024832, "tokens_per_sec": 16272.96151950679, "vram_gb": 6.154519552, "lambda": 0.19779709385087094, "lambda_min": 0.0785118117928505, "lambda_max": 0.3107856810092926, "S_mean": 0.25341796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.868654727935791, "grad_norm": 0.7436133623123169, "tokens_seen": 28188672, "tokens_per_sec": 16274.027417342895, "vram_gb": 6.154519552, "lambda": 0.19838039732227722, "lambda_min": 0.07830723375082016, "lambda_max": 0.3121813237667084, "S_mean": 0.24049886067708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.989455938339233, "grad_norm": 0.731795072555542, "tokens_seen": 28352512, "tokens_per_sec": 16275.218815836244, "vram_gb": 6.154519552, "lambda": 0.19891245880474648, "lambda_min": 0.07808849960565567, "lambda_max": 0.3134089410305023, "S_mean": 0.24615478515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.890238046646118, "grad_norm": 0.7322102785110474, "tokens_seen": 28516352, "tokens_per_sec": 16276.592055851637, "vram_gb": 6.154519552, "lambda": 0.19942132538805404, "lambda_min": 0.07794693857431412, "lambda_max": 0.3150203227996826, "S_mean": 0.24952189127604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.895116806030273, "grad_norm": 0.7262390851974487, "tokens_seen": 28680192, "tokens_per_sec": 16278.237354472898, "vram_gb": 6.154519552, "lambda": 0.1999991238117218, "lambda_min": 0.07789339870214462, "lambda_max": 0.31622833013534546, "S_mean": 0.2686564127604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3500, "val_loss": 4.996367776393891, "val_ppl": 147.87506714936364, "lambda": 0.2000302088757356, "lambda_min": 0.0778772160410881, "lambda_max": 0.3163028657436371, "lambda_per_layer": [0.15747515857219696, 0.1320183277130127, 0.13939286768436432, 0.13629448413848877, 0.14326684176921844, 0.13057374954223633, 0.10138188302516937, 0.0778772160410881, 0.12223594635725021, 0.18337751924991608, 0.17616724967956543, 0.22721950709819794, 0.3163028657436371, 0.314935564994812, 0.2520880699157715, 0.30677881836891174, 0.2912578284740448, 0.30479180812835693, 0.19147372245788574, 0.21808430552482605, 0.24082471430301666, 0.24791745841503143, 0.2079491913318634, 0.18103991448879242], "S_mean": 0.248138427734375, "G_mean": null, "G_std": null, "tokens_seen": 28680192} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.840384006500244, "grad_norm": 0.6971070170402527, "tokens_seen": 28844032, "tokens_per_sec": 16245.794598924278, "vram_gb": 6.154519552, "lambda": 0.20054473169147968, "lambda_min": 0.07752298563718796, "lambda_max": 0.31747835874557495, "S_mean": 0.250152587890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.955134630203247, "grad_norm": 0.7271576523780823, "tokens_seen": 29007872, "tokens_per_sec": 16246.564285017028, "vram_gb": 6.154519552, "lambda": 0.20109170333792767, "lambda_min": 0.07726453989744186, "lambda_max": 0.3190154731273651, "S_mean": 0.22868855794270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.934858322143555, "grad_norm": 0.7620800733566284, "tokens_seen": 29171712, "tokens_per_sec": 16247.202878249398, "vram_gb": 6.154519552, "lambda": 0.20170255781461796, "lambda_min": 0.07703029364347458, "lambda_max": 0.3205130696296692, "S_mean": 0.22832234700520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.983853340148926, "grad_norm": 0.7271220088005066, "tokens_seen": 29335552, "tokens_per_sec": 16248.17182179955, "vram_gb": 6.154519552, "lambda": 0.20238710846751928, "lambda_min": 0.07701568305492401, "lambda_max": 0.3220974802970886, "S_mean": 0.25311279296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 5.049828767776489, "grad_norm": 0.780859649181366, "tokens_seen": 29499392, "tokens_per_sec": 16249.198322030099, "vram_gb": 6.154519552, "lambda": 0.20292103787263235, "lambda_min": 0.07694470137357712, "lambda_max": 0.32353484630584717, "S_mean": 0.23094685872395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.922071933746338, "grad_norm": 0.7196110486984253, "tokens_seen": 29663232, "tokens_per_sec": 16249.830525800025, "vram_gb": 6.154519552, "lambda": 0.20331392840792736, "lambda_min": 0.07695632427930832, "lambda_max": 0.32451027631759644, "S_mean": 0.2519632975260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.799077272415161, "grad_norm": 0.7211406230926514, "tokens_seen": 29827072, "tokens_per_sec": 16250.133383151911, "vram_gb": 6.154519552, "lambda": 0.20394288407017788, "lambda_min": 0.07689084857702255, "lambda_max": 0.3260304033756256, "S_mean": 0.2432861328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.745235919952393, "grad_norm": 0.7229773998260498, "tokens_seen": 29990912, "tokens_per_sec": 16250.648716490809, "vram_gb": 6.154519552, "lambda": 0.20456965640187263, "lambda_min": 0.07671806216239929, "lambda_max": 0.3274499177932739, "S_mean": 0.24737548828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.751861095428467, "grad_norm": 0.7826448082923889, "tokens_seen": 30154752, "tokens_per_sec": 16251.142430785958, "vram_gb": 6.154519552, "lambda": 0.20514997405310473, "lambda_min": 0.07656002044677734, "lambda_max": 0.3286992013454437, "S_mean": 0.24216715494791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.966046333312988, "grad_norm": 0.7341580986976624, "tokens_seen": 30318592, "tokens_per_sec": 16251.559308921323, "vram_gb": 6.154519552, "lambda": 0.20566142257303, "lambda_min": 0.0766485258936882, "lambda_max": 0.3300359547138214, "S_mean": 0.254302978515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.916021823883057, "grad_norm": 0.8195959329605103, "tokens_seen": 30482432, "tokens_per_sec": 16252.351888429464, "vram_gb": 6.154519552, "lambda": 0.20612394716590643, "lambda_min": 0.07654007524251938, "lambda_max": 0.3313526213169098, "S_mean": 0.258270263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.902668476104736, "grad_norm": 0.7260521650314331, "tokens_seen": 30646272, "tokens_per_sec": 16252.75982101426, "vram_gb": 6.154519552, "lambda": 0.20660605188459158, "lambda_min": 0.07628419250249863, "lambda_max": 0.332491010427475, "S_mean": 0.24238077799479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.883121490478516, "grad_norm": 0.7686042189598083, "tokens_seen": 30810112, "tokens_per_sec": 16253.491142844648, "vram_gb": 6.154519552, "lambda": 0.20713374329109988, "lambda_min": 0.07606866210699081, "lambda_max": 0.33373725414276123, "S_mean": 0.2537943522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.79516339302063, "grad_norm": 0.7509250640869141, "tokens_seen": 30973952, "tokens_per_sec": 16253.801867144333, "vram_gb": 6.154519552, "lambda": 0.2076346476872762, "lambda_min": 0.07584746181964874, "lambda_max": 0.33512717485427856, "S_mean": 0.24922688802083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.794163227081299, "grad_norm": 0.7525796890258789, "tokens_seen": 31137792, "tokens_per_sec": 16254.319711492306, "vram_gb": 6.154519552, "lambda": 0.20815570993969837, "lambda_min": 0.07569745928049088, "lambda_max": 0.33628004789352417, "S_mean": 0.259979248046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.868371963500977, "grad_norm": 0.6851819157600403, "tokens_seen": 31301632, "tokens_per_sec": 16254.863942873022, "vram_gb": 6.154519552, "lambda": 0.2086173960318168, "lambda_min": 0.07544635981321335, "lambda_max": 0.33667701482772827, "S_mean": 0.24591064453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.944833040237427, "grad_norm": 0.7311252951622009, "tokens_seen": 31465472, "tokens_per_sec": 16255.553439598198, "vram_gb": 6.154519552, "lambda": 0.20917892704407373, "lambda_min": 0.07534705102443695, "lambda_max": 0.33803874254226685, "S_mean": 0.2559000651041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.827237606048584, "grad_norm": 0.7734097242355347, "tokens_seen": 31629312, "tokens_per_sec": 16255.461513753124, "vram_gb": 6.154519552, "lambda": 0.20966031743834415, "lambda_min": 0.07521450519561768, "lambda_max": 0.3389441668987274, "S_mean": 0.26190185546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.6796042919158936, "grad_norm": 0.6952609419822693, "tokens_seen": 31793152, "tokens_per_sec": 16256.04846922244, "vram_gb": 6.154519552, "lambda": 0.21018888366719088, "lambda_min": 0.07493285089731216, "lambda_max": 0.34017810225486755, "S_mean": 0.2514139811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.803133010864258, "grad_norm": 0.8177988529205322, "tokens_seen": 31956992, "tokens_per_sec": 16256.516209526999, "vram_gb": 6.154519552, "lambda": 0.21072336720923582, "lambda_min": 0.07474512606859207, "lambda_max": 0.34140318632125854, "S_mean": 0.2581787109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.922438144683838, "grad_norm": 0.846493661403656, "tokens_seen": 32120832, "tokens_per_sec": 16256.810986576127, "vram_gb": 6.154519552, "lambda": 0.2112676159789165, "lambda_min": 0.07440361380577087, "lambda_max": 0.34270381927490234, "S_mean": 0.244964599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.755136251449585, "grad_norm": 0.7266095876693726, "tokens_seen": 32284672, "tokens_per_sec": 16256.952217774891, "vram_gb": 6.154519552, "lambda": 0.21177825517952442, "lambda_min": 0.0741056427359581, "lambda_max": 0.3439673185348511, "S_mean": 0.258392333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.985456943511963, "grad_norm": 0.74531090259552, "tokens_seen": 32448512, "tokens_per_sec": 16256.79790555019, "vram_gb": 6.154519552, "lambda": 0.21231603622436523, "lambda_min": 0.07386016845703125, "lambda_max": 0.34559309482574463, "S_mean": 0.24173990885416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 5.029740810394287, "grad_norm": 0.7819800972938538, "tokens_seen": 32612352, "tokens_per_sec": 16257.324861006637, "vram_gb": 6.154519552, "lambda": 0.21281759782383838, "lambda_min": 0.07349838316440582, "lambda_max": 0.3470463752746582, "S_mean": 0.2579752604166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.9292097091674805, "grad_norm": 0.7587428092956543, "tokens_seen": 32776192, "tokens_per_sec": 16258.055291775803, "vram_gb": 6.154519552, "lambda": 0.2132228616004189, "lambda_min": 0.07330727577209473, "lambda_max": 0.34811991453170776, "S_mean": 0.24440511067708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.864401817321777, "val_ppl": 129.59339489742817, "lambda": 0.21324207385381064, "lambda_min": 0.0733039379119873, "lambda_max": 0.34815794229507446, "lambda_per_layer": [0.1594613641500473, 0.13207955658435822, 0.14032410085201263, 0.1375948041677475, 0.14625485241413116, 0.13124704360961914, 0.09916602820158005, 0.0733039379119873, 0.12161853164434433, 0.18757064640522003, 0.1828758865594864, 0.23859137296676636, 0.34815794229507446, 0.345590740442276, 0.2758559584617615, 0.34354859590530396, 0.3192957043647766, 0.3438606858253479, 0.20630715787410736, 0.23730316758155823, 0.26238420605659485, 0.27338674664497375, 0.22299076616764069, 0.18903997540473938], "S_mean": 0.2646382649739583, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.577678203582764, "grad_norm": 0.691815972328186, "tokens_seen": 32940032, "tokens_per_sec": 16228.467211764146, "vram_gb": 6.154519552, "lambda": 0.2137232068926096, "lambda_min": 0.07321333140134811, "lambda_max": 0.348874568939209, "S_mean": 0.24029541015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.857381343841553, "grad_norm": 0.7903532981872559, "tokens_seen": 33103872, "tokens_per_sec": 16229.359079711896, "vram_gb": 6.154519552, "lambda": 0.21418657122800747, "lambda_min": 0.07303395867347717, "lambda_max": 0.34965264797210693, "S_mean": 0.25128173828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.917026042938232, "grad_norm": 0.7820358276367188, "tokens_seen": 33267712, "tokens_per_sec": 16230.319580521797, "vram_gb": 6.154519552, "lambda": 0.21473546574513117, "lambda_min": 0.07287885993719101, "lambda_max": 0.35088181495666504, "S_mean": 0.23343912760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.720263242721558, "grad_norm": 0.7710794806480408, "tokens_seen": 33431552, "tokens_per_sec": 16231.391042225316, "vram_gb": 6.154519552, "lambda": 0.2152970445652803, "lambda_min": 0.07266461104154587, "lambda_max": 0.3518112599849701, "S_mean": 0.2371826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.841686248779297, "grad_norm": 0.782950222492218, "tokens_seen": 33595392, "tokens_per_sec": 16232.160604355433, "vram_gb": 6.154519552, "lambda": 0.21576747857034206, "lambda_min": 0.07258330285549164, "lambda_max": 0.3525983691215515, "S_mean": 0.24863688151041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.8390793800354, "grad_norm": 0.7871049642562866, "tokens_seen": 33759232, "tokens_per_sec": 16232.664737227584, "vram_gb": 6.154519552, "lambda": 0.2162958082432548, "lambda_min": 0.07256065309047699, "lambda_max": 0.3534393906593323, "S_mean": 0.2603352864583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.911729335784912, "grad_norm": 0.7533875703811646, "tokens_seen": 33923072, "tokens_per_sec": 16233.044716091175, "vram_gb": 6.154519552, "lambda": 0.21682898017267385, "lambda_min": 0.07239784300327301, "lambda_max": 0.35476741194725037, "S_mean": 0.24961344401041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.741734504699707, "grad_norm": 0.7474606037139893, "tokens_seen": 34086912, "tokens_per_sec": 16233.380402512572, "vram_gb": 6.154519552, "lambda": 0.21729009101788202, "lambda_min": 0.07219776511192322, "lambda_max": 0.3558574616909027, "S_mean": 0.23454793294270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.98402738571167, "grad_norm": 0.7860008478164673, "tokens_seen": 34250752, "tokens_per_sec": 16234.207077402323, "vram_gb": 6.154519552, "lambda": 0.21780867657313743, "lambda_min": 0.07211277633905411, "lambda_max": 0.3569672703742981, "S_mean": 0.239410400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.692324638366699, "grad_norm": 0.7673478722572327, "tokens_seen": 34414592, "tokens_per_sec": 16234.808189590634, "vram_gb": 6.154519552, "lambda": 0.21833004957685867, "lambda_min": 0.07196816802024841, "lambda_max": 0.3583115041255951, "S_mean": 0.2713216145833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.758995532989502, "grad_norm": 0.7472672462463379, "tokens_seen": 34578432, "tokens_per_sec": 16235.498633070674, "vram_gb": 6.154519552, "lambda": 0.21881539188325405, "lambda_min": 0.07178131490945816, "lambda_max": 0.3599940538406372, "S_mean": 0.24808756510416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.719583749771118, "grad_norm": 0.7206050157546997, "tokens_seen": 34742272, "tokens_per_sec": 16236.116872840796, "vram_gb": 6.154519552, "lambda": 0.21940650790929794, "lambda_min": 0.07154162228107452, "lambda_max": 0.3618519604206085, "S_mean": 0.24169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.743911266326904, "grad_norm": 0.7293166518211365, "tokens_seen": 34906112, "tokens_per_sec": 16236.940060635638, "vram_gb": 6.154519552, "lambda": 0.21987265317390361, "lambda_min": 0.07125705480575562, "lambda_max": 0.3633803427219391, "S_mean": 0.24998982747395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.870424747467041, "grad_norm": 0.8097853660583496, "tokens_seen": 35069952, "tokens_per_sec": 16237.671245582924, "vram_gb": 6.154519552, "lambda": 0.2204120603079597, "lambda_min": 0.07104115188121796, "lambda_max": 0.3648662865161896, "S_mean": 0.22956339518229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.937473297119141, "grad_norm": 0.7357210516929626, "tokens_seen": 35233792, "tokens_per_sec": 16238.228879625854, "vram_gb": 6.154519552, "lambda": 0.22104367365439734, "lambda_min": 0.07102549076080322, "lambda_max": 0.36674439907073975, "S_mean": 0.23555501302083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.719420671463013, "grad_norm": 0.7278839349746704, "tokens_seen": 35397632, "tokens_per_sec": 16238.713653680878, "vram_gb": 6.154519552, "lambda": 0.221507434733212, "lambda_min": 0.07101048529148102, "lambda_max": 0.3682144284248352, "S_mean": 0.24973551432291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.774783134460449, "grad_norm": 0.7479718327522278, "tokens_seen": 35561472, "tokens_per_sec": 16239.220739666272, "vram_gb": 6.154519552, "lambda": 0.22193144106616577, "lambda_min": 0.07100722193717957, "lambda_max": 0.36938634514808655, "S_mean": 0.2506917317708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.851709365844727, "grad_norm": 0.7840186357498169, "tokens_seen": 35725312, "tokens_per_sec": 16239.60047771467, "vram_gb": 6.154519552, "lambda": 0.22243090563764176, "lambda_min": 0.07100442796945572, "lambda_max": 0.37068015336990356, "S_mean": 0.2664591471354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.713218688964844, "grad_norm": 0.7585341930389404, "tokens_seen": 35889152, "tokens_per_sec": 16239.872004451046, "vram_gb": 6.154519552, "lambda": 0.22293827403336763, "lambda_min": 0.07094036787748337, "lambda_max": 0.3717705309391022, "S_mean": 0.246337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.8308610916137695, "grad_norm": 0.7723831534385681, "tokens_seen": 36052992, "tokens_per_sec": 16240.416900358332, "vram_gb": 6.154519552, "lambda": 0.2234069760888815, "lambda_min": 0.07086686789989471, "lambda_max": 0.37296271324157715, "S_mean": 0.24135335286458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.81393837928772, "grad_norm": 0.8020808100700378, "tokens_seen": 36216832, "tokens_per_sec": 16241.202366963822, "vram_gb": 6.154519552, "lambda": 0.22393617716928324, "lambda_min": 0.07068101316690445, "lambda_max": 0.3743850290775299, "S_mean": 0.24834187825520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.719849586486816, "grad_norm": 0.7923833727836609, "tokens_seen": 36380672, "tokens_per_sec": 16241.834439500255, "vram_gb": 6.154519552, "lambda": 0.22445128268251816, "lambda_min": 0.07057821750640869, "lambda_max": 0.3758847713470459, "S_mean": 0.2508443196614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.536275863647461, "grad_norm": 0.7572776079177856, "tokens_seen": 36544512, "tokens_per_sec": 16242.513326664384, "vram_gb": 6.154519552, "lambda": 0.22499836919208369, "lambda_min": 0.07049476355314255, "lambda_max": 0.37727221846580505, "S_mean": 0.2435302734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.683795690536499, "grad_norm": 0.7593286633491516, "tokens_seen": 36708352, "tokens_per_sec": 16243.358064839284, "vram_gb": 6.154519552, "lambda": 0.22555278396854797, "lambda_min": 0.07026432454586029, "lambda_max": 0.3787287771701813, "S_mean": 0.2583821614583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.9711737632751465, "grad_norm": 0.7699804306030273, "tokens_seen": 36872192, "tokens_per_sec": 16243.887518019612, "vram_gb": 6.154519552, "lambda": 0.22603931122769913, "lambda_min": 0.06993934512138367, "lambda_max": 0.3802622854709625, "S_mean": 0.24249267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4500, "val_loss": 4.7724477529525755, "val_ppl": 118.20823267799565, "lambda": 0.22606702676663795, "lambda_min": 0.06992577016353607, "lambda_max": 0.3802940845489502, "lambda_per_layer": [0.16115140914916992, 0.13226322829723358, 0.14080588519573212, 0.13918431103229523, 0.1485854983329773, 0.13257060945034027, 0.09785889834165573, 0.06992577016353607, 0.12108935415744781, 0.19317848980426788, 0.18888013064861298, 0.2485523670911789, 0.37741512060165405, 0.3764360845088959, 0.29714035987854004, 0.3790624737739563, 0.3448445796966553, 0.3802940845489502, 0.21928401291370392, 0.2591095566749573, 0.28593558073043823, 0.30157312750816345, 0.23360782861709595, 0.19685988128185272], "S_mean": 0.247528076171875, "G_mean": null, "G_std": null, "tokens_seen": 36872192} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.832908391952515, "grad_norm": 0.7515859007835388, "tokens_seen": 37036032, "tokens_per_sec": 16217.65097584657, "vram_gb": 6.154519552, "lambda": 0.22653679456561804, "lambda_min": 0.0698089599609375, "lambda_max": 0.3813716173171997, "S_mean": 0.24830118815104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.80275297164917, "grad_norm": 0.7629834413528442, "tokens_seen": 37199872, "tokens_per_sec": 16218.545411391526, "vram_gb": 6.154519552, "lambda": 0.22691284585744143, "lambda_min": 0.06959357112646103, "lambda_max": 0.3826881945133209, "S_mean": 0.2527974446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.6709935665130615, "grad_norm": 0.7680772542953491, "tokens_seen": 37363712, "tokens_per_sec": 16219.351941379435, "vram_gb": 6.154519552, "lambda": 0.22744173867007098, "lambda_min": 0.06949552893638611, "lambda_max": 0.3843936622142792, "S_mean": 0.2528177897135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.836400032043457, "grad_norm": 0.7570936679840088, "tokens_seen": 37527552, "tokens_per_sec": 16219.982747840439, "vram_gb": 6.154519552, "lambda": 0.22795684480418762, "lambda_min": 0.06940599530935287, "lambda_max": 0.38581669330596924, "S_mean": 0.242279052734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.68128228187561, "grad_norm": 0.8114765882492065, "tokens_seen": 37691392, "tokens_per_sec": 16220.381290749774, "vram_gb": 6.154519552, "lambda": 0.22850086353719234, "lambda_min": 0.06930039077997208, "lambda_max": 0.3877165615558624, "S_mean": 0.24397786458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.685129404067993, "grad_norm": 0.9161442518234253, "tokens_seen": 37855232, "tokens_per_sec": 16220.989954108316, "vram_gb": 6.154519552, "lambda": 0.22903884605815014, "lambda_min": 0.06932996213436127, "lambda_max": 0.38937944173812866, "S_mean": 0.24955240885416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.845596790313721, "grad_norm": 0.7646124362945557, "tokens_seen": 38019072, "tokens_per_sec": 16221.779658895915, "vram_gb": 6.154519552, "lambda": 0.22954097048689923, "lambda_min": 0.0692199170589447, "lambda_max": 0.391278475522995, "S_mean": 0.24372355143229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.724863529205322, "grad_norm": 0.7711188197135925, "tokens_seen": 38182912, "tokens_per_sec": 16222.451129299112, "vram_gb": 6.154519552, "lambda": 0.2300376029064258, "lambda_min": 0.06902633607387543, "lambda_max": 0.3930031955242157, "S_mean": 0.24864705403645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.826744556427002, "grad_norm": 0.7846159934997559, "tokens_seen": 38346752, "tokens_per_sec": 16222.842431026482, "vram_gb": 6.154519552, "lambda": 0.23048847634345293, "lambda_min": 0.06883736699819565, "lambda_max": 0.39473235607147217, "S_mean": 0.24761962890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.774476528167725, "grad_norm": 0.8471785187721252, "tokens_seen": 38510592, "tokens_per_sec": 16223.241987544345, "vram_gb": 6.154519552, "lambda": 0.2309194632495443, "lambda_min": 0.06878989189863205, "lambda_max": 0.39619016647338867, "S_mean": 0.2618815104166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.795904636383057, "grad_norm": 0.8862677216529846, "tokens_seen": 38674432, "tokens_per_sec": 16223.816961910557, "vram_gb": 6.154519552, "lambda": 0.23139036322633424, "lambda_min": 0.06880854070186615, "lambda_max": 0.39770010113716125, "S_mean": 0.24602254231770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.749355792999268, "grad_norm": 0.8370123505592346, "tokens_seen": 38838272, "tokens_per_sec": 16224.497859564359, "vram_gb": 6.154519552, "lambda": 0.23179769857476154, "lambda_min": 0.06869062036275864, "lambda_max": 0.39871546626091003, "S_mean": 0.2589619954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.83586859703064, "grad_norm": 0.8039195537567139, "tokens_seen": 39002112, "tokens_per_sec": 16225.341305999144, "vram_gb": 6.154519552, "lambda": 0.23223973655452332, "lambda_min": 0.06858791410923004, "lambda_max": 0.4002397060394287, "S_mean": 0.2537027994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.621494770050049, "grad_norm": 0.8970749378204346, "tokens_seen": 39165952, "tokens_per_sec": 16226.101649245682, "vram_gb": 6.154519552, "lambda": 0.23278148161868253, "lambda_min": 0.06842991709709167, "lambda_max": 0.4016866981983185, "S_mean": 0.2658284505208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.588753700256348, "grad_norm": 0.765558660030365, "tokens_seen": 39329792, "tokens_per_sec": 16226.67368985849, "vram_gb": 6.154519552, "lambda": 0.23327693218986192, "lambda_min": 0.06831780821084976, "lambda_max": 0.40328750014305115, "S_mean": 0.2681681315104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.760311841964722, "grad_norm": 0.7353352904319763, "tokens_seen": 39493632, "tokens_per_sec": 16227.149684695436, "vram_gb": 6.154519552, "lambda": 0.23378719513614973, "lambda_min": 0.06813644617795944, "lambda_max": 0.40513119101524353, "S_mean": 0.258331298828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.774720668792725, "grad_norm": 0.8377525806427002, "tokens_seen": 39657472, "tokens_per_sec": 16227.560738744845, "vram_gb": 6.154519552, "lambda": 0.2342853986968597, "lambda_min": 0.0680248811841011, "lambda_max": 0.40712347626686096, "S_mean": 0.2547098795572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.848317623138428, "grad_norm": 0.7616749405860901, "tokens_seen": 39821312, "tokens_per_sec": 16227.810270712833, "vram_gb": 6.154519552, "lambda": 0.2348183145125707, "lambda_min": 0.06787098944187164, "lambda_max": 0.4087322950363159, "S_mean": 0.251495361328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.615990400314331, "grad_norm": 0.8031917810440063, "tokens_seen": 39985152, "tokens_per_sec": 16228.264101485229, "vram_gb": 6.154519552, "lambda": 0.23521187342703342, "lambda_min": 0.06775777786970139, "lambda_max": 0.40992701053619385, "S_mean": 0.24626668294270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.734536170959473, "grad_norm": 0.7724632620811462, "tokens_seen": 40148992, "tokens_per_sec": 16228.417199418245, "vram_gb": 6.154519552, "lambda": 0.23574173512558141, "lambda_min": 0.06774501502513885, "lambda_max": 0.4113864600658417, "S_mean": 0.245452880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.761580228805542, "grad_norm": 0.8012819290161133, "tokens_seen": 40312832, "tokens_per_sec": 16228.538023660483, "vram_gb": 6.154519552, "lambda": 0.23624510938922563, "lambda_min": 0.0676555261015892, "lambda_max": 0.4129214286804199, "S_mean": 0.253570556640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.523453950881958, "grad_norm": 0.8250341415405273, "tokens_seen": 40476672, "tokens_per_sec": 16229.40712546211, "vram_gb": 6.154519552, "lambda": 0.236639233926932, "lambda_min": 0.06750230491161346, "lambda_max": 0.4145740270614624, "S_mean": 0.2523295084635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.7172768115997314, "grad_norm": 0.9194927215576172, "tokens_seen": 40640512, "tokens_per_sec": 16230.059802521066, "vram_gb": 6.154519552, "lambda": 0.23703490725407997, "lambda_min": 0.067384272813797, "lambda_max": 0.41557061672210693, "S_mean": 0.2625732421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.8505706787109375, "grad_norm": 0.8272122144699097, "tokens_seen": 40804352, "tokens_per_sec": 16230.90208424246, "vram_gb": 6.154519552, "lambda": 0.23747386752317348, "lambda_min": 0.06724369525909424, "lambda_max": 0.41735273599624634, "S_mean": 0.25372314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.730246305465698, "grad_norm": 0.8174875378608704, "tokens_seen": 40968192, "tokens_per_sec": 16231.550541639697, "vram_gb": 6.154519552, "lambda": 0.237977827899158, "lambda_min": 0.06709271669387817, "lambda_max": 0.4181760251522064, "S_mean": 0.2564290364583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.744834578037262, "val_ppl": 114.98878225966733, "lambda": 0.23800618201494217, "lambda_min": 0.06708510220050812, "lambda_max": 0.4182511866092682, "lambda_per_layer": [0.16241435706615448, 0.13229188323020935, 0.1414710432291031, 0.14069072902202606, 0.15094155073165894, 0.13361331820487976, 0.09751665592193604, 0.06708510220050812, 0.12096431851387024, 0.19690538942813873, 0.19381439685821533, 0.25771909952163696, 0.4026379883289337, 0.40654265880584717, 0.31570401787757874, 0.41370686888694763, 0.3679419755935669, 0.4182511866092682, 0.23186282813549042, 0.2762187719345093, 0.3064519464969635, 0.3246179223060608, 0.24781925976276398, 0.20496509969234467], "S_mean": 0.2551472981770833, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.681543827056885, "grad_norm": 0.9227752685546875, "tokens_seen": 41132032, "tokens_per_sec": 16208.261963654277, "vram_gb": 6.154519552, "lambda": 0.23852366767823696, "lambda_min": 0.06696979701519012, "lambda_max": 0.4197487533092499, "S_mean": 0.2544453938802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.623260498046875, "grad_norm": 0.789301872253418, "tokens_seen": 41295872, "tokens_per_sec": 16209.015126522801, "vram_gb": 6.154519552, "lambda": 0.23905244500686726, "lambda_min": 0.06681132316589355, "lambda_max": 0.4210064709186554, "S_mean": 0.2630615234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.760819435119629, "grad_norm": 0.7716865539550781, "tokens_seen": 41459712, "tokens_per_sec": 16209.78280604341, "vram_gb": 6.154519552, "lambda": 0.2395200508957108, "lambda_min": 0.06664618849754333, "lambda_max": 0.42231911420822144, "S_mean": 0.2533467610677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.752926826477051, "grad_norm": 0.8126078844070435, "tokens_seen": 41623552, "tokens_per_sec": 16210.471931645872, "vram_gb": 6.154519552, "lambda": 0.2399194591368238, "lambda_min": 0.06662292033433914, "lambda_max": 0.42350584268569946, "S_mean": 0.2581990559895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.645122051239014, "grad_norm": 0.8521847128868103, "tokens_seen": 41787392, "tokens_per_sec": 16211.186981737012, "vram_gb": 6.154519552, "lambda": 0.24033786139140526, "lambda_min": 0.06661580502986908, "lambda_max": 0.42472201585769653, "S_mean": 0.2609761555989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.765066862106323, "grad_norm": 0.8067538738250732, "tokens_seen": 41951232, "tokens_per_sec": 16211.885507854073, "vram_gb": 6.154519552, "lambda": 0.24083295185118914, "lambda_min": 0.06656888127326965, "lambda_max": 0.42605262994766235, "S_mean": 0.248138427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.7321717739105225, "grad_norm": 0.7954753637313843, "tokens_seen": 42115072, "tokens_per_sec": 16212.414066610589, "vram_gb": 6.154519552, "lambda": 0.24127883557230234, "lambda_min": 0.06636397540569305, "lambda_max": 0.4271916449069977, "S_mean": 0.254180908203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.739030599594116, "grad_norm": 0.8610605597496033, "tokens_seen": 42278912, "tokens_per_sec": 16212.90675547423, "vram_gb": 6.154519552, "lambda": 0.24180774359653393, "lambda_min": 0.06622723489999771, "lambda_max": 0.4284364879131317, "S_mean": 0.259307861328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.799110412597656, "grad_norm": 0.949059784412384, "tokens_seen": 42442752, "tokens_per_sec": 16213.423982438344, "vram_gb": 6.154519552, "lambda": 0.24235093065847954, "lambda_min": 0.06613966822624207, "lambda_max": 0.4300011396408081, "S_mean": 0.28009033203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.737751007080078, "grad_norm": 0.8235440850257874, "tokens_seen": 42606592, "tokens_per_sec": 16214.044105932273, "vram_gb": 6.154519552, "lambda": 0.24274292991807064, "lambda_min": 0.06603624671697617, "lambda_max": 0.43115362524986267, "S_mean": 0.2523600260416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.551467180252075, "grad_norm": 0.8125039935112, "tokens_seen": 42770432, "tokens_per_sec": 16214.604951381561, "vram_gb": 6.154519552, "lambda": 0.24322882946580648, "lambda_min": 0.06602735072374344, "lambda_max": 0.4325566291809082, "S_mean": 0.2628173828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.887998580932617, "grad_norm": 0.8841028213500977, "tokens_seen": 42934272, "tokens_per_sec": 16215.182187249418, "vram_gb": 6.154519552, "lambda": 0.24372390968104204, "lambda_min": 0.06597550958395004, "lambda_max": 0.43411681056022644, "S_mean": 0.23179117838541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.633621692657471, "grad_norm": 0.8049760460853577, "tokens_seen": 43098112, "tokens_per_sec": 16215.976739285847, "vram_gb": 6.154519552, "lambda": 0.24423549976199865, "lambda_min": 0.06589848548173904, "lambda_max": 0.4359673261642456, "S_mean": 0.260467529296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.532226324081421, "grad_norm": 0.7999568581581116, "tokens_seen": 43261952, "tokens_per_sec": 16216.575435700031, "vram_gb": 6.154519552, "lambda": 0.24468839230636755, "lambda_min": 0.06572525203227997, "lambda_max": 0.43753913044929504, "S_mean": 0.24824015299479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.665964126586914, "grad_norm": 0.8219097852706909, "tokens_seen": 43425792, "tokens_per_sec": 16217.08393759905, "vram_gb": 6.154519552, "lambda": 0.24515169827888408, "lambda_min": 0.06559553742408752, "lambda_max": 0.43905404210090637, "S_mean": 0.24825032552083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.6418616771698, "grad_norm": 0.8535506129264832, "tokens_seen": 43589632, "tokens_per_sec": 16217.740110583081, "vram_gb": 6.154519552, "lambda": 0.2455571498721838, "lambda_min": 0.06567837297916412, "lambda_max": 0.4401669204235077, "S_mean": 0.256622314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.744360446929932, "grad_norm": 0.8254343867301941, "tokens_seen": 43753472, "tokens_per_sec": 16218.250040398974, "vram_gb": 6.154519552, "lambda": 0.2460312486315767, "lambda_min": 0.06556420773267746, "lambda_max": 0.4415265619754791, "S_mean": 0.2561136881510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.500592470169067, "grad_norm": 0.7671940326690674, "tokens_seen": 43917312, "tokens_per_sec": 16218.798208715418, "vram_gb": 6.154519552, "lambda": 0.24649443663656712, "lambda_min": 0.06546778976917267, "lambda_max": 0.4428494870662689, "S_mean": 0.255889892578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.547616481781006, "grad_norm": 0.8758147954940796, "tokens_seen": 44081152, "tokens_per_sec": 16219.416021948577, "vram_gb": 6.154519552, "lambda": 0.2468827199190855, "lambda_min": 0.06532659381628036, "lambda_max": 0.4441666007041931, "S_mean": 0.260955810546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.56443977355957, "grad_norm": 0.793893039226532, "tokens_seen": 44244992, "tokens_per_sec": 16220.153538727753, "vram_gb": 6.154519552, "lambda": 0.24734110354135433, "lambda_min": 0.06515397131443024, "lambda_max": 0.445615291595459, "S_mean": 0.2655537923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.64199686050415, "grad_norm": 0.7834300398826599, "tokens_seen": 44408832, "tokens_per_sec": 16220.745583480553, "vram_gb": 6.154519552, "lambda": 0.24781132334222397, "lambda_min": 0.06506958603858948, "lambda_max": 0.44686758518218994, "S_mean": 0.249359130859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.780999183654785, "grad_norm": 0.8301006555557251, "tokens_seen": 44572672, "tokens_per_sec": 16221.505544721243, "vram_gb": 6.154519552, "lambda": 0.24836311272035042, "lambda_min": 0.06508789956569672, "lambda_max": 0.44800466299057007, "S_mean": 0.2576700846354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.663688898086548, "grad_norm": 0.8443921804428101, "tokens_seen": 44736512, "tokens_per_sec": 16222.06232955593, "vram_gb": 6.154519552, "lambda": 0.24892379343509674, "lambda_min": 0.06495321542024612, "lambda_max": 0.4496225118637085, "S_mean": 0.2538045247395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.667542457580566, "grad_norm": 0.8118110299110413, "tokens_seen": 44900352, "tokens_per_sec": 16222.745473481862, "vram_gb": 6.154519552, "lambda": 0.2494250281403462, "lambda_min": 0.06485599279403687, "lambda_max": 0.4509666860103607, "S_mean": 0.26593017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.646533489227295, "grad_norm": 0.8413621783256531, "tokens_seen": 45064192, "tokens_per_sec": 16223.491130382585, "vram_gb": 6.154519552, "lambda": 0.24991634860634804, "lambda_min": 0.06472979485988617, "lambda_max": 0.4528164565563202, "S_mean": 0.2559102376302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5500, "val_loss": 4.683860051631927, "val_ppl": 108.18687453692733, "lambda": 0.24993297768135866, "lambda_min": 0.06472418457269669, "lambda_max": 0.45288389921188354, "lambda_per_layer": [0.16385726630687714, 0.13207106292247772, 0.14201706647872925, 0.14241550862789154, 0.15406624972820282, 0.1341884285211563, 0.09652497619390488, 0.06472418457269669, 0.1207667887210846, 0.20015621185302734, 0.19936655461788177, 0.2664603590965271, 0.42726930975914, 0.4331006705760956, 0.33460110425949097, 0.4501304626464844, 0.3914826810359955, 0.45288389921188354, 0.24379673600196838, 0.2959742844104767, 0.3263658881187439, 0.35221728682518005, 0.26092103123664856, 0.21303345263004303], "S_mean": 0.247650146484375, "G_mean": null, "G_std": null, "tokens_seen": 45064192} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.505629301071167, "grad_norm": 0.8350470066070557, "tokens_seen": 45228032, "tokens_per_sec": 16202.243407148979, "vram_gb": 6.154519552, "lambda": 0.25030915811657906, "lambda_min": 0.06456470489501953, "lambda_max": 0.4543485641479492, "S_mean": 0.24489339192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.6566245555877686, "grad_norm": 0.8747561573982239, "tokens_seen": 45391872, "tokens_per_sec": 16202.791909154634, "vram_gb": 6.154519552, "lambda": 0.25082471687346697, "lambda_min": 0.0644925981760025, "lambda_max": 0.4557873606681824, "S_mean": 0.2565104166666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.642815113067627, "grad_norm": 0.8085522651672363, "tokens_seen": 45555712, "tokens_per_sec": 16203.226557769294, "vram_gb": 6.154519552, "lambda": 0.2511963254461686, "lambda_min": 0.06436097621917725, "lambda_max": 0.4573098421096802, "S_mean": 0.242156982421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.50322961807251, "grad_norm": 0.8369893431663513, "tokens_seen": 45719552, "tokens_per_sec": 16203.931828519808, "vram_gb": 6.154519552, "lambda": 0.2516166179751356, "lambda_min": 0.06430009752511978, "lambda_max": 0.4588339924812317, "S_mean": 0.2602030436197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.709650993347168, "grad_norm": 0.8758800625801086, "tokens_seen": 45883392, "tokens_per_sec": 16204.427987977222, "vram_gb": 6.154519552, "lambda": 0.2521317082767685, "lambda_min": 0.06437185406684875, "lambda_max": 0.4604600667953491, "S_mean": 0.24643961588541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.7062273025512695, "grad_norm": 0.855781614780426, "tokens_seen": 46047232, "tokens_per_sec": 16204.878146593748, "vram_gb": 6.154519552, "lambda": 0.25256990641355515, "lambda_min": 0.06437170505523682, "lambda_max": 0.46191805601119995, "S_mean": 0.2470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.6473143100738525, "grad_norm": 0.8139832019805908, "tokens_seen": 46211072, "tokens_per_sec": 16205.251898730428, "vram_gb": 6.154519552, "lambda": 0.25302438189586, "lambda_min": 0.0643095150589943, "lambda_max": 0.4633444547653198, "S_mean": 0.243255615234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.75836968421936, "grad_norm": 0.8772860169410706, "tokens_seen": 46374912, "tokens_per_sec": 16205.950917057908, "vram_gb": 6.154519552, "lambda": 0.2534272037446499, "lambda_min": 0.06426504999399185, "lambda_max": 0.4646584987640381, "S_mean": 0.255950927734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.7788777351379395, "grad_norm": 0.8661662340164185, "tokens_seen": 46538752, "tokens_per_sec": 16206.615334507951, "vram_gb": 6.154519552, "lambda": 0.2538827977453669, "lambda_min": 0.06427305191755295, "lambda_max": 0.4665466547012329, "S_mean": 0.24222819010416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.61507511138916, "grad_norm": 0.7824345827102661, "tokens_seen": 46702592, "tokens_per_sec": 16206.884522203576, "vram_gb": 6.154519552, "lambda": 0.25425380654633045, "lambda_min": 0.06419636309146881, "lambda_max": 0.4675702452659607, "S_mean": 0.2539469401041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.5079004764556885, "grad_norm": 0.8556483387947083, "tokens_seen": 46866432, "tokens_per_sec": 16207.510820682612, "vram_gb": 6.154519552, "lambda": 0.254712109764417, "lambda_min": 0.06404180079698563, "lambda_max": 0.46888405084609985, "S_mean": 0.261077880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.617286682128906, "grad_norm": 0.8255743384361267, "tokens_seen": 47030272, "tokens_per_sec": 16207.885396618172, "vram_gb": 6.154519552, "lambda": 0.255151907602946, "lambda_min": 0.06405339390039444, "lambda_max": 0.470359742641449, "S_mean": 0.2596232096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.678725957870483, "grad_norm": 0.7964732646942139, "tokens_seen": 47194112, "tokens_per_sec": 16208.459007699536, "vram_gb": 6.154519552, "lambda": 0.2556258620073398, "lambda_min": 0.06421490758657455, "lambda_max": 0.47200584411621094, "S_mean": 0.261260986328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.609393358230591, "grad_norm": 0.8465920090675354, "tokens_seen": 47357952, "tokens_per_sec": 16208.880153561477, "vram_gb": 6.154519552, "lambda": 0.2559850973387559, "lambda_min": 0.06425647437572479, "lambda_max": 0.47289279103279114, "S_mean": 0.2527567545572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.59264612197876, "grad_norm": 0.8877922892570496, "tokens_seen": 47521792, "tokens_per_sec": 16209.418925001812, "vram_gb": 6.154519552, "lambda": 0.25638340413570404, "lambda_min": 0.06417005509138107, "lambda_max": 0.4743528962135315, "S_mean": 0.2509765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.571768760681152, "grad_norm": 0.8115904331207275, "tokens_seen": 47685632, "tokens_per_sec": 16209.846369007308, "vram_gb": 6.154519552, "lambda": 0.25676216805974644, "lambda_min": 0.06411000341176987, "lambda_max": 0.4755658507347107, "S_mean": 0.2603352864583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.558274745941162, "grad_norm": 0.8137060403823853, "tokens_seen": 47849472, "tokens_per_sec": 16210.432789280589, "vram_gb": 6.154519552, "lambda": 0.2571030988668402, "lambda_min": 0.06396752595901489, "lambda_max": 0.4768730103969574, "S_mean": 0.2534281412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.565598726272583, "grad_norm": 0.809879720211029, "tokens_seen": 48013312, "tokens_per_sec": 16211.038468641076, "vram_gb": 6.154519552, "lambda": 0.25747053666661185, "lambda_min": 0.06388717889785767, "lambda_max": 0.4779377281665802, "S_mean": 0.2737019856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.541209936141968, "grad_norm": 0.8352710008621216, "tokens_seen": 48177152, "tokens_per_sec": 16211.694998914629, "vram_gb": 6.154519552, "lambda": 0.25787291023880243, "lambda_min": 0.06380289047956467, "lambda_max": 0.4792748689651489, "S_mean": 0.2711893717447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.529501438140869, "grad_norm": 0.8239332437515259, "tokens_seen": 48340992, "tokens_per_sec": 16212.244900820791, "vram_gb": 6.154519552, "lambda": 0.2583777088051041, "lambda_min": 0.06381218880414963, "lambda_max": 0.4804856777191162, "S_mean": 0.2558797200520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.641293287277222, "grad_norm": 0.8380759954452515, "tokens_seen": 48504832, "tokens_per_sec": 16212.901253749817, "vram_gb": 6.154519552, "lambda": 0.25876234006136656, "lambda_min": 0.06380119174718857, "lambda_max": 0.4815825819969177, "S_mean": 0.24959309895833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.5995213985443115, "grad_norm": 0.8931443691253662, "tokens_seen": 48668672, "tokens_per_sec": 16213.582957379547, "vram_gb": 6.154519552, "lambda": 0.2591073689982295, "lambda_min": 0.0636359229683876, "lambda_max": 0.4829137623310089, "S_mean": 0.2493896484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.751959562301636, "grad_norm": 0.8487215638160706, "tokens_seen": 48832512, "tokens_per_sec": 16214.512967706509, "vram_gb": 6.154519552, "lambda": 0.25953002894918126, "lambda_min": 0.06353243440389633, "lambda_max": 0.48446720838546753, "S_mean": 0.24500528971354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.535435676574707, "grad_norm": 0.7797529101371765, "tokens_seen": 48996352, "tokens_per_sec": 16215.146639422224, "vram_gb": 6.154519552, "lambda": 0.25990053980300826, "lambda_min": 0.06345907598733902, "lambda_max": 0.4861277639865875, "S_mean": 0.24974568684895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.57901668548584, "grad_norm": 0.8493542075157166, "tokens_seen": 49160192, "tokens_per_sec": 16215.65529657955, "vram_gb": 6.154519552, "lambda": 0.2603245973587036, "lambda_min": 0.06330696493387222, "lambda_max": 0.4872055649757385, "S_mean": 0.2639872233072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.6978115916252134, "val_ppl": 109.70682622291639, "lambda": 0.2603438567991058, "lambda_min": 0.06330691277980804, "lambda_max": 0.48722660541534424, "lambda_per_layer": [0.16471652686595917, 0.13192534446716309, 0.14203990995883942, 0.1438390463590622, 0.1560533344745636, 0.13470828533172607, 0.09588299691677094, 0.06330691277980804, 0.12026246637105942, 0.2027755081653595, 0.20444399118423462, 0.27128440141677856, 0.44304242730140686, 0.45481061935424805, 0.3506627678871155, 0.4844608008861542, 0.411498099565506, 0.48722660541534424, 0.25398749113082886, 0.3153471350669861, 0.3446629047393799, 0.37598228454589844, 0.27530696988105774, 0.22002573311328888], "S_mean": 0.2584737141927083, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.642596006393433, "grad_norm": 0.7978529930114746, "tokens_seen": 49324032, "tokens_per_sec": 16196.39515805236, "vram_gb": 6.154519552, "lambda": 0.26074495911598206, "lambda_min": 0.06336798518896103, "lambda_max": 0.4883217215538025, "S_mean": 0.26861572265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.509869813919067, "grad_norm": 0.8252304196357727, "tokens_seen": 49487872, "tokens_per_sec": 16197.065075625056, "vram_gb": 6.154519552, "lambda": 0.2611030812064807, "lambda_min": 0.06340288370847702, "lambda_max": 0.4893918037414551, "S_mean": 0.2635498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.42608118057251, "grad_norm": 0.7736191153526306, "tokens_seen": 49651712, "tokens_per_sec": 16197.518875079437, "vram_gb": 6.154519552, "lambda": 0.26152166817337275, "lambda_min": 0.06333027780056, "lambda_max": 0.4905732274055481, "S_mean": 0.2612406412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.4628095626831055, "grad_norm": 0.8652042150497437, "tokens_seen": 49815552, "tokens_per_sec": 16198.034160263178, "vram_gb": 6.154519552, "lambda": 0.26188795051227015, "lambda_min": 0.0633351057767868, "lambda_max": 0.491264283657074, "S_mean": 0.2607320149739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.472555875778198, "grad_norm": 0.8086479902267456, "tokens_seen": 49979392, "tokens_per_sec": 16198.49746227903, "vram_gb": 6.154519552, "lambda": 0.2622650337095062, "lambda_min": 0.06336239725351334, "lambda_max": 0.49187934398651123, "S_mean": 0.2739969889322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.578312397003174, "grad_norm": 0.81666499376297, "tokens_seen": 50143232, "tokens_per_sec": 16199.12290453691, "vram_gb": 6.154519552, "lambda": 0.2626294791698456, "lambda_min": 0.06330970674753189, "lambda_max": 0.492912232875824, "S_mean": 0.2807210286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.48899507522583, "grad_norm": 0.9365677237510681, "tokens_seen": 50307072, "tokens_per_sec": 16199.340994191994, "vram_gb": 6.154519552, "lambda": 0.26305302884429693, "lambda_min": 0.06326874345541, "lambda_max": 0.4945535361766815, "S_mean": 0.2738850911458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.496440172195435, "grad_norm": 0.8902000784873962, "tokens_seen": 50470912, "tokens_per_sec": 16199.871972753788, "vram_gb": 6.154519552, "lambda": 0.26349415195484954, "lambda_min": 0.06323456019163132, "lambda_max": 0.4960046410560608, "S_mean": 0.263519287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.624850749969482, "grad_norm": 0.8859990239143372, "tokens_seen": 50634752, "tokens_per_sec": 16200.63350804494, "vram_gb": 6.154519552, "lambda": 0.26389257827152807, "lambda_min": 0.06315774470567703, "lambda_max": 0.4972587525844574, "S_mean": 0.2612508138020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.561227321624756, "grad_norm": 0.891685962677002, "tokens_seen": 50798592, "tokens_per_sec": 16201.18678492292, "vram_gb": 6.154519552, "lambda": 0.2642594479645292, "lambda_min": 0.06303928047418594, "lambda_max": 0.49827128648757935, "S_mean": 0.2697245279947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.589185953140259, "grad_norm": 0.9754356145858765, "tokens_seen": 50962432, "tokens_per_sec": 16201.591835531712, "vram_gb": 6.154519552, "lambda": 0.26466826753069955, "lambda_min": 0.06289183348417282, "lambda_max": 0.49954867362976074, "S_mean": 0.2636210123697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.5298686027526855, "grad_norm": 0.9168383479118347, "tokens_seen": 51126272, "tokens_per_sec": 16201.928404879014, "vram_gb": 6.154519552, "lambda": 0.26514975478251773, "lambda_min": 0.062810979783535, "lambda_max": 0.5010726451873779, "S_mean": 0.2764485677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.566164970397949, "grad_norm": 0.965155839920044, "tokens_seen": 51290112, "tokens_per_sec": 16202.288136602676, "vram_gb": 6.154519552, "lambda": 0.26555513062824804, "lambda_min": 0.0627184510231018, "lambda_max": 0.5020886063575745, "S_mean": 0.2550252278645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.613541841506958, "grad_norm": 0.8690904974937439, "tokens_seen": 51453952, "tokens_per_sec": 16202.444204401007, "vram_gb": 6.154519552, "lambda": 0.26597001248349744, "lambda_min": 0.06273336708545685, "lambda_max": 0.5035136938095093, "S_mean": 0.264556884765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.463554620742798, "grad_norm": 0.8270480632781982, "tokens_seen": 51617792, "tokens_per_sec": 16203.03742775753, "vram_gb": 6.154519552, "lambda": 0.2663515542323391, "lambda_min": 0.06270215660333633, "lambda_max": 0.5048296451568604, "S_mean": 0.2667338053385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.645268440246582, "grad_norm": 0.889580249786377, "tokens_seen": 51781632, "tokens_per_sec": 16203.288018832343, "vram_gb": 6.154519552, "lambda": 0.26670451493312913, "lambda_min": 0.06266410648822784, "lambda_max": 0.5059502720832825, "S_mean": 0.26165771484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.4820263385772705, "grad_norm": 0.8686846494674683, "tokens_seen": 51945472, "tokens_per_sec": 16203.668914659038, "vram_gb": 6.154519552, "lambda": 0.26708447095006704, "lambda_min": 0.0626768171787262, "lambda_max": 0.5073681473731995, "S_mean": 0.25238037109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.54833459854126, "grad_norm": 0.8952516317367554, "tokens_seen": 52109312, "tokens_per_sec": 16204.282540634564, "vram_gb": 6.154519552, "lambda": 0.2673929035663605, "lambda_min": 0.06260193884372711, "lambda_max": 0.5082328915596008, "S_mean": 0.26690673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.587063550949097, "grad_norm": 0.8387759327888489, "tokens_seen": 52273152, "tokens_per_sec": 16204.957622669255, "vram_gb": 6.154519552, "lambda": 0.26778790323684615, "lambda_min": 0.06261605769395828, "lambda_max": 0.5095519423484802, "S_mean": 0.2632039388020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.517025947570801, "grad_norm": 0.9492153525352478, "tokens_seen": 52436992, "tokens_per_sec": 16205.573096020034, "vram_gb": 6.154519552, "lambda": 0.2682243551438053, "lambda_min": 0.06264989823102951, "lambda_max": 0.5110073089599609, "S_mean": 0.266357421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.593286037445068, "grad_norm": 0.8950157761573792, "tokens_seen": 52600832, "tokens_per_sec": 16206.10879467153, "vram_gb": 6.154519552, "lambda": 0.26853790258367854, "lambda_min": 0.06255214661359787, "lambda_max": 0.5125097036361694, "S_mean": 0.2608439127604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.543951988220215, "grad_norm": 0.8587701320648193, "tokens_seen": 52764672, "tokens_per_sec": 16206.837215906913, "vram_gb": 6.154519552, "lambda": 0.26889505749568343, "lambda_min": 0.06244346871972084, "lambda_max": 0.5134550333023071, "S_mean": 0.2632344563802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.62633490562439, "grad_norm": 0.9154355525970459, "tokens_seen": 52928512, "tokens_per_sec": 16207.282522842734, "vram_gb": 6.154519552, "lambda": 0.26922307815402746, "lambda_min": 0.06233184039592743, "lambda_max": 0.5142398476600647, "S_mean": 0.2679239908854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.42255711555481, "grad_norm": 0.878433346748352, "tokens_seen": 53092352, "tokens_per_sec": 16207.750068199324, "vram_gb": 6.154519552, "lambda": 0.2696030301352342, "lambda_min": 0.06220914423465729, "lambda_max": 0.5151765942573547, "S_mean": 0.2651468912760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.513074159622192, "grad_norm": 0.8673679828643799, "tokens_seen": 53256192, "tokens_per_sec": 16208.265514294442, "vram_gb": 6.154519552, "lambda": 0.2699735634960234, "lambda_min": 0.06209566071629524, "lambda_max": 0.5163394212722778, "S_mean": 0.2623799641927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6500, "val_loss": 4.6381396651268005, "val_ppl": 103.35189948700433, "lambda": 0.26999205226699513, "lambda_min": 0.06209273636341095, "lambda_max": 0.5163785219192505, "lambda_per_layer": [0.16581490635871887, 0.13199633359909058, 0.14211399853229523, 0.14518257975578308, 0.15813100337982178, 0.13556994497776031, 0.09520145505666733, 0.06209273636341095, 0.12001427263021469, 0.20472438633441925, 0.2083403766155243, 0.2771773934364319, 0.46029019355773926, 0.4776530861854553, 0.3665343225002289, 0.5141192674636841, 0.4293222725391388, 0.5163785219192505, 0.26330187916755676, 0.33104172348976135, 0.36153820157051086, 0.399057000875473, 0.28646931052207947, 0.22774408757686615], "S_mean": 0.2617289225260417, "G_mean": null, "G_std": null, "tokens_seen": 53256192} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.4885759353637695, "grad_norm": 0.886269211769104, "tokens_seen": 53420032, "tokens_per_sec": 16190.441745883982, "vram_gb": 6.154519552, "lambda": 0.27029684589554864, "lambda_min": 0.06198734790086746, "lambda_max": 0.517460286617279, "S_mean": 0.2575276692708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.637819051742554, "grad_norm": 0.9210312962532043, "tokens_seen": 53583872, "tokens_per_sec": 16191.074645108201, "vram_gb": 6.154519552, "lambda": 0.2706656969773273, "lambda_min": 0.06191374734044075, "lambda_max": 0.5183017253875732, "S_mean": 0.249786376953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.588910102844238, "grad_norm": 0.8479650616645813, "tokens_seen": 53747712, "tokens_per_sec": 16191.546066723286, "vram_gb": 6.154519552, "lambda": 0.2710023927502334, "lambda_min": 0.06195933744311333, "lambda_max": 0.5190743803977966, "S_mean": 0.261627197265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.540533065795898, "grad_norm": 0.8561347126960754, "tokens_seen": 53911552, "tokens_per_sec": 16192.004752950164, "vram_gb": 6.154519552, "lambda": 0.2714395858347416, "lambda_min": 0.061835743486881256, "lambda_max": 0.5203429460525513, "S_mean": 0.27337646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.347044229507446, "grad_norm": 0.9139407873153687, "tokens_seen": 54075392, "tokens_per_sec": 16192.69600292473, "vram_gb": 6.154519552, "lambda": 0.2717702404285471, "lambda_min": 0.06181351840496063, "lambda_max": 0.5215951204299927, "S_mean": 0.259002685546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.761108636856079, "grad_norm": 0.9090878963470459, "tokens_seen": 54239232, "tokens_per_sec": 16193.181002492765, "vram_gb": 6.154519552, "lambda": 0.27213627596696216, "lambda_min": 0.06176931411027908, "lambda_max": 0.522571325302124, "S_mean": 0.2567240397135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.5550148487091064, "grad_norm": 0.9185671806335449, "tokens_seen": 54403072, "tokens_per_sec": 16193.755701283055, "vram_gb": 6.154519552, "lambda": 0.2725620820807914, "lambda_min": 0.0618293397128582, "lambda_max": 0.5239078402519226, "S_mean": 0.2689921061197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.512359142303467, "grad_norm": 0.895561695098877, "tokens_seen": 54566912, "tokens_per_sec": 16194.155107940913, "vram_gb": 6.154519552, "lambda": 0.2729447220141689, "lambda_min": 0.06188765913248062, "lambda_max": 0.5248638391494751, "S_mean": 0.2598470052083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.494341850280762, "grad_norm": 0.8548930287361145, "tokens_seen": 54730752, "tokens_per_sec": 16194.38533068137, "vram_gb": 6.154519552, "lambda": 0.2733023804612458, "lambda_min": 0.06185830757021904, "lambda_max": 0.5260869264602661, "S_mean": 0.2627970377604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.516329765319824, "grad_norm": 0.8994311690330505, "tokens_seen": 54894592, "tokens_per_sec": 16194.779264141816, "vram_gb": 6.154519552, "lambda": 0.27375338536997634, "lambda_min": 0.06180967390537262, "lambda_max": 0.5275888442993164, "S_mean": 0.24692281087239584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.514349460601807, "grad_norm": 0.893934428691864, "tokens_seen": 55058432, "tokens_per_sec": 16195.139919241923, "vram_gb": 6.154519552, "lambda": 0.2741064333046476, "lambda_min": 0.06174265593290329, "lambda_max": 0.5292118787765503, "S_mean": 0.2606608072916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.513088703155518, "grad_norm": 0.8471900820732117, "tokens_seen": 55222272, "tokens_per_sec": 16195.494772036287, "vram_gb": 6.154519552, "lambda": 0.27435077851017314, "lambda_min": 0.06172899901866913, "lambda_max": 0.5303924679756165, "S_mean": 0.28118896484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.56866717338562, "grad_norm": 0.9011436700820923, "tokens_seen": 55386112, "tokens_per_sec": 16195.927513473978, "vram_gb": 6.154519552, "lambda": 0.2746713110245764, "lambda_min": 0.061805400997400284, "lambda_max": 0.5313345193862915, "S_mean": 0.261871337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.56551718711853, "grad_norm": 0.8784312605857849, "tokens_seen": 55549952, "tokens_per_sec": 16196.352819716667, "vram_gb": 6.154519552, "lambda": 0.2749913861043751, "lambda_min": 0.0618167482316494, "lambda_max": 0.5325250625610352, "S_mean": 0.263092041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.5283777713775635, "grad_norm": 1.0035525560379028, "tokens_seen": 55713792, "tokens_per_sec": 16196.646765911068, "vram_gb": 6.154519552, "lambda": 0.27527797035872936, "lambda_min": 0.061785824596881866, "lambda_max": 0.5336117148399353, "S_mean": 0.271484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.6766517162323, "grad_norm": 0.9585226774215698, "tokens_seen": 55877632, "tokens_per_sec": 16196.986226890123, "vram_gb": 6.154519552, "lambda": 0.2756367122444014, "lambda_min": 0.06173206493258476, "lambda_max": 0.5349892377853394, "S_mean": 0.2648213704427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.571314334869385, "grad_norm": 0.8538182377815247, "tokens_seen": 56041472, "tokens_per_sec": 16197.4867181266, "vram_gb": 6.154519552, "lambda": 0.2760385259365042, "lambda_min": 0.06173987686634064, "lambda_max": 0.5362710356712341, "S_mean": 0.2621866861979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.464107036590576, "grad_norm": 0.8751707673072815, "tokens_seen": 56205312, "tokens_per_sec": 16198.154189213756, "vram_gb": 6.154519552, "lambda": 0.27639530630161363, "lambda_min": 0.06171178072690964, "lambda_max": 0.5375372767448425, "S_mean": 0.267669677734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.513344049453735, "grad_norm": 0.8707103729248047, "tokens_seen": 56369152, "tokens_per_sec": 16198.638264840481, "vram_gb": 6.154519552, "lambda": 0.27673674483473104, "lambda_min": 0.06155970320105553, "lambda_max": 0.538336455821991, "S_mean": 0.2572428385416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.496648550033569, "grad_norm": 0.9794366359710693, "tokens_seen": 56532992, "tokens_per_sec": 16199.222920986906, "vram_gb": 6.154519552, "lambda": 0.27704178014149267, "lambda_min": 0.06136525422334671, "lambda_max": 0.5392508506774902, "S_mean": 0.24820963541666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.4332275390625, "grad_norm": 0.8801221251487732, "tokens_seen": 56696832, "tokens_per_sec": 16199.83036265127, "vram_gb": 6.154519552, "lambda": 0.2773617124184966, "lambda_min": 0.06127806752920151, "lambda_max": 0.540231466293335, "S_mean": 0.2735493977864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.436108112335205, "grad_norm": 0.8788508176803589, "tokens_seen": 56860672, "tokens_per_sec": 16200.100476192192, "vram_gb": 6.154519552, "lambda": 0.2776341244267921, "lambda_min": 0.06120072677731514, "lambda_max": 0.541478157043457, "S_mean": 0.2514546712239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.497340679168701, "grad_norm": 0.8833326697349548, "tokens_seen": 57024512, "tokens_per_sec": 16200.544378388755, "vram_gb": 6.154519552, "lambda": 0.27790067018941045, "lambda_min": 0.06113118305802345, "lambda_max": 0.5425779819488525, "S_mean": 0.2588602701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.36282205581665, "grad_norm": 0.8600112795829773, "tokens_seen": 57188352, "tokens_per_sec": 16200.88397389881, "vram_gb": 6.154519552, "lambda": 0.27827820073192316, "lambda_min": 0.0611315555870533, "lambda_max": 0.5439088344573975, "S_mean": 0.24104817708333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.363704681396484, "grad_norm": 0.8835076093673706, "tokens_seen": 57352192, "tokens_per_sec": 16201.317850776222, "vram_gb": 6.154519552, "lambda": 0.2786052088874082, "lambda_min": 0.061101969331502914, "lambda_max": 0.5446767210960388, "S_mean": 0.2518310546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.579020261764526, "val_ppl": 97.41890241265982, "lambda": 0.2786225004432102, "lambda_min": 0.06109807267785072, "lambda_max": 0.544724702835083, "lambda_per_layer": [0.16762222349643707, 0.1320413500070572, 0.14312781393527985, 0.1463707834482193, 0.1598927080631256, 0.13581253588199615, 0.09431511163711548, 0.06109807267785072, 0.1203918382525444, 0.20726120471954346, 0.21154609322547913, 0.28242358565330505, 0.471446692943573, 0.497214138507843, 0.38206493854522705, 0.539254367351532, 0.4456176161766052, 0.544724702835083, 0.27276140451431274, 0.3461684286594391, 0.37907567620277405, 0.41725224256515503, 0.29580432176589966, 0.23365215957164764], "S_mean": 0.25537109375, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.52805233001709, "grad_norm": 0.9404250979423523, "tokens_seen": 57516032, "tokens_per_sec": 16184.829954346646, "vram_gb": 6.154519552, "lambda": 0.27888333471491933, "lambda_min": 0.06103361025452614, "lambda_max": 0.5454318523406982, "S_mean": 0.257598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.625948905944824, "grad_norm": 0.8803851008415222, "tokens_seen": 57679872, "tokens_per_sec": 16184.94685948814, "vram_gb": 6.154519552, "lambda": 0.2791189434938133, "lambda_min": 0.06095518544316292, "lambda_max": 0.5460649728775024, "S_mean": 0.2670491536458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.5330822467803955, "grad_norm": 0.9522498846054077, "tokens_seen": 57843712, "tokens_per_sec": 16185.383820805442, "vram_gb": 6.154519552, "lambda": 0.27952013180280727, "lambda_min": 0.06085729971528053, "lambda_max": 0.5472217798233032, "S_mean": 0.2501118977864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.430135488510132, "grad_norm": 0.907394528388977, "tokens_seen": 58007552, "tokens_per_sec": 16185.859423758753, "vram_gb": 6.154519552, "lambda": 0.2798736188560724, "lambda_min": 0.060823164880275726, "lambda_max": 0.5480471849441528, "S_mean": 0.25262451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.577326536178589, "grad_norm": 1.0023304224014282, "tokens_seen": 58171392, "tokens_per_sec": 16186.25036673083, "vram_gb": 6.154519552, "lambda": 0.2802042147765557, "lambda_min": 0.06074230372905731, "lambda_max": 0.5488049387931824, "S_mean": 0.25360107421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.490359544754028, "grad_norm": 0.9394949078559875, "tokens_seen": 58335232, "tokens_per_sec": 16186.942099793294, "vram_gb": 6.154519552, "lambda": 0.2805185269874831, "lambda_min": 0.06073319539427757, "lambda_max": 0.5498912334442139, "S_mean": 0.2542622884114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.429049015045166, "grad_norm": 0.9316082000732422, "tokens_seen": 58499072, "tokens_per_sec": 16187.568521356076, "vram_gb": 6.154519552, "lambda": 0.28072876498724025, "lambda_min": 0.06067746505141258, "lambda_max": 0.5503755807876587, "S_mean": 0.2516072591145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.37964940071106, "grad_norm": 1.0010581016540527, "tokens_seen": 58662912, "tokens_per_sec": 16187.976370166622, "vram_gb": 6.154519552, "lambda": 0.2810599375516176, "lambda_min": 0.06071941554546356, "lambda_max": 0.5513746738433838, "S_mean": 0.2753092447916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.590371608734131, "grad_norm": 0.8957452774047852, "tokens_seen": 58826752, "tokens_per_sec": 16188.503211154475, "vram_gb": 6.154519552, "lambda": 0.28139493831743795, "lambda_min": 0.06070517748594284, "lambda_max": 0.5524142384529114, "S_mean": 0.256195068359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.484694004058838, "grad_norm": 0.9109411835670471, "tokens_seen": 58990592, "tokens_per_sec": 16188.996690367636, "vram_gb": 6.154519552, "lambda": 0.28168125186736387, "lambda_min": 0.060671981424093246, "lambda_max": 0.5532233715057373, "S_mean": 0.251373291015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.570205211639404, "grad_norm": 0.9779280424118042, "tokens_seen": 59154432, "tokens_per_sec": 16189.41588075735, "vram_gb": 6.154519552, "lambda": 0.28198602857689065, "lambda_min": 0.060593970119953156, "lambda_max": 0.5542044043540955, "S_mean": 0.26055908203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.487489223480225, "grad_norm": 0.9999345541000366, "tokens_seen": 59318272, "tokens_per_sec": 16189.86680107531, "vram_gb": 6.154519552, "lambda": 0.28232393444826204, "lambda_min": 0.06057099997997284, "lambda_max": 0.5553224682807922, "S_mean": 0.2533467610677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.513172626495361, "grad_norm": 0.880693793296814, "tokens_seen": 59482112, "tokens_per_sec": 16190.304187184503, "vram_gb": 6.154519552, "lambda": 0.28259801616271335, "lambda_min": 0.060557298362255096, "lambda_max": 0.5564683675765991, "S_mean": 0.259124755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.598325967788696, "grad_norm": 0.8921072483062744, "tokens_seen": 59645952, "tokens_per_sec": 16191.028644290942, "vram_gb": 6.154519552, "lambda": 0.28293947301184136, "lambda_min": 0.060573089867830276, "lambda_max": 0.5577859878540039, "S_mean": 0.24827067057291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.431775808334351, "grad_norm": 0.9170940518379211, "tokens_seen": 59809792, "tokens_per_sec": 16191.655698871224, "vram_gb": 6.154519552, "lambda": 0.28323067740226787, "lambda_min": 0.06054415926337242, "lambda_max": 0.5586976408958435, "S_mean": 0.2526346842447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.4898762702941895, "grad_norm": 0.8680316209793091, "tokens_seen": 59973632, "tokens_per_sec": 16192.007374446948, "vram_gb": 6.154519552, "lambda": 0.28351221652701497, "lambda_min": 0.060472529381513596, "lambda_max": 0.5596863627433777, "S_mean": 0.2573750813802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.4970245361328125, "grad_norm": 0.8778571486473083, "tokens_seen": 60137472, "tokens_per_sec": 16192.70625458431, "vram_gb": 6.154519552, "lambda": 0.2837839404431482, "lambda_min": 0.06042158976197243, "lambda_max": 0.560521125793457, "S_mean": 0.2515055338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.527292966842651, "grad_norm": 0.8845223784446716, "tokens_seen": 60301312, "tokens_per_sec": 16193.31664615409, "vram_gb": 6.154519552, "lambda": 0.2840099895062546, "lambda_min": 0.06032334640622139, "lambda_max": 0.5613710880279541, "S_mean": 0.259613037109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.364248514175415, "grad_norm": 0.9026106595993042, "tokens_seen": 60465152, "tokens_per_sec": 16193.92663459588, "vram_gb": 6.154519552, "lambda": 0.2843011942071219, "lambda_min": 0.060215119272470474, "lambda_max": 0.5622521042823792, "S_mean": 0.2665201822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.3892436027526855, "grad_norm": 0.85634446144104, "tokens_seen": 60628992, "tokens_per_sec": 16194.366609946472, "vram_gb": 6.154519552, "lambda": 0.28454958569879335, "lambda_min": 0.06015359237790108, "lambda_max": 0.5632667541503906, "S_mean": 0.24778238932291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.546669244766235, "grad_norm": 0.9057910442352295, "tokens_seen": 60792832, "tokens_per_sec": 16194.800674488162, "vram_gb": 6.154519552, "lambda": 0.284821090909342, "lambda_min": 0.06006333976984024, "lambda_max": 0.5645735859870911, "S_mean": 0.249786376953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.5749671459198, "grad_norm": 0.9791732430458069, "tokens_seen": 60956672, "tokens_per_sec": 16195.367491062674, "vram_gb": 6.154519552, "lambda": 0.2850601420116921, "lambda_min": 0.05999905988574028, "lambda_max": 0.5653277039527893, "S_mean": 0.252532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.448013782501221, "grad_norm": 0.8643819093704224, "tokens_seen": 61120512, "tokens_per_sec": 16195.861441619407, "vram_gb": 6.154519552, "lambda": 0.28534231210748356, "lambda_min": 0.059998899698257446, "lambda_max": 0.5662006139755249, "S_mean": 0.252349853515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.672851085662842, "grad_norm": 0.9335200786590576, "tokens_seen": 61284352, "tokens_per_sec": 16196.259235828295, "vram_gb": 6.154519552, "lambda": 0.28561265968407196, "lambda_min": 0.05998316779732704, "lambda_max": 0.5672631859779358, "S_mean": 0.2526041666666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.303086519241333, "grad_norm": 0.8854817748069763, "tokens_seen": 61448192, "tokens_per_sec": 16196.676535996401, "vram_gb": 6.154519552, "lambda": 0.2858827984891832, "lambda_min": 0.05996892973780632, "lambda_max": 0.5683767795562744, "S_mean": 0.2667744954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7500, "val_loss": 4.556157279014587, "val_ppl": 95.21688400012043, "lambda": 0.2858982547186315, "lambda_min": 0.059969913214445114, "lambda_max": 0.5684323310852051, "lambda_per_layer": [0.16942332684993744, 0.13208308815956116, 0.1429680734872818, 0.14771200716495514, 0.161539226770401, 0.1370784491300583, 0.09396723657846451, 0.059969913214445114, 0.12054750323295593, 0.20826943218708038, 0.21352791786193848, 0.2866143584251404, 0.4835173189640045, 0.5122911930084229, 0.3920135200023651, 0.5617762207984924, 0.4583127498626709, 0.5684323310852051, 0.280910462141037, 0.3600142300128937, 0.39305490255355835, 0.43405669927597046, 0.30349001288414, 0.23998793959617615], "S_mean": 0.2543131510416667, "G_mean": null, "G_std": null, "tokens_seen": 61448192} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.54892635345459, "grad_norm": 0.8860533237457275, "tokens_seen": 61612032, "tokens_per_sec": 16181.127166797169, "vram_gb": 6.154519552, "lambda": 0.2861437917066117, "lambda_min": 0.059957344084978104, "lambda_max": 0.5692746639251709, "S_mean": 0.23981730143229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.404925107955933, "grad_norm": 0.8878945112228394, "tokens_seen": 61775872, "tokens_per_sec": 16181.343650243498, "vram_gb": 6.154519552, "lambda": 0.28641958596805733, "lambda_min": 0.05995101481676102, "lambda_max": 0.5699814558029175, "S_mean": 0.258209228515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.42266058921814, "grad_norm": 0.9357117414474487, "tokens_seen": 61939712, "tokens_per_sec": 16181.529787690231, "vram_gb": 6.154519552, "lambda": 0.2867658364896973, "lambda_min": 0.059860602021217346, "lambda_max": 0.5708104372024536, "S_mean": 0.266357421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.390897750854492, "grad_norm": 0.9147933721542358, "tokens_seen": 62103552, "tokens_per_sec": 16182.072621011535, "vram_gb": 6.154519552, "lambda": 0.28706618460516137, "lambda_min": 0.0598328560590744, "lambda_max": 0.5717641115188599, "S_mean": 0.260833740234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.4006123542785645, "grad_norm": 0.9549149870872498, "tokens_seen": 62267392, "tokens_per_sec": 16182.576818997899, "vram_gb": 6.154519552, "lambda": 0.28733425370107096, "lambda_min": 0.05983014404773712, "lambda_max": 0.5727653503417969, "S_mean": 0.2561136881510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.388133525848389, "grad_norm": 0.9222473502159119, "tokens_seen": 62431232, "tokens_per_sec": 16183.099186716767, "vram_gb": 6.154519552, "lambda": 0.2875923849642277, "lambda_min": 0.05987311154603958, "lambda_max": 0.5738889575004578, "S_mean": 0.25054931640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.425553321838379, "grad_norm": 0.8915309906005859, "tokens_seen": 62595072, "tokens_per_sec": 16183.546353958156, "vram_gb": 6.154519552, "lambda": 0.28787522250786424, "lambda_min": 0.059939365833997726, "lambda_max": 0.57453453540802, "S_mean": 0.248748779296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.518048524856567, "grad_norm": 1.106918454170227, "tokens_seen": 62758912, "tokens_per_sec": 16183.756239136683, "vram_gb": 6.154519552, "lambda": 0.2881760687256853, "lambda_min": 0.059968024492263794, "lambda_max": 0.5749492049217224, "S_mean": 0.26300048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.3803956508636475, "grad_norm": 1.0927765369415283, "tokens_seen": 62922752, "tokens_per_sec": 16183.901572696723, "vram_gb": 6.154519552, "lambda": 0.2884117648936808, "lambda_min": 0.05997713282704353, "lambda_max": 0.5754518508911133, "S_mean": 0.2763875325520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.492150545120239, "grad_norm": 0.9665156602859497, "tokens_seen": 63086592, "tokens_per_sec": 16183.81624061221, "vram_gb": 6.154519552, "lambda": 0.28867070640747744, "lambda_min": 0.05992943421006203, "lambda_max": 0.5764672756195068, "S_mean": 0.2532145182291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.533488750457764, "grad_norm": 0.9804385304450989, "tokens_seen": 63250432, "tokens_per_sec": 16184.212431601389, "vram_gb": 6.154519552, "lambda": 0.28893172089010477, "lambda_min": 0.05984026938676834, "lambda_max": 0.5774366855621338, "S_mean": 0.24769083658854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.369755268096924, "grad_norm": 0.9903513193130493, "tokens_seen": 63414272, "tokens_per_sec": 16184.484695342482, "vram_gb": 6.154519552, "lambda": 0.2891350717594226, "lambda_min": 0.05980294197797775, "lambda_max": 0.5778917074203491, "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.583830118179321, "grad_norm": 0.951499342918396, "tokens_seen": 63578112, "tokens_per_sec": 16184.593462270008, "vram_gb": 6.154519552, "lambda": 0.2893577837385237, "lambda_min": 0.05974528566002846, "lambda_max": 0.5787910223007202, "S_mean": 0.2659403483072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.4458746910095215, "grad_norm": 0.9300122857093811, "tokens_seen": 63741952, "tokens_per_sec": 16185.154473119039, "vram_gb": 6.154519552, "lambda": 0.2896332307718694, "lambda_min": 0.0597221665084362, "lambda_max": 0.579505443572998, "S_mean": 0.26531982421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.488610744476318, "grad_norm": 0.9409655332565308, "tokens_seen": 63905792, "tokens_per_sec": 16185.249553019858, "vram_gb": 6.154519552, "lambda": 0.28990340682988364, "lambda_min": 0.05970853939652443, "lambda_max": 0.5800424814224243, "S_mean": 0.2606404622395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.381647348403931, "grad_norm": 0.9682251811027527, "tokens_seen": 64069632, "tokens_per_sec": 16185.428345782902, "vram_gb": 6.154519552, "lambda": 0.29018339949349564, "lambda_min": 0.059690482914447784, "lambda_max": 0.5808634757995605, "S_mean": 0.24644978841145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.501993179321289, "grad_norm": 0.9863818287849426, "tokens_seen": 64233472, "tokens_per_sec": 16185.689069913738, "vram_gb": 6.154519552, "lambda": 0.290459793061018, "lambda_min": 0.05970555543899536, "lambda_max": 0.5814621448516846, "S_mean": 0.2513834635416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.449368238449097, "grad_norm": 0.9514556527137756, "tokens_seen": 64397312, "tokens_per_sec": 16185.827318412454, "vram_gb": 6.154519552, "lambda": 0.29071293693656725, "lambda_min": 0.05972885712981224, "lambda_max": 0.5819910764694214, "S_mean": 0.26116943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.403618574142456, "grad_norm": 1.0527524948120117, "tokens_seen": 64561152, "tokens_per_sec": 16185.75955548038, "vram_gb": 6.154519552, "lambda": 0.290951491954426, "lambda_min": 0.05967531353235245, "lambda_max": 0.5829887986183167, "S_mean": 0.2505696614583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.578453063964844, "grad_norm": 1.0703307390213013, "tokens_seen": 64724992, "tokens_per_sec": 16186.049438046417, "vram_gb": 6.154519552, "lambda": 0.2912028799764812, "lambda_min": 0.05964219197630882, "lambda_max": 0.583729088306427, "S_mean": 0.2674357096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.427182674407959, "grad_norm": 0.9958019852638245, "tokens_seen": 64888832, "tokens_per_sec": 16186.024156859045, "vram_gb": 6.154519552, "lambda": 0.2914314012353619, "lambda_min": 0.05966337025165558, "lambda_max": 0.5841571688652039, "S_mean": 0.2585856119791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.475267171859741, "grad_norm": 0.9523515701293945, "tokens_seen": 65052672, "tokens_per_sec": 16185.625717551336, "vram_gb": 6.154519552, "lambda": 0.2916813792350392, "lambda_min": 0.05967477336525917, "lambda_max": 0.5848977565765381, "S_mean": 0.2562255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.535253047943115, "grad_norm": 0.9912096261978149, "tokens_seen": 65216512, "tokens_per_sec": 16185.611248547712, "vram_gb": 6.154519552, "lambda": 0.29196102793018025, "lambda_min": 0.05967903137207031, "lambda_max": 0.5857663154602051, "S_mean": 0.2654825846354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.2700135707855225, "grad_norm": 0.9479875564575195, "tokens_seen": 65380352, "tokens_per_sec": 16185.823531460514, "vram_gb": 6.154519552, "lambda": 0.29219992536430556, "lambda_min": 0.05965457484126091, "lambda_max": 0.5863051414489746, "S_mean": 0.26348876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.344963788986206, "grad_norm": 0.9408001899719238, "tokens_seen": 65544192, "tokens_per_sec": 16185.731662123753, "vram_gb": 6.154519552, "lambda": 0.2924469136632979, "lambda_min": 0.05961301550269127, "lambda_max": 0.5871569514274597, "S_mean": 0.23903401692708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.549648344516754, "val_ppl": 94.59913616015808, "lambda": 0.29246091314901906, "lambda_min": 0.05961371958255768, "lambda_max": 0.5872304439544678, "lambda_per_layer": [0.1711413711309433, 0.13211573660373688, 0.14350685477256775, 0.14867518842220306, 0.16311024129390717, 0.13740618526935577, 0.09407760947942734, 0.05961371958255768, 0.12031197547912598, 0.20957493782043457, 0.21596212685108185, 0.2897161543369293, 0.4913256764411926, 0.5266557335853577, 0.40075623989105225, 0.5830572247505188, 0.46971386671066284, 0.5872304439544678, 0.2879216969013214, 0.3717412054538727, 0.40734097361564636, 0.44975119829177856, 0.3134097754955292, 0.24494577944278717], "S_mean": 0.2638651529947917, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.63200569152832, "grad_norm": 1.0743147134780884, "tokens_seen": 65708032, "tokens_per_sec": 16170.960152534253, "vram_gb": 6.154519552, "lambda": 0.29267588490620255, "lambda_min": 0.05959579721093178, "lambda_max": 0.5880899429321289, "S_mean": 0.24458821614583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.419168472290039, "grad_norm": 0.9139220714569092, "tokens_seen": 65871872, "tokens_per_sec": 16170.99599677388, "vram_gb": 6.154519552, "lambda": 0.29289949949209887, "lambda_min": 0.0595921166241169, "lambda_max": 0.5887627005577087, "S_mean": 0.2704060872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.306946516036987, "grad_norm": 0.8995296359062195, "tokens_seen": 66035712, "tokens_per_sec": 16170.973469519904, "vram_gb": 6.154519552, "lambda": 0.29312398346761864, "lambda_min": 0.059604816138744354, "lambda_max": 0.5891768336296082, "S_mean": 0.24846394856770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.406766891479492, "grad_norm": 0.9730721712112427, "tokens_seen": 66199552, "tokens_per_sec": 16171.206399087792, "vram_gb": 6.154519552, "lambda": 0.29332011255125207, "lambda_min": 0.05957157909870148, "lambda_max": 0.5896920561790466, "S_mean": 0.2542928059895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.41324520111084, "grad_norm": 0.9946528077125549, "tokens_seen": 66363392, "tokens_per_sec": 16171.657247102363, "vram_gb": 6.154519552, "lambda": 0.2935580853372812, "lambda_min": 0.05952560156583786, "lambda_max": 0.590231716632843, "S_mean": 0.2534586588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.448856830596924, "grad_norm": 0.8954397439956665, "tokens_seen": 66527232, "tokens_per_sec": 16171.857419330003, "vram_gb": 6.154519552, "lambda": 0.2937704309200247, "lambda_min": 0.05947566032409668, "lambda_max": 0.5908098220825195, "S_mean": 0.2599995930989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.405802011489868, "grad_norm": 0.9305103421211243, "tokens_seen": 66691072, "tokens_per_sec": 16172.025433255503, "vram_gb": 6.154519552, "lambda": 0.2940021494093041, "lambda_min": 0.05949528142809868, "lambda_max": 0.5915259718894958, "S_mean": 0.2639973958333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.519175291061401, "grad_norm": 1.1323201656341553, "tokens_seen": 66854912, "tokens_per_sec": 16172.373319426693, "vram_gb": 6.154519552, "lambda": 0.29422547575086355, "lambda_min": 0.05952735245227814, "lambda_max": 0.5924079418182373, "S_mean": 0.262359619140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.448177337646484, "grad_norm": 0.8964563608169556, "tokens_seen": 67018752, "tokens_per_sec": 16172.730539262706, "vram_gb": 6.154519552, "lambda": 0.2944705636861424, "lambda_min": 0.05953573063015938, "lambda_max": 0.5933336019515991, "S_mean": 0.2664388020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.525083065032959, "grad_norm": 1.044051170349121, "tokens_seen": 67182592, "tokens_per_sec": 16173.106902513566, "vram_gb": 6.154519552, "lambda": 0.2946924924229582, "lambda_min": 0.059493452310562134, "lambda_max": 0.5941275358200073, "S_mean": 0.255279541015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.374570369720459, "grad_norm": 0.9064874649047852, "tokens_seen": 67346432, "tokens_per_sec": 16173.442224714698, "vram_gb": 6.154519552, "lambda": 0.2949130347309013, "lambda_min": 0.05950620397925377, "lambda_max": 0.5949220061302185, "S_mean": 0.2524922688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.449120044708252, "grad_norm": 1.016631007194519, "tokens_seen": 67510272, "tokens_per_sec": 16174.00695822679, "vram_gb": 6.154519552, "lambda": 0.29514892228568596, "lambda_min": 0.05955871567130089, "lambda_max": 0.5956525206565857, "S_mean": 0.24876912434895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.342777967453003, "grad_norm": 0.9444773197174072, "tokens_seen": 67674112, "tokens_per_sec": 16174.503506306892, "vram_gb": 6.154519552, "lambda": 0.2953907987102866, "lambda_min": 0.0595923513174057, "lambda_max": 0.5964257717132568, "S_mean": 0.247650146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.422708034515381, "grad_norm": 0.9251338243484497, "tokens_seen": 67837952, "tokens_per_sec": 16175.019311195936, "vram_gb": 6.154519552, "lambda": 0.2956513344931106, "lambda_min": 0.059615608304739, "lambda_max": 0.5972661375999451, "S_mean": 0.24604288736979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.593639373779297, "grad_norm": 1.0643765926361084, "tokens_seen": 68001792, "tokens_per_sec": 16175.646647101548, "vram_gb": 6.154519552, "lambda": 0.2958928888353209, "lambda_min": 0.059600312262773514, "lambda_max": 0.5981725454330444, "S_mean": 0.24845377604166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.227223873138428, "grad_norm": 0.897145688533783, "tokens_seen": 68165632, "tokens_per_sec": 16176.137684750105, "vram_gb": 6.154519552, "lambda": 0.29607979394495487, "lambda_min": 0.05959039181470871, "lambda_max": 0.5988491773605347, "S_mean": 0.24654134114583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.424773931503296, "grad_norm": 0.9786860942840576, "tokens_seen": 68329472, "tokens_per_sec": 16176.713716229777, "vram_gb": 6.154519552, "lambda": 0.2962655096004407, "lambda_min": 0.05960740149021149, "lambda_max": 0.5996211767196655, "S_mean": 0.24909464518229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.420524597167969, "grad_norm": 0.9429067969322205, "tokens_seen": 68493312, "tokens_per_sec": 16177.018506388175, "vram_gb": 6.154519552, "lambda": 0.2964733244540791, "lambda_min": 0.05959488824009895, "lambda_max": 0.600233793258667, "S_mean": 0.2717692057291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.363699913024902, "grad_norm": 0.9120973944664001, "tokens_seen": 68657152, "tokens_per_sec": 16177.271113403718, "vram_gb": 6.154519552, "lambda": 0.2966975133555631, "lambda_min": 0.059556830674409866, "lambda_max": 0.6009758710861206, "S_mean": 0.2578226725260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.558582305908203, "grad_norm": 0.9380890130996704, "tokens_seen": 68820992, "tokens_per_sec": 16177.712593041078, "vram_gb": 6.154519552, "lambda": 0.29683464315409464, "lambda_min": 0.05953944846987724, "lambda_max": 0.6016086339950562, "S_mean": 0.2576090494791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.3721535205841064, "grad_norm": 0.902398407459259, "tokens_seen": 68984832, "tokens_per_sec": 16178.143635871496, "vram_gb": 6.154519552, "lambda": 0.2970579563019176, "lambda_min": 0.059487197548151016, "lambda_max": 0.6023596525192261, "S_mean": 0.255462646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.3763744831085205, "grad_norm": 0.8959020972251892, "tokens_seen": 69148672, "tokens_per_sec": 16178.485563911598, "vram_gb": 6.154519552, "lambda": 0.2972819013521075, "lambda_min": 0.05946756899356842, "lambda_max": 0.6030038595199585, "S_mean": 0.25250244140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.410777568817139, "grad_norm": 0.8989619016647339, "tokens_seen": 69312512, "tokens_per_sec": 16179.054550869572, "vram_gb": 6.154519552, "lambda": 0.2975311658034722, "lambda_min": 0.05951007455587387, "lambda_max": 0.6037319898605347, "S_mean": 0.24803670247395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.531337261199951, "grad_norm": 1.0205600261688232, "tokens_seen": 69476352, "tokens_per_sec": 16179.219346863194, "vram_gb": 6.154519552, "lambda": 0.29772764382263023, "lambda_min": 0.05950026959180832, "lambda_max": 0.6041349768638611, "S_mean": 0.2509663899739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.501974821090698, "grad_norm": 0.967903196811676, "tokens_seen": 69640192, "tokens_per_sec": 16179.443814188178, "vram_gb": 6.154519552, "lambda": 0.29791661196698743, "lambda_min": 0.059491924941539764, "lambda_max": 0.6048151254653931, "S_mean": 0.2669881184895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8500, "val_loss": 4.493940734863282, "val_ppl": 89.47334277829381, "lambda": 0.29792948129276436, "lambda_min": 0.05948854982852936, "lambda_max": 0.6048558950424194, "lambda_per_layer": [0.17253781855106354, 0.13231898844242096, 0.14354175329208374, 0.14964985847473145, 0.16454872488975525, 0.13752242922782898, 0.09393367171287537, 0.05948854982852936, 0.12065935134887695, 0.2104623168706894, 0.21862341463565826, 0.292435884475708, 0.49894872307777405, 0.5371968746185303, 0.4092136323451996, 0.5991372466087341, 0.4769445061683655, 0.6048558950424194, 0.2937760651111603, 0.38182562589645386, 0.4192259907722473, 0.46206697821617126, 0.3213309347629547, 0.25006231665611267], "S_mean": 0.26470947265625, "G_mean": null, "G_std": null, "tokens_seen": 69640192} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.421479225158691, "grad_norm": 1.0676175355911255, "tokens_seen": 69804032, "tokens_per_sec": 16165.972738789618, "vram_gb": 6.154519552, "lambda": 0.29813544374580186, "lambda_min": 0.0594228096306324, "lambda_max": 0.60542231798172, "S_mean": 0.24786376953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.379856586456299, "grad_norm": 0.975906491279602, "tokens_seen": 69967872, "tokens_per_sec": 16166.281716478845, "vram_gb": 6.154519552, "lambda": 0.29832247536008555, "lambda_min": 0.05941518768668175, "lambda_max": 0.6058329939842224, "S_mean": 0.2577412923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.366225957870483, "grad_norm": 1.0405511856079102, "tokens_seen": 70131712, "tokens_per_sec": 16166.73859288313, "vram_gb": 6.154519552, "lambda": 0.29848232166841626, "lambda_min": 0.059428777545690536, "lambda_max": 0.6061303615570068, "S_mean": 0.24599202473958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.491957187652588, "grad_norm": 0.935315728187561, "tokens_seen": 70295552, "tokens_per_sec": 16167.065896461761, "vram_gb": 6.154519552, "lambda": 0.298711233617117, "lambda_min": 0.05946708843111992, "lambda_max": 0.6066057682037354, "S_mean": 0.24936930338541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.536584854125977, "grad_norm": 0.9618468880653381, "tokens_seen": 70459392, "tokens_per_sec": 16167.56822175606, "vram_gb": 6.154519552, "lambda": 0.2989866834444304, "lambda_min": 0.059433016926050186, "lambda_max": 0.6075351238250732, "S_mean": 0.24567667643229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.500193357467651, "grad_norm": 0.9370812773704529, "tokens_seen": 70623232, "tokens_per_sec": 16168.067810512828, "vram_gb": 6.154519552, "lambda": 0.29916213965043426, "lambda_min": 0.05939963832497597, "lambda_max": 0.6083744168281555, "S_mean": 0.2547810872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.3989245891571045, "grad_norm": 0.989224910736084, "tokens_seen": 70787072, "tokens_per_sec": 16168.44313733197, "vram_gb": 6.154519552, "lambda": 0.2993601846198241, "lambda_min": 0.05936592072248459, "lambda_max": 0.6088151335716248, "S_mean": 0.24828084309895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.43950080871582, "grad_norm": 0.9954469203948975, "tokens_seen": 70950912, "tokens_per_sec": 16168.76142334495, "vram_gb": 6.154519552, "lambda": 0.29952491509417695, "lambda_min": 0.05936751514673233, "lambda_max": 0.6093780994415283, "S_mean": 0.257598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.38052773475647, "grad_norm": 0.9593409299850464, "tokens_seen": 71114752, "tokens_per_sec": 16169.154455680487, "vram_gb": 6.154519552, "lambda": 0.29966902174055576, "lambda_min": 0.059364184737205505, "lambda_max": 0.610004186630249, "S_mean": 0.24489339192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.359419822692871, "grad_norm": 1.041999101638794, "tokens_seen": 71278592, "tokens_per_sec": 16169.598838754993, "vram_gb": 6.154519552, "lambda": 0.2998958514072001, "lambda_min": 0.05935530737042427, "lambda_max": 0.6106451749801636, "S_mean": 0.2500406901041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.461224555969238, "grad_norm": 0.9085447788238525, "tokens_seen": 71442432, "tokens_per_sec": 16170.07347164201, "vram_gb": 6.154519552, "lambda": 0.30011184234172106, "lambda_min": 0.05937863141298294, "lambda_max": 0.6111884117126465, "S_mean": 0.27313232421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.580142498016357, "grad_norm": 0.926604688167572, "tokens_seen": 71606272, "tokens_per_sec": 16170.566326244953, "vram_gb": 6.154519552, "lambda": 0.30031712834412855, "lambda_min": 0.05934044346213341, "lambda_max": 0.611781656742096, "S_mean": 0.241943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.416372776031494, "grad_norm": 1.0902036428451538, "tokens_seen": 71770112, "tokens_per_sec": 16171.03195564489, "vram_gb": 6.154519552, "lambda": 0.30051958793774247, "lambda_min": 0.0593215636909008, "lambda_max": 0.6124813556671143, "S_mean": 0.252471923828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.145340204238892, "grad_norm": 0.9703395962715149, "tokens_seen": 71933952, "tokens_per_sec": 16171.570219981752, "vram_gb": 6.154519552, "lambda": 0.3007090220538278, "lambda_min": 0.05931920185685158, "lambda_max": 0.6129850745201111, "S_mean": 0.2556355794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.417508602142334, "grad_norm": 1.074939250946045, "tokens_seen": 72097792, "tokens_per_sec": 16172.03005778864, "vram_gb": 6.154519552, "lambda": 0.30085172643885016, "lambda_min": 0.0592559315264225, "lambda_max": 0.6135064959526062, "S_mean": 0.25860595703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.3888490200042725, "grad_norm": 0.9976732134819031, "tokens_seen": 72261632, "tokens_per_sec": 16172.448927745265, "vram_gb": 6.154519552, "lambda": 0.30100381762410205, "lambda_min": 0.059224892407655716, "lambda_max": 0.61397385597229, "S_mean": 0.24961344401041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.243165016174316, "grad_norm": 0.9946999549865723, "tokens_seen": 72425472, "tokens_per_sec": 16172.912100050664, "vram_gb": 6.154519552, "lambda": 0.301206838650008, "lambda_min": 0.05923084914684296, "lambda_max": 0.6147474646568298, "S_mean": 0.2589619954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.3952436447143555, "grad_norm": 0.9807591438293457, "tokens_seen": 72589312, "tokens_per_sec": 16173.231186588075, "vram_gb": 6.154519552, "lambda": 0.30141823552548885, "lambda_min": 0.05921207368373871, "lambda_max": 0.6154550909996033, "S_mean": 0.248687744140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.509157180786133, "grad_norm": 1.0414716005325317, "tokens_seen": 72753152, "tokens_per_sec": 16173.532276453583, "vram_gb": 6.154519552, "lambda": 0.3015864448000987, "lambda_min": 0.05922595411539078, "lambda_max": 0.6158993244171143, "S_mean": 0.241668701171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.559986591339111, "grad_norm": 0.9387983679771423, "tokens_seen": 72916992, "tokens_per_sec": 16173.98374462042, "vram_gb": 6.154519552, "lambda": 0.3017045119777322, "lambda_min": 0.05925010144710541, "lambda_max": 0.6162803769111633, "S_mean": 0.24623616536458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.38383150100708, "grad_norm": 0.9530970454216003, "tokens_seen": 73080832, "tokens_per_sec": 16174.333947265319, "vram_gb": 6.154519552, "lambda": 0.3019007472321391, "lambda_min": 0.05929524451494217, "lambda_max": 0.6169787645339966, "S_mean": 0.24831136067708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.407990455627441, "grad_norm": 1.0057909488677979, "tokens_seen": 73244672, "tokens_per_sec": 16174.571599777582, "vram_gb": 6.154519552, "lambda": 0.30209623742848635, "lambda_min": 0.05929142236709595, "lambda_max": 0.6176190376281738, "S_mean": 0.24738566080729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.408384323120117, "grad_norm": 0.9716637134552002, "tokens_seen": 73408512, "tokens_per_sec": 16174.501708694948, "vram_gb": 6.154519552, "lambda": 0.3022719824997087, "lambda_min": 0.0592748187482357, "lambda_max": 0.6180837750434875, "S_mean": 0.2528788248697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.445981979370117, "grad_norm": 0.9963840842247009, "tokens_seen": 73572352, "tokens_per_sec": 16174.84368918769, "vram_gb": 6.154519552, "lambda": 0.302461097172151, "lambda_min": 0.059244077652692795, "lambda_max": 0.6187438368797302, "S_mean": 0.248931884765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.427779674530029, "grad_norm": 0.964931070804596, "tokens_seen": 73736192, "tokens_per_sec": 16175.324141488036, "vram_gb": 6.154519552, "lambda": 0.30265433123956126, "lambda_min": 0.05920238792896271, "lambda_max": 0.619613528251648, "S_mean": 0.24602254231770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.464624345302582, "val_ppl": 86.88838337892709, "lambda": 0.30266399231428903, "lambda_min": 0.059201743453741074, "lambda_max": 0.6196582317352295, "lambda_per_layer": [0.17444467544555664, 0.13223543763160706, 0.1440790742635727, 0.15057438611984253, 0.16562877595424652, 0.13817650079727173, 0.09415359050035477, 0.059201743453741074, 0.12054381519556046, 0.2114810049533844, 0.22028779983520508, 0.2942538261413574, 0.503126323223114, 0.5459662675857544, 0.4157366454601288, 0.611379086971283, 0.48502713441848755, 0.6196582317352295, 0.29860949516296387, 0.39152857661247253, 0.42978712916374207, 0.47472694516181946, 0.32843300700187683, 0.254896342754364], "S_mean": 0.246551513671875, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.444535732269287, "grad_norm": 0.9706581234931946, "tokens_seen": 73900032, "tokens_per_sec": 16162.573377134673, "vram_gb": 6.154519552, "lambda": 0.3028355611798664, "lambda_min": 0.05920116975903511, "lambda_max": 0.6201651096343994, "S_mean": 0.24995930989583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.4095776081085205, "grad_norm": 1.1115012168884277, "tokens_seen": 74063872, "tokens_per_sec": 16163.050867326348, "vram_gb": 6.154519552, "lambda": 0.30303765755767625, "lambda_min": 0.059196945279836655, "lambda_max": 0.6206650137901306, "S_mean": 0.2550252278645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.44542670249939, "grad_norm": 1.0281951427459717, "tokens_seen": 74227712, "tokens_per_sec": 16163.528926051822, "vram_gb": 6.154519552, "lambda": 0.3032073308713734, "lambda_min": 0.05916455760598183, "lambda_max": 0.6210970878601074, "S_mean": 0.24986775716145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.431211471557617, "grad_norm": 1.0605125427246094, "tokens_seen": 74391552, "tokens_per_sec": 16163.925433265558, "vram_gb": 6.154519552, "lambda": 0.3033461334804694, "lambda_min": 0.059106454253196716, "lambda_max": 0.6214426159858704, "S_mean": 0.250274658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.3486223220825195, "grad_norm": 1.0521785020828247, "tokens_seen": 74555392, "tokens_per_sec": 16164.290377761858, "vram_gb": 6.154519552, "lambda": 0.3035028548911214, "lambda_min": 0.05904404819011688, "lambda_max": 0.6222090721130371, "S_mean": 0.2537027994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.233337163925171, "grad_norm": 0.9858140349388123, "tokens_seen": 74719232, "tokens_per_sec": 16164.527829044353, "vram_gb": 6.154519552, "lambda": 0.30368954051906866, "lambda_min": 0.05902598425745964, "lambda_max": 0.6229608654975891, "S_mean": 0.2470245361328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.480536937713623, "grad_norm": 0.9983611702919006, "tokens_seen": 74883072, "tokens_per_sec": 16164.823019579466, "vram_gb": 6.154519552, "lambda": 0.30388032101715606, "lambda_min": 0.05903344228863716, "lambda_max": 0.6236031651496887, "S_mean": 0.2543436686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.551690101623535, "grad_norm": 0.9266571998596191, "tokens_seen": 75046912, "tokens_per_sec": 16165.242020652922, "vram_gb": 6.154519552, "lambda": 0.30407132146259147, "lambda_min": 0.059022799134254456, "lambda_max": 0.6238992810249329, "S_mean": 0.252532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.344302654266357, "grad_norm": 1.0526666641235352, "tokens_seen": 75210752, "tokens_per_sec": 16165.664857631722, "vram_gb": 6.154519552, "lambda": 0.304265855345875, "lambda_min": 0.059012044221162796, "lambda_max": 0.6243686079978943, "S_mean": 0.24953206380208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.450013160705566, "grad_norm": 1.07662832736969, "tokens_seen": 75374592, "tokens_per_sec": 16166.022048365541, "vram_gb": 6.154519552, "lambda": 0.3043803380181392, "lambda_min": 0.05902959406375885, "lambda_max": 0.6247485876083374, "S_mean": 0.2572733561197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.348285913467407, "grad_norm": 0.9831506013870239, "tokens_seen": 75538432, "tokens_per_sec": 16166.621066411331, "vram_gb": 6.154519552, "lambda": 0.3045242913067341, "lambda_min": 0.059041112661361694, "lambda_max": 0.6252920627593994, "S_mean": 0.2510630289713542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.3740458488464355, "grad_norm": 1.0013519525527954, "tokens_seen": 75702272, "tokens_per_sec": 16167.077247510359, "vram_gb": 6.154519552, "lambda": 0.30468938561777276, "lambda_min": 0.05901841074228287, "lambda_max": 0.6260765194892883, "S_mean": 0.25408935546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.463359117507935, "grad_norm": 1.0118619203567505, "tokens_seen": 75866112, "tokens_per_sec": 16167.676188019148, "vram_gb": 6.154519552, "lambda": 0.3048354503698647, "lambda_min": 0.05896248295903206, "lambda_max": 0.6265944242477417, "S_mean": 0.2593587239583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.473551034927368, "grad_norm": 1.069131851196289, "tokens_seen": 76029952, "tokens_per_sec": 16168.121470521966, "vram_gb": 6.154519552, "lambda": 0.30499903702487546, "lambda_min": 0.05893487483263016, "lambda_max": 0.627204418182373, "S_mean": 0.2583414713541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.536921262741089, "grad_norm": 1.0305054187774658, "tokens_seen": 76193792, "tokens_per_sec": 16168.572169775414, "vram_gb": 6.154519552, "lambda": 0.30516315003236133, "lambda_min": 0.05891667306423187, "lambda_max": 0.6278270483016968, "S_mean": 0.2530110677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.286567211151123, "grad_norm": 0.958324670791626, "tokens_seen": 76357632, "tokens_per_sec": 16168.764916064392, "vram_gb": 6.154519552, "lambda": 0.3053437997587025, "lambda_min": 0.058939289301633835, "lambda_max": 0.6284860372543335, "S_mean": 0.26470947265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.298105001449585, "grad_norm": 1.0198951959609985, "tokens_seen": 76521472, "tokens_per_sec": 16169.2159403831, "vram_gb": 6.154519552, "lambda": 0.30546039699887234, "lambda_min": 0.058946963399648666, "lambda_max": 0.6291163563728333, "S_mean": 0.2615559895833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.479494094848633, "grad_norm": 1.0754224061965942, "tokens_seen": 76685312, "tokens_per_sec": 16169.547203701959, "vram_gb": 6.154519552, "lambda": 0.3055997472256422, "lambda_min": 0.05891761928796768, "lambda_max": 0.6294331550598145, "S_mean": 0.2508748372395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.376665830612183, "grad_norm": 0.9986034035682678, "tokens_seen": 76849152, "tokens_per_sec": 16170.072568379905, "vram_gb": 6.154519552, "lambda": 0.3057355503551662, "lambda_min": 0.058920685201883316, "lambda_max": 0.6298892498016357, "S_mean": 0.266632080078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.35053825378418, "grad_norm": 1.0720103979110718, "tokens_seen": 77012992, "tokens_per_sec": 16170.449705062098, "vram_gb": 6.154519552, "lambda": 0.30588876688852906, "lambda_min": 0.058926645666360855, "lambda_max": 0.6302472949028015, "S_mean": 0.2770589192708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.235093355178833, "grad_norm": 1.0074113607406616, "tokens_seen": 77176832, "tokens_per_sec": 16170.855762318477, "vram_gb": 6.154519552, "lambda": 0.30604788412650424, "lambda_min": 0.058952465653419495, "lambda_max": 0.6305251121520996, "S_mean": 0.24425252278645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.333272933959961, "grad_norm": 0.9730798602104187, "tokens_seen": 77340672, "tokens_per_sec": 16171.286827480588, "vram_gb": 6.154519552, "lambda": 0.30621867704515654, "lambda_min": 0.05894998088479042, "lambda_max": 0.6308637857437134, "S_mean": 0.2622884114583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.315628528594971, "grad_norm": 1.191809892654419, "tokens_seen": 77504512, "tokens_per_sec": 16171.700214611981, "vram_gb": 6.154519552, "lambda": 0.306339383435746, "lambda_min": 0.05893368273973465, "lambda_max": 0.6311687231063843, "S_mean": 0.2664693196614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.399661302566528, "grad_norm": 0.9622207880020142, "tokens_seen": 77668352, "tokens_per_sec": 16172.052142721657, "vram_gb": 6.154519552, "lambda": 0.3064597112437089, "lambda_min": 0.05889902263879776, "lambda_max": 0.6314432621002197, "S_mean": 0.2517903645833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.368473291397095, "grad_norm": 0.9960479736328125, "tokens_seen": 77832192, "tokens_per_sec": 16172.55938797481, "vram_gb": 6.154519552, "lambda": 0.30658378048489493, "lambda_min": 0.05886705219745636, "lambda_max": 0.6318197250366211, "S_mean": 0.2669270833333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9500, "val_loss": 4.456770765781402, "val_ppl": 86.20867113133234, "lambda": 0.30658894528945285, "lambda_min": 0.058862678706645966, "lambda_max": 0.6318413019180298, "lambda_per_layer": [0.17625610530376434, 0.13255129754543304, 0.1440557837486267, 0.15110468864440918, 0.16649924218654633, 0.13829556107521057, 0.094334676861763, 0.058862678706645966, 0.12073422223329544, 0.21191023290157318, 0.22193922102451324, 0.2955801486968994, 0.5078802704811096, 0.5533016324043274, 0.4210989773273468, 0.6217753291130066, 0.49179548025131226, 0.6318413019180298, 0.3033279478549957, 0.3993108570575714, 0.43862634897232056, 0.48486125469207764, 0.33427223563194275, 0.25791919231414795], "S_mean": 0.2515055338541667, "G_mean": null, "G_std": null, "tokens_seen": 77832192} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.4192728996276855, "grad_norm": 1.0800559520721436, "tokens_seen": 77996032, "tokens_per_sec": 16160.481696535979, "vram_gb": 6.154519552, "lambda": 0.30672946432605386, "lambda_min": 0.058803778141736984, "lambda_max": 0.6322546005249023, "S_mean": 0.2596333821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.298245429992676, "grad_norm": 1.0219107866287231, "tokens_seen": 78159872, "tokens_per_sec": 16160.90308666238, "vram_gb": 6.154519552, "lambda": 0.30691562484328944, "lambda_min": 0.05880945548415184, "lambda_max": 0.6327486038208008, "S_mean": 0.24906412760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.417131662368774, "grad_norm": 1.1955502033233643, "tokens_seen": 78323712, "tokens_per_sec": 16161.269578841522, "vram_gb": 6.154519552, "lambda": 0.30705101881176233, "lambda_min": 0.05881570279598236, "lambda_max": 0.6330872178077698, "S_mean": 0.254180908203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.442913770675659, "grad_norm": 1.063269853591919, "tokens_seen": 78487552, "tokens_per_sec": 16161.329902655707, "vram_gb": 6.154519552, "lambda": 0.30719937989488244, "lambda_min": 0.05881386622786522, "lambda_max": 0.6335673332214355, "S_mean": 0.2512919108072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.341745615005493, "grad_norm": 1.01902174949646, "tokens_seen": 78651392, "tokens_per_sec": 16161.731899284165, "vram_gb": 6.154519552, "lambda": 0.30734188699473936, "lambda_min": 0.05880261957645416, "lambda_max": 0.6339291930198669, "S_mean": 0.2635498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.365823030471802, "grad_norm": 1.0307503938674927, "tokens_seen": 78815232, "tokens_per_sec": 16162.201876403893, "vram_gb": 6.154519552, "lambda": 0.30748422909528017, "lambda_min": 0.05876944959163666, "lambda_max": 0.6344404816627502, "S_mean": 0.2604777018229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.285102128982544, "grad_norm": 0.990737795829773, "tokens_seen": 78979072, "tokens_per_sec": 16162.627716584057, "vram_gb": 6.154519552, "lambda": 0.30764527525752783, "lambda_min": 0.05876930058002472, "lambda_max": 0.6350839138031006, "S_mean": 0.2559407552083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.546819686889648, "grad_norm": 1.0088346004486084, "tokens_seen": 79142912, "tokens_per_sec": 16163.106587393911, "vram_gb": 6.154519552, "lambda": 0.30776598552862805, "lambda_min": 0.05873697251081467, "lambda_max": 0.6356107592582703, "S_mean": 0.2506306966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.337275981903076, "grad_norm": 0.9945391416549683, "tokens_seen": 79306752, "tokens_per_sec": 16163.451487423989, "vram_gb": 6.154519552, "lambda": 0.30789166238779825, "lambda_min": 0.058726418763399124, "lambda_max": 0.6357775330543518, "S_mean": 0.24896240234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.432814121246338, "grad_norm": 0.9849459528923035, "tokens_seen": 79470592, "tokens_per_sec": 16164.065962962997, "vram_gb": 6.154519552, "lambda": 0.3079889953757326, "lambda_min": 0.05876803398132324, "lambda_max": 0.6359463334083557, "S_mean": 0.24762980143229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.4983673095703125, "grad_norm": 1.2211092710494995, "tokens_seen": 79634432, "tokens_per_sec": 16164.555624473538, "vram_gb": 6.154519552, "lambda": 0.30813327291980386, "lambda_min": 0.05880073085427284, "lambda_max": 0.6363279223442078, "S_mean": 0.24711100260416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.232617378234863, "grad_norm": 0.9987059235572815, "tokens_seen": 79798272, "tokens_per_sec": 16165.158200775779, "vram_gb": 6.154519552, "lambda": 0.30829782349367935, "lambda_min": 0.05884101241827011, "lambda_max": 0.6367421746253967, "S_mean": 0.2605997721354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.456574440002441, "grad_norm": 0.9791383743286133, "tokens_seen": 79962112, "tokens_per_sec": 16165.560687795745, "vram_gb": 6.154519552, "lambda": 0.3084360828312735, "lambda_min": 0.058875370770692825, "lambda_max": 0.6371283531188965, "S_mean": 0.24680582682291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.368430852890015, "grad_norm": 1.0000954866409302, "tokens_seen": 80125952, "tokens_per_sec": 16165.970976666453, "vram_gb": 6.154519552, "lambda": 0.3085886087889473, "lambda_min": 0.05891125649213791, "lambda_max": 0.6377620697021484, "S_mean": 0.23548380533854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.4422829151153564, "grad_norm": 0.9685609936714172, "tokens_seen": 80289792, "tokens_per_sec": 16166.148464146987, "vram_gb": 6.154519552, "lambda": 0.30873542713622254, "lambda_min": 0.05894353985786438, "lambda_max": 0.6381456255912781, "S_mean": 0.2526346842447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.2852396965026855, "grad_norm": 1.0275017023086548, "tokens_seen": 80453632, "tokens_per_sec": 16166.566764719264, "vram_gb": 6.154519552, "lambda": 0.30887045664712787, "lambda_min": 0.0589464046061039, "lambda_max": 0.638627290725708, "S_mean": 0.255157470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.493949890136719, "grad_norm": 1.0449968576431274, "tokens_seen": 80617472, "tokens_per_sec": 16166.81650862021, "vram_gb": 6.154519552, "lambda": 0.3090095343068242, "lambda_min": 0.058951400220394135, "lambda_max": 0.6391379237174988, "S_mean": 0.2694193522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.52844500541687, "grad_norm": 1.0856293439865112, "tokens_seen": 80781312, "tokens_per_sec": 16167.153236686889, "vram_gb": 6.154519552, "lambda": 0.3091390375047922, "lambda_min": 0.05894634872674942, "lambda_max": 0.6394264698028564, "S_mean": 0.24866739908854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.364814281463623, "grad_norm": 1.032950758934021, "tokens_seen": 80945152, "tokens_per_sec": 16167.43341006354, "vram_gb": 6.154519552, "lambda": 0.3092653672210872, "lambda_min": 0.05895597115159035, "lambda_max": 0.6396888494491577, "S_mean": 0.2566324869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.336815595626831, "grad_norm": 0.9988683462142944, "tokens_seen": 81108992, "tokens_per_sec": 16167.814954208363, "vram_gb": 6.154519552, "lambda": 0.3093430567532778, "lambda_min": 0.0589420348405838, "lambda_max": 0.640038788318634, "S_mean": 0.24821980794270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.303224802017212, "grad_norm": 1.0675979852676392, "tokens_seen": 81272832, "tokens_per_sec": 16168.264261991335, "vram_gb": 6.154519552, "lambda": 0.3094260774863263, "lambda_min": 0.058942120522260666, "lambda_max": 0.6404475569725037, "S_mean": 0.2534586588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.2798192501068115, "grad_norm": 1.058598279953003, "tokens_seen": 81436672, "tokens_per_sec": 16168.582777115564, "vram_gb": 6.154519552, "lambda": 0.3095025531947613, "lambda_min": 0.05894659459590912, "lambda_max": 0.640670120716095, "S_mean": 0.2664896647135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.343724727630615, "grad_norm": 1.0537248849868774, "tokens_seen": 81600512, "tokens_per_sec": 16168.924158018792, "vram_gb": 6.154519552, "lambda": 0.3096251782650749, "lambda_min": 0.05895276367664337, "lambda_max": 0.6409982442855835, "S_mean": 0.25860595703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.394404649734497, "grad_norm": 0.9766550064086914, "tokens_seen": 81764352, "tokens_per_sec": 16169.11639579545, "vram_gb": 6.154519552, "lambda": 0.309730507278194, "lambda_min": 0.05894747003912926, "lambda_max": 0.6415027379989624, "S_mean": 0.24397786458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.334595203399658, "grad_norm": 1.0394740104675293, "tokens_seen": 81928192, "tokens_per_sec": 16169.481224714125, "vram_gb": 6.154519552, "lambda": 0.3098534542756776, "lambda_min": 0.0589635856449604, "lambda_max": 0.64206862449646, "S_mean": 0.24422200520833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.423740541934967, "val_ppl": 83.40769255110658, "lambda": 0.3098575497666995, "lambda_min": 0.05896572768688202, "lambda_max": 0.6420803666114807, "lambda_per_layer": [0.17726796865463257, 0.13249485194683075, 0.14449071884155273, 0.15175101161003113, 0.16720746457576752, 0.1384592056274414, 0.0943450927734375, 0.05896572768688202, 0.12082283198833466, 0.2123839259147644, 0.22336804866790771, 0.2969750165939331, 0.5114628076553345, 0.5591105818748474, 0.4255587160587311, 0.6303460001945496, 0.49718421697616577, 0.6420803666114807, 0.30670249462127686, 0.4050804078578949, 0.4462074041366577, 0.493591845035553, 0.33944252133369446, 0.26128196716308594], "S_mean": 0.2557779947916667, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.6527674198150635, "grad_norm": 1.0918148756027222, "tokens_seen": 82092032, "tokens_per_sec": 16157.945800526943, "vram_gb": 6.154519552, "lambda": 0.30996352480724454, "lambda_min": 0.05899806693196297, "lambda_max": 0.6424871683120728, "S_mean": 0.2549336751302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.25642204284668, "grad_norm": 1.0996732711791992, "tokens_seen": 82255872, "tokens_per_sec": 16158.073756767188, "vram_gb": 6.154519552, "lambda": 0.3100249149526159, "lambda_min": 0.05900256335735321, "lambda_max": 0.6426653861999512, "S_mean": 0.2642110188802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.477592468261719, "grad_norm": 1.0642693042755127, "tokens_seen": 82419712, "tokens_per_sec": 16158.430189258534, "vram_gb": 6.154519552, "lambda": 0.31014164723455906, "lambda_min": 0.05900408327579498, "lambda_max": 0.6430066227912903, "S_mean": 0.25469970703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.2596986293792725, "grad_norm": 1.0649150609970093, "tokens_seen": 82583552, "tokens_per_sec": 16158.791034258506, "vram_gb": 6.154519552, "lambda": 0.3102500283469756, "lambda_min": 0.058970823884010315, "lambda_max": 0.6431912183761597, "S_mean": 0.2642618815104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.307148456573486, "grad_norm": 1.095449686050415, "tokens_seen": 82747392, "tokens_per_sec": 16159.126769440436, "vram_gb": 6.154519552, "lambda": 0.31035940255969763, "lambda_min": 0.05894096940755844, "lambda_max": 0.643488347530365, "S_mean": 0.2601318359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.327325344085693, "grad_norm": 1.0320136547088623, "tokens_seen": 82911232, "tokens_per_sec": 16159.521578983577, "vram_gb": 6.154519552, "lambda": 0.31048025159786147, "lambda_min": 0.05893392115831375, "lambda_max": 0.6439139246940613, "S_mean": 0.256134033203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.268190145492554, "grad_norm": 1.202529788017273, "tokens_seen": 83075072, "tokens_per_sec": 16160.015423115996, "vram_gb": 6.154519552, "lambda": 0.31059078375498456, "lambda_min": 0.05896413326263428, "lambda_max": 0.6442016363143921, "S_mean": 0.2454376220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.376405715942383, "grad_norm": 1.0942546129226685, "tokens_seen": 83238912, "tokens_per_sec": 16160.510276511932, "vram_gb": 6.154519552, "lambda": 0.31069645363216597, "lambda_min": 0.05898554250597954, "lambda_max": 0.6446232199668884, "S_mean": 0.2514546712239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.278775691986084, "grad_norm": 0.9552472829818726, "tokens_seen": 83402752, "tokens_per_sec": 16160.945390737676, "vram_gb": 6.154519552, "lambda": 0.3107927044232686, "lambda_min": 0.05898105353116989, "lambda_max": 0.6449690461158752, "S_mean": 0.2579447428385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.469871997833252, "grad_norm": 1.0784646272659302, "tokens_seen": 83566592, "tokens_per_sec": 16161.478742360438, "vram_gb": 6.154519552, "lambda": 0.31091234227642417, "lambda_min": 0.058975670486688614, "lambda_max": 0.6453151702880859, "S_mean": 0.24933878580729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.397300720214844, "grad_norm": 0.9841864705085754, "tokens_seen": 83730432, "tokens_per_sec": 16161.975475850453, "vram_gb": 6.154519552, "lambda": 0.31099399241308373, "lambda_min": 0.05897195637226105, "lambda_max": 0.6456063389778137, "S_mean": 0.2591145833333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.3390467166900635, "grad_norm": 1.0904959440231323, "tokens_seen": 83894272, "tokens_per_sec": 16162.356330934668, "vram_gb": 6.154519552, "lambda": 0.31114252827440697, "lambda_min": 0.059001948684453964, "lambda_max": 0.6460400223731995, "S_mean": 0.24039713541666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.442246198654175, "grad_norm": 0.9787020087242126, "tokens_seen": 84058112, "tokens_per_sec": 16162.578625511298, "vram_gb": 6.154519552, "lambda": 0.311273121740669, "lambda_min": 0.058982815593481064, "lambda_max": 0.6464645862579346, "S_mean": 0.24363199869791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.346367120742798, "grad_norm": 0.9862992167472839, "tokens_seen": 84221952, "tokens_per_sec": 16162.955005579555, "vram_gb": 6.154519552, "lambda": 0.31137941405177116, "lambda_min": 0.05896229296922684, "lambda_max": 0.646760106086731, "S_mean": 0.2464599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.260626792907715, "grad_norm": 1.1103743314743042, "tokens_seen": 84385792, "tokens_per_sec": 16163.330815492727, "vram_gb": 6.154519552, "lambda": 0.3114518163104852, "lambda_min": 0.058957308530807495, "lambda_max": 0.6470484733581543, "S_mean": 0.24843343098958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.312319755554199, "grad_norm": 1.025649070739746, "tokens_seen": 84549632, "tokens_per_sec": 16163.663862300773, "vram_gb": 6.154519552, "lambda": 0.3115803723533948, "lambda_min": 0.05892683565616608, "lambda_max": 0.6473683714866638, "S_mean": 0.252838134765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.462673664093018, "grad_norm": 1.140591025352478, "tokens_seen": 84713472, "tokens_per_sec": 16164.10069980218, "vram_gb": 6.154519552, "lambda": 0.3117086680916448, "lambda_min": 0.05889930948615074, "lambda_max": 0.6478672027587891, "S_mean": 0.246673583984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.355411052703857, "grad_norm": 0.9868873357772827, "tokens_seen": 84877312, "tokens_per_sec": 16164.179046343708, "vram_gb": 6.154519552, "lambda": 0.3118272772990167, "lambda_min": 0.058909159153699875, "lambda_max": 0.648160994052887, "S_mean": 0.2541097005208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.360039710998535, "grad_norm": 1.0468288660049438, "tokens_seen": 85041152, "tokens_per_sec": 16164.532144928375, "vram_gb": 6.154519552, "lambda": 0.31192218729605276, "lambda_min": 0.05890711396932602, "lambda_max": 0.6484544277191162, "S_mean": 0.254974365234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.474289894104004, "grad_norm": 1.2591983079910278, "tokens_seen": 85204992, "tokens_per_sec": 16165.015552632674, "vram_gb": 6.154519552, "lambda": 0.3119936046811442, "lambda_min": 0.058891404420137405, "lambda_max": 0.6488014459609985, "S_mean": 0.2559916178385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.410906553268433, "grad_norm": 1.177729606628418, "tokens_seen": 85368832, "tokens_per_sec": 16165.445254109425, "vram_gb": 6.154519552, "lambda": 0.31210906136160094, "lambda_min": 0.05890001729130745, "lambda_max": 0.6490575075149536, "S_mean": 0.2753702799479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.424168109893799, "grad_norm": 1.038081407546997, "tokens_seen": 85532672, "tokens_per_sec": 16165.942163951659, "vram_gb": 6.154519552, "lambda": 0.3121997270112236, "lambda_min": 0.058896422386169434, "lambda_max": 0.6493422389030457, "S_mean": 0.2602742513020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.2530012130737305, "grad_norm": 1.068894386291504, "tokens_seen": 85696512, "tokens_per_sec": 16166.365413784404, "vram_gb": 6.154519552, "lambda": 0.31231579557061195, "lambda_min": 0.058908216655254364, "lambda_max": 0.649707555770874, "S_mean": 0.2621256510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.351237535476685, "grad_norm": 1.1720219850540161, "tokens_seen": 85860352, "tokens_per_sec": 16166.776776133587, "vram_gb": 6.154519552, "lambda": 0.3124179888206224, "lambda_min": 0.05891064181923866, "lambda_max": 0.6500126719474792, "S_mean": 0.254608154296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.539949655532837, "grad_norm": 1.3587533235549927, "tokens_seen": 86024192, "tokens_per_sec": 16167.23903448985, "vram_gb": 6.154519552, "lambda": 0.3125395557532708, "lambda_min": 0.05888114869594574, "lambda_max": 0.6504735946655273, "S_mean": 0.248687744140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10500, "val_loss": 4.416546714305878, "val_ppl": 82.80982504420771, "lambda": 0.31254472661142546, "lambda_min": 0.05887966230511665, "lambda_max": 0.6504966020584106, "lambda_per_layer": [0.1786544770002365, 0.13246890902519226, 0.14465448260307312, 0.1522686928510666, 0.16791430115699768, 0.13881239295005798, 0.0942545086145401, 0.05887966230511665, 0.12109208106994629, 0.21278643608093262, 0.2242373675107956, 0.29718703031539917, 0.5131042003631592, 0.5645844340324402, 0.4295453429222107, 0.6377023458480835, 0.5023094415664673, 0.6504966020584106, 0.3091861307621002, 0.409963995218277, 0.452420711517334, 0.5007922649383545, 0.34380418062210083, 0.26395344734191895], "S_mean": 0.24944559733072916, "G_mean": null, "G_std": null, "tokens_seen": 86024192} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.36548638343811, "grad_norm": 1.07090425491333, "tokens_seen": 86188032, "tokens_per_sec": 16156.237629641852, "vram_gb": 6.154519552, "lambda": 0.312620484114935, "lambda_min": 0.058859873563051224, "lambda_max": 0.6508182883262634, "S_mean": 0.2519022623697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.448176383972168, "grad_norm": 1.1948976516723633, "tokens_seen": 86351872, "tokens_per_sec": 16156.61350984744, "vram_gb": 6.154519552, "lambda": 0.31271016225218773, "lambda_min": 0.05883997678756714, "lambda_max": 0.6510190963745117, "S_mean": 0.2567952473958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.43323540687561, "grad_norm": 1.1703537702560425, "tokens_seen": 86515712, "tokens_per_sec": 16157.014888395666, "vram_gb": 6.154519552, "lambda": 0.31277095278104144, "lambda_min": 0.05885232239961624, "lambda_max": 0.6512783765792847, "S_mean": 0.242828369140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.328307390213013, "grad_norm": 1.0560977458953857, "tokens_seen": 86679552, "tokens_per_sec": 16157.51254394799, "vram_gb": 6.154519552, "lambda": 0.3128591288502018, "lambda_min": 0.05885414779186249, "lambda_max": 0.6515753865242004, "S_mean": 0.24473063151041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.242422103881836, "grad_norm": 1.1190170049667358, "tokens_seen": 86843392, "tokens_per_sec": 16157.92986858583, "vram_gb": 6.154519552, "lambda": 0.31294214849670726, "lambda_min": 0.05886123329401016, "lambda_max": 0.6517082452774048, "S_mean": 0.2511698404947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.348520278930664, "grad_norm": 1.165487289428711, "tokens_seen": 87007232, "tokens_per_sec": 16158.32085341834, "vram_gb": 6.154519552, "lambda": 0.3130654306150973, "lambda_min": 0.05888033285737038, "lambda_max": 0.6520227193832397, "S_mean": 0.262451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.45808219909668, "grad_norm": 1.043105125427246, "tokens_seen": 87171072, "tokens_per_sec": 16158.748593004657, "vram_gb": 6.154519552, "lambda": 0.31318632746115327, "lambda_min": 0.05887584760785103, "lambda_max": 0.6524178981781006, "S_mean": 0.2531026204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.274129867553711, "grad_norm": 1.0420323610305786, "tokens_seen": 87334912, "tokens_per_sec": 16159.286107631031, "vram_gb": 6.154519552, "lambda": 0.31331028742715716, "lambda_min": 0.058862026780843735, "lambda_max": 0.6528170108795166, "S_mean": 0.2635599772135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.286383152008057, "grad_norm": 1.0866029262542725, "tokens_seen": 87498752, "tokens_per_sec": 16160.068404718517, "vram_gb": 6.154519552, "lambda": 0.3134273610388239, "lambda_min": 0.05884546786546707, "lambda_max": 0.6531738042831421, "S_mean": 0.2701009114583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.331996917724609, "grad_norm": 1.0074564218521118, "tokens_seen": 87662592, "tokens_per_sec": 16160.776920679764, "vram_gb": 6.154519552, "lambda": 0.3135062408012648, "lambda_min": 0.058828819543123245, "lambda_max": 0.6534337401390076, "S_mean": 0.247314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.354703664779663, "grad_norm": 1.185947299003601, "tokens_seen": 87826432, "tokens_per_sec": 16161.138160871877, "vram_gb": 6.154519552, "lambda": 0.31357066659256816, "lambda_min": 0.05880894884467125, "lambda_max": 0.6536464095115662, "S_mean": 0.2611287434895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.4195404052734375, "grad_norm": 1.1057829856872559, "tokens_seen": 87990272, "tokens_per_sec": 16161.469189367815, "vram_gb": 6.154519552, "lambda": 0.3136758793455859, "lambda_min": 0.05880646035075188, "lambda_max": 0.6540353894233704, "S_mean": 0.24347941080729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.310503005981445, "grad_norm": 1.0523302555084229, "tokens_seen": 88154112, "tokens_per_sec": 16161.869551492611, "vram_gb": 6.154519552, "lambda": 0.31378814407313865, "lambda_min": 0.058796998113393784, "lambda_max": 0.6543342471122742, "S_mean": 0.2535298665364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.3426666259765625, "grad_norm": 1.0629782676696777, "tokens_seen": 88317952, "tokens_per_sec": 16162.275201353339, "vram_gb": 6.154519552, "lambda": 0.313869867939502, "lambda_min": 0.05878717079758644, "lambda_max": 0.6546642780303955, "S_mean": 0.242767333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.419018507003784, "grad_norm": 1.1615185737609863, "tokens_seen": 88481792, "tokens_per_sec": 16162.639811345076, "vram_gb": 6.154519552, "lambda": 0.31393232801929116, "lambda_min": 0.05878124758601189, "lambda_max": 0.6549323797225952, "S_mean": 0.252777099609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.506511926651001, "grad_norm": 1.0620183944702148, "tokens_seen": 88645632, "tokens_per_sec": 16162.982269596241, "vram_gb": 6.154519552, "lambda": 0.31401774551098544, "lambda_min": 0.0587802417576313, "lambda_max": 0.6553073525428772, "S_mean": 0.2565816243489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.194761276245117, "grad_norm": 1.0358160734176636, "tokens_seen": 88809472, "tokens_per_sec": 16163.389015631998, "vram_gb": 6.154519552, "lambda": 0.31410947705929476, "lambda_min": 0.058781858533620834, "lambda_max": 0.6555948257446289, "S_mean": 0.247161865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.371954441070557, "grad_norm": 1.0348118543624878, "tokens_seen": 88973312, "tokens_per_sec": 16163.734889613663, "vram_gb": 6.154519552, "lambda": 0.3141969363205135, "lambda_min": 0.05876951292157173, "lambda_max": 0.6558333039283752, "S_mean": 0.243438720703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.200717210769653, "grad_norm": 1.1364519596099854, "tokens_seen": 89137152, "tokens_per_sec": 16164.017167658256, "vram_gb": 6.154519552, "lambda": 0.3142947376084824, "lambda_min": 0.058776263147592545, "lambda_max": 0.6560938358306885, "S_mean": 0.250396728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.343573093414307, "grad_norm": 1.0384345054626465, "tokens_seen": 89300992, "tokens_per_sec": 16164.457059227427, "vram_gb": 6.154519552, "lambda": 0.31438156589865685, "lambda_min": 0.05878125876188278, "lambda_max": 0.6563131213188171, "S_mean": 0.2534281412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.351819753646851, "grad_norm": 1.0778608322143555, "tokens_seen": 89464832, "tokens_per_sec": 16164.862451456896, "vram_gb": 6.154519552, "lambda": 0.31448253119985264, "lambda_min": 0.05878305435180664, "lambda_max": 0.6565413475036621, "S_mean": 0.251953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.288221597671509, "grad_norm": 1.0533968210220337, "tokens_seen": 89628672, "tokens_per_sec": 16165.276545835846, "vram_gb": 6.154519552, "lambda": 0.31458225365107256, "lambda_min": 0.05880073830485344, "lambda_max": 0.6568055152893066, "S_mean": 0.2536265055338542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.30670690536499, "grad_norm": 1.0214155912399292, "tokens_seen": 89792512, "tokens_per_sec": 16165.676792744785, "vram_gb": 6.154519552, "lambda": 0.31466511357575655, "lambda_min": 0.05880197882652283, "lambda_max": 0.6570610404014587, "S_mean": 0.2412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.5007288455963135, "grad_norm": 1.2241029739379883, "tokens_seen": 89956352, "tokens_per_sec": 16165.99718275913, "vram_gb": 6.154519552, "lambda": 0.3147580255754292, "lambda_min": 0.05880400910973549, "lambda_max": 0.6573865413665771, "S_mean": 0.2543131510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.302428483963013, "grad_norm": 1.036862850189209, "tokens_seen": 90120192, "tokens_per_sec": 16166.237096631383, "vram_gb": 6.154519552, "lambda": 0.31484835284451645, "lambda_min": 0.05878438800573349, "lambda_max": 0.6577079892158508, "S_mean": 0.2539469401041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.399602925777435, "val_ppl": 81.41853304486119, "lambda": 0.31485191856821376, "lambda_min": 0.058783285319805145, "lambda_max": 0.6577256321907043, "lambda_per_layer": [0.17984019219875336, 0.13260158896446228, 0.14483506977558136, 0.15273012220859528, 0.1685105860233307, 0.13904817402362823, 0.09438108652830124, 0.058783285319805145, 0.1212691068649292, 0.21328967809677124, 0.22468824684619904, 0.29817864298820496, 0.5152888894081116, 0.568993091583252, 0.43263930082321167, 0.6435520648956299, 0.5059911608695984, 0.6577256321907043, 0.3121890127658844, 0.4144328534603119, 0.4573392868041992, 0.5066177248954773, 0.3470335304737091, 0.266487717628479], "S_mean": 0.2567952473958333, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.121327638626099, "grad_norm": 1.0793097019195557, "tokens_seen": 90284032, "tokens_per_sec": 16155.828098649596, "vram_gb": 6.154519552, "lambda": 0.3149232864379883, "lambda_min": 0.058772921562194824, "lambda_max": 0.6580298542976379, "S_mean": 0.2530619303385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.340213060379028, "grad_norm": 1.0940210819244385, "tokens_seen": 90447872, "tokens_per_sec": 16156.246027591647, "vram_gb": 6.154519552, "lambda": 0.3150128290678064, "lambda_min": 0.05876714736223221, "lambda_max": 0.6582486033439636, "S_mean": 0.269012451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.381588697433472, "grad_norm": 1.1859121322631836, "tokens_seen": 90611712, "tokens_per_sec": 16156.575629268045, "vram_gb": 6.154519552, "lambda": 0.31513188205038506, "lambda_min": 0.05875619873404503, "lambda_max": 0.6586161255836487, "S_mean": 0.2504984537760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.398100852966309, "grad_norm": 1.085322380065918, "tokens_seen": 90775552, "tokens_per_sec": 16157.100081590435, "vram_gb": 6.154519552, "lambda": 0.3152274911602338, "lambda_min": 0.05873687565326691, "lambda_max": 0.6588354110717773, "S_mean": 0.2590230305989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.4048285484313965, "grad_norm": 1.1053965091705322, "tokens_seen": 90939392, "tokens_per_sec": 16157.427633477762, "vram_gb": 6.154519552, "lambda": 0.3153082470720013, "lambda_min": 0.05872651934623718, "lambda_max": 0.6589347124099731, "S_mean": 0.2550557454427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.393081903457642, "grad_norm": 1.0616968870162964, "tokens_seen": 91103232, "tokens_per_sec": 16157.835807838419, "vram_gb": 6.154519552, "lambda": 0.31538375342885655, "lambda_min": 0.05873260647058487, "lambda_max": 0.6590872406959534, "S_mean": 0.27294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.258970260620117, "grad_norm": 1.1060668230056763, "tokens_seen": 91267072, "tokens_per_sec": 16158.126594181631, "vram_gb": 6.154519552, "lambda": 0.3154756515286863, "lambda_min": 0.058718111366033554, "lambda_max": 0.6595132946968079, "S_mean": 0.2507731119791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.289479494094849, "grad_norm": 1.0724601745605469, "tokens_seen": 91430912, "tokens_per_sec": 16158.467377171275, "vram_gb": 6.154519552, "lambda": 0.31555634643882513, "lambda_min": 0.058700114488601685, "lambda_max": 0.6598288416862488, "S_mean": 0.2615559895833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.326247930526733, "grad_norm": 1.024353265762329, "tokens_seen": 91594752, "tokens_per_sec": 16158.873679968365, "vram_gb": 6.154519552, "lambda": 0.3156367408422132, "lambda_min": 0.05869463458657265, "lambda_max": 0.659997284412384, "S_mean": 0.24810791015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.491636276245117, "grad_norm": 1.208600640296936, "tokens_seen": 91758592, "tokens_per_sec": 16159.11855244479, "vram_gb": 6.154519552, "lambda": 0.31571987783536315, "lambda_min": 0.058675315231084824, "lambda_max": 0.6602331399917603, "S_mean": 0.252044677734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.350372076034546, "grad_norm": 1.1267948150634766, "tokens_seen": 91922432, "tokens_per_sec": 16159.561987073577, "vram_gb": 6.154519552, "lambda": 0.31580875456954044, "lambda_min": 0.05867099389433861, "lambda_max": 0.6604766845703125, "S_mean": 0.24956258138020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.2690160274505615, "grad_norm": 1.1160309314727783, "tokens_seen": 92086272, "tokens_per_sec": 16159.970624935728, "vram_gb": 6.154519552, "lambda": 0.3159041058582564, "lambda_min": 0.05867866799235344, "lambda_max": 0.6607021689414978, "S_mean": 0.2508443196614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.295346260070801, "grad_norm": 1.0365296602249146, "tokens_seen": 92250112, "tokens_per_sec": 16160.51131517206, "vram_gb": 6.154519552, "lambda": 0.31598934189726907, "lambda_min": 0.05869782716035843, "lambda_max": 0.6609659194946289, "S_mean": 0.2596435546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.195167541503906, "grad_norm": 1.2146022319793701, "tokens_seen": 92413952, "tokens_per_sec": 16160.997160394807, "vram_gb": 6.154519552, "lambda": 0.3160843026513855, "lambda_min": 0.05868126451969147, "lambda_max": 0.6612551212310791, "S_mean": 0.2526753743489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.390127420425415, "grad_norm": 1.0854005813598633, "tokens_seen": 92577792, "tokens_per_sec": 16161.392649502865, "vram_gb": 6.154519552, "lambda": 0.31617185349265736, "lambda_min": 0.058671168982982635, "lambda_max": 0.6615631580352783, "S_mean": 0.254119873046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.224596977233887, "grad_norm": 1.1289573907852173, "tokens_seen": 92741632, "tokens_per_sec": 16161.784470024937, "vram_gb": 6.154519552, "lambda": 0.3162553041862945, "lambda_min": 0.058662544935941696, "lambda_max": 0.6618025898933411, "S_mean": 0.2828369140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.400710344314575, "grad_norm": 1.1649013757705688, "tokens_seen": 92905472, "tokens_per_sec": 16162.260455921665, "vram_gb": 6.154519552, "lambda": 0.3163252801944812, "lambda_min": 0.058640606701374054, "lambda_max": 0.6620303392410278, "S_mean": 0.24701944986979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.270214557647705, "grad_norm": 1.1802634000778198, "tokens_seen": 93069312, "tokens_per_sec": 16162.624401501844, "vram_gb": 6.154519552, "lambda": 0.3164070590088765, "lambda_min": 0.05863258242607117, "lambda_max": 0.6623739004135132, "S_mean": 0.24655659993489584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.338282108306885, "grad_norm": 1.2416560649871826, "tokens_seen": 93233152, "tokens_per_sec": 16163.048631834345, "vram_gb": 6.154519552, "lambda": 0.31648378695050877, "lambda_min": 0.05864176154136658, "lambda_max": 0.6625869870185852, "S_mean": 0.25592041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.401852369308472, "grad_norm": 1.1150466203689575, "tokens_seen": 93396992, "tokens_per_sec": 16163.436567882374, "vram_gb": 6.154519552, "lambda": 0.31654646232103306, "lambda_min": 0.05864531919360161, "lambda_max": 0.6627156138420105, "S_mean": 0.241912841796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.074006915092468, "grad_norm": 1.0493344068527222, "tokens_seen": 93560832, "tokens_per_sec": 16163.659260418686, "vram_gb": 6.154519552, "lambda": 0.3166309637017548, "lambda_min": 0.05864236131310463, "lambda_max": 0.662891149520874, "S_mean": 0.2494964599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.188676357269287, "grad_norm": 1.1317088603973389, "tokens_seen": 93724672, "tokens_per_sec": 16164.009681664571, "vram_gb": 6.154519552, "lambda": 0.3167249468776087, "lambda_min": 0.058632176369428635, "lambda_max": 0.6630886793136597, "S_mean": 0.249908447265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.322199583053589, "grad_norm": 1.149908185005188, "tokens_seen": 93888512, "tokens_per_sec": 16164.17511998647, "vram_gb": 6.154519552, "lambda": 0.31683856016024947, "lambda_min": 0.05863628163933754, "lambda_max": 0.6633075475692749, "S_mean": 0.2533365885416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.373352527618408, "grad_norm": 1.108832597732544, "tokens_seen": 94052352, "tokens_per_sec": 16164.506007228603, "vram_gb": 6.154519552, "lambda": 0.31691893469542265, "lambda_min": 0.05865372717380524, "lambda_max": 0.6634109616279602, "S_mean": 0.24808756510416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.522469520568848, "grad_norm": 1.450771450996399, "tokens_seen": 94216192, "tokens_per_sec": 16164.913360942199, "vram_gb": 6.154519552, "lambda": 0.31699860421940684, "lambda_min": 0.05866527184844017, "lambda_max": 0.6636894345283508, "S_mean": 0.24240620930989584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11500, "val_loss": 4.397210085391999, "val_ppl": 81.22394439350633, "lambda": 0.31700122837598127, "lambda_min": 0.05866597965359688, "lambda_max": 0.6636961102485657, "lambda_per_layer": [0.18105733394622803, 0.13262121379375458, 0.14525507390499115, 0.15309399366378784, 0.1690768450498581, 0.13920189440250397, 0.09457358717918396, 0.05866597965359688, 0.12133503705263138, 0.21350783109664917, 0.22564631700515747, 0.29870277643203735, 0.5167880058288574, 0.5730273127555847, 0.43562051653862, 0.6492998600006104, 0.5096819400787354, 0.6636961102485657, 0.3146803081035614, 0.4191684424877167, 0.4624413251876831, 0.5120781660079956, 0.3499574065208435, 0.26885220408439636], "S_mean": 0.24878946940104166, "G_mean": null, "G_std": null, "tokens_seen": 94216192} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.2918009757995605, "grad_norm": 1.080014944076538, "tokens_seen": 94380032, "tokens_per_sec": 16154.784329274571, "vram_gb": 6.154519552, "lambda": 0.3170652633222441, "lambda_min": 0.05867539718747139, "lambda_max": 0.663827657699585, "S_mean": 0.24607340494791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.200302362442017, "grad_norm": 1.1554142236709595, "tokens_seen": 94543872, "tokens_per_sec": 16155.187450677831, "vram_gb": 6.154519552, "lambda": 0.317131075852861, "lambda_min": 0.05867258086800575, "lambda_max": 0.6641530990600586, "S_mean": 0.2507273356119792, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.329330682754517, "grad_norm": 1.0994219779968262, "tokens_seen": 94707712, "tokens_per_sec": 16155.613384801447, "vram_gb": 6.154519552, "lambda": 0.31719852921863395, "lambda_min": 0.058673836290836334, "lambda_max": 0.6644182801246643, "S_mean": 0.24755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.3580920696258545, "grad_norm": 1.1040372848510742, "tokens_seen": 94871552, "tokens_per_sec": 16156.143581184378, "vram_gb": 6.154519552, "lambda": 0.3172711847970883, "lambda_min": 0.058684542775154114, "lambda_max": 0.6646113395690918, "S_mean": 0.255340576171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.182127952575684, "grad_norm": 1.098736047744751, "tokens_seen": 95035392, "tokens_per_sec": 16156.48158922216, "vram_gb": 6.154519552, "lambda": 0.3173346972713868, "lambda_min": 0.0586794838309288, "lambda_max": 0.6647747159004211, "S_mean": 0.24424235026041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.4416186809539795, "grad_norm": 1.12491774559021, "tokens_seen": 95199232, "tokens_per_sec": 16156.993120067522, "vram_gb": 6.154519552, "lambda": 0.3174167543960114, "lambda_min": 0.05867746099829674, "lambda_max": 0.6651209592819214, "S_mean": 0.24645487467447916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.376117944717407, "grad_norm": 1.2050518989562988, "tokens_seen": 95363072, "tokens_per_sec": 16157.452772094226, "vram_gb": 6.154519552, "lambda": 0.31748952344059944, "lambda_min": 0.05870075523853302, "lambda_max": 0.6653324365615845, "S_mean": 0.24726359049479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.368203163146973, "grad_norm": 1.0429097414016724, "tokens_seen": 95526912, "tokens_per_sec": 16157.849038250013, "vram_gb": 6.154519552, "lambda": 0.31755445928623277, "lambda_min": 0.058699220418930054, "lambda_max": 0.6655187606811523, "S_mean": 0.24729410807291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.274087190628052, "grad_norm": 1.0371888875961304, "tokens_seen": 95690752, "tokens_per_sec": 16158.265390278808, "vram_gb": 6.154519552, "lambda": 0.31763995019719005, "lambda_min": 0.05871380493044853, "lambda_max": 0.6657116413116455, "S_mean": 0.26593017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.303732872009277, "grad_norm": 1.062308669090271, "tokens_seen": 95854592, "tokens_per_sec": 16158.70780391129, "vram_gb": 6.154519552, "lambda": 0.3177475438763698, "lambda_min": 0.05872512608766556, "lambda_max": 0.6659604907035828, "S_mean": 0.24651082356770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.16709041595459, "grad_norm": 1.0522147417068481, "tokens_seen": 96018432, "tokens_per_sec": 16158.946185746732, "vram_gb": 6.154519552, "lambda": 0.31783172860741615, "lambda_min": 0.058732420206069946, "lambda_max": 0.6661943197250366, "S_mean": 0.2553914388020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.394485235214233, "grad_norm": 1.1505255699157715, "tokens_seen": 96182272, "tokens_per_sec": 16159.38575001894, "vram_gb": 6.154519552, "lambda": 0.3178861439228058, "lambda_min": 0.058732785284519196, "lambda_max": 0.666391134262085, "S_mean": 0.257232666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.269838809967041, "grad_norm": 1.1924883127212524, "tokens_seen": 96346112, "tokens_per_sec": 16159.802644582882, "vram_gb": 6.154519552, "lambda": 0.31793792685493827, "lambda_min": 0.05872931703925133, "lambda_max": 0.6664943099021912, "S_mean": 0.25067138671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.257530212402344, "grad_norm": 1.1523418426513672, "tokens_seen": 96509952, "tokens_per_sec": 16160.178577312143, "vram_gb": 6.154519552, "lambda": 0.3180032406623165, "lambda_min": 0.058741867542266846, "lambda_max": 0.6667690277099609, "S_mean": 0.24709065755208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.250849962234497, "grad_norm": 1.1295812129974365, "tokens_seen": 96673792, "tokens_per_sec": 16160.552568540432, "vram_gb": 6.154519552, "lambda": 0.31810388589898747, "lambda_min": 0.058736495673656464, "lambda_max": 0.667058527469635, "S_mean": 0.24591064453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.327347993850708, "grad_norm": 1.0771729946136475, "tokens_seen": 96837632, "tokens_per_sec": 16160.935108623678, "vram_gb": 6.154519552, "lambda": 0.31819315332298476, "lambda_min": 0.05871133133769035, "lambda_max": 0.6673831939697266, "S_mean": 0.2487945556640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.299740791320801, "grad_norm": 1.070960521697998, "tokens_seen": 97001472, "tokens_per_sec": 16161.329289005787, "vram_gb": 6.154519552, "lambda": 0.31826623653372127, "lambda_min": 0.05871815234422684, "lambda_max": 0.6676263809204102, "S_mean": 0.24867757161458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.307725667953491, "grad_norm": 1.1526795625686646, "tokens_seen": 97165312, "tokens_per_sec": 16161.701273540128, "vram_gb": 6.154519552, "lambda": 0.31833786983042955, "lambda_min": 0.058702901005744934, "lambda_max": 0.6677777767181396, "S_mean": 0.24881998697916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.2761311531066895, "grad_norm": 1.119242548942566, "tokens_seen": 97329152, "tokens_per_sec": 16162.213823490834, "vram_gb": 6.154519552, "lambda": 0.3184195230714977, "lambda_min": 0.05871060863137245, "lambda_max": 0.6680411696434021, "S_mean": 0.2555338541666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.395169496536255, "grad_norm": 1.2053351402282715, "tokens_seen": 97492992, "tokens_per_sec": 16162.448703334312, "vram_gb": 6.154519552, "lambda": 0.3184930654242635, "lambda_min": 0.0586995854973793, "lambda_max": 0.6683670282363892, "S_mean": 0.24784342447916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.24543309211731, "grad_norm": 1.2097294330596924, "tokens_seen": 97656832, "tokens_per_sec": 16162.93759322974, "vram_gb": 6.154519552, "lambda": 0.3185646793184181, "lambda_min": 0.058678749948740005, "lambda_max": 0.6685283184051514, "S_mean": 0.2551778157552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.3118767738342285, "grad_norm": 1.0894191265106201, "tokens_seen": 97820672, "tokens_per_sec": 16163.321618418333, "vram_gb": 6.154519552, "lambda": 0.31862772318224114, "lambda_min": 0.058677904307842255, "lambda_max": 0.6686750054359436, "S_mean": 0.2479248046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.216917514801025, "grad_norm": 1.1176687479019165, "tokens_seen": 97984512, "tokens_per_sec": 16163.68263534709, "vram_gb": 6.154519552, "lambda": 0.3186955444204311, "lambda_min": 0.05869504436850548, "lambda_max": 0.6687683463096619, "S_mean": 0.2534688313802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.313776254653931, "grad_norm": 1.1351501941680908, "tokens_seen": 98148352, "tokens_per_sec": 16164.103936140758, "vram_gb": 6.154519552, "lambda": 0.3187686977908015, "lambda_min": 0.05869493633508682, "lambda_max": 0.6691104173660278, "S_mean": 0.2575480143229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.392780220508575, "val_ppl": 80.86492907631813, "lambda": 0.31883614308511216, "lambda_min": 0.058697257190942764, "lambda_max": 0.6693422198295593, "lambda_per_layer": [0.1821700781583786, 0.13264930248260498, 0.1452958583831787, 0.15345242619514465, 0.1694808453321457, 0.1393585205078125, 0.09471017867326736, 0.058697257190942764, 0.12137100100517273, 0.21377667784690857, 0.22598251700401306, 0.2991520166397095, 0.5182293653488159, 0.5768132209777832, 0.43797433376312256, 0.6536375284194946, 0.5130853652954102, 0.6693422198295593, 0.31673625111579895, 0.42259830236434937, 0.46644294261932373, 0.5169222354888916, 0.35338646173477173, 0.2708025276660919], "S_mean": 0.2588094075520833, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 6085.201226711273, "nan_failures": 0} diff --git a/s_only_s2/config.json b/s_only_s2/config.json new file mode 100644 index 0000000000000000000000000000000000000000..42b167872a079f5bc14dc0e94cfb2f7b6a211029 --- /dev/null +++ b/s_only_s2/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 2, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s2", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 2001343, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/s_only_s2/final.pt b/s_only_s2/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..ff9f5db7e3de2fc8a463f935fbfa68a381fa521d --- /dev/null +++ b/s_only_s2/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4252bc6d9a898cef24e62aa25450e809611b3cdb5609347cd592effcae995ef +size 255945415 diff --git a/s_only_s2/metrics.jsonl b/s_only_s2/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..87ef41f182a2ef76337ab4a7b80c94e9df0e6014 --- /dev/null +++ b/s_only_s2/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 2, "config": {"variant": "s_only", "seed": 2, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s2", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 2001343, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.940910339355469, "grad_norm": 14.17945671081543, "tokens_seen": 8192, "tokens_per_sec": 8126.723540288846, "vram_gb": 5.017291776, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.32544199625651044, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.887802124023438, "grad_norm": 1.9021415710449219, "tokens_seen": 172032, "tokens_per_sec": 19565.617683357475, "vram_gb": 5.521900544, "lambda": 0.12692837851742902, "lambda_min": 0.126896470785141, "lambda_max": 0.12696804106235504, "S_mean": 0.4959665934244792, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.077728271484375, "grad_norm": 1.7641005516052246, "tokens_seen": 335872, "tokens_per_sec": 20317.55622064328, "vram_gb": 5.521900544, "lambda": 0.12686570671697459, "lambda_min": 0.12679675221443176, "lambda_max": 0.1270575374364853, "S_mean": 0.5110371907552084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.17807388305664, "grad_norm": 1.5157204866409302, "tokens_seen": 499712, "tokens_per_sec": 20584.547422234664, "vram_gb": 5.521900544, "lambda": 0.1267942531655232, "lambda_min": 0.1266922503709793, "lambda_max": 0.12716424465179443, "S_mean": 0.5029754638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.560121536254883, "grad_norm": 0.8668480515480042, "tokens_seen": 663552, "tokens_per_sec": 20698.12435119302, "vram_gb": 5.521900544, "lambda": 0.1266943079729875, "lambda_min": 0.1264900267124176, "lambda_max": 0.1273195743560791, "S_mean": 0.4011739095052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.249262809753418, "grad_norm": 0.6871859431266785, "tokens_seen": 827392, "tokens_per_sec": 20788.5478545669, "vram_gb": 5.521900544, "lambda": 0.12660098386307558, "lambda_min": 0.1262539029121399, "lambda_max": 0.1275126039981842, "S_mean": 0.3141428629557292, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.149725914001465, "grad_norm": 0.6239500045776367, "tokens_seen": 991232, "tokens_per_sec": 20835.101469399608, "vram_gb": 5.521900544, "lambda": 0.1265279327829679, "lambda_min": 0.12597014009952545, "lambda_max": 0.12780530750751495, "S_mean": 0.312652587890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 7.032435655593872, "grad_norm": 0.6659835577011108, "tokens_seen": 1155072, "tokens_per_sec": 20864.4712518204, "vram_gb": 5.521900544, "lambda": 0.12647979520261288, "lambda_min": 0.12577685713768005, "lambda_max": 0.12812694907188416, "S_mean": 0.2666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.73395848274231, "grad_norm": 0.5692887306213379, "tokens_seen": 1318912, "tokens_per_sec": 20897.54701672416, "vram_gb": 5.521900544, "lambda": 0.1264337282627821, "lambda_min": 0.12570977210998535, "lambda_max": 0.12830787897109985, "S_mean": 0.2583414713541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.86512565612793, "grad_norm": 0.6951659917831421, "tokens_seen": 1482752, "tokens_per_sec": 20925.82780568318, "vram_gb": 5.521900544, "lambda": 0.12645062804222107, "lambda_min": 0.1256422996520996, "lambda_max": 0.12855954468250275, "S_mean": 0.2732035319010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.6360228061676025, "grad_norm": 0.5788378715515137, "tokens_seen": 1646592, "tokens_per_sec": 20942.147624040048, "vram_gb": 5.521900544, "lambda": 0.12645902298390865, "lambda_min": 0.12539386749267578, "lambda_max": 0.12879851460456848, "S_mean": 0.2760416666666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.646700382232666, "grad_norm": 0.5835967063903809, "tokens_seen": 1810432, "tokens_per_sec": 20954.97773570612, "vram_gb": 5.521900544, "lambda": 0.12642931938171387, "lambda_min": 0.12525337934494019, "lambda_max": 0.12901996076107025, "S_mean": 0.2564595540364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.5953497886657715, "grad_norm": 0.5198419094085693, "tokens_seen": 1974272, "tokens_per_sec": 20979.75283403231, "vram_gb": 5.521900544, "lambda": 0.12646159219245115, "lambda_min": 0.12501215934753418, "lambda_max": 0.12923212349414825, "S_mean": 0.2689463297526042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.427461862564087, "grad_norm": 0.5166700482368469, "tokens_seen": 2138112, "tokens_per_sec": 20986.815475519183, "vram_gb": 5.521900544, "lambda": 0.1264669212202231, "lambda_min": 0.12467905133962631, "lambda_max": 0.12956039607524872, "S_mean": 0.24896240234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.516310214996338, "grad_norm": 0.5005192756652832, "tokens_seen": 2301952, "tokens_per_sec": 21002.03485034918, "vram_gb": 5.521900544, "lambda": 0.1264281782011191, "lambda_min": 0.12450550496578217, "lambda_max": 0.12967883050441742, "S_mean": 0.2620951334635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.665858030319214, "grad_norm": 0.4919309616088867, "tokens_seen": 2465792, "tokens_per_sec": 21011.185024691014, "vram_gb": 5.521900544, "lambda": 0.12640829725811878, "lambda_min": 0.12450889497995377, "lambda_max": 0.1298515498638153, "S_mean": 0.2158203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.613470792770386, "grad_norm": 0.5214915871620178, "tokens_seen": 2629632, "tokens_per_sec": 21016.78333925963, "vram_gb": 5.521900544, "lambda": 0.12643759263058504, "lambda_min": 0.1245390772819519, "lambda_max": 0.12994547188282013, "S_mean": 0.266510009765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.484394550323486, "grad_norm": 0.53399258852005, "tokens_seen": 2793472, "tokens_per_sec": 21026.26440577899, "vram_gb": 5.521900544, "lambda": 0.12648962500194708, "lambda_min": 0.12459962069988251, "lambda_max": 0.13026843965053558, "S_mean": 0.25390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.406179428100586, "grad_norm": 0.5364183187484741, "tokens_seen": 2957312, "tokens_per_sec": 21028.546644303966, "vram_gb": 5.521900544, "lambda": 0.12656068118909994, "lambda_min": 0.1245930939912796, "lambda_max": 0.13040074706077576, "S_mean": 0.2698771158854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.278898000717163, "grad_norm": 0.5227168202400208, "tokens_seen": 3121152, "tokens_per_sec": 21038.608592531862, "vram_gb": 5.521900544, "lambda": 0.1266042236238718, "lambda_min": 0.12460063397884369, "lambda_max": 0.13068358600139618, "S_mean": 0.28125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.486760139465332, "grad_norm": 0.5379012823104858, "tokens_seen": 3284992, "tokens_per_sec": 21040.00679798616, "vram_gb": 5.521900544, "lambda": 0.12663555362572274, "lambda_min": 0.12455307692289352, "lambda_max": 0.1311202049255371, "S_mean": 0.3297220865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.3156561851501465, "grad_norm": 0.5471468567848206, "tokens_seen": 3448832, "tokens_per_sec": 21045.37085168395, "vram_gb": 5.521900544, "lambda": 0.12671906097481647, "lambda_min": 0.12448056787252426, "lambda_max": 0.13146552443504333, "S_mean": 0.2650655110677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.308719873428345, "grad_norm": 0.5140279531478882, "tokens_seen": 3612672, "tokens_per_sec": 21047.8792062594, "vram_gb": 5.521900544, "lambda": 0.1267982693389058, "lambda_min": 0.12450366467237473, "lambda_max": 0.13205154240131378, "S_mean": 0.2958475748697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.259105682373047, "grad_norm": 0.4980272948741913, "tokens_seen": 3776512, "tokens_per_sec": 21053.741638189455, "vram_gb": 5.521900544, "lambda": 0.12690138320128122, "lambda_min": 0.1245124489068985, "lambda_max": 0.1324598640203476, "S_mean": 0.2747090657552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.497291326522827, "grad_norm": 0.49034297466278076, "tokens_seen": 3940352, "tokens_per_sec": 21052.920631374243, "vram_gb": 5.521900544, "lambda": 0.12700510739038387, "lambda_min": 0.12470205873250961, "lambda_max": 0.1331263929605484, "S_mean": 0.3069661458333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.339718341827393, "grad_norm": 0.5607737898826599, "tokens_seen": 4104192, "tokens_per_sec": 21040.400943071178, "vram_gb": 5.521900544, "lambda": 0.12715404046078524, "lambda_min": 0.1250620037317276, "lambda_max": 0.1334594041109085, "S_mean": 0.2963409423828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.358003616333008, "grad_norm": 0.5227742791175842, "tokens_seen": 4268032, "tokens_per_sec": 21047.44280401729, "vram_gb": 5.521900544, "lambda": 0.12734212602178255, "lambda_min": 0.12536467611789703, "lambda_max": 0.13385812938213348, "S_mean": 0.2760009765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.0998804569244385, "grad_norm": 0.49015119671821594, "tokens_seen": 4431872, "tokens_per_sec": 21050.60311129521, "vram_gb": 5.521900544, "lambda": 0.12745685689151287, "lambda_min": 0.12541185319423676, "lambda_max": 0.13426068425178528, "S_mean": 0.3201192220052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.061691045761108, "grad_norm": 0.5646761059761047, "tokens_seen": 4595712, "tokens_per_sec": 21054.756785927246, "vram_gb": 5.521900544, "lambda": 0.12762061941126981, "lambda_min": 0.12548519670963287, "lambda_max": 0.13503426313400269, "S_mean": 0.3044840494791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 5.927958011627197, "grad_norm": 0.5279897451400757, "tokens_seen": 4759552, "tokens_per_sec": 21056.796905814263, "vram_gb": 5.521900544, "lambda": 0.12780083467562994, "lambda_min": 0.12569676339626312, "lambda_max": 0.13587962090969086, "S_mean": 0.33209228515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.06975531578064, "grad_norm": 0.5308067798614502, "tokens_seen": 4923392, "tokens_per_sec": 21061.05660725932, "vram_gb": 5.521900544, "lambda": 0.12803988282879195, "lambda_min": 0.12587372958660126, "lambda_max": 0.13650861382484436, "S_mean": 0.2992757161458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.231961250305176, "grad_norm": 0.5483922958374023, "tokens_seen": 5087232, "tokens_per_sec": 21064.38991017719, "vram_gb": 5.521900544, "lambda": 0.12824469866851965, "lambda_min": 0.1259113848209381, "lambda_max": 0.13683469593524933, "S_mean": 0.2866007486979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.153791427612305, "grad_norm": 0.5336629152297974, "tokens_seen": 5251072, "tokens_per_sec": 21067.030310990314, "vram_gb": 5.521900544, "lambda": 0.12845907794932523, "lambda_min": 0.1256241798400879, "lambda_max": 0.13764558732509613, "S_mean": 0.267822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.143269777297974, "grad_norm": 0.532779335975647, "tokens_seen": 5414912, "tokens_per_sec": 21069.663914370867, "vram_gb": 5.521900544, "lambda": 0.1287163874755303, "lambda_min": 0.1251278817653656, "lambda_max": 0.13825350999832153, "S_mean": 0.30084228515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.079803228378296, "grad_norm": 0.5073944926261902, "tokens_seen": 5578752, "tokens_per_sec": 21069.266120772863, "vram_gb": 5.521900544, "lambda": 0.1289759629095594, "lambda_min": 0.1247439906001091, "lambda_max": 0.13897578418254852, "S_mean": 0.302642822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.980813980102539, "grad_norm": 0.4944467842578888, "tokens_seen": 5742592, "tokens_per_sec": 21072.119162105406, "vram_gb": 5.521900544, "lambda": 0.1292309695854783, "lambda_min": 0.12459900230169296, "lambda_max": 0.13952676951885223, "S_mean": 0.3010203043619792, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 6.0470335483551025, "grad_norm": 0.6655526161193848, "tokens_seen": 5906432, "tokens_per_sec": 21073.31373298245, "vram_gb": 5.521900544, "lambda": 0.12961956827590862, "lambda_min": 0.12443649023771286, "lambda_max": 0.14013735949993134, "S_mean": 0.302490234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.022952556610107, "grad_norm": 0.5854817032814026, "tokens_seen": 6070272, "tokens_per_sec": 21069.653693232576, "vram_gb": 5.521900544, "lambda": 0.12988734431564808, "lambda_min": 0.12436233460903168, "lambda_max": 0.1407901793718338, "S_mean": 0.24860636393229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.03270149230957, "grad_norm": 0.5492920875549316, "tokens_seen": 6234112, "tokens_per_sec": 21071.37971299345, "vram_gb": 5.521900544, "lambda": 0.13020737345019975, "lambda_min": 0.12405246496200562, "lambda_max": 0.14133715629577637, "S_mean": 0.2968953450520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 5.793671607971191, "grad_norm": 0.499827116727829, "tokens_seen": 6397952, "tokens_per_sec": 21071.809275284897, "vram_gb": 5.521900544, "lambda": 0.13058073415110508, "lambda_min": 0.12382002919912338, "lambda_max": 0.14181604981422424, "S_mean": 0.30230712890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.907901287078857, "grad_norm": 0.5497183203697205, "tokens_seen": 6561792, "tokens_per_sec": 21073.93178004644, "vram_gb": 5.521900544, "lambda": 0.13093745801597834, "lambda_min": 0.12350677698850632, "lambda_max": 0.14235356450080872, "S_mean": 0.2940266927083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.862401485443115, "grad_norm": 0.5372743010520935, "tokens_seen": 6725632, "tokens_per_sec": 21072.595916068687, "vram_gb": 5.521900544, "lambda": 0.1313253886376818, "lambda_min": 0.12332051247358322, "lambda_max": 0.14271970093250275, "S_mean": 0.2866414388020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.926397800445557, "grad_norm": 0.502406120300293, "tokens_seen": 6889472, "tokens_per_sec": 21074.899509074145, "vram_gb": 5.521900544, "lambda": 0.1317227923621734, "lambda_min": 0.12308110296726227, "lambda_max": 0.1431252658367157, "S_mean": 0.3131510416666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.897989273071289, "grad_norm": 0.5240710377693176, "tokens_seen": 7053312, "tokens_per_sec": 21077.17989367485, "vram_gb": 5.521900544, "lambda": 0.132078694490095, "lambda_min": 0.12292221933603287, "lambda_max": 0.14358076453208923, "S_mean": 0.2827555338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.9965901374816895, "grad_norm": 0.566005527973175, "tokens_seen": 7217152, "tokens_per_sec": 21080.199884331683, "vram_gb": 5.521900544, "lambda": 0.1324861655011773, "lambda_min": 0.12256816774606705, "lambda_max": 0.14421744644641876, "S_mean": 0.2577311197916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.7908971309661865, "grad_norm": 0.5408899188041687, "tokens_seen": 7380992, "tokens_per_sec": 21080.136441801853, "vram_gb": 5.521900544, "lambda": 0.1329219943533341, "lambda_min": 0.12220194935798645, "lambda_max": 0.14491970837116241, "S_mean": 0.2764485677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.946794509887695, "grad_norm": 0.54109787940979, "tokens_seen": 7544832, "tokens_per_sec": 21082.57700798504, "vram_gb": 5.521900544, "lambda": 0.13340729009360075, "lambda_min": 0.12179190665483475, "lambda_max": 0.14544272422790527, "S_mean": 0.232086181640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.857127904891968, "grad_norm": 0.5280145406723022, "tokens_seen": 7708672, "tokens_per_sec": 21082.91353321915, "vram_gb": 5.521900544, "lambda": 0.1338638523593545, "lambda_min": 0.12189460545778275, "lambda_max": 0.14588074386119843, "S_mean": 0.2534586588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.750457048416138, "grad_norm": 0.5277897119522095, "tokens_seen": 7872512, "tokens_per_sec": 21079.650126281496, "vram_gb": 5.521900544, "lambda": 0.1343357814475894, "lambda_min": 0.12184678763151169, "lambda_max": 0.14696729183197021, "S_mean": 0.258392333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.84183144569397, "grad_norm": 0.5459370017051697, "tokens_seen": 8036352, "tokens_per_sec": 21073.783963830254, "vram_gb": 5.521900544, "lambda": 0.13474212369571129, "lambda_min": 0.12159087508916855, "lambda_max": 0.1480288952589035, "S_mean": 0.2551472981770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.747198104858398, "grad_norm": 0.5480197072029114, "tokens_seen": 8200192, "tokens_per_sec": 21065.3786687759, "vram_gb": 5.521900544, "lambda": 0.13519146665930748, "lambda_min": 0.12135887145996094, "lambda_max": 0.14923088252544403, "S_mean": 0.250518798828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.743636226654052, "val_ppl": 312.19757064785534, "lambda": 0.13521475034455457, "lambda_min": 0.12134642899036407, "lambda_max": 0.14929410815238953, "lambda_per_layer": [0.14671875536441803, 0.1286826729774475, 0.13114769756793976, 0.13256579637527466, 0.1306869387626648, 0.13137969374656677, 0.12614168226718903, 0.12134642899036407, 0.12807786464691162, 0.1268569827079773, 0.13529770076274872, 0.13849486410617828, 0.14929410815238953, 0.14384785294532776, 0.1379346251487732, 0.14105764031410217, 0.1434709131717682, 0.13984769582748413, 0.1360543966293335, 0.13610123097896576, 0.14292842149734497, 0.13214069604873657, 0.1346127837896347, 0.13046656548976898], "S_mean": 0.2661641438802083, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.697205305099487, "grad_norm": 0.5668137669563293, "tokens_seen": 8364032, "tokens_per_sec": 20990.78880031881, "vram_gb": 5.521900544, "lambda": 0.13562108296900988, "lambda_min": 0.12138866633176804, "lambda_max": 0.1502181738615036, "S_mean": 0.23955281575520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.832383394241333, "grad_norm": 0.5442060828208923, "tokens_seen": 8527872, "tokens_per_sec": 20989.8311458755, "vram_gb": 5.521900544, "lambda": 0.1361229456961155, "lambda_min": 0.12129652500152588, "lambda_max": 0.15151257812976837, "S_mean": 0.2644144694010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.689237356185913, "grad_norm": 0.7116553783416748, "tokens_seen": 8691712, "tokens_per_sec": 20990.286860763863, "vram_gb": 5.521900544, "lambda": 0.1366096499065558, "lambda_min": 0.12117882072925568, "lambda_max": 0.1527521163225174, "S_mean": 0.2726236979166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.776704788208008, "grad_norm": 0.5467809438705444, "tokens_seen": 8855552, "tokens_per_sec": 20985.01241525418, "vram_gb": 5.521900544, "lambda": 0.13713131193071604, "lambda_min": 0.12091637402772903, "lambda_max": 0.15403181314468384, "S_mean": 0.2727457682291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.625854969024658, "grad_norm": 0.5425832271575928, "tokens_seen": 9019392, "tokens_per_sec": 20983.230315853227, "vram_gb": 5.521900544, "lambda": 0.1376005799199144, "lambda_min": 0.12040521949529648, "lambda_max": 0.15531624853610992, "S_mean": 0.2470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.698369264602661, "grad_norm": 0.5471133589744568, "tokens_seen": 9183232, "tokens_per_sec": 20981.34268961062, "vram_gb": 5.521900544, "lambda": 0.13808964751660824, "lambda_min": 0.12017412483692169, "lambda_max": 0.15655361115932465, "S_mean": 0.23573811848958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.765463590621948, "grad_norm": 0.6168978810310364, "tokens_seen": 9347072, "tokens_per_sec": 20975.75286399924, "vram_gb": 5.521900544, "lambda": 0.1385601075987021, "lambda_min": 0.11985014379024506, "lambda_max": 0.1577371060848236, "S_mean": 0.260498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.684749126434326, "grad_norm": 0.5302802920341492, "tokens_seen": 9510912, "tokens_per_sec": 20979.19217766798, "vram_gb": 5.521900544, "lambda": 0.13905498230208954, "lambda_min": 0.11986780911684036, "lambda_max": 0.1591331511735916, "S_mean": 0.2526448567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.520455837249756, "grad_norm": 0.5226300954818726, "tokens_seen": 9674752, "tokens_per_sec": 20971.105704374306, "vram_gb": 5.521900544, "lambda": 0.1395608065649867, "lambda_min": 0.11975014954805374, "lambda_max": 0.16053654253482819, "S_mean": 0.22606404622395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.56164813041687, "grad_norm": 0.5307884812355042, "tokens_seen": 9838592, "tokens_per_sec": 20973.46293436042, "vram_gb": 5.521900544, "lambda": 0.14009722167005143, "lambda_min": 0.1197165921330452, "lambda_max": 0.16174548864364624, "S_mean": 0.23789469401041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.623223066329956, "grad_norm": 0.5147273540496826, "tokens_seen": 10002432, "tokens_per_sec": 20975.211200925944, "vram_gb": 5.521900544, "lambda": 0.14061182737350464, "lambda_min": 0.11933453381061554, "lambda_max": 0.1630220115184784, "S_mean": 0.25848388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.566812038421631, "grad_norm": 0.531353235244751, "tokens_seen": 10166272, "tokens_per_sec": 20976.621891314822, "vram_gb": 5.521900544, "lambda": 0.14108367574711642, "lambda_min": 0.11917144060134888, "lambda_max": 0.1641102135181427, "S_mean": 0.24958292643229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.488171100616455, "grad_norm": 0.5438312888145447, "tokens_seen": 10330112, "tokens_per_sec": 20977.465036455505, "vram_gb": 5.521900544, "lambda": 0.14163944280395904, "lambda_min": 0.11905292421579361, "lambda_max": 0.16533435881137848, "S_mean": 0.24213663736979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.482810974121094, "grad_norm": 0.6880329251289368, "tokens_seen": 10493952, "tokens_per_sec": 20979.00557492922, "vram_gb": 5.521900544, "lambda": 0.14213442026327053, "lambda_min": 0.11890880018472672, "lambda_max": 0.16664983332157135, "S_mean": 0.25634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.627620220184326, "grad_norm": 0.548370897769928, "tokens_seen": 10657792, "tokens_per_sec": 20980.302410237222, "vram_gb": 5.521900544, "lambda": 0.14264978220065436, "lambda_min": 0.11853624880313873, "lambda_max": 0.16793791949748993, "S_mean": 0.2722574869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.48348069190979, "grad_norm": 0.6402755975723267, "tokens_seen": 10821632, "tokens_per_sec": 20980.886970098276, "vram_gb": 5.521900544, "lambda": 0.14318145190676054, "lambda_min": 0.11838540434837341, "lambda_max": 0.16921266913414001, "S_mean": 0.24357096354166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.678401708602905, "grad_norm": 0.5726685523986816, "tokens_seen": 10985472, "tokens_per_sec": 20982.434589678724, "vram_gb": 5.521900544, "lambda": 0.14363846431175867, "lambda_min": 0.1178605705499649, "lambda_max": 0.17037267982959747, "S_mean": 0.2481689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.502448320388794, "grad_norm": 0.5855535268783569, "tokens_seen": 11149312, "tokens_per_sec": 20983.634128988244, "vram_gb": 5.521900544, "lambda": 0.14412274025380611, "lambda_min": 0.1176280826330185, "lambda_max": 0.1717272847890854, "S_mean": 0.2502950032552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.504179000854492, "grad_norm": 0.5672062635421753, "tokens_seen": 11313152, "tokens_per_sec": 20986.020592600722, "vram_gb": 5.521900544, "lambda": 0.14457225458075604, "lambda_min": 0.11747925728559494, "lambda_max": 0.17277538776397705, "S_mean": 0.2646891276041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.417607545852661, "grad_norm": 0.5922394394874573, "tokens_seen": 11476992, "tokens_per_sec": 20988.14922926675, "vram_gb": 5.521900544, "lambda": 0.1451071264843146, "lambda_min": 0.1173199862241745, "lambda_max": 0.17393341660499573, "S_mean": 0.2578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.4802563190460205, "grad_norm": 0.6413223147392273, "tokens_seen": 11640832, "tokens_per_sec": 20990.064090394608, "vram_gb": 5.521900544, "lambda": 0.14556187018752098, "lambda_min": 0.11676037311553955, "lambda_max": 0.17513199150562286, "S_mean": 0.24886067708333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.524144172668457, "grad_norm": 0.566851019859314, "tokens_seen": 11804672, "tokens_per_sec": 20991.957051568206, "vram_gb": 5.521900544, "lambda": 0.1460501728579402, "lambda_min": 0.11641188710927963, "lambda_max": 0.1764477640390396, "S_mean": 0.22600301106770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.43556547164917, "grad_norm": 0.5919942855834961, "tokens_seen": 11968512, "tokens_per_sec": 20993.867171696373, "vram_gb": 5.521900544, "lambda": 0.14660542582472166, "lambda_min": 0.11619351804256439, "lambda_max": 0.177712544798851, "S_mean": 0.2303466796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.3875815868377686, "grad_norm": 0.5825634598731995, "tokens_seen": 12132352, "tokens_per_sec": 20997.201296621824, "vram_gb": 5.521900544, "lambda": 0.14709274005144835, "lambda_min": 0.11571208387613297, "lambda_max": 0.179034024477005, "S_mean": 0.2692972819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.647464036941528, "grad_norm": 0.5768333077430725, "tokens_seen": 12296192, "tokens_per_sec": 20997.52848069394, "vram_gb": 5.521900544, "lambda": 0.147624634206295, "lambda_min": 0.11550340056419373, "lambda_max": 0.1806046962738037, "S_mean": 0.24982706705729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.422910451889038, "grad_norm": 0.6050505638122559, "tokens_seen": 12460032, "tokens_per_sec": 21000.078643155215, "vram_gb": 5.521900544, "lambda": 0.14811959645400444, "lambda_min": 0.11516507714986801, "lambda_max": 0.18198588490486145, "S_mean": 0.2772216796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.423418760299683, "grad_norm": 0.5457274913787842, "tokens_seen": 12623872, "tokens_per_sec": 21001.168164094575, "vram_gb": 5.521900544, "lambda": 0.14862913514177004, "lambda_min": 0.11488646268844604, "lambda_max": 0.18318058550357819, "S_mean": 0.23914591471354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.6252219676971436, "grad_norm": 0.5817556977272034, "tokens_seen": 12787712, "tokens_per_sec": 21003.181358321344, "vram_gb": 5.521900544, "lambda": 0.14908674856026968, "lambda_min": 0.11470375955104828, "lambda_max": 0.1843733936548233, "S_mean": 0.24924723307291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.525305986404419, "grad_norm": 0.5601052641868591, "tokens_seen": 12951552, "tokens_per_sec": 21005.70342362864, "vram_gb": 5.521900544, "lambda": 0.1496563327188293, "lambda_min": 0.11447154730558395, "lambda_max": 0.18562546372413635, "S_mean": 0.2597554524739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.42660665512085, "grad_norm": 0.595329225063324, "tokens_seen": 13115392, "tokens_per_sec": 21006.637868357357, "vram_gb": 5.521900544, "lambda": 0.15021418469647566, "lambda_min": 0.11414867639541626, "lambda_max": 0.18677009642124176, "S_mean": 0.26324462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.397246360778809, "grad_norm": 0.5599926114082336, "tokens_seen": 13279232, "tokens_per_sec": 21008.5309536106, "vram_gb": 5.521900544, "lambda": 0.1507384612535437, "lambda_min": 0.1137157455086708, "lambda_max": 0.18791000545024872, "S_mean": 0.250579833984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.467061281204224, "grad_norm": 0.5679962635040283, "tokens_seen": 13443072, "tokens_per_sec": 21010.805408429966, "vram_gb": 5.521900544, "lambda": 0.15124856370190778, "lambda_min": 0.1131165474653244, "lambda_max": 0.18909236788749695, "S_mean": 0.2651468912760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.183326959609985, "grad_norm": 0.6318773627281189, "tokens_seen": 13606912, "tokens_per_sec": 21011.56259618732, "vram_gb": 5.521900544, "lambda": 0.15179214936991534, "lambda_min": 0.112566277384758, "lambda_max": 0.19023823738098145, "S_mean": 0.235931396484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.4532365798950195, "grad_norm": 0.5508665442466736, "tokens_seen": 13770752, "tokens_per_sec": 21014.242521474964, "vram_gb": 5.521900544, "lambda": 0.15230250606934229, "lambda_min": 0.11217691004276276, "lambda_max": 0.19162912666797638, "S_mean": 0.2529805501302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.5286266803741455, "grad_norm": 0.6019228100776672, "tokens_seen": 13934592, "tokens_per_sec": 21016.903187391836, "vram_gb": 5.521900544, "lambda": 0.152803473174572, "lambda_min": 0.11200012266635895, "lambda_max": 0.1929139792919159, "S_mean": 0.24444580078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.329761028289795, "grad_norm": 0.604013204574585, "tokens_seen": 14098432, "tokens_per_sec": 21018.919042747802, "vram_gb": 5.521900544, "lambda": 0.1533876716469725, "lambda_min": 0.11178160458803177, "lambda_max": 0.1941300928592682, "S_mean": 0.2567240397135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.384696006774902, "grad_norm": 0.5773354768753052, "tokens_seen": 14262272, "tokens_per_sec": 21018.436080385003, "vram_gb": 5.521900544, "lambda": 0.15392273850739002, "lambda_min": 0.11158411204814911, "lambda_max": 0.19526273012161255, "S_mean": 0.2665608723958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.258469104766846, "grad_norm": 0.597111701965332, "tokens_seen": 14426112, "tokens_per_sec": 21019.78963568946, "vram_gb": 5.521900544, "lambda": 0.15444788864503303, "lambda_min": 0.11142920702695847, "lambda_max": 0.19679230451583862, "S_mean": 0.2502339680989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.2922682762146, "grad_norm": 0.6161434650421143, "tokens_seen": 14589952, "tokens_per_sec": 21020.649951752996, "vram_gb": 5.521900544, "lambda": 0.15498756027470031, "lambda_min": 0.11102693527936935, "lambda_max": 0.19801783561706543, "S_mean": 0.2618001302083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.462880373001099, "grad_norm": 0.6358092427253723, "tokens_seen": 14753792, "tokens_per_sec": 21023.368719920596, "vram_gb": 5.521900544, "lambda": 0.1555917477235198, "lambda_min": 0.11038070172071457, "lambda_max": 0.19943362474441528, "S_mean": 0.24458821614583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.492534399032593, "grad_norm": 0.5939339399337769, "tokens_seen": 14917632, "tokens_per_sec": 21024.820397252486, "vram_gb": 5.521900544, "lambda": 0.1561817629262805, "lambda_min": 0.1100708469748497, "lambda_max": 0.20082895457744598, "S_mean": 0.2572835286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.438765287399292, "grad_norm": 0.5968295335769653, "tokens_seen": 15081472, "tokens_per_sec": 21026.25113426463, "vram_gb": 5.521900544, "lambda": 0.15670432150363922, "lambda_min": 0.10969178378582001, "lambda_max": 0.20200687646865845, "S_mean": 0.240203857421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.434476613998413, "grad_norm": 0.6183770895004272, "tokens_seen": 15245312, "tokens_per_sec": 21027.01450021307, "vram_gb": 5.521900544, "lambda": 0.15726616761336723, "lambda_min": 0.10896385461091995, "lambda_max": 0.20328611135482788, "S_mean": 0.2654622395833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.214544773101807, "grad_norm": 0.6036406755447388, "tokens_seen": 15409152, "tokens_per_sec": 21028.14285063378, "vram_gb": 5.521900544, "lambda": 0.15779087785631418, "lambda_min": 0.10843724757432938, "lambda_max": 0.2042580246925354, "S_mean": 0.2528584798177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.255739212036133, "grad_norm": 0.6018016338348389, "tokens_seen": 15572992, "tokens_per_sec": 21029.141233691727, "vram_gb": 5.521900544, "lambda": 0.15831719214717546, "lambda_min": 0.10833524167537689, "lambda_max": 0.2053947150707245, "S_mean": 0.2716878255208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.432646751403809, "grad_norm": 0.5844281911849976, "tokens_seen": 15736832, "tokens_per_sec": 21029.66215765072, "vram_gb": 5.521900544, "lambda": 0.15885187809666, "lambda_min": 0.10800279676914215, "lambda_max": 0.20679761469364166, "S_mean": 0.24094645182291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.568269968032837, "grad_norm": 0.7189407348632812, "tokens_seen": 15900672, "tokens_per_sec": 21030.042043845766, "vram_gb": 5.521900544, "lambda": 0.15938908948252598, "lambda_min": 0.10770126432180405, "lambda_max": 0.20806817710399628, "S_mean": 0.2678629557291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.18530535697937, "grad_norm": 0.5475291013717651, "tokens_seen": 16064512, "tokens_per_sec": 21031.366909590655, "vram_gb": 5.521900544, "lambda": 0.15991612430661917, "lambda_min": 0.10739385336637497, "lambda_max": 0.20929689705371857, "S_mean": 0.261932373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.1713385581970215, "grad_norm": 0.5929754376411438, "tokens_seen": 16228352, "tokens_per_sec": 21032.561622895028, "vram_gb": 5.521900544, "lambda": 0.16047911842664084, "lambda_min": 0.10688523948192596, "lambda_max": 0.21030175685882568, "S_mean": 0.2690836588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.131040096282959, "grad_norm": 0.6333627104759216, "tokens_seen": 16392192, "tokens_per_sec": 21033.78091754578, "vram_gb": 5.521900544, "lambda": 0.16100293211638927, "lambda_min": 0.10671386122703552, "lambda_max": 0.2115161269903183, "S_mean": 0.286163330078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.301946091651916, "val_ppl": 200.7270633795354, "lambda": 0.16103048560520014, "lambda_min": 0.10670404136180878, "lambda_max": 0.21159283816814423, "lambda_per_layer": [0.1591344028711319, 0.13065916299819946, 0.13395334780216217, 0.13465067744255066, 0.13241636753082275, 0.13285702466964722, 0.12649807333946228, 0.10670404136180878, 0.1258278787136078, 0.12564757466316223, 0.16479334235191345, 0.17208857834339142, 0.2114151567220688, 0.20672394335269928, 0.19813333451747894, 0.1928296834230423, 0.21159283816814423, 0.1788555532693863, 0.1747819185256958, 0.17094576358795166, 0.20293650031089783, 0.1578391045331955, 0.16475926339626312, 0.14868812263011932], "S_mean": 0.2626851399739583, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.219664573669434, "grad_norm": 0.6496083736419678, "tokens_seen": 16556032, "tokens_per_sec": 20997.45760599856, "vram_gb": 5.521900544, "lambda": 0.1615820483615001, "lambda_min": 0.10643129050731659, "lambda_max": 0.21312758326530457, "S_mean": 0.2654622395833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.324448585510254, "grad_norm": 0.625651478767395, "tokens_seen": 16719872, "tokens_per_sec": 20998.677985275746, "vram_gb": 5.521900544, "lambda": 0.16213112759093443, "lambda_min": 0.1057814210653305, "lambda_max": 0.21469856798648834, "S_mean": 0.2542317708333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.38261866569519, "grad_norm": 0.6397023797035217, "tokens_seen": 16883712, "tokens_per_sec": 20999.758658396884, "vram_gb": 5.521900544, "lambda": 0.16268985780576864, "lambda_min": 0.1054164320230484, "lambda_max": 0.21643657982349396, "S_mean": 0.2512105305989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.405380964279175, "grad_norm": 0.6380612254142761, "tokens_seen": 17047552, "tokens_per_sec": 20999.92541211187, "vram_gb": 5.521900544, "lambda": 0.1632603518664837, "lambda_min": 0.10510353744029999, "lambda_max": 0.21831835806369781, "S_mean": 0.2566324869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.213055610656738, "grad_norm": 0.6645368337631226, "tokens_seen": 17211392, "tokens_per_sec": 21000.374181504412, "vram_gb": 5.521900544, "lambda": 0.16379532249023518, "lambda_min": 0.1047784611582756, "lambda_max": 0.22006572782993317, "S_mean": 0.23985799153645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.3849053382873535, "grad_norm": 0.735051691532135, "tokens_seen": 17375232, "tokens_per_sec": 21001.397494546567, "vram_gb": 5.521900544, "lambda": 0.16432241164147854, "lambda_min": 0.10456813126802444, "lambda_max": 0.22151832282543182, "S_mean": 0.2640584309895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.146137475967407, "grad_norm": 0.602116048336029, "tokens_seen": 17539072, "tokens_per_sec": 21002.95514804275, "vram_gb": 5.521900544, "lambda": 0.16498092034210762, "lambda_min": 0.10442020744085312, "lambda_max": 0.22319389879703522, "S_mean": 0.24817911783854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.184512615203857, "grad_norm": 0.5649759769439697, "tokens_seen": 17702912, "tokens_per_sec": 21004.109592904533, "vram_gb": 5.521900544, "lambda": 0.16553983620057502, "lambda_min": 0.1040424257516861, "lambda_max": 0.22496499121189117, "S_mean": 0.249755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.313716888427734, "grad_norm": 0.6442745327949524, "tokens_seen": 17866752, "tokens_per_sec": 21005.669883087878, "vram_gb": 5.521900544, "lambda": 0.16605860833078623, "lambda_min": 0.10359000414609909, "lambda_max": 0.2265259474515915, "S_mean": 0.23968505859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.08708381652832, "grad_norm": 0.6652660369873047, "tokens_seen": 18030592, "tokens_per_sec": 21006.44544171424, "vram_gb": 5.521900544, "lambda": 0.16659335481623808, "lambda_min": 0.10319667309522629, "lambda_max": 0.22801868617534637, "S_mean": 0.287628173828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.200078964233398, "grad_norm": 0.6268738508224487, "tokens_seen": 18194432, "tokens_per_sec": 21007.723551400602, "vram_gb": 5.521900544, "lambda": 0.1671676750605305, "lambda_min": 0.10285954177379608, "lambda_max": 0.2295995056629181, "S_mean": 0.2728678385416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.306542873382568, "grad_norm": 0.6544014811515808, "tokens_seen": 18358272, "tokens_per_sec": 21009.164711395773, "vram_gb": 5.521900544, "lambda": 0.1677684352422754, "lambda_min": 0.10245911777019501, "lambda_max": 0.2313539832830429, "S_mean": 0.2571004231770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.0935235023498535, "grad_norm": 0.6762507557868958, "tokens_seen": 18522112, "tokens_per_sec": 21010.637090133983, "vram_gb": 5.521900544, "lambda": 0.16831384226679802, "lambda_min": 0.1022062823176384, "lambda_max": 0.23308397829532623, "S_mean": 0.283721923828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.152268886566162, "grad_norm": 0.6540051102638245, "tokens_seen": 18685952, "tokens_per_sec": 21012.206470807818, "vram_gb": 5.521900544, "lambda": 0.16890539011607567, "lambda_min": 0.10201045125722885, "lambda_max": 0.23480753600597382, "S_mean": 0.269287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.34054970741272, "grad_norm": 0.6160057187080383, "tokens_seen": 18849792, "tokens_per_sec": 21012.665817573274, "vram_gb": 5.521900544, "lambda": 0.16950044532616934, "lambda_min": 0.10180673748254776, "lambda_max": 0.2366323471069336, "S_mean": 0.264068603515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.299043655395508, "grad_norm": 0.6713581085205078, "tokens_seen": 19013632, "tokens_per_sec": 21013.882692088042, "vram_gb": 5.521900544, "lambda": 0.17006774774442115, "lambda_min": 0.10151758790016174, "lambda_max": 0.2382810115814209, "S_mean": 0.23860677083333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.131752967834473, "grad_norm": 0.665687084197998, "tokens_seen": 19177472, "tokens_per_sec": 21014.895926248733, "vram_gb": 5.521900544, "lambda": 0.17066771630197763, "lambda_min": 0.10127690434455872, "lambda_max": 0.24026913940906525, "S_mean": 0.2633260091145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.134856462478638, "grad_norm": 0.6074668169021606, "tokens_seen": 19341312, "tokens_per_sec": 21015.962017543177, "vram_gb": 5.521900544, "lambda": 0.17124903885026774, "lambda_min": 0.10090501606464386, "lambda_max": 0.24187707901000977, "S_mean": 0.2735087076822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.113281011581421, "grad_norm": 0.7524777054786682, "tokens_seen": 19505152, "tokens_per_sec": 21016.380420869526, "vram_gb": 5.521900544, "lambda": 0.17179550509899855, "lambda_min": 0.10062805563211441, "lambda_max": 0.24337074160575867, "S_mean": 0.2611592610677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.117660760879517, "grad_norm": 0.648444414138794, "tokens_seen": 19668992, "tokens_per_sec": 21017.684049053074, "vram_gb": 5.521900544, "lambda": 0.17235980027665695, "lambda_min": 0.10005726665258408, "lambda_max": 0.24496926367282867, "S_mean": 0.265899658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.262939929962158, "grad_norm": 0.7131848931312561, "tokens_seen": 19832832, "tokens_per_sec": 21018.519346170266, "vram_gb": 5.521900544, "lambda": 0.17294246703386307, "lambda_min": 0.09952493757009506, "lambda_max": 0.24665632843971252, "S_mean": 0.2636311848958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.138291120529175, "grad_norm": 0.6062823534011841, "tokens_seen": 19996672, "tokens_per_sec": 21019.822435349004, "vram_gb": 5.521900544, "lambda": 0.17352001337955394, "lambda_min": 0.09890876710414886, "lambda_max": 0.24840602278709412, "S_mean": 0.24962361653645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.069446802139282, "grad_norm": 0.6328402161598206, "tokens_seen": 20160512, "tokens_per_sec": 21020.231637827885, "vram_gb": 5.521900544, "lambda": 0.17409420013427734, "lambda_min": 0.09854447841644287, "lambda_max": 0.2501862943172455, "S_mean": 0.2534281412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.061570167541504, "grad_norm": 0.623177707195282, "tokens_seen": 20324352, "tokens_per_sec": 21021.07956892281, "vram_gb": 5.521900544, "lambda": 0.1747010381271442, "lambda_min": 0.09852444380521774, "lambda_max": 0.2520023286342621, "S_mean": 0.246368408203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.171238899230957, "grad_norm": 0.6770996451377869, "tokens_seen": 20488192, "tokens_per_sec": 21020.405651151337, "vram_gb": 5.521900544, "lambda": 0.17530828590194383, "lambda_min": 0.09821640700101852, "lambda_max": 0.2538931667804718, "S_mean": 0.25860595703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.000216007232666, "grad_norm": 0.6412043571472168, "tokens_seen": 20652032, "tokens_per_sec": 21021.461763206717, "vram_gb": 5.521900544, "lambda": 0.17590311573197445, "lambda_min": 0.09808970987796783, "lambda_max": 0.25568026304244995, "S_mean": 0.2601521809895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.081989049911499, "grad_norm": 0.6501253247261047, "tokens_seen": 20815872, "tokens_per_sec": 21022.16909535379, "vram_gb": 5.521900544, "lambda": 0.17645126394927502, "lambda_min": 0.09771029651165009, "lambda_max": 0.25741806626319885, "S_mean": 0.23529052734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.18791389465332, "grad_norm": 0.7190420031547546, "tokens_seen": 20979712, "tokens_per_sec": 21023.222882286067, "vram_gb": 5.521900544, "lambda": 0.177005589629213, "lambda_min": 0.09727200865745544, "lambda_max": 0.2590079605579376, "S_mean": 0.23970540364583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.171807289123535, "grad_norm": 0.7023465633392334, "tokens_seen": 21143552, "tokens_per_sec": 21024.016442706095, "vram_gb": 5.521900544, "lambda": 0.17756234792371592, "lambda_min": 0.09711935371160507, "lambda_max": 0.2609117329120636, "S_mean": 0.2659200032552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.059714078903198, "grad_norm": 0.6349670886993408, "tokens_seen": 21307392, "tokens_per_sec": 21025.289414709692, "vram_gb": 5.521900544, "lambda": 0.17819232710947594, "lambda_min": 0.09697677195072174, "lambda_max": 0.26260313391685486, "S_mean": 0.2526753743489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 4.984404802322388, "grad_norm": 0.6499555110931396, "tokens_seen": 21471232, "tokens_per_sec": 21025.410975345923, "vram_gb": 5.521900544, "lambda": 0.17872155426690975, "lambda_min": 0.09649573266506195, "lambda_max": 0.2643715441226959, "S_mean": 0.24352010091145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.104703903198242, "grad_norm": 0.6616628170013428, "tokens_seen": 21635072, "tokens_per_sec": 21027.381455994113, "vram_gb": 5.521900544, "lambda": 0.17935810051858425, "lambda_min": 0.09603579342365265, "lambda_max": 0.26629307866096497, "S_mean": 0.2573140462239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.157007455825806, "grad_norm": 0.6657440662384033, "tokens_seen": 21798912, "tokens_per_sec": 21028.28529865096, "vram_gb": 5.521900544, "lambda": 0.17990141020466885, "lambda_min": 0.09565228968858719, "lambda_max": 0.26801908016204834, "S_mean": 0.2638956705729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.1196129322052, "grad_norm": 0.7889326214790344, "tokens_seen": 21962752, "tokens_per_sec": 21029.515708326493, "vram_gb": 5.521900544, "lambda": 0.18049528046200672, "lambda_min": 0.09539739042520523, "lambda_max": 0.26992711424827576, "S_mean": 0.250946044921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.147268772125244, "grad_norm": 0.7088968753814697, "tokens_seen": 22126592, "tokens_per_sec": 21030.637911953247, "vram_gb": 5.521900544, "lambda": 0.1810732992986838, "lambda_min": 0.0951036810874939, "lambda_max": 0.2715701162815094, "S_mean": 0.2550150553385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.011564254760742, "grad_norm": 0.6893517374992371, "tokens_seen": 22290432, "tokens_per_sec": 21032.455376355323, "vram_gb": 5.521900544, "lambda": 0.18164366949349642, "lambda_min": 0.094750314950943, "lambda_max": 0.27324235439300537, "S_mean": 0.2609354654947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.095144748687744, "grad_norm": 0.6850308775901794, "tokens_seen": 22454272, "tokens_per_sec": 21033.036662453786, "vram_gb": 5.521900544, "lambda": 0.1821500655884544, "lambda_min": 0.09437283128499985, "lambda_max": 0.27521559596061707, "S_mean": 0.2729899088541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.044555425643921, "grad_norm": 0.7223242521286011, "tokens_seen": 22618112, "tokens_per_sec": 21033.769718175285, "vram_gb": 5.521900544, "lambda": 0.1827440056949854, "lambda_min": 0.09390445798635483, "lambda_max": 0.27739983797073364, "S_mean": 0.24979654947916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 4.9542036056518555, "grad_norm": 0.6348716616630554, "tokens_seen": 22781952, "tokens_per_sec": 21034.45442250544, "vram_gb": 5.521900544, "lambda": 0.18333740656574568, "lambda_min": 0.09350192546844482, "lambda_max": 0.27945420145988464, "S_mean": 0.266021728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.00756311416626, "grad_norm": 0.6763122081756592, "tokens_seen": 22945792, "tokens_per_sec": 21034.579446700896, "vram_gb": 5.521900544, "lambda": 0.18387206022938093, "lambda_min": 0.09324568510055542, "lambda_max": 0.28104013204574585, "S_mean": 0.2606608072916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.995766639709473, "grad_norm": 0.7042557597160339, "tokens_seen": 23109632, "tokens_per_sec": 21032.02702005829, "vram_gb": 5.521900544, "lambda": 0.1844573626294732, "lambda_min": 0.09294265508651733, "lambda_max": 0.28283557295799255, "S_mean": 0.257720947265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.122816801071167, "grad_norm": 0.6733028888702393, "tokens_seen": 23273472, "tokens_per_sec": 21030.599782115, "vram_gb": 5.521900544, "lambda": 0.18507957799981037, "lambda_min": 0.09261730313301086, "lambda_max": 0.28499314188957214, "S_mean": 0.25946044921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.887555360794067, "grad_norm": 0.6462814807891846, "tokens_seen": 23437312, "tokens_per_sec": 21031.340827748074, "vram_gb": 5.521900544, "lambda": 0.18568130830923715, "lambda_min": 0.09219318628311157, "lambda_max": 0.28689417243003845, "S_mean": 0.24592081705729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.034095764160156, "grad_norm": 0.6812773942947388, "tokens_seen": 23601152, "tokens_per_sec": 21032.971019221404, "vram_gb": 5.521900544, "lambda": 0.18624555133283138, "lambda_min": 0.09198728948831558, "lambda_max": 0.2888345718383789, "S_mean": 0.266387939453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.015749216079712, "grad_norm": 0.7255266308784485, "tokens_seen": 23764992, "tokens_per_sec": 21033.6458123325, "vram_gb": 5.521900544, "lambda": 0.18677870370447636, "lambda_min": 0.09167956560850143, "lambda_max": 0.2901635766029358, "S_mean": 0.2531026204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.065702199935913, "grad_norm": 0.6749770641326904, "tokens_seen": 23928832, "tokens_per_sec": 21034.489836401688, "vram_gb": 5.521900544, "lambda": 0.18737626199920973, "lambda_min": 0.09127126634120941, "lambda_max": 0.2922079861164093, "S_mean": 0.2666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.166755437850952, "grad_norm": 0.6896979212760925, "tokens_seen": 24092672, "tokens_per_sec": 21035.284058693585, "vram_gb": 5.521900544, "lambda": 0.18794211776306233, "lambda_min": 0.09112706035375595, "lambda_max": 0.29419827461242676, "S_mean": 0.26763916015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.901217937469482, "grad_norm": 0.7058941125869751, "tokens_seen": 24256512, "tokens_per_sec": 21036.2689516346, "vram_gb": 5.521900544, "lambda": 0.1885195386906465, "lambda_min": 0.09103390574455261, "lambda_max": 0.29620540142059326, "S_mean": 0.2764485677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.989386320114136, "grad_norm": 0.6619486808776855, "tokens_seen": 24420352, "tokens_per_sec": 21037.023427784163, "vram_gb": 5.521900544, "lambda": 0.18904351287831864, "lambda_min": 0.09097430109977722, "lambda_max": 0.2980048656463623, "S_mean": 0.260955810546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.891732692718506, "grad_norm": 0.6868851780891418, "tokens_seen": 24584192, "tokens_per_sec": 21038.167682705363, "vram_gb": 5.521900544, "lambda": 0.18962034489959478, "lambda_min": 0.09083078801631927, "lambda_max": 0.29988235235214233, "S_mean": 0.2693277994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.0578796625137326, "val_ppl": 157.25672524405098, "lambda": 0.18964788348724446, "lambda_min": 0.09081396460533142, "lambda_max": 0.2999572157859802, "lambda_per_layer": [0.16288013756275177, 0.13350120186805725, 0.1366758644580841, 0.13668598234653473, 0.13475970923900604, 0.13508595526218414, 0.1257753223180771, 0.09081396460533142, 0.12338511645793915, 0.12395390123128891, 0.1823585331439972, 0.19262032210826874, 0.2694442570209503, 0.2732173204421997, 0.2597869634628296, 0.26220977306365967, 0.2999572157859802, 0.2331470251083374, 0.2295781970024109, 0.2238394021987915, 0.2781452238559723, 0.18546293675899506, 0.1913135051727295, 0.16695137321949005], "S_mean": 0.2621256510416667, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.086152076721191, "grad_norm": 0.7158778309822083, "tokens_seen": 24748032, "tokens_per_sec": 21013.70179738299, "vram_gb": 5.521900544, "lambda": 0.1902163171519836, "lambda_min": 0.09071638435125351, "lambda_max": 0.30161252617836, "S_mean": 0.2588602701822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 5.038458824157715, "grad_norm": 0.7168376445770264, "tokens_seen": 24911872, "tokens_per_sec": 21014.437967023434, "vram_gb": 5.521900544, "lambda": 0.19070918609698614, "lambda_min": 0.09067774564027786, "lambda_max": 0.30338042974472046, "S_mean": 0.26751708984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.972569704055786, "grad_norm": 0.7057023048400879, "tokens_seen": 25075712, "tokens_per_sec": 21015.899093060212, "vram_gb": 5.521900544, "lambda": 0.19126395508646965, "lambda_min": 0.09054834395647049, "lambda_max": 0.3048861026763916, "S_mean": 0.2709554036458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.867860555648804, "grad_norm": 0.6956775784492493, "tokens_seen": 25239552, "tokens_per_sec": 21015.94224137996, "vram_gb": 5.521900544, "lambda": 0.1918899022663633, "lambda_min": 0.0902174785733223, "lambda_max": 0.3067871034145355, "S_mean": 0.2598978678385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 4.99083948135376, "grad_norm": 0.6743100881576538, "tokens_seen": 25403392, "tokens_per_sec": 21016.688305301537, "vram_gb": 5.521900544, "lambda": 0.19249269304176173, "lambda_min": 0.0901639312505722, "lambda_max": 0.308457612991333, "S_mean": 0.2569376627604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.742072582244873, "grad_norm": 0.7474048733711243, "tokens_seen": 25567232, "tokens_per_sec": 21016.95419499321, "vram_gb": 5.521900544, "lambda": 0.1930630775168538, "lambda_min": 0.08993921428918839, "lambda_max": 0.3100951611995697, "S_mean": 0.2645467122395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.880253553390503, "grad_norm": 0.6529266834259033, "tokens_seen": 25731072, "tokens_per_sec": 21017.587366062304, "vram_gb": 5.521900544, "lambda": 0.19371144690861306, "lambda_min": 0.08974722027778625, "lambda_max": 0.31176334619522095, "S_mean": 0.2591552734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.859967470169067, "grad_norm": 0.6994131803512573, "tokens_seen": 25894912, "tokens_per_sec": 21018.230528008833, "vram_gb": 5.521900544, "lambda": 0.19428702723234892, "lambda_min": 0.08951126039028168, "lambda_max": 0.31338560581207275, "S_mean": 0.2621968587239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.948193073272705, "grad_norm": 0.6338233947753906, "tokens_seen": 26058752, "tokens_per_sec": 21019.206672915676, "vram_gb": 5.521900544, "lambda": 0.1949081982796391, "lambda_min": 0.08926308900117874, "lambda_max": 0.3155263662338257, "S_mean": 0.2625020345052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.890424013137817, "grad_norm": 0.6631647348403931, "tokens_seen": 26222592, "tokens_per_sec": 21019.275482064804, "vram_gb": 5.521900544, "lambda": 0.19549890824904045, "lambda_min": 0.08926519006490707, "lambda_max": 0.31725427508354187, "S_mean": 0.24634806315104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.962120294570923, "grad_norm": 0.7172770500183105, "tokens_seen": 26386432, "tokens_per_sec": 21019.513316901546, "vram_gb": 5.521900544, "lambda": 0.19612916931509972, "lambda_min": 0.08928343653678894, "lambda_max": 0.31912851333618164, "S_mean": 0.2719014485677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 5.01789927482605, "grad_norm": 0.7366092205047607, "tokens_seen": 26550272, "tokens_per_sec": 21020.18209787336, "vram_gb": 5.521900544, "lambda": 0.19662492039302984, "lambda_min": 0.08909297734498978, "lambda_max": 0.32085418701171875, "S_mean": 0.2527974446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.071333408355713, "grad_norm": 0.6999775767326355, "tokens_seen": 26714112, "tokens_per_sec": 21021.273571056663, "vram_gb": 5.521900544, "lambda": 0.19715873307238022, "lambda_min": 0.08901939541101456, "lambda_max": 0.32262247800827026, "S_mean": 0.26251220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.871652603149414, "grad_norm": 0.8425275683403015, "tokens_seen": 26877952, "tokens_per_sec": 21019.490919089516, "vram_gb": 5.521900544, "lambda": 0.19774328420559564, "lambda_min": 0.08878134191036224, "lambda_max": 0.3242979943752289, "S_mean": 0.2592264811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.773456335067749, "grad_norm": 0.7481575608253479, "tokens_seen": 27041792, "tokens_per_sec": 21019.998858750325, "vram_gb": 5.521900544, "lambda": 0.19831778233249983, "lambda_min": 0.08849591016769409, "lambda_max": 0.3260425925254822, "S_mean": 0.24907430013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.881996154785156, "grad_norm": 0.9211469888687134, "tokens_seen": 27205632, "tokens_per_sec": 21020.86103858221, "vram_gb": 5.521900544, "lambda": 0.19882962697496018, "lambda_min": 0.08829955756664276, "lambda_max": 0.32738471031188965, "S_mean": 0.2756754557291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.917588710784912, "grad_norm": 0.7280186414718628, "tokens_seen": 27369472, "tokens_per_sec": 21021.863510864125, "vram_gb": 5.521900544, "lambda": 0.19934438013782105, "lambda_min": 0.08831391483545303, "lambda_max": 0.3290003538131714, "S_mean": 0.2551981608072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.855993747711182, "grad_norm": 0.7415102124214172, "tokens_seen": 27533312, "tokens_per_sec": 21021.950062091146, "vram_gb": 5.521900544, "lambda": 0.19995553512126207, "lambda_min": 0.08806685358285904, "lambda_max": 0.33087506890296936, "S_mean": 0.269866943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.987443685531616, "grad_norm": 0.7199220061302185, "tokens_seen": 27697152, "tokens_per_sec": 21022.439481511803, "vram_gb": 5.521900544, "lambda": 0.20045332113901773, "lambda_min": 0.08780940622091293, "lambda_max": 0.33303967118263245, "S_mean": 0.2607828776041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.893886566162109, "grad_norm": 0.7700580954551697, "tokens_seen": 27860992, "tokens_per_sec": 21022.722114144643, "vram_gb": 5.521900544, "lambda": 0.20099328427265087, "lambda_min": 0.08754158020019531, "lambda_max": 0.33500564098358154, "S_mean": 0.251617431640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 5.017784357070923, "grad_norm": 0.6939494013786316, "tokens_seen": 28024832, "tokens_per_sec": 21023.705861895614, "vram_gb": 5.521900544, "lambda": 0.2015475214769443, "lambda_min": 0.08729725331068039, "lambda_max": 0.33659297227859497, "S_mean": 0.2532552083333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.929032564163208, "grad_norm": 0.7658638954162598, "tokens_seen": 28188672, "tokens_per_sec": 21024.582284257485, "vram_gb": 5.521900544, "lambda": 0.20206240937113762, "lambda_min": 0.08733267337083817, "lambda_max": 0.33828282356262207, "S_mean": 0.2578938802083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.935873985290527, "grad_norm": 0.7009369730949402, "tokens_seen": 28352512, "tokens_per_sec": 21024.72301590057, "vram_gb": 5.521900544, "lambda": 0.20268456917256117, "lambda_min": 0.08719664067029953, "lambda_max": 0.3402349650859833, "S_mean": 0.2575887044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.955014228820801, "grad_norm": 0.7169913053512573, "tokens_seen": 28516352, "tokens_per_sec": 21025.723551852556, "vram_gb": 5.521900544, "lambda": 0.20332708023488522, "lambda_min": 0.08696708083152771, "lambda_max": 0.34196290373802185, "S_mean": 0.2526041666666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.85381007194519, "grad_norm": 0.7064260244369507, "tokens_seen": 28680192, "tokens_per_sec": 21026.513418298553, "vram_gb": 5.521900544, "lambda": 0.20391590365519127, "lambda_min": 0.086766317486763, "lambda_max": 0.34370356798171997, "S_mean": 0.267852783203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.906928300857544, "grad_norm": 0.7213712930679321, "tokens_seen": 28844032, "tokens_per_sec": 21027.43411165361, "vram_gb": 5.521900544, "lambda": 0.20442970904211202, "lambda_min": 0.08635158836841583, "lambda_max": 0.345065176486969, "S_mean": 0.259490966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.876380205154419, "grad_norm": 0.7214331030845642, "tokens_seen": 29007872, "tokens_per_sec": 21027.99767203771, "vram_gb": 5.521900544, "lambda": 0.2050004784638683, "lambda_min": 0.08620293438434601, "lambda_max": 0.34697628021240234, "S_mean": 0.2506917317708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.817455291748047, "grad_norm": 0.7574647068977356, "tokens_seen": 29171712, "tokens_per_sec": 21029.29097814463, "vram_gb": 5.521900544, "lambda": 0.20555343634138504, "lambda_min": 0.08610847592353821, "lambda_max": 0.3486388325691223, "S_mean": 0.25933837890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.899237394332886, "grad_norm": 0.6962270140647888, "tokens_seen": 29335552, "tokens_per_sec": 21029.886565087447, "vram_gb": 5.521900544, "lambda": 0.20615917971978584, "lambda_min": 0.08596692979335785, "lambda_max": 0.3504815101623535, "S_mean": 0.2634480794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.764387130737305, "grad_norm": 0.6929406523704529, "tokens_seen": 29499392, "tokens_per_sec": 21030.982821927653, "vram_gb": 5.521900544, "lambda": 0.20671730612715086, "lambda_min": 0.08605406433343887, "lambda_max": 0.35195037722587585, "S_mean": 0.2808125813802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.88019871711731, "grad_norm": 0.8454177379608154, "tokens_seen": 29663232, "tokens_per_sec": 21031.995661037625, "vram_gb": 5.521900544, "lambda": 0.2072356352582574, "lambda_min": 0.08596229553222656, "lambda_max": 0.3537806272506714, "S_mean": 0.24315388997395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.81608510017395, "grad_norm": 0.8013958930969238, "tokens_seen": 29827072, "tokens_per_sec": 21033.09445928508, "vram_gb": 5.521900544, "lambda": 0.20776245091110468, "lambda_min": 0.08584663271903992, "lambda_max": 0.3556530177593231, "S_mean": 0.2667439778645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.875288724899292, "grad_norm": 0.759914755821228, "tokens_seen": 29990912, "tokens_per_sec": 21033.983181136977, "vram_gb": 5.521900544, "lambda": 0.2083150368804733, "lambda_min": 0.08568087965250015, "lambda_max": 0.35720089077949524, "S_mean": 0.24076334635416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.904686450958252, "grad_norm": 0.7996959686279297, "tokens_seen": 30154752, "tokens_per_sec": 21034.247736212736, "vram_gb": 5.521900544, "lambda": 0.20880986719081798, "lambda_min": 0.08537231385707855, "lambda_max": 0.3589865267276764, "S_mean": 0.26397705078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.8081769943237305, "grad_norm": 0.7984825372695923, "tokens_seen": 30318592, "tokens_per_sec": 21035.071852778347, "vram_gb": 5.521900544, "lambda": 0.2093805857002735, "lambda_min": 0.08519076555967331, "lambda_max": 0.36081060767173767, "S_mean": 0.2584431966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.994701147079468, "grad_norm": 0.743453323841095, "tokens_seen": 30482432, "tokens_per_sec": 21035.64467920102, "vram_gb": 5.521900544, "lambda": 0.20986641198396683, "lambda_min": 0.08517900109291077, "lambda_max": 0.3621048629283905, "S_mean": 0.2609965006510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.879343509674072, "grad_norm": 0.7750759720802307, "tokens_seen": 30646272, "tokens_per_sec": 21036.291678021993, "vram_gb": 5.521900544, "lambda": 0.210326025262475, "lambda_min": 0.08507082611322403, "lambda_max": 0.3634847402572632, "S_mean": 0.2638753255208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.813467025756836, "grad_norm": 0.8032552599906921, "tokens_seen": 30810112, "tokens_per_sec": 21036.841737093422, "vram_gb": 5.521900544, "lambda": 0.21089446265250444, "lambda_min": 0.08499491214752197, "lambda_max": 0.36518964171409607, "S_mean": 0.261077880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.799360752105713, "grad_norm": 0.7573422789573669, "tokens_seen": 30973952, "tokens_per_sec": 21036.680383082956, "vram_gb": 5.521900544, "lambda": 0.21149151865392923, "lambda_min": 0.08490762859582901, "lambda_max": 0.3670315742492676, "S_mean": 0.2523193359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.9176483154296875, "grad_norm": 0.7228620052337646, "tokens_seen": 31137792, "tokens_per_sec": 21036.956316037864, "vram_gb": 5.521900544, "lambda": 0.2120856357117494, "lambda_min": 0.0845789685845375, "lambda_max": 0.36872634291648865, "S_mean": 0.2779439290364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.863619804382324, "grad_norm": 0.7184105515480042, "tokens_seen": 31301632, "tokens_per_sec": 21037.379904383197, "vram_gb": 5.521900544, "lambda": 0.21253252495080233, "lambda_min": 0.08436375111341476, "lambda_max": 0.3705838918685913, "S_mean": 0.25115966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 5.0156731605529785, "grad_norm": 0.7673572897911072, "tokens_seen": 31465472, "tokens_per_sec": 21038.315831302854, "vram_gb": 5.521900544, "lambda": 0.2130688981463512, "lambda_min": 0.08429688215255737, "lambda_max": 0.3720509111881256, "S_mean": 0.263885498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.653599262237549, "grad_norm": 0.7587129473686218, "tokens_seen": 31629312, "tokens_per_sec": 21038.82671937237, "vram_gb": 5.521900544, "lambda": 0.21374133291343847, "lambda_min": 0.08434822410345078, "lambda_max": 0.37375277280807495, "S_mean": 0.246337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.775684595108032, "grad_norm": 0.7584585547447205, "tokens_seen": 31793152, "tokens_per_sec": 21039.269210053015, "vram_gb": 5.521900544, "lambda": 0.21432387673606476, "lambda_min": 0.08425422012805939, "lambda_max": 0.3753492534160614, "S_mean": 0.25830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.7757248878479, "grad_norm": 0.686314582824707, "tokens_seen": 31956992, "tokens_per_sec": 21039.813977989124, "vram_gb": 5.521900544, "lambda": 0.21485952070603767, "lambda_min": 0.08405190706253052, "lambda_max": 0.3768911063671112, "S_mean": 0.2521870930989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.834439992904663, "grad_norm": 0.75373375415802, "tokens_seen": 32120832, "tokens_per_sec": 21040.117354284084, "vram_gb": 5.521900544, "lambda": 0.21542129199951887, "lambda_min": 0.08407144993543625, "lambda_max": 0.3786602318286896, "S_mean": 0.2652791341145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.961619853973389, "grad_norm": 0.7265402674674988, "tokens_seen": 32284672, "tokens_per_sec": 21040.829555153105, "vram_gb": 5.521900544, "lambda": 0.21584463367859522, "lambda_min": 0.083821140229702, "lambda_max": 0.38015392422676086, "S_mean": 0.250732421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.7114341259002686, "grad_norm": 0.7337124943733215, "tokens_seen": 32448512, "tokens_per_sec": 21041.243542462096, "vram_gb": 5.521900544, "lambda": 0.21637693171699843, "lambda_min": 0.08368939906358719, "lambda_max": 0.3816676437854767, "S_mean": 0.2566121419270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.654743909835815, "grad_norm": 0.7905656695365906, "tokens_seen": 32612352, "tokens_per_sec": 21042.018508339843, "vram_gb": 5.521900544, "lambda": 0.21690085747589669, "lambda_min": 0.08353964239358902, "lambda_max": 0.383192777633667, "S_mean": 0.2633260091145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.869159698486328, "grad_norm": 0.7608336210250854, "tokens_seen": 32776192, "tokens_per_sec": 21042.45956368148, "vram_gb": 5.521900544, "lambda": 0.21754630841314793, "lambda_min": 0.08359510451555252, "lambda_max": 0.38521790504455566, "S_mean": 0.2618815104166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.8797660827636715, "val_ppl": 131.5998768056727, "lambda": 0.21757264590511718, "lambda_min": 0.08358197659254074, "lambda_max": 0.38532155752182007, "lambda_per_layer": [0.16384178400039673, 0.13396182656288147, 0.13903474807739258, 0.1396590918302536, 0.13804909586906433, 0.13679389655590057, 0.12844429910182953, 0.08358197659254074, 0.12193242460489273, 0.12288063019514084, 0.1953974962234497, 0.21273069083690643, 0.3179672956466675, 0.3367652893066406, 0.31456610560417175, 0.3328508734703064, 0.38532155752182007, 0.28326690196990967, 0.2882486879825592, 0.2838100790977478, 0.35194218158721924, 0.21108900010585785, 0.21733328700065613, 0.18227428197860718], "S_mean": 0.2479248046875, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.906997203826904, "grad_norm": 0.7535441517829895, "tokens_seen": 32940032, "tokens_per_sec": 21023.98332403252, "vram_gb": 5.521900544, "lambda": 0.21807953622192144, "lambda_min": 0.08346851170063019, "lambda_max": 0.3870004713535309, "S_mean": 0.2605997721354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.668352127075195, "grad_norm": 0.756783664226532, "tokens_seen": 33103872, "tokens_per_sec": 21022.11421519639, "vram_gb": 5.521900544, "lambda": 0.21866723832984766, "lambda_min": 0.08335290104150772, "lambda_max": 0.3890349864959717, "S_mean": 0.265960693359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.722046136856079, "grad_norm": 0.7205480933189392, "tokens_seen": 33267712, "tokens_per_sec": 21022.601122775653, "vram_gb": 5.521900544, "lambda": 0.2192204132055243, "lambda_min": 0.08326198160648346, "lambda_max": 0.3907965123653412, "S_mean": 0.2595316569010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.8480353355407715, "grad_norm": 0.8263683915138245, "tokens_seen": 33431552, "tokens_per_sec": 21023.096042312958, "vram_gb": 5.521900544, "lambda": 0.21977955009788275, "lambda_min": 0.08309238404035568, "lambda_max": 0.39210042357444763, "S_mean": 0.2646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.733654022216797, "grad_norm": 0.747093677520752, "tokens_seen": 33595392, "tokens_per_sec": 21023.250770441475, "vram_gb": 5.521900544, "lambda": 0.22024597072352967, "lambda_min": 0.08294744044542313, "lambda_max": 0.3934299349784851, "S_mean": 0.2635701497395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.946314573287964, "grad_norm": 0.7951809763908386, "tokens_seen": 33759232, "tokens_per_sec": 21023.85525818773, "vram_gb": 5.521900544, "lambda": 0.22079447687913975, "lambda_min": 0.08277112245559692, "lambda_max": 0.3951098620891571, "S_mean": 0.2634684244791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.84526252746582, "grad_norm": 0.8297885656356812, "tokens_seen": 33923072, "tokens_per_sec": 21024.369903095394, "vram_gb": 5.521900544, "lambda": 0.22136733680963516, "lambda_min": 0.0826304629445076, "lambda_max": 0.3966906666755676, "S_mean": 0.242828369140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.755382061004639, "grad_norm": 0.7021905183792114, "tokens_seen": 34086912, "tokens_per_sec": 21024.83569340646, "vram_gb": 5.521900544, "lambda": 0.22182494215667248, "lambda_min": 0.08262965828180313, "lambda_max": 0.3984946608543396, "S_mean": 0.2514851888020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.828708171844482, "grad_norm": 0.7315245270729065, "tokens_seen": 34250752, "tokens_per_sec": 21025.603103388712, "vram_gb": 5.521900544, "lambda": 0.22229902384181818, "lambda_min": 0.08262220770120621, "lambda_max": 0.39987921714782715, "S_mean": 0.26025390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.691140651702881, "grad_norm": 0.7630362510681152, "tokens_seen": 34414592, "tokens_per_sec": 21026.389081531095, "vram_gb": 5.521900544, "lambda": 0.22284341820826134, "lambda_min": 0.08270275592803955, "lambda_max": 0.4016357660293579, "S_mean": 0.268707275390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.718368053436279, "grad_norm": 0.7429161667823792, "tokens_seen": 34578432, "tokens_per_sec": 21027.191247017756, "vram_gb": 5.521900544, "lambda": 0.22335255021850267, "lambda_min": 0.08291006833314896, "lambda_max": 0.4035462737083435, "S_mean": 0.2647705078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.746517896652222, "grad_norm": 0.7337663173675537, "tokens_seen": 34742272, "tokens_per_sec": 21027.175447882397, "vram_gb": 5.521900544, "lambda": 0.2238574189444383, "lambda_min": 0.08304167538881302, "lambda_max": 0.4053543508052826, "S_mean": 0.2581583658854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.628326892852783, "grad_norm": 0.761822521686554, "tokens_seen": 34906112, "tokens_per_sec": 21028.171278553924, "vram_gb": 5.521900544, "lambda": 0.22441158692042032, "lambda_min": 0.08273181319236755, "lambda_max": 0.4069983661174774, "S_mean": 0.2803141276041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.737081050872803, "grad_norm": 0.7304466366767883, "tokens_seen": 35069952, "tokens_per_sec": 21028.316505304414, "vram_gb": 5.521900544, "lambda": 0.22498197760432959, "lambda_min": 0.08238067477941513, "lambda_max": 0.40882882475852966, "S_mean": 0.2552591959635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.781616449356079, "grad_norm": 0.8481020927429199, "tokens_seen": 35233792, "tokens_per_sec": 21028.03808582203, "vram_gb": 5.521900544, "lambda": 0.22546239228298268, "lambda_min": 0.08221511542797089, "lambda_max": 0.41043800115585327, "S_mean": 0.2537943522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.8513829708099365, "grad_norm": 0.8181490898132324, "tokens_seen": 35397632, "tokens_per_sec": 21028.44780735635, "vram_gb": 5.521900544, "lambda": 0.2260184483602643, "lambda_min": 0.0820697620511055, "lambda_max": 0.4121939241886139, "S_mean": 0.2593587239583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.8133885860443115, "grad_norm": 0.7695834040641785, "tokens_seen": 35561472, "tokens_per_sec": 21028.60968524079, "vram_gb": 5.521900544, "lambda": 0.22659847171356282, "lambda_min": 0.08194588124752045, "lambda_max": 0.4141802489757538, "S_mean": 0.24708048502604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.682611703872681, "grad_norm": 0.7841764092445374, "tokens_seen": 35725312, "tokens_per_sec": 21029.220306629664, "vram_gb": 5.521900544, "lambda": 0.2270673637588819, "lambda_min": 0.08187609165906906, "lambda_max": 0.41563600301742554, "S_mean": 0.2516682942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.677201509475708, "grad_norm": 0.7685333490371704, "tokens_seen": 35889152, "tokens_per_sec": 21029.464172822194, "vram_gb": 5.521900544, "lambda": 0.22754385291288295, "lambda_min": 0.08171404898166656, "lambda_max": 0.4170634150505066, "S_mean": 0.2608540852864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.727282285690308, "grad_norm": 0.8615970611572266, "tokens_seen": 36052992, "tokens_per_sec": 21030.055960178695, "vram_gb": 5.521900544, "lambda": 0.2280224555482467, "lambda_min": 0.08178645372390747, "lambda_max": 0.41876205801963806, "S_mean": 0.276275634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.7703857421875, "grad_norm": 0.7456344962120056, "tokens_seen": 36216832, "tokens_per_sec": 21030.586820087457, "vram_gb": 5.521900544, "lambda": 0.22852246773739657, "lambda_min": 0.08180549740791321, "lambda_max": 0.4199601113796234, "S_mean": 0.2523549397786458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.752436637878418, "grad_norm": 0.7778792977333069, "tokens_seen": 36380672, "tokens_per_sec": 21030.896077266472, "vram_gb": 5.521900544, "lambda": 0.22891566498825947, "lambda_min": 0.08179818093776703, "lambda_max": 0.42127764225006104, "S_mean": 0.2502237955729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.898783206939697, "grad_norm": 0.7883918285369873, "tokens_seen": 36544512, "tokens_per_sec": 21031.483340877912, "vram_gb": 5.521900544, "lambda": 0.22933202485243478, "lambda_min": 0.08163371682167053, "lambda_max": 0.4231305718421936, "S_mean": 0.2737630208333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.848790168762207, "grad_norm": 0.7834096550941467, "tokens_seen": 36708352, "tokens_per_sec": 21031.75829306542, "vram_gb": 5.521900544, "lambda": 0.22984327748417854, "lambda_min": 0.08152902871370316, "lambda_max": 0.4245762526988983, "S_mean": 0.25494384765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.687731742858887, "grad_norm": 0.7249013781547546, "tokens_seen": 36872192, "tokens_per_sec": 21032.387828116025, "vram_gb": 5.521900544, "lambda": 0.23033169532815614, "lambda_min": 0.08153116703033447, "lambda_max": 0.42638537287712097, "S_mean": 0.2673848470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.823530673980713, "grad_norm": 0.8183988332748413, "tokens_seen": 37036032, "tokens_per_sec": 21032.935582164995, "vram_gb": 5.521900544, "lambda": 0.23096610171099505, "lambda_min": 0.08163487166166306, "lambda_max": 0.4284690320491791, "S_mean": 0.2550048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.771129608154297, "grad_norm": 0.8349397778511047, "tokens_seen": 37199872, "tokens_per_sec": 21033.477751152437, "vram_gb": 5.521900544, "lambda": 0.23148034668217102, "lambda_min": 0.08177126944065094, "lambda_max": 0.43061092495918274, "S_mean": 0.2579142252604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.795761585235596, "grad_norm": 0.7960843443870544, "tokens_seen": 37363712, "tokens_per_sec": 21033.418251273728, "vram_gb": 5.521900544, "lambda": 0.23198267072439194, "lambda_min": 0.08179100602865219, "lambda_max": 0.43215838074684143, "S_mean": 0.2562662760416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.626948595046997, "grad_norm": 0.7685580849647522, "tokens_seen": 37527552, "tokens_per_sec": 21033.311664540655, "vram_gb": 5.521900544, "lambda": 0.23249882366508245, "lambda_min": 0.08179903030395508, "lambda_max": 0.4337143003940582, "S_mean": 0.2652180989583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.6341328620910645, "grad_norm": 0.7947863936424255, "tokens_seen": 37691392, "tokens_per_sec": 21032.631499713156, "vram_gb": 5.521900544, "lambda": 0.23296099218229452, "lambda_min": 0.08187711238861084, "lambda_max": 0.4351901113986969, "S_mean": 0.2523396809895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.683456182479858, "grad_norm": 0.758617103099823, "tokens_seen": 37855232, "tokens_per_sec": 21032.60625106187, "vram_gb": 5.521900544, "lambda": 0.23343835522731146, "lambda_min": 0.0817805677652359, "lambda_max": 0.43669724464416504, "S_mean": 0.2606404622395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.70827841758728, "grad_norm": 0.8159248232841492, "tokens_seen": 38019072, "tokens_per_sec": 21032.97878812758, "vram_gb": 5.521900544, "lambda": 0.23392422745625177, "lambda_min": 0.08174359053373337, "lambda_max": 0.4381808340549469, "S_mean": 0.265716552734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.866164445877075, "grad_norm": 0.7813473343849182, "tokens_seen": 38182912, "tokens_per_sec": 21033.067039490543, "vram_gb": 5.521900544, "lambda": 0.23433506612976393, "lambda_min": 0.08161663264036179, "lambda_max": 0.43961185216903687, "S_mean": 0.2654012044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.881277799606323, "grad_norm": 0.8655605912208557, "tokens_seen": 38346752, "tokens_per_sec": 21033.15318968874, "vram_gb": 5.521900544, "lambda": 0.23492382590969405, "lambda_min": 0.08151140809059143, "lambda_max": 0.44163504242897034, "S_mean": 0.2518208821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.706395387649536, "grad_norm": 0.7984678745269775, "tokens_seen": 38510592, "tokens_per_sec": 21033.349409314716, "vram_gb": 5.521900544, "lambda": 0.23534104165931544, "lambda_min": 0.08145502209663391, "lambda_max": 0.4433979392051697, "S_mean": 0.24609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.606186628341675, "grad_norm": 1.3577735424041748, "tokens_seen": 38674432, "tokens_per_sec": 21033.13235533856, "vram_gb": 5.521900544, "lambda": 0.2357162625218431, "lambda_min": 0.08120476454496384, "lambda_max": 0.4447980523109436, "S_mean": 0.2523295084635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.9093523025512695, "grad_norm": 0.7722316384315491, "tokens_seen": 38838272, "tokens_per_sec": 21033.317727093392, "vram_gb": 5.521900544, "lambda": 0.23618849646300077, "lambda_min": 0.08103106170892715, "lambda_max": 0.44631433486938477, "S_mean": 0.2565104166666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.542900323867798, "grad_norm": 0.8945040702819824, "tokens_seen": 39002112, "tokens_per_sec": 21033.829550687096, "vram_gb": 5.521900544, "lambda": 0.23679729861517748, "lambda_min": 0.0809781402349472, "lambda_max": 0.4482657015323639, "S_mean": 0.26239013671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.50583553314209, "grad_norm": 0.8300411105155945, "tokens_seen": 39165952, "tokens_per_sec": 21033.987022382626, "vram_gb": 5.521900544, "lambda": 0.2373175024986267, "lambda_min": 0.08101161569356918, "lambda_max": 0.44996821880340576, "S_mean": 0.268463134765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.782957315444946, "grad_norm": 0.784521222114563, "tokens_seen": 39329792, "tokens_per_sec": 21034.683233257416, "vram_gb": 5.521900544, "lambda": 0.23780154964576164, "lambda_min": 0.08087332546710968, "lambda_max": 0.451496422290802, "S_mean": 0.2579142252604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.687962055206299, "grad_norm": 0.792803943157196, "tokens_seen": 39493632, "tokens_per_sec": 21034.277884579624, "vram_gb": 5.521900544, "lambda": 0.23826238668213287, "lambda_min": 0.08066941797733307, "lambda_max": 0.453030526638031, "S_mean": 0.2722676595052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.768572092056274, "grad_norm": 0.7902020215988159, "tokens_seen": 39657472, "tokens_per_sec": 21034.60753592518, "vram_gb": 5.521900544, "lambda": 0.23875076106439033, "lambda_min": 0.08063606917858124, "lambda_max": 0.4544188976287842, "S_mean": 0.2640279134114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.769334316253662, "grad_norm": 0.8792814016342163, "tokens_seen": 39821312, "tokens_per_sec": 21035.55333434379, "vram_gb": 5.521900544, "lambda": 0.2391461500277122, "lambda_min": 0.08065182715654373, "lambda_max": 0.4558795690536499, "S_mean": 0.26580810546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.6182050704956055, "grad_norm": 0.8290342092514038, "tokens_seen": 39985152, "tokens_per_sec": 21036.27255212702, "vram_gb": 5.521900544, "lambda": 0.23952643014490604, "lambda_min": 0.08064354956150055, "lambda_max": 0.45760220289230347, "S_mean": 0.26385498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.709840536117554, "grad_norm": 0.7698827981948853, "tokens_seen": 40148992, "tokens_per_sec": 21036.909480632712, "vram_gb": 5.521900544, "lambda": 0.2400260434175531, "lambda_min": 0.08068361133337021, "lambda_max": 0.45961570739746094, "S_mean": 0.2680765787760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.6795618534088135, "grad_norm": 0.8355493545532227, "tokens_seen": 40312832, "tokens_per_sec": 21037.54935285587, "vram_gb": 5.521900544, "lambda": 0.24049460298071304, "lambda_min": 0.08065345138311386, "lambda_max": 0.46106085181236267, "S_mean": 0.260986328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.649513244628906, "grad_norm": 0.7710692882537842, "tokens_seen": 40476672, "tokens_per_sec": 21037.845392081013, "vram_gb": 5.521900544, "lambda": 0.2409783893575271, "lambda_min": 0.08046675473451614, "lambda_max": 0.4623158276081085, "S_mean": 0.2660420735677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.638274669647217, "grad_norm": 0.735122561454773, "tokens_seen": 40640512, "tokens_per_sec": 21038.207578202713, "vram_gb": 5.521900544, "lambda": 0.24148680735379457, "lambda_min": 0.08024893701076508, "lambda_max": 0.4637064039707184, "S_mean": 0.2546641031901042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.665129899978638, "grad_norm": 0.8111314177513123, "tokens_seen": 40804352, "tokens_per_sec": 21038.665421753314, "vram_gb": 5.521900544, "lambda": 0.24200281966477633, "lambda_min": 0.0800628587603569, "lambda_max": 0.4651983082294464, "S_mean": 0.2647349039713542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.562907934188843, "grad_norm": 0.7834972739219666, "tokens_seen": 40968192, "tokens_per_sec": 21039.00464574296, "vram_gb": 5.521900544, "lambda": 0.24247589986771345, "lambda_min": 0.0799592062830925, "lambda_max": 0.4666408896446228, "S_mean": 0.2622782389322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.74664568901062, "val_ppl": 115.19722840760649, "lambda": 0.2424992167701324, "lambda_min": 0.0799526646733284, "lambda_max": 0.4667205214500427, "lambda_per_layer": [0.16599756479263306, 0.13391883671283722, 0.14146554470062256, 0.1422308385372162, 0.1417510062456131, 0.1379062831401825, 0.128786101937294, 0.0799526646733284, 0.1206812709569931, 0.12229829281568527, 0.20598584413528442, 0.2259252369403839, 0.352483868598938, 0.387184739112854, 0.35921698808670044, 0.3921685814857483, 0.4667205214500427, 0.3300205171108246, 0.3475002944469452, 0.3402738571166992, 0.4179558753967285, 0.23502030968666077, 0.24624551832675934, 0.1982906460762024], "S_mean": 0.2540384928385417, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.82145094871521, "grad_norm": 0.8063393831253052, "tokens_seen": 41132032, "tokens_per_sec": 21024.271749024127, "vram_gb": 5.521900544, "lambda": 0.24291378383835158, "lambda_min": 0.07998401671648026, "lambda_max": 0.46791017055511475, "S_mean": 0.2815958658854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.632046222686768, "grad_norm": 0.8521965742111206, "tokens_seen": 41295872, "tokens_per_sec": 21024.35149632011, "vram_gb": 5.521900544, "lambda": 0.2433711833630999, "lambda_min": 0.0798724889755249, "lambda_max": 0.4695896804332733, "S_mean": 0.2652587890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.653580188751221, "grad_norm": 0.842842161655426, "tokens_seen": 41459712, "tokens_per_sec": 21023.79671316068, "vram_gb": 5.521900544, "lambda": 0.24388939204315344, "lambda_min": 0.0797014981508255, "lambda_max": 0.470733106136322, "S_mean": 0.2726847330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.559514760971069, "grad_norm": 0.8157907128334045, "tokens_seen": 41623552, "tokens_per_sec": 21023.7768381797, "vram_gb": 5.521900544, "lambda": 0.244293423059086, "lambda_min": 0.07954314351081848, "lambda_max": 0.47129392623901367, "S_mean": 0.2518819173177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.661698579788208, "grad_norm": 0.8076314330101013, "tokens_seen": 41787392, "tokens_per_sec": 21023.89390898042, "vram_gb": 5.521900544, "lambda": 0.2447816121081511, "lambda_min": 0.07938404381275177, "lambda_max": 0.4731781482696533, "S_mean": 0.2613728841145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.505528211593628, "grad_norm": 0.8218358159065247, "tokens_seen": 41951232, "tokens_per_sec": 21024.16484960644, "vram_gb": 5.521900544, "lambda": 0.24521219109495482, "lambda_min": 0.07925452291965485, "lambda_max": 0.4745129942893982, "S_mean": 0.2649332682291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.81694221496582, "grad_norm": 0.9326336979866028, "tokens_seen": 42115072, "tokens_per_sec": 21024.29530902314, "vram_gb": 5.521900544, "lambda": 0.24565046653151512, "lambda_min": 0.07911323755979538, "lambda_max": 0.4757401943206787, "S_mean": 0.258544921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.852541923522949, "grad_norm": 0.8361801505088806, "tokens_seen": 42278912, "tokens_per_sec": 21024.5483411713, "vram_gb": 5.521900544, "lambda": 0.24613430630415678, "lambda_min": 0.07922326028347015, "lambda_max": 0.47684770822525024, "S_mean": 0.266571044921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.640559434890747, "grad_norm": 0.7828181982040405, "tokens_seen": 42442752, "tokens_per_sec": 21025.00709985795, "vram_gb": 5.521900544, "lambda": 0.24664599324266115, "lambda_min": 0.07929357886314392, "lambda_max": 0.478359580039978, "S_mean": 0.245452880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.681687116622925, "grad_norm": 0.863317608833313, "tokens_seen": 42606592, "tokens_per_sec": 21025.239416359163, "vram_gb": 5.521900544, "lambda": 0.24718484623978534, "lambda_min": 0.07918626815080643, "lambda_max": 0.48028984665870667, "S_mean": 0.2641092936197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.744274854660034, "grad_norm": 0.9024051427841187, "tokens_seen": 42770432, "tokens_per_sec": 21025.65512477814, "vram_gb": 5.521900544, "lambda": 0.24770667621244988, "lambda_min": 0.07920380681753159, "lambda_max": 0.48203566670417786, "S_mean": 0.24920654296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.608425617218018, "grad_norm": 0.7984424233436584, "tokens_seen": 42934272, "tokens_per_sec": 21025.746743913936, "vram_gb": 5.521900544, "lambda": 0.24814249544094005, "lambda_min": 0.07907772064208984, "lambda_max": 0.4838503897190094, "S_mean": 0.26953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.48192024230957, "grad_norm": 0.9024948477745056, "tokens_seen": 43098112, "tokens_per_sec": 21025.591533441027, "vram_gb": 5.521900544, "lambda": 0.24867582973092794, "lambda_min": 0.07888898998498917, "lambda_max": 0.48553699254989624, "S_mean": 0.2630615234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.812798261642456, "grad_norm": 0.874380886554718, "tokens_seen": 43261952, "tokens_per_sec": 21026.21102313462, "vram_gb": 5.521900544, "lambda": 0.24914405308663845, "lambda_min": 0.07869501411914825, "lambda_max": 0.487008273601532, "S_mean": 0.2480621337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.705305337905884, "grad_norm": 0.8670281171798706, "tokens_seen": 43425792, "tokens_per_sec": 21026.57445308428, "vram_gb": 5.521900544, "lambda": 0.24957690046479306, "lambda_min": 0.07872924953699112, "lambda_max": 0.48855727910995483, "S_mean": 0.26068115234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.726988315582275, "grad_norm": 0.8185871839523315, "tokens_seen": 43589632, "tokens_per_sec": 21026.720776707334, "vram_gb": 5.521900544, "lambda": 0.2500547183056672, "lambda_min": 0.07876270264387131, "lambda_max": 0.4900992214679718, "S_mean": 0.2628987630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.495291233062744, "grad_norm": 0.8122828602790833, "tokens_seen": 43753472, "tokens_per_sec": 21027.02563065167, "vram_gb": 5.521900544, "lambda": 0.2504766738663117, "lambda_min": 0.0788281038403511, "lambda_max": 0.4912148118019104, "S_mean": 0.2593180338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.721809387207031, "grad_norm": 0.8776823878288269, "tokens_seen": 43917312, "tokens_per_sec": 21027.673417111335, "vram_gb": 5.521900544, "lambda": 0.25097494665533304, "lambda_min": 0.07875429093837738, "lambda_max": 0.4926953911781311, "S_mean": 0.261749267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.493478775024414, "grad_norm": 0.907353401184082, "tokens_seen": 44081152, "tokens_per_sec": 21027.657443344444, "vram_gb": 5.521900544, "lambda": 0.2514585942650835, "lambda_min": 0.07875744998455048, "lambda_max": 0.4945734739303589, "S_mean": 0.247894287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.544688701629639, "grad_norm": 0.8428261280059814, "tokens_seen": 44244992, "tokens_per_sec": 21027.931975279964, "vram_gb": 5.521900544, "lambda": 0.2519325663646062, "lambda_min": 0.07874183356761932, "lambda_max": 0.49603068828582764, "S_mean": 0.24619547526041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.717109441757202, "grad_norm": 0.8223717212677002, "tokens_seen": 44408832, "tokens_per_sec": 21028.070379185196, "vram_gb": 5.521900544, "lambda": 0.25239215915401775, "lambda_min": 0.07855641841888428, "lambda_max": 0.49757421016693115, "S_mean": 0.24755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.543424606323242, "grad_norm": 0.7872406840324402, "tokens_seen": 44572672, "tokens_per_sec": 21027.724866631303, "vram_gb": 5.521900544, "lambda": 0.25276057111720246, "lambda_min": 0.07853633910417557, "lambda_max": 0.4988422393798828, "S_mean": 0.24959309895833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.774578809738159, "grad_norm": 0.8433037996292114, "tokens_seen": 44736512, "tokens_per_sec": 21028.41386460893, "vram_gb": 5.521900544, "lambda": 0.2531544156372547, "lambda_min": 0.07843828946352005, "lambda_max": 0.500220537185669, "S_mean": 0.2590535481770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.549947738647461, "grad_norm": 0.8429269790649414, "tokens_seen": 44900352, "tokens_per_sec": 21028.439413193413, "vram_gb": 5.521900544, "lambda": 0.2536056600511074, "lambda_min": 0.07832766324281693, "lambda_max": 0.5010876655578613, "S_mean": 0.256591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.88735556602478, "grad_norm": 0.8712621927261353, "tokens_seen": 45064192, "tokens_per_sec": 21028.820360828075, "vram_gb": 5.521900544, "lambda": 0.25403946234534186, "lambda_min": 0.07821284979581833, "lambda_max": 0.5024632215499878, "S_mean": 0.2528483072916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.61163067817688, "grad_norm": 0.8968179225921631, "tokens_seen": 45228032, "tokens_per_sec": 21029.231338721416, "vram_gb": 5.521900544, "lambda": 0.2545148752008875, "lambda_min": 0.07813604176044464, "lambda_max": 0.504351019859314, "S_mean": 0.24951171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.591519117355347, "grad_norm": 0.8613353967666626, "tokens_seen": 45391872, "tokens_per_sec": 21029.488131422077, "vram_gb": 5.521900544, "lambda": 0.2550393861408035, "lambda_min": 0.07811812311410904, "lambda_max": 0.506089985370636, "S_mean": 0.2641092936197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.654024124145508, "grad_norm": 0.8200214505195618, "tokens_seen": 45555712, "tokens_per_sec": 21029.329052101933, "vram_gb": 5.521900544, "lambda": 0.2554669848953684, "lambda_min": 0.07803870737552643, "lambda_max": 0.5076268911361694, "S_mean": 0.24700927734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.5966575145721436, "grad_norm": 0.8115018606185913, "tokens_seen": 45719552, "tokens_per_sec": 21026.705709362723, "vram_gb": 5.521900544, "lambda": 0.2559390064949791, "lambda_min": 0.0779443085193634, "lambda_max": 0.5089365839958191, "S_mean": 0.23930867513020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.782689094543457, "grad_norm": 0.8136021494865417, "tokens_seen": 45883392, "tokens_per_sec": 21024.90256794899, "vram_gb": 5.521900544, "lambda": 0.256461293126146, "lambda_min": 0.07793907821178436, "lambda_max": 0.5107187032699585, "S_mean": 0.24702962239583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.6910927295684814, "grad_norm": 0.8761401176452637, "tokens_seen": 46047232, "tokens_per_sec": 21023.433886675157, "vram_gb": 5.521900544, "lambda": 0.2569750587766369, "lambda_min": 0.07794471085071564, "lambda_max": 0.5121843218803406, "S_mean": 0.24871826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.529286861419678, "grad_norm": 0.8522616624832153, "tokens_seen": 46211072, "tokens_per_sec": 21013.291847664106, "vram_gb": 5.521900544, "lambda": 0.25740332963565987, "lambda_min": 0.07788220047950745, "lambda_max": 0.5138466358184814, "S_mean": 0.2638346354166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.589537858963013, "grad_norm": 0.8186948299407959, "tokens_seen": 46374912, "tokens_per_sec": 21007.726072593076, "vram_gb": 5.521900544, "lambda": 0.2579368744045496, "lambda_min": 0.07791464775800705, "lambda_max": 0.5154319405555725, "S_mean": 0.2604878743489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.6626482009887695, "grad_norm": 0.817253828048706, "tokens_seen": 46538752, "tokens_per_sec": 21005.34375530045, "vram_gb": 5.521900544, "lambda": 0.2583879418671131, "lambda_min": 0.07795436680316925, "lambda_max": 0.5166732668876648, "S_mean": 0.2603658040364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.580737590789795, "grad_norm": 0.8841962814331055, "tokens_seen": 46702592, "tokens_per_sec": 21006.058675882578, "vram_gb": 5.521900544, "lambda": 0.25863895534227294, "lambda_min": 0.0778636783361435, "lambda_max": 0.5173439383506775, "S_mean": 0.2636820475260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.542194604873657, "grad_norm": 0.8361031413078308, "tokens_seen": 46866432, "tokens_per_sec": 21005.823087980636, "vram_gb": 5.521900544, "lambda": 0.25900476550062496, "lambda_min": 0.07787317782640457, "lambda_max": 0.5185262560844421, "S_mean": 0.2618408203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.5705647468566895, "grad_norm": 0.8994027972221375, "tokens_seen": 47030272, "tokens_per_sec": 21006.55917354732, "vram_gb": 5.521900544, "lambda": 0.2594257574528456, "lambda_min": 0.07779750972986221, "lambda_max": 0.5203155875205994, "S_mean": 0.267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.350321531295776, "grad_norm": 0.8325281143188477, "tokens_seen": 47194112, "tokens_per_sec": 21006.151108797345, "vram_gb": 5.521900544, "lambda": 0.2598218002046148, "lambda_min": 0.07771237194538116, "lambda_max": 0.521714985370636, "S_mean": 0.24973551432291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.554800033569336, "grad_norm": 0.8061785697937012, "tokens_seen": 47357952, "tokens_per_sec": 21004.2761903876, "vram_gb": 5.521900544, "lambda": 0.2602734516064326, "lambda_min": 0.0777624249458313, "lambda_max": 0.5230146646499634, "S_mean": 0.2548828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.632845640182495, "grad_norm": 0.8693532347679138, "tokens_seen": 47521792, "tokens_per_sec": 21004.43050030272, "vram_gb": 5.521900544, "lambda": 0.2606243460128705, "lambda_min": 0.07772160321474075, "lambda_max": 0.5237258076667786, "S_mean": 0.2562764485677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.5982818603515625, "grad_norm": 0.8634690046310425, "tokens_seen": 47685632, "tokens_per_sec": 21004.430442761466, "vram_gb": 5.521900544, "lambda": 0.2610365664586425, "lambda_min": 0.07771320641040802, "lambda_max": 0.5249308347702026, "S_mean": 0.25921630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.619499683380127, "grad_norm": 0.7951598167419434, "tokens_seen": 47849472, "tokens_per_sec": 21004.692506256175, "vram_gb": 5.521900544, "lambda": 0.2614124271397789, "lambda_min": 0.07772956788539886, "lambda_max": 0.5261465311050415, "S_mean": 0.2672220865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.645881175994873, "grad_norm": 0.8728010058403015, "tokens_seen": 48013312, "tokens_per_sec": 21004.85774266006, "vram_gb": 5.521900544, "lambda": 0.2618840370948116, "lambda_min": 0.07764781266450882, "lambda_max": 0.5276288390159607, "S_mean": 0.248504638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.638088226318359, "grad_norm": 0.9124292731285095, "tokens_seen": 48177152, "tokens_per_sec": 21005.291197570936, "vram_gb": 5.521900544, "lambda": 0.26229391650607187, "lambda_min": 0.07761894166469574, "lambda_max": 0.5289912223815918, "S_mean": 0.2575887044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.800175428390503, "grad_norm": 0.911965012550354, "tokens_seen": 48340992, "tokens_per_sec": 21004.827441335714, "vram_gb": 5.521900544, "lambda": 0.2626849866161744, "lambda_min": 0.07766011357307434, "lambda_max": 0.5302199721336365, "S_mean": 0.24810791015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.63848090171814, "grad_norm": 0.8983819484710693, "tokens_seen": 48504832, "tokens_per_sec": 21004.148335069607, "vram_gb": 5.521900544, "lambda": 0.26311251396934193, "lambda_min": 0.07776705175638199, "lambda_max": 0.531619668006897, "S_mean": 0.2515055338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.625113010406494, "grad_norm": 0.9507989287376404, "tokens_seen": 48668672, "tokens_per_sec": 21004.35383627612, "vram_gb": 5.521900544, "lambda": 0.2635484794154763, "lambda_min": 0.0777222216129303, "lambda_max": 0.5331202745437622, "S_mean": 0.248291015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.672646999359131, "grad_norm": 0.8398948311805725, "tokens_seen": 48832512, "tokens_per_sec": 21003.83679449904, "vram_gb": 5.521900544, "lambda": 0.2639696712916096, "lambda_min": 0.0776435062289238, "lambda_max": 0.5343731045722961, "S_mean": 0.2593892415364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.51628303527832, "grad_norm": 0.8715479969978333, "tokens_seen": 48996352, "tokens_per_sec": 21003.9725226955, "vram_gb": 5.521900544, "lambda": 0.2643654244020581, "lambda_min": 0.07764007896184921, "lambda_max": 0.5354424715042114, "S_mean": 0.2606404622395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.547879219055176, "grad_norm": 0.8180282115936279, "tokens_seen": 49160192, "tokens_per_sec": 21003.87178358726, "vram_gb": 5.521900544, "lambda": 0.2646653664608796, "lambda_min": 0.077611543238163, "lambda_max": 0.536422848701477, "S_mean": 0.24412027994791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.634347748756409, "val_ppl": 102.96073981849861, "lambda": 0.2646775919323166, "lambda_min": 0.07760842889547348, "lambda_max": 0.53645920753479, "lambda_per_layer": [0.16776569187641144, 0.13461729884147644, 0.14429053664207458, 0.1454031765460968, 0.14594635367393494, 0.1390601545572281, 0.1301766335964203, 0.07760842889547348, 0.12043832242488861, 0.12184834480285645, 0.21345481276512146, 0.23767457902431488, 0.3807317018508911, 0.4271261692047119, 0.39656782150268555, 0.4455895721912384, 0.53645920753479, 0.37184906005859375, 0.40012913942337036, 0.3915764093399048, 0.4821963608264923, 0.25466036796569824, 0.2731783092021942, 0.21391375362873077], "S_mean": 0.2621968587239583, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.476563215255737, "grad_norm": 0.8334056735038757, "tokens_seen": 49324032, "tokens_per_sec": 20991.543288767472, "vram_gb": 5.521900544, "lambda": 0.26500156583885354, "lambda_min": 0.07759051024913788, "lambda_max": 0.537612795829773, "S_mean": 0.27154541015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.616102695465088, "grad_norm": 0.8395498991012573, "tokens_seen": 49487872, "tokens_per_sec": 20992.373305259473, "vram_gb": 5.521900544, "lambda": 0.2653660637636979, "lambda_min": 0.07748289406299591, "lambda_max": 0.5394488573074341, "S_mean": 0.2623799641927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.52627158164978, "grad_norm": 0.8364176750183105, "tokens_seen": 49651712, "tokens_per_sec": 20993.325014826125, "vram_gb": 5.521900544, "lambda": 0.26578407268971205, "lambda_min": 0.07753220200538635, "lambda_max": 0.540688157081604, "S_mean": 0.2557881673177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.6074559688568115, "grad_norm": 0.8752039074897766, "tokens_seen": 49815552, "tokens_per_sec": 20993.56134527191, "vram_gb": 5.521900544, "lambda": 0.2662042152757446, "lambda_min": 0.07767844945192337, "lambda_max": 0.541655957698822, "S_mean": 0.256622314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.558631181716919, "grad_norm": 0.8517488241195679, "tokens_seen": 49979392, "tokens_per_sec": 20994.453979088343, "vram_gb": 5.521900544, "lambda": 0.2666562845309575, "lambda_min": 0.07778725028038025, "lambda_max": 0.5428798198699951, "S_mean": 0.2506306966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.467627048492432, "grad_norm": 0.8170798420906067, "tokens_seen": 50143232, "tokens_per_sec": 20994.712811767182, "vram_gb": 5.521900544, "lambda": 0.26710641880830127, "lambda_min": 0.07783570885658264, "lambda_max": 0.5442941188812256, "S_mean": 0.2505696614583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.5324625968933105, "grad_norm": 0.8970842957496643, "tokens_seen": 50307072, "tokens_per_sec": 20995.484873714795, "vram_gb": 5.521900544, "lambda": 0.2673940894504388, "lambda_min": 0.07788753509521484, "lambda_max": 0.5455114245414734, "S_mean": 0.23821512858072916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.639607191085815, "grad_norm": 0.8349180817604065, "tokens_seen": 50470912, "tokens_per_sec": 20995.971390727944, "vram_gb": 5.521900544, "lambda": 0.26776444508383673, "lambda_min": 0.07777039706707001, "lambda_max": 0.5467790365219116, "S_mean": 0.245391845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.5401506423950195, "grad_norm": 0.8842869997024536, "tokens_seen": 50634752, "tokens_per_sec": 20995.721044651098, "vram_gb": 5.521900544, "lambda": 0.26814319379627705, "lambda_min": 0.07759563624858856, "lambda_max": 0.5478295087814331, "S_mean": 0.2576192220052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.594408273696899, "grad_norm": 0.8196466565132141, "tokens_seen": 50798592, "tokens_per_sec": 20996.31664907308, "vram_gb": 5.521900544, "lambda": 0.2685485426336527, "lambda_min": 0.07754255086183548, "lambda_max": 0.5493040680885315, "S_mean": 0.2759297688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.488748550415039, "grad_norm": 0.833413302898407, "tokens_seen": 50962432, "tokens_per_sec": 20996.61642956707, "vram_gb": 5.521900544, "lambda": 0.2689185906201601, "lambda_min": 0.07745563238859177, "lambda_max": 0.5504308938980103, "S_mean": 0.248382568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.450593709945679, "grad_norm": 0.8892465829849243, "tokens_seen": 51126272, "tokens_per_sec": 20996.856632797597, "vram_gb": 5.521900544, "lambda": 0.2693471970657508, "lambda_min": 0.07743692398071289, "lambda_max": 0.5518105030059814, "S_mean": 0.24530029296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.452655553817749, "grad_norm": 0.8811444044113159, "tokens_seen": 51290112, "tokens_per_sec": 20996.848673724042, "vram_gb": 5.521900544, "lambda": 0.2697340085481604, "lambda_min": 0.077440045773983, "lambda_max": 0.5530691146850586, "S_mean": 0.24659220377604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.748134136199951, "grad_norm": 0.852196216583252, "tokens_seen": 51453952, "tokens_per_sec": 20997.07234969942, "vram_gb": 5.521900544, "lambda": 0.27000636979937553, "lambda_min": 0.07733768224716187, "lambda_max": 0.5543496012687683, "S_mean": 0.24190775553385416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.720000267028809, "grad_norm": 0.8281876444816589, "tokens_seen": 51617792, "tokens_per_sec": 20997.614449344925, "vram_gb": 5.521900544, "lambda": 0.2703995270033677, "lambda_min": 0.07727450877428055, "lambda_max": 0.5556161403656006, "S_mean": 0.24133809407552084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.480774402618408, "grad_norm": 0.889539897441864, "tokens_seen": 51781632, "tokens_per_sec": 20997.74235288757, "vram_gb": 5.521900544, "lambda": 0.2707295262565215, "lambda_min": 0.07730094343423843, "lambda_max": 0.5569981932640076, "S_mean": 0.2524922688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.630643606185913, "grad_norm": 0.8809701204299927, "tokens_seen": 51945472, "tokens_per_sec": 20997.736924907425, "vram_gb": 5.521900544, "lambda": 0.271109270863235, "lambda_min": 0.07721446454524994, "lambda_max": 0.5583141446113586, "S_mean": 0.24718221028645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.504629611968994, "grad_norm": 0.8271266222000122, "tokens_seen": 52109312, "tokens_per_sec": 20998.11032950158, "vram_gb": 5.521900544, "lambda": 0.27148185949772596, "lambda_min": 0.0772743970155716, "lambda_max": 0.5593894720077515, "S_mean": 0.2524922688802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.557825088500977, "grad_norm": 0.9138264060020447, "tokens_seen": 52273152, "tokens_per_sec": 20998.23195263446, "vram_gb": 5.521900544, "lambda": 0.2719736456250151, "lambda_min": 0.07729335874319077, "lambda_max": 0.5607637166976929, "S_mean": 0.2577921549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.365574836730957, "grad_norm": 0.8950985074043274, "tokens_seen": 52436992, "tokens_per_sec": 20998.766290861393, "vram_gb": 5.521900544, "lambda": 0.2724094359825055, "lambda_min": 0.07730908691883087, "lambda_max": 0.5621052980422974, "S_mean": 0.2398834228515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.6034910678863525, "grad_norm": 0.8782233595848083, "tokens_seen": 52600832, "tokens_per_sec": 20998.761929643093, "vram_gb": 5.521900544, "lambda": 0.27273204891632, "lambda_min": 0.07724232971668243, "lambda_max": 0.5634368658065796, "S_mean": 0.2556355794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.735718727111816, "grad_norm": 0.9360345602035522, "tokens_seen": 52764672, "tokens_per_sec": 20998.936186903484, "vram_gb": 5.521900544, "lambda": 0.27313968787590664, "lambda_min": 0.07726757973432541, "lambda_max": 0.5652233362197876, "S_mean": 0.2550455729166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.527852773666382, "grad_norm": 0.9422896504402161, "tokens_seen": 52928512, "tokens_per_sec": 20999.165627277634, "vram_gb": 5.521900544, "lambda": 0.2734393747523427, "lambda_min": 0.07728207111358643, "lambda_max": 0.566531240940094, "S_mean": 0.24306233723958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.590263843536377, "grad_norm": 0.9869821667671204, "tokens_seen": 53092352, "tokens_per_sec": 20999.745682893383, "vram_gb": 5.521900544, "lambda": 0.2737533909579118, "lambda_min": 0.07718835026025772, "lambda_max": 0.5673669576644897, "S_mean": 0.2821044921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.410860061645508, "grad_norm": 0.9550554156303406, "tokens_seen": 53256192, "tokens_per_sec": 21000.246876494315, "vram_gb": 5.521900544, "lambda": 0.27401096125443775, "lambda_min": 0.0770534947514534, "lambda_max": 0.5685001611709595, "S_mean": 0.2511444091796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.670938491821289, "grad_norm": 0.9126531481742859, "tokens_seen": 53420032, "tokens_per_sec": 21000.67488296147, "vram_gb": 5.521900544, "lambda": 0.27432002623875934, "lambda_min": 0.07705412805080414, "lambda_max": 0.5689448118209839, "S_mean": 0.24625651041666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.573750972747803, "grad_norm": 0.9659491181373596, "tokens_seen": 53583872, "tokens_per_sec": 21001.20246279591, "vram_gb": 5.521900544, "lambda": 0.2746452623978257, "lambda_min": 0.07702364027500153, "lambda_max": 0.5699514746665955, "S_mean": 0.24717203776041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.583317756652832, "grad_norm": 0.9225878715515137, "tokens_seen": 53747712, "tokens_per_sec": 21001.626335352823, "vram_gb": 5.521900544, "lambda": 0.2749651127184431, "lambda_min": 0.07690777629613876, "lambda_max": 0.5710507035255432, "S_mean": 0.256805419921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.456587553024292, "grad_norm": 0.9489147067070007, "tokens_seen": 53911552, "tokens_per_sec": 21001.563408386653, "vram_gb": 5.521900544, "lambda": 0.27529759611934423, "lambda_min": 0.07688973844051361, "lambda_max": 0.5721182227134705, "S_mean": 0.257171630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.628332138061523, "grad_norm": 0.9494166374206543, "tokens_seen": 54075392, "tokens_per_sec": 21001.86250198404, "vram_gb": 5.521900544, "lambda": 0.27562828455120325, "lambda_min": 0.07682278752326965, "lambda_max": 0.5731645822525024, "S_mean": 0.2604573567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.485793113708496, "grad_norm": 0.9247280955314636, "tokens_seen": 54239232, "tokens_per_sec": 21002.03680588541, "vram_gb": 5.521900544, "lambda": 0.2759853570411603, "lambda_min": 0.07684366405010223, "lambda_max": 0.5745106935501099, "S_mean": 0.24155171712239584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.583664178848267, "grad_norm": 0.8700988292694092, "tokens_seen": 54403072, "tokens_per_sec": 21002.677388010332, "vram_gb": 5.521900544, "lambda": 0.2763743670657277, "lambda_min": 0.07684170454740524, "lambda_max": 0.5753359794616699, "S_mean": 0.24903361002604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.68224024772644, "grad_norm": 0.9249603152275085, "tokens_seen": 54566912, "tokens_per_sec": 21003.21990565619, "vram_gb": 5.521900544, "lambda": 0.2767472357178728, "lambda_min": 0.07685194909572601, "lambda_max": 0.5766309499740601, "S_mean": 0.249267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.51323938369751, "grad_norm": 0.8728731274604797, "tokens_seen": 54730752, "tokens_per_sec": 21003.730616579054, "vram_gb": 5.521900544, "lambda": 0.27713999586800736, "lambda_min": 0.07674669474363327, "lambda_max": 0.577942430973053, "S_mean": 0.23869832356770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.603126049041748, "grad_norm": 0.8843225240707397, "tokens_seen": 54894592, "tokens_per_sec": 21004.07427856895, "vram_gb": 5.521900544, "lambda": 0.27746275905519724, "lambda_min": 0.07666768878698349, "lambda_max": 0.5785545110702515, "S_mean": 0.24112955729166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.6036951541900635, "grad_norm": 0.8335806131362915, "tokens_seen": 55058432, "tokens_per_sec": 21004.258621490375, "vram_gb": 5.521900544, "lambda": 0.27782616081337136, "lambda_min": 0.0765976831316948, "lambda_max": 0.5797258615493774, "S_mean": 0.239898681640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.565648794174194, "grad_norm": 0.9203588962554932, "tokens_seen": 55222272, "tokens_per_sec": 21004.92163767456, "vram_gb": 5.521900544, "lambda": 0.2781501868739724, "lambda_min": 0.07664020359516144, "lambda_max": 0.5806710720062256, "S_mean": 0.26043701171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.596033334732056, "grad_norm": 0.8534631133079529, "tokens_seen": 55386112, "tokens_per_sec": 21005.356564517024, "vram_gb": 5.521900544, "lambda": 0.2785565185671051, "lambda_min": 0.07663098722696304, "lambda_max": 0.5816119313240051, "S_mean": 0.24934895833333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.54971170425415, "grad_norm": 0.8640565872192383, "tokens_seen": 55549952, "tokens_per_sec": 21005.581438107813, "vram_gb": 5.521900544, "lambda": 0.2789055447404583, "lambda_min": 0.07658301293849945, "lambda_max": 0.5821797847747803, "S_mean": 0.2486724853515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.60128378868103, "grad_norm": 0.9419265389442444, "tokens_seen": 55713792, "tokens_per_sec": 21005.58884795895, "vram_gb": 5.521900544, "lambda": 0.27922236422697705, "lambda_min": 0.07659851014614105, "lambda_max": 0.5827837586402893, "S_mean": 0.2623189290364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.550258636474609, "grad_norm": 0.9152454733848572, "tokens_seen": 55877632, "tokens_per_sec": 21005.91363580464, "vram_gb": 5.521900544, "lambda": 0.27953907412787277, "lambda_min": 0.07662130147218704, "lambda_max": 0.5839051604270935, "S_mean": 0.2562255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.618995666503906, "grad_norm": 0.8828350901603699, "tokens_seen": 56041472, "tokens_per_sec": 21005.963755529214, "vram_gb": 5.521900544, "lambda": 0.27984337353458005, "lambda_min": 0.07670364528894424, "lambda_max": 0.5850725173950195, "S_mean": 0.24241129557291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.589201927185059, "grad_norm": 0.8993854522705078, "tokens_seen": 56205312, "tokens_per_sec": 21006.420966515714, "vram_gb": 5.521900544, "lambda": 0.2801496246829629, "lambda_min": 0.07662082463502884, "lambda_max": 0.585774838924408, "S_mean": 0.24235026041666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.494658470153809, "grad_norm": 0.9450801610946655, "tokens_seen": 56369152, "tokens_per_sec": 21006.196749685823, "vram_gb": 5.521900544, "lambda": 0.28054239694029093, "lambda_min": 0.07649689167737961, "lambda_max": 0.5867701172828674, "S_mean": 0.243865966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.557375907897949, "grad_norm": 0.9792022109031677, "tokens_seen": 56532992, "tokens_per_sec": 21006.33573142434, "vram_gb": 5.521900544, "lambda": 0.2808959602067868, "lambda_min": 0.07658033072948456, "lambda_max": 0.5879719257354736, "S_mean": 0.23731486002604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.418922424316406, "grad_norm": 0.8961890935897827, "tokens_seen": 56696832, "tokens_per_sec": 21006.605878263297, "vram_gb": 5.521900544, "lambda": 0.2812091599528988, "lambda_min": 0.07658444344997406, "lambda_max": 0.5894690752029419, "S_mean": 0.248504638671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.749907493591309, "grad_norm": 0.994726836681366, "tokens_seen": 56860672, "tokens_per_sec": 21006.52549618563, "vram_gb": 5.521900544, "lambda": 0.2814779194692771, "lambda_min": 0.07662054896354675, "lambda_max": 0.5906894207000732, "S_mean": 0.24101765950520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.521517515182495, "grad_norm": 0.8625763654708862, "tokens_seen": 57024512, "tokens_per_sec": 21006.96416315753, "vram_gb": 5.521900544, "lambda": 0.2817836885030071, "lambda_min": 0.07663040608167648, "lambda_max": 0.5919874310493469, "S_mean": 0.2524312337239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.460691452026367, "grad_norm": 0.9129204154014587, "tokens_seen": 57188352, "tokens_per_sec": 21006.79356061745, "vram_gb": 5.521900544, "lambda": 0.2820315894981225, "lambda_min": 0.07657523453235626, "lambda_max": 0.5924459099769592, "S_mean": 0.24870808919270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.578240871429443, "grad_norm": 0.9498896598815918, "tokens_seen": 57352192, "tokens_per_sec": 21007.15487331532, "vram_gb": 5.521900544, "lambda": 0.2823164627576868, "lambda_min": 0.07651125639677048, "lambda_max": 0.5935391783714294, "S_mean": 0.24042765299479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.582757043838501, "val_ppl": 97.78361662523585, "lambda": 0.2823305871958534, "lambda_min": 0.07651334255933762, "lambda_max": 0.5935846567153931, "lambda_per_layer": [0.1705840528011322, 0.1345832198858261, 0.14726439118385315, 0.14814245700836182, 0.1494782418012619, 0.13999414443969727, 0.13227835297584534, 0.07651334255933762, 0.11891961842775345, 0.12210863083600998, 0.21922408044338226, 0.24772706627845764, 0.4009004533290863, 0.45417752861976624, 0.4271252751350403, 0.48703306913375854, 0.5935846567153931, 0.4027571380138397, 0.444810152053833, 0.4336664080619812, 0.5322326421737671, 0.27297088503837585, 0.29359254240989685, 0.22626574337482452], "S_mean": 0.2517801920572917, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.452367067337036, "grad_norm": 0.8931667804718018, "tokens_seen": 57516032, "tokens_per_sec": 20996.039637906495, "vram_gb": 5.521900544, "lambda": 0.2826612290615837, "lambda_min": 0.07660288363695145, "lambda_max": 0.5946482419967651, "S_mean": 0.24988810221354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.575768232345581, "grad_norm": 0.8955281972885132, "tokens_seen": 57679872, "tokens_per_sec": 20994.247779569283, "vram_gb": 5.521900544, "lambda": 0.28294239720950526, "lambda_min": 0.07664068043231964, "lambda_max": 0.5954871773719788, "S_mean": 0.24937947591145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.618493556976318, "grad_norm": 0.9263421297073364, "tokens_seen": 57843712, "tokens_per_sec": 20992.847494980277, "vram_gb": 5.521900544, "lambda": 0.2831963129962484, "lambda_min": 0.07658321410417557, "lambda_max": 0.5961657762527466, "S_mean": 0.2503662109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.479926347732544, "grad_norm": 0.9665198922157288, "tokens_seen": 58007552, "tokens_per_sec": 20991.872595150056, "vram_gb": 5.521900544, "lambda": 0.2834468698129058, "lambda_min": 0.07655341178178787, "lambda_max": 0.5971065163612366, "S_mean": 0.24774169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.489597797393799, "grad_norm": 0.9767932891845703, "tokens_seen": 58171392, "tokens_per_sec": 20990.359172355373, "vram_gb": 5.521900544, "lambda": 0.2837455316136281, "lambda_min": 0.0764935314655304, "lambda_max": 0.5981261134147644, "S_mean": 0.23902384440104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.425444602966309, "grad_norm": 0.9928594827651978, "tokens_seen": 58335232, "tokens_per_sec": 20988.796292188337, "vram_gb": 5.521900544, "lambda": 0.2840733025223017, "lambda_min": 0.07644543796777725, "lambda_max": 0.5990197658538818, "S_mean": 0.24736531575520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.567398548126221, "grad_norm": 0.8691080808639526, "tokens_seen": 58499072, "tokens_per_sec": 20989.107795112584, "vram_gb": 5.521900544, "lambda": 0.2844091371322672, "lambda_min": 0.07648061960935593, "lambda_max": 0.6000031232833862, "S_mean": 0.24375406901041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.438683748245239, "grad_norm": 0.8812302350997925, "tokens_seen": 58662912, "tokens_per_sec": 20989.39820462946, "vram_gb": 5.521900544, "lambda": 0.2847208607320984, "lambda_min": 0.07654275000095367, "lambda_max": 0.6010574698448181, "S_mean": 0.2574564615885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.507849931716919, "grad_norm": 0.9583150148391724, "tokens_seen": 58826752, "tokens_per_sec": 20990.29831788983, "vram_gb": 5.521900544, "lambda": 0.2850275399784247, "lambda_min": 0.07660739123821259, "lambda_max": 0.6018834114074707, "S_mean": 0.24564615885416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.55437159538269, "grad_norm": 0.9795733690261841, "tokens_seen": 58990592, "tokens_per_sec": 20991.226229488086, "vram_gb": 5.521900544, "lambda": 0.2853563331688444, "lambda_min": 0.07660141587257385, "lambda_max": 0.6028048992156982, "S_mean": 0.24048868815104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.567473649978638, "grad_norm": 0.9204486012458801, "tokens_seen": 59154432, "tokens_per_sec": 20991.234162479934, "vram_gb": 5.521900544, "lambda": 0.2857344712441166, "lambda_min": 0.07665927708148956, "lambda_max": 0.6040074825286865, "S_mean": 0.24483235677083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.389625072479248, "grad_norm": 0.9716787338256836, "tokens_seen": 59318272, "tokens_per_sec": 20990.482112597827, "vram_gb": 5.521900544, "lambda": 0.28605805803090334, "lambda_min": 0.07672470808029175, "lambda_max": 0.604713499546051, "S_mean": 0.24371337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.581831932067871, "grad_norm": 1.0353671312332153, "tokens_seen": 59482112, "tokens_per_sec": 20990.620933272825, "vram_gb": 5.521900544, "lambda": 0.28628035355359316, "lambda_min": 0.07679647207260132, "lambda_max": 0.6053560972213745, "S_mean": 0.24590047200520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.465623140335083, "grad_norm": 0.9965075254440308, "tokens_seen": 59645952, "tokens_per_sec": 20990.568232806494, "vram_gb": 5.521900544, "lambda": 0.28663209453225136, "lambda_min": 0.07680471986532211, "lambda_max": 0.6065207719802856, "S_mean": 0.24376424153645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.457931280136108, "grad_norm": 0.8980460166931152, "tokens_seen": 59809792, "tokens_per_sec": 20990.93222613317, "vram_gb": 5.521900544, "lambda": 0.28688729864855606, "lambda_min": 0.07679255306720734, "lambda_max": 0.6073369383811951, "S_mean": 0.262664794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.626759767532349, "grad_norm": 0.9472367763519287, "tokens_seen": 59973632, "tokens_per_sec": 20990.50704931141, "vram_gb": 5.521900544, "lambda": 0.2871602090696494, "lambda_min": 0.07670102268457413, "lambda_max": 0.6084466576576233, "S_mean": 0.2559305826822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.400681972503662, "grad_norm": 1.0102753639221191, "tokens_seen": 60137472, "tokens_per_sec": 20990.67377054566, "vram_gb": 5.521900544, "lambda": 0.2874564950664838, "lambda_min": 0.0766654834151268, "lambda_max": 0.6096882820129395, "S_mean": 0.2591145833333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.3809778690338135, "grad_norm": 0.9346611499786377, "tokens_seen": 60301312, "tokens_per_sec": 20990.456514717876, "vram_gb": 5.521900544, "lambda": 0.2877280519654353, "lambda_min": 0.07655274122953415, "lambda_max": 0.6105517148971558, "S_mean": 0.2686360677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.58553147315979, "grad_norm": 0.920255184173584, "tokens_seen": 60465152, "tokens_per_sec": 20990.190304691256, "vram_gb": 5.521900544, "lambda": 0.2880757578338186, "lambda_min": 0.07647854089736938, "lambda_max": 0.6118577122688293, "S_mean": 0.24689737955729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.446789264678955, "grad_norm": 0.8812698721885681, "tokens_seen": 60628992, "tokens_per_sec": 20990.35270487269, "vram_gb": 5.521900544, "lambda": 0.28835653358449537, "lambda_min": 0.07644188404083252, "lambda_max": 0.6128456592559814, "S_mean": 0.23802693684895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.665526390075684, "grad_norm": 0.9851827025413513, "tokens_seen": 60792832, "tokens_per_sec": 20990.240943204055, "vram_gb": 5.521900544, "lambda": 0.28862268570810556, "lambda_min": 0.07643037289381027, "lambda_max": 0.6135574579238892, "S_mean": 0.24760945638020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.437721252441406, "grad_norm": 0.9006943106651306, "tokens_seen": 60956672, "tokens_per_sec": 20990.215312355995, "vram_gb": 5.521900544, "lambda": 0.2889397374043862, "lambda_min": 0.07639704644680023, "lambda_max": 0.6145004034042358, "S_mean": 0.24247233072916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.533536434173584, "grad_norm": 0.9450875520706177, "tokens_seen": 61120512, "tokens_per_sec": 20990.205309129287, "vram_gb": 5.521900544, "lambda": 0.2891930139934023, "lambda_min": 0.07632408291101456, "lambda_max": 0.6149102449417114, "S_mean": 0.24543253580729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.50966215133667, "grad_norm": 0.9199025630950928, "tokens_seen": 61284352, "tokens_per_sec": 20990.025255037526, "vram_gb": 5.521900544, "lambda": 0.2894794186577201, "lambda_min": 0.07635139673948288, "lambda_max": 0.6155206561088562, "S_mean": 0.23990885416666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.459616661071777, "grad_norm": 0.9562729597091675, "tokens_seen": 61448192, "tokens_per_sec": 20989.91093269167, "vram_gb": 5.521900544, "lambda": 0.28979209375878173, "lambda_min": 0.07645697146654129, "lambda_max": 0.616686999797821, "S_mean": 0.23893229166666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.365613698959351, "grad_norm": 0.9099746346473694, "tokens_seen": 61612032, "tokens_per_sec": 20990.01038325582, "vram_gb": 5.521900544, "lambda": 0.2900073140238722, "lambda_min": 0.0764126405119896, "lambda_max": 0.6176714301109314, "S_mean": 0.24718221028645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.47144341468811, "grad_norm": 0.9488985538482666, "tokens_seen": 61775872, "tokens_per_sec": 20989.551839722455, "vram_gb": 5.521900544, "lambda": 0.2902587990586956, "lambda_min": 0.07642605900764465, "lambda_max": 0.6184967160224915, "S_mean": 0.23467000325520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.59016227722168, "grad_norm": 0.9228143692016602, "tokens_seen": 61939712, "tokens_per_sec": 20989.496192629555, "vram_gb": 5.521900544, "lambda": 0.29052977822721004, "lambda_min": 0.07647448778152466, "lambda_max": 0.619623601436615, "S_mean": 0.24394734700520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.481133937835693, "grad_norm": 1.0150285959243774, "tokens_seen": 62103552, "tokens_per_sec": 20989.3242101514, "vram_gb": 5.521900544, "lambda": 0.29086567151049775, "lambda_min": 0.07641967386007309, "lambda_max": 0.6209071278572083, "S_mean": 0.24073282877604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.466585636138916, "grad_norm": 0.9175503849983215, "tokens_seen": 62267392, "tokens_per_sec": 20989.29446516566, "vram_gb": 5.521900544, "lambda": 0.2911681147913138, "lambda_min": 0.07637491822242737, "lambda_max": 0.6215183138847351, "S_mean": 0.230682373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.712766170501709, "grad_norm": 1.0421432256698608, "tokens_seen": 62431232, "tokens_per_sec": 20989.327492623048, "vram_gb": 5.521900544, "lambda": 0.2914774042243759, "lambda_min": 0.07638417929410934, "lambda_max": 0.6227366328239441, "S_mean": 0.24003092447916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.4441492557525635, "grad_norm": 0.9761121273040771, "tokens_seen": 62595072, "tokens_per_sec": 20989.02151425599, "vram_gb": 5.521900544, "lambda": 0.29175439663231373, "lambda_min": 0.07635574787855148, "lambda_max": 0.6234110593795776, "S_mean": 0.24015299479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.382313013076782, "grad_norm": 0.9200542569160461, "tokens_seen": 62758912, "tokens_per_sec": 20989.146861350546, "vram_gb": 5.521900544, "lambda": 0.2920082062482834, "lambda_min": 0.07632937282323837, "lambda_max": 0.624238908290863, "S_mean": 0.23844401041666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.340981960296631, "grad_norm": 0.936614990234375, "tokens_seen": 62922752, "tokens_per_sec": 20988.662282211044, "vram_gb": 5.521900544, "lambda": 0.29227580533673364, "lambda_min": 0.0763014480471611, "lambda_max": 0.6248446702957153, "S_mean": 0.2510274251302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.488765478134155, "grad_norm": 0.9040800333023071, "tokens_seen": 63086592, "tokens_per_sec": 20988.801324024567, "vram_gb": 5.521900544, "lambda": 0.29256780569752056, "lambda_min": 0.07624354958534241, "lambda_max": 0.6255508065223694, "S_mean": 0.241851806640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.536593914031982, "grad_norm": 0.9515636563301086, "tokens_seen": 63250432, "tokens_per_sec": 20988.453989547153, "vram_gb": 5.521900544, "lambda": 0.29287401648859185, "lambda_min": 0.07609773427248001, "lambda_max": 0.6264732480049133, "S_mean": 0.2522176106770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.426661014556885, "grad_norm": 1.0229848623275757, "tokens_seen": 63414272, "tokens_per_sec": 20988.11188948589, "vram_gb": 5.521900544, "lambda": 0.2931306092068553, "lambda_min": 0.07612019777297974, "lambda_max": 0.6273450255393982, "S_mean": 0.236846923828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.567556381225586, "grad_norm": 1.0779813528060913, "tokens_seen": 63578112, "tokens_per_sec": 20988.147881870478, "vram_gb": 5.521900544, "lambda": 0.29338400283207494, "lambda_min": 0.07609177380800247, "lambda_max": 0.6284981966018677, "S_mean": 0.24384562174479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.609992265701294, "grad_norm": 0.9195167422294617, "tokens_seen": 63741952, "tokens_per_sec": 20988.65412483328, "vram_gb": 5.521900544, "lambda": 0.2936224543179075, "lambda_min": 0.07606706023216248, "lambda_max": 0.6292149424552917, "S_mean": 0.2366943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.50096869468689, "grad_norm": 1.002974271774292, "tokens_seen": 63905792, "tokens_per_sec": 20989.14497142689, "vram_gb": 5.521900544, "lambda": 0.29387457203119993, "lambda_min": 0.07614676654338837, "lambda_max": 0.6299969553947449, "S_mean": 0.23728434244791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.460200786590576, "grad_norm": 0.9056123495101929, "tokens_seen": 64069632, "tokens_per_sec": 20988.471231247608, "vram_gb": 5.521900544, "lambda": 0.2942239890495936, "lambda_min": 0.07622422277927399, "lambda_max": 0.6309391856193542, "S_mean": 0.23534138997395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.372146368026733, "grad_norm": 0.8934492468833923, "tokens_seen": 64233472, "tokens_per_sec": 20987.226734154123, "vram_gb": 5.521900544, "lambda": 0.2944861330712835, "lambda_min": 0.07626847922801971, "lambda_max": 0.6318079829216003, "S_mean": 0.24201456705729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.413793563842773, "grad_norm": 0.8693607449531555, "tokens_seen": 64397312, "tokens_per_sec": 20986.830505830607, "vram_gb": 5.521900544, "lambda": 0.29470052880545455, "lambda_min": 0.07621199637651443, "lambda_max": 0.632738471031189, "S_mean": 0.23617045084635416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.484452724456787, "grad_norm": 0.9824521541595459, "tokens_seen": 64561152, "tokens_per_sec": 20985.215134137834, "vram_gb": 5.521900544, "lambda": 0.2949250877524416, "lambda_min": 0.07623840123414993, "lambda_max": 0.6334982514381409, "S_mean": 0.2523600260416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.478623390197754, "grad_norm": 0.9193552732467651, "tokens_seen": 64724992, "tokens_per_sec": 20984.86676921598, "vram_gb": 5.521900544, "lambda": 0.2951602293178439, "lambda_min": 0.07624311000108719, "lambda_max": 0.6337451934814453, "S_mean": 0.229156494140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.555516004562378, "grad_norm": 0.966046929359436, "tokens_seen": 64888832, "tokens_per_sec": 20984.865144626758, "vram_gb": 5.521900544, "lambda": 0.29533701390028, "lambda_min": 0.07621689140796661, "lambda_max": 0.6342320442199707, "S_mean": 0.2440185546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.451541900634766, "grad_norm": 0.950050413608551, "tokens_seen": 65052672, "tokens_per_sec": 20985.18636950843, "vram_gb": 5.521900544, "lambda": 0.2955960410957535, "lambda_min": 0.07614309340715408, "lambda_max": 0.6351743340492249, "S_mean": 0.23748779296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.505908012390137, "grad_norm": 0.9793927073478699, "tokens_seen": 65216512, "tokens_per_sec": 20985.029817760696, "vram_gb": 5.521900544, "lambda": 0.2958082702631752, "lambda_min": 0.07606036961078644, "lambda_max": 0.6358441710472107, "S_mean": 0.24039713541666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.47735333442688, "grad_norm": 1.0077736377716064, "tokens_seen": 65380352, "tokens_per_sec": 20985.082020905484, "vram_gb": 5.521900544, "lambda": 0.29603456892073154, "lambda_min": 0.07603120803833008, "lambda_max": 0.6366280913352966, "S_mean": 0.23503621419270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.432013034820557, "grad_norm": 0.9998961091041565, "tokens_seen": 65544192, "tokens_per_sec": 20985.023128385867, "vram_gb": 5.521900544, "lambda": 0.2962665806214015, "lambda_min": 0.07600079476833344, "lambda_max": 0.6371954083442688, "S_mean": 0.2502034505208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.4830653190612795, "val_ppl": 88.50555505802998, "lambda": 0.296280054996411, "lambda_min": 0.07600070536136627, "lambda_max": 0.6372044682502747, "lambda_per_layer": [0.17419704794883728, 0.13475356996059418, 0.15000605583190918, 0.15062811970710754, 0.15252478420734406, 0.14065797626972198, 0.13488830626010895, 0.07600070536136627, 0.1190544068813324, 0.12204170227050781, 0.22364765405654907, 0.25345945358276367, 0.4156079888343811, 0.4744282364845276, 0.44891148805618286, 0.5182527303695679, 0.6372044682502747, 0.4269509017467499, 0.48097464442253113, 0.4690539538860321, 0.5708337426185608, 0.28692862391471863, 0.312137633562088, 0.23757712543010712], "S_mean": 0.2574055989583333, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.490249395370483, "grad_norm": 0.981256902217865, "tokens_seen": 65708032, "tokens_per_sec": 20974.891181901825, "vram_gb": 5.521900544, "lambda": 0.2965219095349312, "lambda_min": 0.07605072855949402, "lambda_max": 0.6375287771224976, "S_mean": 0.23492431640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.5089521408081055, "grad_norm": 0.9052578210830688, "tokens_seen": 65871872, "tokens_per_sec": 20974.64256331163, "vram_gb": 5.521900544, "lambda": 0.29672919772565365, "lambda_min": 0.07603046298027039, "lambda_max": 0.6382721662521362, "S_mean": 0.24082438151041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.381044149398804, "grad_norm": 0.9501718282699585, "tokens_seen": 66035712, "tokens_per_sec": 20974.074264369425, "vram_gb": 5.521900544, "lambda": 0.29699624174584943, "lambda_min": 0.07607544213533401, "lambda_max": 0.6393752694129944, "S_mean": 0.23759969075520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.464479207992554, "grad_norm": 0.9439469575881958, "tokens_seen": 66199552, "tokens_per_sec": 20973.173311867246, "vram_gb": 5.521900544, "lambda": 0.2972711365049084, "lambda_min": 0.07615528255701065, "lambda_max": 0.640232503414154, "S_mean": 0.248046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.498940944671631, "grad_norm": 0.9846131801605225, "tokens_seen": 66363392, "tokens_per_sec": 20972.70734345746, "vram_gb": 5.521900544, "lambda": 0.2974848508213957, "lambda_min": 0.0761481449007988, "lambda_max": 0.6407225131988525, "S_mean": 0.24668375651041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.524741172790527, "grad_norm": 0.9446843862533569, "tokens_seen": 66527232, "tokens_per_sec": 20972.49265912289, "vram_gb": 5.521900544, "lambda": 0.29773142592360574, "lambda_min": 0.07611876726150513, "lambda_max": 0.641547679901123, "S_mean": 0.2503865559895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.446952819824219, "grad_norm": 0.9422426819801331, "tokens_seen": 66691072, "tokens_per_sec": 20971.661319807477, "vram_gb": 5.521900544, "lambda": 0.29799348457405966, "lambda_min": 0.07609741389751434, "lambda_max": 0.642227292060852, "S_mean": 0.23415120442708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.562380313873291, "grad_norm": 0.9329805374145508, "tokens_seen": 66854912, "tokens_per_sec": 20970.70774355516, "vram_gb": 5.521900544, "lambda": 0.29824015218764544, "lambda_min": 0.076158307492733, "lambda_max": 0.6431262493133545, "S_mean": 0.23675537109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.542923450469971, "grad_norm": 1.1165767908096313, "tokens_seen": 67018752, "tokens_per_sec": 20969.900846007782, "vram_gb": 5.521900544, "lambda": 0.29847285089393455, "lambda_min": 0.0762253850698471, "lambda_max": 0.6440573930740356, "S_mean": 0.24818929036458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.502199649810791, "grad_norm": 0.9237786531448364, "tokens_seen": 67182592, "tokens_per_sec": 20969.221867570384, "vram_gb": 5.521900544, "lambda": 0.2987157544121146, "lambda_min": 0.07626041769981384, "lambda_max": 0.645091712474823, "S_mean": 0.24091593424479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.513559341430664, "grad_norm": 0.96515953540802, "tokens_seen": 67346432, "tokens_per_sec": 20968.638533807134, "vram_gb": 5.521900544, "lambda": 0.29893200534085435, "lambda_min": 0.07628937810659409, "lambda_max": 0.6459217667579651, "S_mean": 0.228973388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.418086528778076, "grad_norm": 1.0104515552520752, "tokens_seen": 67510272, "tokens_per_sec": 20968.71730973835, "vram_gb": 5.521900544, "lambda": 0.29914099754144746, "lambda_min": 0.07638643682003021, "lambda_max": 0.6463048458099365, "S_mean": 0.23685709635416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.473601818084717, "grad_norm": 0.9271848797798157, "tokens_seen": 67674112, "tokens_per_sec": 20968.976193549828, "vram_gb": 5.521900544, "lambda": 0.29931600391864777, "lambda_min": 0.07639520615339279, "lambda_max": 0.6466472148895264, "S_mean": 0.24396769205729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.46466064453125, "grad_norm": 1.016013741493225, "tokens_seen": 67837952, "tokens_per_sec": 20969.03259241891, "vram_gb": 5.521900544, "lambda": 0.2995398637528221, "lambda_min": 0.07641635090112686, "lambda_max": 0.6473056674003601, "S_mean": 0.23454793294270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.487486839294434, "grad_norm": 1.0037347078323364, "tokens_seen": 68001792, "tokens_per_sec": 20969.046517910236, "vram_gb": 5.521900544, "lambda": 0.2997386644904812, "lambda_min": 0.07640954852104187, "lambda_max": 0.6480770707130432, "S_mean": 0.240081787109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.428094148635864, "grad_norm": 0.9794841408729553, "tokens_seen": 68165632, "tokens_per_sec": 20969.106698605927, "vram_gb": 5.521900544, "lambda": 0.29993723581234616, "lambda_min": 0.07638678699731827, "lambda_max": 0.6486943960189819, "S_mean": 0.24983723958333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.471280813217163, "grad_norm": 1.0133216381072998, "tokens_seen": 68329472, "tokens_per_sec": 20969.306554899802, "vram_gb": 5.521900544, "lambda": 0.3001044470195969, "lambda_min": 0.07627888768911362, "lambda_max": 0.6494334936141968, "S_mean": 0.23073323567708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.48143744468689, "grad_norm": 1.019004225730896, "tokens_seen": 68493312, "tokens_per_sec": 20969.558032819306, "vram_gb": 5.521900544, "lambda": 0.30030416045337915, "lambda_min": 0.07624761760234833, "lambda_max": 0.6501614451408386, "S_mean": 0.23405965169270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.438899517059326, "grad_norm": 0.9173959493637085, "tokens_seen": 68657152, "tokens_per_sec": 20969.316026135562, "vram_gb": 5.521900544, "lambda": 0.30053397330145043, "lambda_min": 0.07622719556093216, "lambda_max": 0.6507177948951721, "S_mean": 0.23793538411458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.383547306060791, "grad_norm": 1.051161766052246, "tokens_seen": 68820992, "tokens_per_sec": 20969.31804005714, "vram_gb": 5.521900544, "lambda": 0.3007811267549793, "lambda_min": 0.07623957097530365, "lambda_max": 0.6512848138809204, "S_mean": 0.2502644856770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.5864598751068115, "grad_norm": 1.092644214630127, "tokens_seen": 68984832, "tokens_per_sec": 20969.39071178424, "vram_gb": 5.521900544, "lambda": 0.30098992958664894, "lambda_min": 0.07623449712991714, "lambda_max": 0.652075469493866, "S_mean": 0.239227294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.479170083999634, "grad_norm": 1.1110717058181763, "tokens_seen": 69148672, "tokens_per_sec": 20969.497005116256, "vram_gb": 5.521900544, "lambda": 0.3011693594356378, "lambda_min": 0.07626684010028839, "lambda_max": 0.6529619097709656, "S_mean": 0.23785400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.427712678909302, "grad_norm": 1.0550438165664673, "tokens_seen": 69312512, "tokens_per_sec": 20969.692234291502, "vram_gb": 5.521900544, "lambda": 0.3013659439360102, "lambda_min": 0.07624931633472443, "lambda_max": 0.6535404324531555, "S_mean": 0.2556559244791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.367966175079346, "grad_norm": 0.930753767490387, "tokens_seen": 69476352, "tokens_per_sec": 20969.340598259885, "vram_gb": 5.521900544, "lambda": 0.30158438937117654, "lambda_min": 0.07620888203382492, "lambda_max": 0.6544128656387329, "S_mean": 0.23964436848958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.396753787994385, "grad_norm": 1.0333187580108643, "tokens_seen": 69640192, "tokens_per_sec": 20969.6743500467, "vram_gb": 5.521900544, "lambda": 0.3017741885657112, "lambda_min": 0.07615108042955399, "lambda_max": 0.6550071239471436, "S_mean": 0.24544270833333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.530926942825317, "grad_norm": 0.9933605194091797, "tokens_seen": 69804032, "tokens_per_sec": 20969.62466871518, "vram_gb": 5.521900544, "lambda": 0.30195903964340687, "lambda_min": 0.07615228742361069, "lambda_max": 0.6556300520896912, "S_mean": 0.24163818359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.57752799987793, "grad_norm": 1.065354347229004, "tokens_seen": 69967872, "tokens_per_sec": 20970.084485576917, "vram_gb": 5.521900544, "lambda": 0.30213839809099835, "lambda_min": 0.07613781839609146, "lambda_max": 0.656378448009491, "S_mean": 0.2550862630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.315752983093262, "grad_norm": 1.0363930463790894, "tokens_seen": 70131712, "tokens_per_sec": 20970.019004296377, "vram_gb": 5.521900544, "lambda": 0.3023693251113097, "lambda_min": 0.07617197930812836, "lambda_max": 0.6571923494338989, "S_mean": 0.24951171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.481890678405762, "grad_norm": 0.9414995312690735, "tokens_seen": 70295552, "tokens_per_sec": 20970.456442706247, "vram_gb": 5.521900544, "lambda": 0.30250710683564347, "lambda_min": 0.07616885751485825, "lambda_max": 0.6576788425445557, "S_mean": 0.24064127604166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.379387855529785, "grad_norm": 0.9941855072975159, "tokens_seen": 70459392, "tokens_per_sec": 20970.51268942752, "vram_gb": 5.521900544, "lambda": 0.3027069090555112, "lambda_min": 0.07613617181777954, "lambda_max": 0.6583737134933472, "S_mean": 0.2573649088541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.407585144042969, "grad_norm": 1.0914164781570435, "tokens_seen": 70623232, "tokens_per_sec": 20970.48409599257, "vram_gb": 5.521900544, "lambda": 0.3029084202523033, "lambda_min": 0.07611434161663055, "lambda_max": 0.6588858366012573, "S_mean": 0.24599202473958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.40426778793335, "grad_norm": 0.983725905418396, "tokens_seen": 70787072, "tokens_per_sec": 20969.935605324878, "vram_gb": 5.521900544, "lambda": 0.30310207574317855, "lambda_min": 0.07616852968931198, "lambda_max": 0.6593411564826965, "S_mean": 0.24019368489583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.45548152923584, "grad_norm": 0.9567357301712036, "tokens_seen": 70950912, "tokens_per_sec": 20970.178435670525, "vram_gb": 5.521900544, "lambda": 0.303305072709918, "lambda_min": 0.07610796391963959, "lambda_max": 0.659898579120636, "S_mean": 0.2609456380208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.329862356185913, "grad_norm": 1.0560972690582275, "tokens_seen": 71114752, "tokens_per_sec": 20970.377740735315, "vram_gb": 5.521900544, "lambda": 0.303483239064614, "lambda_min": 0.07603459805250168, "lambda_max": 0.6606565117835999, "S_mean": 0.2606201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.190553426742554, "grad_norm": 0.927392303943634, "tokens_seen": 71278592, "tokens_per_sec": 20970.468983051636, "vram_gb": 5.521900544, "lambda": 0.3036634648839633, "lambda_min": 0.07605575770139694, "lambda_max": 0.6611190438270569, "S_mean": 0.243194580078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.384555816650391, "grad_norm": 1.0163556337356567, "tokens_seen": 71442432, "tokens_per_sec": 20970.367204755003, "vram_gb": 5.521900544, "lambda": 0.30385657120496035, "lambda_min": 0.07601173967123032, "lambda_max": 0.6615501046180725, "S_mean": 0.24348958333333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.447652101516724, "grad_norm": 1.0110090970993042, "tokens_seen": 71606272, "tokens_per_sec": 20970.421597585464, "vram_gb": 5.521900544, "lambda": 0.30404316044102114, "lambda_min": 0.07593892514705658, "lambda_max": 0.6621783971786499, "S_mean": 0.2530721028645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.536181449890137, "grad_norm": 1.0142543315887451, "tokens_seen": 71770112, "tokens_per_sec": 20970.172885442273, "vram_gb": 5.521900544, "lambda": 0.3042232890923818, "lambda_min": 0.07586604356765747, "lambda_max": 0.6627485752105713, "S_mean": 0.24021402994791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.537852764129639, "grad_norm": 0.9693053960800171, "tokens_seen": 71933952, "tokens_per_sec": 20969.509155542328, "vram_gb": 5.521900544, "lambda": 0.3043933504571517, "lambda_min": 0.07585369795560837, "lambda_max": 0.6632624268531799, "S_mean": 0.23848470052083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.3611016273498535, "grad_norm": 0.9760385155677795, "tokens_seen": 72097792, "tokens_per_sec": 20969.838688058917, "vram_gb": 5.521900544, "lambda": 0.30460307964434225, "lambda_min": 0.07588512450456619, "lambda_max": 0.6639588475227356, "S_mean": 0.25201416015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.3624467849731445, "grad_norm": 0.9625882506370544, "tokens_seen": 72261632, "tokens_per_sec": 20970.60492046577, "vram_gb": 5.521900544, "lambda": 0.30480420341094333, "lambda_min": 0.07590103149414062, "lambda_max": 0.6645277142524719, "S_mean": 0.25592041015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.518042087554932, "grad_norm": 0.9943995475769043, "tokens_seen": 72425472, "tokens_per_sec": 20971.482897294463, "vram_gb": 5.521900544, "lambda": 0.3049757042899728, "lambda_min": 0.07585890591144562, "lambda_max": 0.6650233268737793, "S_mean": 0.23543294270833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.401983261108398, "grad_norm": 1.0117084980010986, "tokens_seen": 72589312, "tokens_per_sec": 20971.501858135543, "vram_gb": 5.521900544, "lambda": 0.30509729745487374, "lambda_min": 0.07581810653209686, "lambda_max": 0.6655596494674683, "S_mean": 0.23729451497395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.3957743644714355, "grad_norm": 0.9774487018585205, "tokens_seen": 72753152, "tokens_per_sec": 20971.34254256682, "vram_gb": 5.521900544, "lambda": 0.3052949759488304, "lambda_min": 0.0757216215133667, "lambda_max": 0.6662176251411438, "S_mean": 0.24515787760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.479803085327148, "grad_norm": 0.9864226579666138, "tokens_seen": 72916992, "tokens_per_sec": 20971.486910736403, "vram_gb": 5.521900544, "lambda": 0.30547469730178517, "lambda_min": 0.07570534944534302, "lambda_max": 0.6665375232696533, "S_mean": 0.23581949869791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.357503175735474, "grad_norm": 1.0412958860397339, "tokens_seen": 73080832, "tokens_per_sec": 20971.462185654284, "vram_gb": 5.521900544, "lambda": 0.30562735131631297, "lambda_min": 0.07567860931158066, "lambda_max": 0.6672713756561279, "S_mean": 0.24267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.4562907218933105, "grad_norm": 0.9977017045021057, "tokens_seen": 73244672, "tokens_per_sec": 20971.256120891103, "vram_gb": 5.521900544, "lambda": 0.30581385393937427, "lambda_min": 0.07570646703243256, "lambda_max": 0.6678526997566223, "S_mean": 0.2393798828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.412344455718994, "grad_norm": 1.0003098249435425, "tokens_seen": 73408512, "tokens_per_sec": 20971.463980687528, "vram_gb": 5.521900544, "lambda": 0.30603296495974064, "lambda_min": 0.0757095217704773, "lambda_max": 0.6685538291931152, "S_mean": 0.22731526692708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.335934162139893, "grad_norm": 0.9732476472854614, "tokens_seen": 73572352, "tokens_per_sec": 20971.300689165764, "vram_gb": 5.521900544, "lambda": 0.3061758146310846, "lambda_min": 0.07574749737977982, "lambda_max": 0.6689170002937317, "S_mean": 0.24153645833333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.364348411560059, "grad_norm": 0.9385374784469604, "tokens_seen": 73736192, "tokens_per_sec": 20971.320773540236, "vram_gb": 5.521900544, "lambda": 0.3063194140171011, "lambda_min": 0.07577456533908844, "lambda_max": 0.6695309281349182, "S_mean": 0.235107421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.5040521144866945, "val_ppl": 90.38263104560718, "lambda": 0.30632765280703705, "lambda_min": 0.07577293366193771, "lambda_max": 0.6695632934570312, "lambda_per_layer": [0.17747843265533447, 0.13475678861141205, 0.15178750455379486, 0.15266048908233643, 0.1547473520040512, 0.14109165966510773, 0.13560894131660461, 0.07577293366193771, 0.11893082410097122, 0.12217305600643158, 0.22566503286361694, 0.25865229964256287, 0.42516759037971497, 0.485757052898407, 0.4643653929233551, 0.5408758521080017, 0.6695632934570312, 0.44393354654312134, 0.5064939856529236, 0.4938566982746124, 0.6005565524101257, 0.2963928282260895, 0.3276655077934265, 0.2479100525379181], "S_mean": 0.24980672200520834, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.3840107917785645, "grad_norm": 0.9948850870132446, "tokens_seen": 73900032, "tokens_per_sec": 20963.076510450046, "vram_gb": 5.521900544, "lambda": 0.30647911100337905, "lambda_min": 0.07580278813838959, "lambda_max": 0.6699435114860535, "S_mean": 0.24454752604166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.538971424102783, "grad_norm": 1.0314075946807861, "tokens_seen": 74063872, "tokens_per_sec": 20963.26597006505, "vram_gb": 5.521900544, "lambda": 0.3066239729523659, "lambda_min": 0.07578171789646149, "lambda_max": 0.6701821088790894, "S_mean": 0.23969523111979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.407773017883301, "grad_norm": 1.0880175828933716, "tokens_seen": 74227712, "tokens_per_sec": 20963.601617488923, "vram_gb": 5.521900544, "lambda": 0.30680373031646013, "lambda_min": 0.07576870918273926, "lambda_max": 0.6706151366233826, "S_mean": 0.22843424479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.325106859207153, "grad_norm": 0.9701098799705505, "tokens_seen": 74391552, "tokens_per_sec": 20963.496433520228, "vram_gb": 5.521900544, "lambda": 0.30694302792350453, "lambda_min": 0.07571180164813995, "lambda_max": 0.6710946559906006, "S_mean": 0.23227945963541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.444163084030151, "grad_norm": 1.0741411447525024, "tokens_seen": 74555392, "tokens_per_sec": 20963.51116541687, "vram_gb": 5.521900544, "lambda": 0.3071571169421077, "lambda_min": 0.07564525306224823, "lambda_max": 0.6718112826347351, "S_mean": 0.2352294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.3842034339904785, "grad_norm": 1.0101797580718994, "tokens_seen": 74719232, "tokens_per_sec": 20963.589031107724, "vram_gb": 5.521900544, "lambda": 0.3073020074516535, "lambda_min": 0.07565490156412125, "lambda_max": 0.6723340749740601, "S_mean": 0.24553426106770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.315794944763184, "grad_norm": 0.9525149464607239, "tokens_seen": 74883072, "tokens_per_sec": 20963.851979421986, "vram_gb": 5.521900544, "lambda": 0.3075289170568188, "lambda_min": 0.07566164433956146, "lambda_max": 0.6730967164039612, "S_mean": 0.22858683268229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.4252989292144775, "grad_norm": 1.083375096321106, "tokens_seen": 75046912, "tokens_per_sec": 20964.106415417067, "vram_gb": 5.521900544, "lambda": 0.3077426416178544, "lambda_min": 0.07562494277954102, "lambda_max": 0.6737455725669861, "S_mean": 0.2412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.372941732406616, "grad_norm": 1.08433198928833, "tokens_seen": 75210752, "tokens_per_sec": 20963.82955692534, "vram_gb": 5.521900544, "lambda": 0.30788091383874416, "lambda_min": 0.07562518119812012, "lambda_max": 0.6741176843643188, "S_mean": 0.23907470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.474557399749756, "grad_norm": 1.1090853214263916, "tokens_seen": 75374592, "tokens_per_sec": 20963.910979731474, "vram_gb": 5.521900544, "lambda": 0.3080643971140186, "lambda_min": 0.07568299770355225, "lambda_max": 0.67453932762146, "S_mean": 0.234344482421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.331881999969482, "grad_norm": 1.145009994506836, "tokens_seen": 75538432, "tokens_per_sec": 20963.650296291413, "vram_gb": 5.521900544, "lambda": 0.3081953537960847, "lambda_min": 0.07573841512203217, "lambda_max": 0.675067126750946, "S_mean": 0.2789103190104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.35661506652832, "grad_norm": 0.994030773639679, "tokens_seen": 75702272, "tokens_per_sec": 20963.665794021155, "vram_gb": 5.521900544, "lambda": 0.30835285782814026, "lambda_min": 0.07580044120550156, "lambda_max": 0.6754992008209229, "S_mean": 0.23628743489583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.354203462600708, "grad_norm": 1.037513017654419, "tokens_seen": 75866112, "tokens_per_sec": 20963.772981850914, "vram_gb": 5.521900544, "lambda": 0.3085153366749485, "lambda_min": 0.07581906765699387, "lambda_max": 0.6759918928146362, "S_mean": 0.24176025390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.246459722518921, "grad_norm": 1.0336641073226929, "tokens_seen": 76029952, "tokens_per_sec": 20963.725807005514, "vram_gb": 5.521900544, "lambda": 0.3086527433867256, "lambda_min": 0.0757986456155777, "lambda_max": 0.6762081384658813, "S_mean": 0.23540242513020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.491252422332764, "grad_norm": 1.1664568185806274, "tokens_seen": 76193792, "tokens_per_sec": 20963.66947449291, "vram_gb": 5.521900544, "lambda": 0.3088077005619804, "lambda_min": 0.07581602036952972, "lambda_max": 0.6765870451927185, "S_mean": 0.24077351888020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.354318141937256, "grad_norm": 1.0648428201675415, "tokens_seen": 76357632, "tokens_per_sec": 20963.690676667095, "vram_gb": 5.521900544, "lambda": 0.3089525445053975, "lambda_min": 0.07580583542585373, "lambda_max": 0.677222490310669, "S_mean": 0.2511189778645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.364654064178467, "grad_norm": 1.0482898950576782, "tokens_seen": 76521472, "tokens_per_sec": 20963.805956829023, "vram_gb": 5.521900544, "lambda": 0.30910664579520625, "lambda_min": 0.07581952214241028, "lambda_max": 0.6776207685470581, "S_mean": 0.2337646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.115331172943115, "grad_norm": 0.9715759754180908, "tokens_seen": 76685312, "tokens_per_sec": 20961.172559333936, "vram_gb": 5.521900544, "lambda": 0.3092357652882735, "lambda_min": 0.07581621408462524, "lambda_max": 0.6780368089675903, "S_mean": 0.242401123046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.514878749847412, "grad_norm": 1.083775281906128, "tokens_seen": 76849152, "tokens_per_sec": 20955.781431845626, "vram_gb": 5.521900544, "lambda": 0.3093980907772978, "lambda_min": 0.0758129358291626, "lambda_max": 0.6786404848098755, "S_mean": 0.24568684895833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.428113698959351, "grad_norm": 1.0080333948135376, "tokens_seen": 77012992, "tokens_per_sec": 20952.57726218483, "vram_gb": 5.521900544, "lambda": 0.30953816665957373, "lambda_min": 0.07583677023649216, "lambda_max": 0.6789249777793884, "S_mean": 0.24976603190104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.376176357269287, "grad_norm": 1.0568193197250366, "tokens_seen": 77176832, "tokens_per_sec": 20950.790635572997, "vram_gb": 5.521900544, "lambda": 0.3097144455338518, "lambda_min": 0.07587699592113495, "lambda_max": 0.6793734431266785, "S_mean": 0.2535196940104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.2773683071136475, "grad_norm": 0.9763369560241699, "tokens_seen": 77340672, "tokens_per_sec": 20950.59532640812, "vram_gb": 5.521900544, "lambda": 0.30990738545854885, "lambda_min": 0.07586031407117844, "lambda_max": 0.6798837780952454, "S_mean": 0.233367919921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.445725202560425, "grad_norm": 0.9804233312606812, "tokens_seen": 77504512, "tokens_per_sec": 20950.045283133935, "vram_gb": 5.521900544, "lambda": 0.31005147751420736, "lambda_min": 0.07584628462791443, "lambda_max": 0.6803316473960876, "S_mean": 0.24289957682291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.270207405090332, "grad_norm": 1.1347088813781738, "tokens_seen": 77668352, "tokens_per_sec": 20950.063569635862, "vram_gb": 5.521900544, "lambda": 0.3101800136889021, "lambda_min": 0.07583954185247421, "lambda_max": 0.6809413433074951, "S_mean": 0.23676554361979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.424214601516724, "grad_norm": 1.0806975364685059, "tokens_seen": 77832192, "tokens_per_sec": 20950.070101090634, "vram_gb": 5.521900544, "lambda": 0.31034424373259145, "lambda_min": 0.07580829411745071, "lambda_max": 0.6815553307533264, "S_mean": 0.24407958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.284170866012573, "grad_norm": 0.9934876561164856, "tokens_seen": 77996032, "tokens_per_sec": 20950.13861227119, "vram_gb": 5.521900544, "lambda": 0.3105243795240919, "lambda_min": 0.07583615183830261, "lambda_max": 0.6822037696838379, "S_mean": 0.23255411783854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.259943723678589, "grad_norm": 1.0540000200271606, "tokens_seen": 78159872, "tokens_per_sec": 20949.566699747793, "vram_gb": 5.521900544, "lambda": 0.3106787158176303, "lambda_min": 0.07584579288959503, "lambda_max": 0.6826390624046326, "S_mean": 0.24365234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.314416170120239, "grad_norm": 0.9674305319786072, "tokens_seen": 78323712, "tokens_per_sec": 20949.324551152637, "vram_gb": 5.521900544, "lambda": 0.3108053067699075, "lambda_min": 0.07582660764455795, "lambda_max": 0.6832332015037537, "S_mean": 0.23998006184895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.240170955657959, "grad_norm": 0.9872175455093384, "tokens_seen": 78487552, "tokens_per_sec": 20949.57749846564, "vram_gb": 5.521900544, "lambda": 0.3109419218574961, "lambda_min": 0.07575716823339462, "lambda_max": 0.6836473345756531, "S_mean": 0.23388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.391080856323242, "grad_norm": 1.115218162536621, "tokens_seen": 78651392, "tokens_per_sec": 20949.52302256122, "vram_gb": 5.521900544, "lambda": 0.3110789271692435, "lambda_min": 0.07578446716070175, "lambda_max": 0.6839339733123779, "S_mean": 0.23046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.481411933898926, "grad_norm": 0.9643661975860596, "tokens_seen": 78815232, "tokens_per_sec": 20949.926663283895, "vram_gb": 5.521900544, "lambda": 0.31120830836395424, "lambda_min": 0.07581822574138641, "lambda_max": 0.6843174695968628, "S_mean": 0.24358113606770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.402187824249268, "grad_norm": 0.9868290424346924, "tokens_seen": 78979072, "tokens_per_sec": 20949.852984643232, "vram_gb": 5.521900544, "lambda": 0.3113517450789611, "lambda_min": 0.07585791498422623, "lambda_max": 0.6847782731056213, "S_mean": 0.24430338541666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.233449220657349, "grad_norm": 0.9713080525398254, "tokens_seen": 79142912, "tokens_per_sec": 20950.251554581442, "vram_gb": 5.521900544, "lambda": 0.3114693419386943, "lambda_min": 0.07588806748390198, "lambda_max": 0.6849526762962341, "S_mean": 0.235687255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.2363245487213135, "grad_norm": 0.9999755620956421, "tokens_seen": 79306752, "tokens_per_sec": 20950.51297353642, "vram_gb": 5.521900544, "lambda": 0.3116099542627732, "lambda_min": 0.07592494040727615, "lambda_max": 0.6852362155914307, "S_mean": 0.23139444986979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.254707336425781, "grad_norm": 1.0664845705032349, "tokens_seen": 79470592, "tokens_per_sec": 20950.462079485427, "vram_gb": 5.521900544, "lambda": 0.311699609272182, "lambda_min": 0.07593560963869095, "lambda_max": 0.6856456995010376, "S_mean": 0.23402913411458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.2703680992126465, "grad_norm": 1.0344364643096924, "tokens_seen": 79634432, "tokens_per_sec": 20950.590539624027, "vram_gb": 5.521900544, "lambda": 0.31183134764432907, "lambda_min": 0.0758863091468811, "lambda_max": 0.6860914826393127, "S_mean": 0.24800618489583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.348379135131836, "grad_norm": 1.1207438707351685, "tokens_seen": 79798272, "tokens_per_sec": 20950.69358295162, "vram_gb": 5.521900544, "lambda": 0.3119480072831114, "lambda_min": 0.07585300505161285, "lambda_max": 0.686335563659668, "S_mean": 0.23530069986979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.2273268699646, "grad_norm": 0.9761785268783569, "tokens_seen": 79962112, "tokens_per_sec": 20950.965235454976, "vram_gb": 5.521900544, "lambda": 0.312070540462931, "lambda_min": 0.07584329694509506, "lambda_max": 0.6865034699440002, "S_mean": 0.23340861002604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.352510213851929, "grad_norm": 1.1332378387451172, "tokens_seen": 80125952, "tokens_per_sec": 20951.35773541337, "vram_gb": 5.521900544, "lambda": 0.3121681148186326, "lambda_min": 0.07587459683418274, "lambda_max": 0.6867877244949341, "S_mean": 0.22802734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.405977010726929, "grad_norm": 1.0354504585266113, "tokens_seen": 80289792, "tokens_per_sec": 20951.455941112064, "vram_gb": 5.521900544, "lambda": 0.31227718697239953, "lambda_min": 0.07588254660367966, "lambda_max": 0.6870288848876953, "S_mean": 0.23627726236979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.423596382141113, "grad_norm": 1.1074275970458984, "tokens_seen": 80453632, "tokens_per_sec": 20951.46581461764, "vram_gb": 5.521900544, "lambda": 0.3124190090845029, "lambda_min": 0.0758843868970871, "lambda_max": 0.6873107552528381, "S_mean": 0.234649658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.16312313079834, "grad_norm": 1.053943157196045, "tokens_seen": 80617472, "tokens_per_sec": 20951.529006574256, "vram_gb": 5.521900544, "lambda": 0.3125620245312651, "lambda_min": 0.07590129971504211, "lambda_max": 0.6878637671470642, "S_mean": 0.23819986979166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.432610988616943, "grad_norm": 1.0096173286437988, "tokens_seen": 80781312, "tokens_per_sec": 20951.546331888072, "vram_gb": 5.521900544, "lambda": 0.3126934288690488, "lambda_min": 0.07589064538478851, "lambda_max": 0.6882174015045166, "S_mean": 0.23112996419270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.509074687957764, "grad_norm": 1.0188393592834473, "tokens_seen": 80945152, "tokens_per_sec": 20951.66555618658, "vram_gb": 5.521900544, "lambda": 0.3128033395235737, "lambda_min": 0.07589405030012131, "lambda_max": 0.6884932518005371, "S_mean": 0.22878011067708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.332850217819214, "grad_norm": 1.0376380681991577, "tokens_seen": 81108992, "tokens_per_sec": 20951.835087830998, "vram_gb": 5.521900544, "lambda": 0.3129155542701483, "lambda_min": 0.07588192075490952, "lambda_max": 0.6888961791992188, "S_mean": 0.24773152669270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.393997430801392, "grad_norm": 1.0686644315719604, "tokens_seen": 81272832, "tokens_per_sec": 20951.813963985285, "vram_gb": 5.521900544, "lambda": 0.3130331278468172, "lambda_min": 0.07590511441230774, "lambda_max": 0.6892292499542236, "S_mean": 0.24022420247395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.346010684967041, "grad_norm": 1.0136126279830933, "tokens_seen": 81436672, "tokens_per_sec": 20951.70540894298, "vram_gb": 5.521900544, "lambda": 0.31312812585383654, "lambda_min": 0.07590151578187943, "lambda_max": 0.6895573735237122, "S_mean": 0.23932902018229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.190823554992676, "grad_norm": 1.0987647771835327, "tokens_seen": 81600512, "tokens_per_sec": 20951.52415385358, "vram_gb": 5.521900544, "lambda": 0.3132575511311491, "lambda_min": 0.07591140270233154, "lambda_max": 0.6899114847183228, "S_mean": 0.23681640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.316827297210693, "grad_norm": 1.0328556299209595, "tokens_seen": 81764352, "tokens_per_sec": 20951.537856599385, "vram_gb": 5.521900544, "lambda": 0.313356245867908, "lambda_min": 0.07592425495386124, "lambda_max": 0.6902447938919067, "S_mean": 0.24332682291666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.209489583969116, "grad_norm": 1.124996304512024, "tokens_seen": 81928192, "tokens_per_sec": 20951.516329142818, "vram_gb": 5.521900544, "lambda": 0.31344156712293625, "lambda_min": 0.07594966888427734, "lambda_max": 0.6904792785644531, "S_mean": 0.244354248046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.4136155366897585, "val_ppl": 82.56745013413533, "lambda": 0.31344715133309364, "lambda_min": 0.07595237344503403, "lambda_max": 0.6905073523521423, "lambda_per_layer": [0.18109829723834991, 0.1350838840007782, 0.15310446918010712, 0.154335156083107, 0.1565958708524704, 0.14151941239833832, 0.13652904331684113, 0.07595237344503403, 0.11867903918027878, 0.12221263349056244, 0.22727006673812866, 0.2608262300491333, 0.43092262744903564, 0.494296669960022, 0.47329193353652954, 0.5559357404708862, 0.6905073523521423, 0.45588892698287964, 0.5246658325195312, 0.5130826830863953, 0.6241251826286316, 0.3042311370372772, 0.3370567560195923, 0.2555203139781952], "S_mean": 0.24493408203125, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.494284629821777, "grad_norm": 1.0835291147232056, "tokens_seen": 82092032, "tokens_per_sec": 20943.90394316278, "vram_gb": 5.521900544, "lambda": 0.3135682161276539, "lambda_min": 0.07595079392194748, "lambda_max": 0.6909369826316833, "S_mean": 0.24407958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.540444374084473, "grad_norm": 1.1059671640396118, "tokens_seen": 82255872, "tokens_per_sec": 20944.41238930978, "vram_gb": 5.521900544, "lambda": 0.31365023273974657, "lambda_min": 0.07593118399381638, "lambda_max": 0.6912762522697449, "S_mean": 0.23402913411458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.5288872718811035, "grad_norm": 1.027239441871643, "tokens_seen": 82419712, "tokens_per_sec": 20944.22262357568, "vram_gb": 5.521900544, "lambda": 0.3137767994776368, "lambda_min": 0.07593941688537598, "lambda_max": 0.6916634440422058, "S_mean": 0.24474080403645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.364016056060791, "grad_norm": 1.0903321504592896, "tokens_seen": 82583552, "tokens_per_sec": 20944.182027221996, "vram_gb": 5.521900544, "lambda": 0.3139068717136979, "lambda_min": 0.07594665139913559, "lambda_max": 0.6919815540313721, "S_mean": 0.231536865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.521641492843628, "grad_norm": 1.0843945741653442, "tokens_seen": 82747392, "tokens_per_sec": 20944.18539088686, "vram_gb": 5.521900544, "lambda": 0.31403287779539824, "lambda_min": 0.0759631022810936, "lambda_max": 0.6922854781150818, "S_mean": 0.231170654296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.296411991119385, "grad_norm": 1.187554121017456, "tokens_seen": 82911232, "tokens_per_sec": 20943.97770564066, "vram_gb": 5.521900544, "lambda": 0.3141213372970621, "lambda_min": 0.07595549523830414, "lambda_max": 0.6926466822624207, "S_mean": 0.23917643229166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.291154861450195, "grad_norm": 1.1111668348312378, "tokens_seen": 83075072, "tokens_per_sec": 20943.640191942104, "vram_gb": 5.521900544, "lambda": 0.31423222211500007, "lambda_min": 0.07592548429965973, "lambda_max": 0.6930835247039795, "S_mean": 0.246307373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.386406183242798, "grad_norm": 1.0625892877578735, "tokens_seen": 83238912, "tokens_per_sec": 20943.49301181915, "vram_gb": 5.521900544, "lambda": 0.3143604143212239, "lambda_min": 0.07591784745454788, "lambda_max": 0.693537712097168, "S_mean": 0.24339803059895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.388834714889526, "grad_norm": 1.155990719795227, "tokens_seen": 83402752, "tokens_per_sec": 20943.60545308318, "vram_gb": 5.521900544, "lambda": 0.31446348937849206, "lambda_min": 0.07594628632068634, "lambda_max": 0.6936993598937988, "S_mean": 0.2534586588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.300122499465942, "grad_norm": 1.0662686824798584, "tokens_seen": 83566592, "tokens_per_sec": 20943.37858636541, "vram_gb": 5.521900544, "lambda": 0.31457167211920023, "lambda_min": 0.0759580209851265, "lambda_max": 0.6941836476325989, "S_mean": 0.23581949869791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.39301323890686, "grad_norm": 1.0381723642349243, "tokens_seen": 83730432, "tokens_per_sec": 20943.136561040392, "vram_gb": 5.521900544, "lambda": 0.314686903419594, "lambda_min": 0.07598552852869034, "lambda_max": 0.6945077180862427, "S_mean": 0.24239095052083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.297338008880615, "grad_norm": 1.098339319229126, "tokens_seen": 83894272, "tokens_per_sec": 20943.267512884955, "vram_gb": 5.521900544, "lambda": 0.31479004366944235, "lambda_min": 0.07600930333137512, "lambda_max": 0.6947270035743713, "S_mean": 0.24489339192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.316145181655884, "grad_norm": 1.0516879558563232, "tokens_seen": 84058112, "tokens_per_sec": 20943.38422476807, "vram_gb": 5.521900544, "lambda": 0.3149356059730053, "lambda_min": 0.07602882385253906, "lambda_max": 0.6950823068618774, "S_mean": 0.24384562174479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.4620678424835205, "grad_norm": 1.0711997747421265, "tokens_seen": 84221952, "tokens_per_sec": 20943.410954634208, "vram_gb": 5.521900544, "lambda": 0.315046984081467, "lambda_min": 0.07602054625749588, "lambda_max": 0.6954342722892761, "S_mean": 0.24573771158854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.328990459442139, "grad_norm": 1.131218671798706, "tokens_seen": 84385792, "tokens_per_sec": 20943.434636262682, "vram_gb": 5.521900544, "lambda": 0.31516568797330063, "lambda_min": 0.07602782547473907, "lambda_max": 0.6958217024803162, "S_mean": 0.23912556966145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.48292875289917, "grad_norm": 1.044818639755249, "tokens_seen": 84549632, "tokens_per_sec": 20943.693895205444, "vram_gb": 5.521900544, "lambda": 0.3152906059597929, "lambda_min": 0.07600995898246765, "lambda_max": 0.6961801648139954, "S_mean": 0.24586995442708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.313994884490967, "grad_norm": 1.1151703596115112, "tokens_seen": 84713472, "tokens_per_sec": 20943.824645381308, "vram_gb": 5.521900544, "lambda": 0.315370158602794, "lambda_min": 0.07599420100450516, "lambda_max": 0.6964869499206543, "S_mean": 0.23392740885416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.324565410614014, "grad_norm": 1.0517181158065796, "tokens_seen": 84877312, "tokens_per_sec": 20944.13529398635, "vram_gb": 5.521900544, "lambda": 0.3154366848369439, "lambda_min": 0.07599352300167084, "lambda_max": 0.6966110467910767, "S_mean": 0.239837646484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.46056866645813, "grad_norm": 1.1581065654754639, "tokens_seen": 85041152, "tokens_per_sec": 20944.455962303764, "vram_gb": 5.521900544, "lambda": 0.3155407576511304, "lambda_min": 0.07599354535341263, "lambda_max": 0.6967421174049377, "S_mean": 0.24370320638020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.44622015953064, "grad_norm": 1.0332860946655273, "tokens_seen": 85204992, "tokens_per_sec": 20944.346177619806, "vram_gb": 5.521900544, "lambda": 0.31563942320644855, "lambda_min": 0.07600587606430054, "lambda_max": 0.696998119354248, "S_mean": 0.23946126302083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.273375511169434, "grad_norm": 1.0537666082382202, "tokens_seen": 85368832, "tokens_per_sec": 20944.16176151514, "vram_gb": 5.521900544, "lambda": 0.3157343876858552, "lambda_min": 0.07599819451570511, "lambda_max": 0.6972917318344116, "S_mean": 0.2513020833333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.209328651428223, "grad_norm": 1.043289303779602, "tokens_seen": 85532672, "tokens_per_sec": 20943.96184680589, "vram_gb": 5.521900544, "lambda": 0.3158433359737198, "lambda_min": 0.07598729431629181, "lambda_max": 0.6976422071456909, "S_mean": 0.24577840169270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.376980304718018, "grad_norm": 1.1669307947158813, "tokens_seen": 85696512, "tokens_per_sec": 20943.736507852707, "vram_gb": 5.521900544, "lambda": 0.3159447442740202, "lambda_min": 0.07597460597753525, "lambda_max": 0.697928786277771, "S_mean": 0.23876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.373384475708008, "grad_norm": 1.068476915359497, "tokens_seen": 85860352, "tokens_per_sec": 20943.387343161405, "vram_gb": 5.521900544, "lambda": 0.3160390534127752, "lambda_min": 0.07594890147447586, "lambda_max": 0.6982132196426392, "S_mean": 0.2565511067708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.280790090560913, "grad_norm": 0.9758613705635071, "tokens_seen": 86024192, "tokens_per_sec": 20942.541096101286, "vram_gb": 5.521900544, "lambda": 0.3161451180155079, "lambda_min": 0.0759812518954277, "lambda_max": 0.6984708905220032, "S_mean": 0.24489339192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.236919403076172, "grad_norm": 0.9963020086288452, "tokens_seen": 86188032, "tokens_per_sec": 20942.30694327501, "vram_gb": 5.521900544, "lambda": 0.3162609313925107, "lambda_min": 0.07599872350692749, "lambda_max": 0.6986179351806641, "S_mean": 0.233428955078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.113057613372803, "grad_norm": 1.040155291557312, "tokens_seen": 86351872, "tokens_per_sec": 20942.006834341606, "vram_gb": 5.521900544, "lambda": 0.31634946446865797, "lambda_min": 0.07599519193172455, "lambda_max": 0.6989537477493286, "S_mean": 0.23865763346354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.347994327545166, "grad_norm": 1.0662428140640259, "tokens_seen": 86515712, "tokens_per_sec": 20941.948476537687, "vram_gb": 5.521900544, "lambda": 0.31647471711039543, "lambda_min": 0.07599222660064697, "lambda_max": 0.699309766292572, "S_mean": 0.2369384765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.4044365882873535, "grad_norm": 1.135418176651001, "tokens_seen": 86679552, "tokens_per_sec": 20941.81795952967, "vram_gb": 5.521900544, "lambda": 0.31658029463142157, "lambda_min": 0.075968436896801, "lambda_max": 0.6995365023612976, "S_mean": 0.24602254231770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.455777406692505, "grad_norm": 1.1414523124694824, "tokens_seen": 86843392, "tokens_per_sec": 20941.917375875986, "vram_gb": 5.521900544, "lambda": 0.31668085791170597, "lambda_min": 0.07591971755027771, "lambda_max": 0.6998434662818909, "S_mean": 0.23738606770833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.382472515106201, "grad_norm": 1.0008915662765503, "tokens_seen": 87007232, "tokens_per_sec": 20941.678533050796, "vram_gb": 5.521900544, "lambda": 0.31675523798912764, "lambda_min": 0.07588331401348114, "lambda_max": 0.7001345157623291, "S_mean": 0.23784383138020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.374981880187988, "grad_norm": 1.0622222423553467, "tokens_seen": 87171072, "tokens_per_sec": 20942.0573309417, "vram_gb": 5.521900544, "lambda": 0.3168450075512131, "lambda_min": 0.07583951950073242, "lambda_max": 0.7003660202026367, "S_mean": 0.23014322916666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.348634243011475, "grad_norm": 1.096805214881897, "tokens_seen": 87334912, "tokens_per_sec": 20942.19123672087, "vram_gb": 5.521900544, "lambda": 0.31692829572906095, "lambda_min": 0.07579644024372101, "lambda_max": 0.7004907727241516, "S_mean": 0.234283447265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.434743165969849, "grad_norm": 1.0508352518081665, "tokens_seen": 87498752, "tokens_per_sec": 20942.492007980898, "vram_gb": 5.521900544, "lambda": 0.3170183689023058, "lambda_min": 0.07578421384096146, "lambda_max": 0.700778603553772, "S_mean": 0.23612467447916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.456961154937744, "grad_norm": 1.1138606071472168, "tokens_seen": 87662592, "tokens_per_sec": 20942.525713071464, "vram_gb": 5.521900544, "lambda": 0.3171346504241228, "lambda_min": 0.07580265402793884, "lambda_max": 0.70119708776474, "S_mean": 0.242218017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.3396735191345215, "grad_norm": 1.0211853981018066, "tokens_seen": 87826432, "tokens_per_sec": 20942.732737996423, "vram_gb": 5.521900544, "lambda": 0.3172450993830959, "lambda_min": 0.07581038028001785, "lambda_max": 0.701500415802002, "S_mean": 0.23161824544270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.2110984325408936, "grad_norm": 0.989276111125946, "tokens_seen": 87990272, "tokens_per_sec": 20942.813487010288, "vram_gb": 5.521900544, "lambda": 0.31733663318057853, "lambda_min": 0.0758071318268776, "lambda_max": 0.7017129063606262, "S_mean": 0.2327880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.338357448577881, "grad_norm": 1.0868417024612427, "tokens_seen": 88154112, "tokens_per_sec": 20942.972492855453, "vram_gb": 5.521900544, "lambda": 0.3174064851676424, "lambda_min": 0.07580315321683884, "lambda_max": 0.701964259147644, "S_mean": 0.24399820963541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.451634168624878, "grad_norm": 1.2384380102157593, "tokens_seen": 88317952, "tokens_per_sec": 20943.069092660295, "vram_gb": 5.521900544, "lambda": 0.31751407341410715, "lambda_min": 0.0758092999458313, "lambda_max": 0.7022495269775391, "S_mean": 0.23581949869791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.397380113601685, "grad_norm": 1.1590652465820312, "tokens_seen": 88481792, "tokens_per_sec": 20943.165031866032, "vram_gb": 5.521900544, "lambda": 0.3176001642520229, "lambda_min": 0.07581730931997299, "lambda_max": 0.7025134563446045, "S_mean": 0.23868815104166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.317004919052124, "grad_norm": 1.0772795677185059, "tokens_seen": 88645632, "tokens_per_sec": 20943.155291915267, "vram_gb": 5.521900544, "lambda": 0.3176762890070677, "lambda_min": 0.07583750784397125, "lambda_max": 0.7025777101516724, "S_mean": 0.24128214518229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.330420970916748, "grad_norm": 1.2027689218521118, "tokens_seen": 88809472, "tokens_per_sec": 20943.32944830285, "vram_gb": 5.521900544, "lambda": 0.31776675426711637, "lambda_min": 0.0758381336927414, "lambda_max": 0.7028961777687073, "S_mean": 0.24226888020833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.3025877475738525, "grad_norm": 1.092318058013916, "tokens_seen": 88973312, "tokens_per_sec": 20943.528034744282, "vram_gb": 5.521900544, "lambda": 0.3178356569260359, "lambda_min": 0.07585643231868744, "lambda_max": 0.7030503749847412, "S_mean": 0.24918619791666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.4387452602386475, "grad_norm": 1.0772404670715332, "tokens_seen": 89137152, "tokens_per_sec": 20943.718008341555, "vram_gb": 5.521900544, "lambda": 0.31792027472207945, "lambda_min": 0.07585423439741135, "lambda_max": 0.703264594078064, "S_mean": 0.230194091796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.493814468383789, "grad_norm": 1.1657543182373047, "tokens_seen": 89300992, "tokens_per_sec": 20943.884013582858, "vram_gb": 5.521900544, "lambda": 0.31800786467889947, "lambda_min": 0.07585559785366058, "lambda_max": 0.7034740447998047, "S_mean": 0.2423095703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.531611680984497, "grad_norm": 1.3615761995315552, "tokens_seen": 89464832, "tokens_per_sec": 20943.532953249738, "vram_gb": 5.521900544, "lambda": 0.3180863031496604, "lambda_min": 0.07588838040828705, "lambda_max": 0.7036715149879456, "S_mean": 0.229766845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.535367965698242, "grad_norm": 1.0540837049484253, "tokens_seen": 89628672, "tokens_per_sec": 20943.52880437214, "vram_gb": 5.521900544, "lambda": 0.31818564763913554, "lambda_min": 0.07590192556381226, "lambda_max": 0.703913152217865, "S_mean": 0.24582926432291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.291231870651245, "grad_norm": 1.0839824676513672, "tokens_seen": 89792512, "tokens_per_sec": 20943.577572000606, "vram_gb": 5.521900544, "lambda": 0.318250703625381, "lambda_min": 0.07588613778352737, "lambda_max": 0.7041938304901123, "S_mean": 0.23665364583333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.173349857330322, "grad_norm": 1.0949193239212036, "tokens_seen": 89956352, "tokens_per_sec": 20943.688367012477, "vram_gb": 5.521900544, "lambda": 0.31832778081297874, "lambda_min": 0.07586377114057541, "lambda_max": 0.7044209837913513, "S_mean": 0.23555501302083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.413402080535889, "grad_norm": 1.1395485401153564, "tokens_seen": 90120192, "tokens_per_sec": 20943.976888456265, "vram_gb": 5.521900544, "lambda": 0.31840884933869046, "lambda_min": 0.0758935958147049, "lambda_max": 0.7045721411705017, "S_mean": 0.23895263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.416970539093017, "val_ppl": 82.84492933918983, "lambda": 0.3184110792353749, "lambda_min": 0.0758952796459198, "lambda_max": 0.7045729160308838, "lambda_per_layer": [0.1841835230588913, 0.13501915335655212, 0.15412716567516327, 0.15563082695007324, 0.15770921111106873, 0.14184556901454926, 0.1369566172361374, 0.0758952796459198, 0.11870025098323822, 0.12235601991415024, 0.2283376008272171, 0.2626231908798218, 0.4341577887535095, 0.49985599517822266, 0.4799751341342926, 0.5672038197517395, 0.7045729160308838, 0.46420586109161377, 0.5385462641716003, 0.5257704257965088, 0.640046238899231, 0.30916285514831543, 0.34396740794181824, 0.2610167860984802], "S_mean": 0.23679606119791666, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.358388662338257, "grad_norm": 1.1814757585525513, "tokens_seen": 90284032, "tokens_per_sec": 20937.33836709902, "vram_gb": 5.521900544, "lambda": 0.3184821304554741, "lambda_min": 0.07590901851654053, "lambda_max": 0.7047363519668579, "S_mean": 0.2722981770833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.276075601577759, "grad_norm": 1.168894648551941, "tokens_seen": 90447872, "tokens_per_sec": 20937.317166436795, "vram_gb": 5.521900544, "lambda": 0.31857701887687045, "lambda_min": 0.07591107487678528, "lambda_max": 0.7051239013671875, "S_mean": 0.24267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.40165901184082, "grad_norm": 1.089553713798523, "tokens_seen": 90611712, "tokens_per_sec": 20937.32184393177, "vram_gb": 5.521900544, "lambda": 0.3186589435984691, "lambda_min": 0.07589314132928848, "lambda_max": 0.7053759694099426, "S_mean": 0.2548624674479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.304551601409912, "grad_norm": 1.1252273321151733, "tokens_seen": 90775552, "tokens_per_sec": 20937.035612345764, "vram_gb": 5.521900544, "lambda": 0.3187442384660244, "lambda_min": 0.0759100615978241, "lambda_max": 0.7056154012680054, "S_mean": 0.23736572265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.450609445571899, "grad_norm": 1.0882940292358398, "tokens_seen": 90939392, "tokens_per_sec": 20937.063682610544, "vram_gb": 5.521900544, "lambda": 0.3188036906843384, "lambda_min": 0.07593376934528351, "lambda_max": 0.7056565880775452, "S_mean": 0.2566324869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.173192262649536, "grad_norm": 1.1132748126983643, "tokens_seen": 91103232, "tokens_per_sec": 20937.203436868494, "vram_gb": 5.521900544, "lambda": 0.31888922955840826, "lambda_min": 0.0759333148598671, "lambda_max": 0.7059245109558105, "S_mean": 0.24308268229166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.3648881912231445, "grad_norm": 1.2845615148544312, "tokens_seen": 91267072, "tokens_per_sec": 20937.278150907565, "vram_gb": 5.521900544, "lambda": 0.3189837612832586, "lambda_min": 0.07593454420566559, "lambda_max": 0.7062949538230896, "S_mean": 0.23486328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.359609842300415, "grad_norm": 1.1107591390609741, "tokens_seen": 91430912, "tokens_per_sec": 20937.0488017015, "vram_gb": 5.521900544, "lambda": 0.3190742504472534, "lambda_min": 0.07591727375984192, "lambda_max": 0.7066128849983215, "S_mean": 0.23335774739583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.352390766143799, "grad_norm": 1.1077057123184204, "tokens_seen": 91594752, "tokens_per_sec": 20937.097722631, "vram_gb": 5.521900544, "lambda": 0.3191437888890505, "lambda_min": 0.07590454816818237, "lambda_max": 0.7068905234336853, "S_mean": 0.23854573567708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.184746026992798, "grad_norm": 1.062608242034912, "tokens_seen": 91758592, "tokens_per_sec": 20937.28363551641, "vram_gb": 5.521900544, "lambda": 0.3192397678891818, "lambda_min": 0.07590790092945099, "lambda_max": 0.7072544693946838, "S_mean": 0.2505086263020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.132447242736816, "grad_norm": 1.0208499431610107, "tokens_seen": 91922432, "tokens_per_sec": 20937.35250681757, "vram_gb": 5.521900544, "lambda": 0.31933032255619764, "lambda_min": 0.075899638235569, "lambda_max": 0.7074978351593018, "S_mean": 0.23289998372395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.351528882980347, "grad_norm": 1.2396187782287598, "tokens_seen": 92086272, "tokens_per_sec": 20937.39308796161, "vram_gb": 5.521900544, "lambda": 0.3194219156478842, "lambda_min": 0.07588179409503937, "lambda_max": 0.7077156901359558, "S_mean": 0.246307373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.374356269836426, "grad_norm": 1.1511762142181396, "tokens_seen": 92250112, "tokens_per_sec": 20937.31845180648, "vram_gb": 5.521900544, "lambda": 0.31950094488759834, "lambda_min": 0.07587604969739914, "lambda_max": 0.7078729867935181, "S_mean": 0.2432861328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.326635122299194, "grad_norm": 1.1505966186523438, "tokens_seen": 92413952, "tokens_per_sec": 20937.390081220736, "vram_gb": 5.521900544, "lambda": 0.319584546610713, "lambda_min": 0.07587103545665741, "lambda_max": 0.7080317139625549, "S_mean": 0.23443603515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.3671183586120605, "grad_norm": 1.0604342222213745, "tokens_seen": 92577792, "tokens_per_sec": 20937.480583440392, "vram_gb": 5.521900544, "lambda": 0.319683985474209, "lambda_min": 0.0758582279086113, "lambda_max": 0.7083640098571777, "S_mean": 0.23722330729166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.245595216751099, "grad_norm": 1.066221833229065, "tokens_seen": 92741632, "tokens_per_sec": 20937.37086857525, "vram_gb": 5.521900544, "lambda": 0.319750570692122, "lambda_min": 0.07588465511798859, "lambda_max": 0.708559513092041, "S_mean": 0.23378499348958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.409071445465088, "grad_norm": 1.2592377662658691, "tokens_seen": 92905472, "tokens_per_sec": 20937.268042988995, "vram_gb": 5.521900544, "lambda": 0.3198445653542876, "lambda_min": 0.07584252208471298, "lambda_max": 0.7089048624038696, "S_mean": 0.241119384765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.322314977645874, "grad_norm": 1.163333535194397, "tokens_seen": 93069312, "tokens_per_sec": 20937.053192766263, "vram_gb": 5.521900544, "lambda": 0.3199012711023291, "lambda_min": 0.07583322376012802, "lambda_max": 0.7090256214141846, "S_mean": 0.23653157552083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.334006071090698, "grad_norm": 1.137975811958313, "tokens_seen": 93233152, "tokens_per_sec": 20937.201333504963, "vram_gb": 5.521900544, "lambda": 0.3199791464333733, "lambda_min": 0.07583372294902802, "lambda_max": 0.7092297673225403, "S_mean": 0.23125203450520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.375930070877075, "grad_norm": 1.4309909343719482, "tokens_seen": 93396992, "tokens_per_sec": 20937.247231817608, "vram_gb": 5.521900544, "lambda": 0.32003639452159405, "lambda_min": 0.0758221298456192, "lambda_max": 0.709321141242981, "S_mean": 0.23554484049479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.3305134773254395, "grad_norm": 1.0281590223312378, "tokens_seen": 93560832, "tokens_per_sec": 20937.475807074694, "vram_gb": 5.521900544, "lambda": 0.32011103971550864, "lambda_min": 0.0758122056722641, "lambda_max": 0.7095022797584534, "S_mean": 0.23313395182291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.201779365539551, "grad_norm": 1.1212308406829834, "tokens_seen": 93724672, "tokens_per_sec": 20937.49605332082, "vram_gb": 5.521900544, "lambda": 0.32020201285680133, "lambda_min": 0.07580024749040604, "lambda_max": 0.7097306251525879, "S_mean": 0.24714152018229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.2671637535095215, "grad_norm": 1.0862423181533813, "tokens_seen": 93888512, "tokens_per_sec": 20937.62476435803, "vram_gb": 5.521900544, "lambda": 0.3202570068339507, "lambda_min": 0.07580944150686264, "lambda_max": 0.709906816482544, "S_mean": 0.23301188151041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.247682809829712, "grad_norm": 1.147290587425232, "tokens_seen": 94052352, "tokens_per_sec": 20937.79582693876, "vram_gb": 5.521900544, "lambda": 0.3203262286260724, "lambda_min": 0.07580754160881042, "lambda_max": 0.7100526094436646, "S_mean": 0.23787434895833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.365674257278442, "grad_norm": 1.14326012134552, "tokens_seen": 94216192, "tokens_per_sec": 20937.868738208104, "vram_gb": 5.521900544, "lambda": 0.3203818431744973, "lambda_min": 0.0757763609290123, "lambda_max": 0.7101386785507202, "S_mean": 0.24918619791666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.469127178192139, "grad_norm": 1.1341936588287354, "tokens_seen": 94380032, "tokens_per_sec": 20938.068983621157, "vram_gb": 5.521900544, "lambda": 0.32046946107099455, "lambda_min": 0.07575260102748871, "lambda_max": 0.7103111743927002, "S_mean": 0.23177083333333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.3443920612335205, "grad_norm": 1.2063556909561157, "tokens_seen": 94543872, "tokens_per_sec": 20938.28439940529, "vram_gb": 5.521900544, "lambda": 0.3205560448889931, "lambda_min": 0.07573428004980087, "lambda_max": 0.7105767130851746, "S_mean": 0.22954305013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.303657531738281, "grad_norm": 1.1610201597213745, "tokens_seen": 94707712, "tokens_per_sec": 20938.48828795468, "vram_gb": 5.521900544, "lambda": 0.3206070481489102, "lambda_min": 0.07573463022708893, "lambda_max": 0.7108243703842163, "S_mean": 0.24876912434895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.485320329666138, "grad_norm": 1.303880214691162, "tokens_seen": 94871552, "tokens_per_sec": 20938.49164118972, "vram_gb": 5.521900544, "lambda": 0.3206480899825692, "lambda_min": 0.07572278380393982, "lambda_max": 0.7110499739646912, "S_mean": 0.23578898111979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.207217454910278, "grad_norm": 1.1148419380187988, "tokens_seen": 95035392, "tokens_per_sec": 20938.642718805222, "vram_gb": 5.521900544, "lambda": 0.3207078240811825, "lambda_min": 0.07575057446956635, "lambda_max": 0.71124267578125, "S_mean": 0.24312337239583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.329600095748901, "grad_norm": 1.0723044872283936, "tokens_seen": 95199232, "tokens_per_sec": 20938.811407985082, "vram_gb": 5.521900544, "lambda": 0.32077139616012573, "lambda_min": 0.07573558390140533, "lambda_max": 0.7114046812057495, "S_mean": 0.24593098958333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.250370025634766, "grad_norm": 1.1280910968780518, "tokens_seen": 95363072, "tokens_per_sec": 20939.16727964421, "vram_gb": 5.521900544, "lambda": 0.32085795638461906, "lambda_min": 0.07574325054883957, "lambda_max": 0.7115621566772461, "S_mean": 0.22974650065104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.28301739692688, "grad_norm": 1.2590723037719727, "tokens_seen": 95526912, "tokens_per_sec": 20939.309972151124, "vram_gb": 5.521900544, "lambda": 0.32095415952305, "lambda_min": 0.07575081288814545, "lambda_max": 0.7117947340011597, "S_mean": 0.2508544921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.31201696395874, "grad_norm": 1.0813121795654297, "tokens_seen": 95690752, "tokens_per_sec": 20939.465601970885, "vram_gb": 5.521900544, "lambda": 0.3210279953976472, "lambda_min": 0.07578305900096893, "lambda_max": 0.7120452523231506, "S_mean": 0.230133056640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.352276563644409, "grad_norm": 1.15703284740448, "tokens_seen": 95854592, "tokens_per_sec": 20938.73196111659, "vram_gb": 5.521900544, "lambda": 0.32109503261744976, "lambda_min": 0.07577698677778244, "lambda_max": 0.7122928500175476, "S_mean": 0.23790486653645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.307147026062012, "grad_norm": 1.1630396842956543, "tokens_seen": 96018432, "tokens_per_sec": 20938.785997815503, "vram_gb": 5.521900544, "lambda": 0.3211668940881888, "lambda_min": 0.07577894628047943, "lambda_max": 0.7124600410461426, "S_mean": 0.23881022135416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.348819971084595, "grad_norm": 1.2478220462799072, "tokens_seen": 96182272, "tokens_per_sec": 20938.59763475112, "vram_gb": 5.521900544, "lambda": 0.3212483761211236, "lambda_min": 0.07576851546764374, "lambda_max": 0.7125773429870605, "S_mean": 0.24164835611979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.2395734786987305, "grad_norm": 1.178272008895874, "tokens_seen": 96346112, "tokens_per_sec": 20938.691748459416, "vram_gb": 5.521900544, "lambda": 0.3213210618123412, "lambda_min": 0.07576356828212738, "lambda_max": 0.7127376794815063, "S_mean": 0.23344930013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.418463706970215, "grad_norm": 1.2715249061584473, "tokens_seen": 96509952, "tokens_per_sec": 20938.729708415372, "vram_gb": 5.521900544, "lambda": 0.3213784700880448, "lambda_min": 0.07574725151062012, "lambda_max": 0.7128928303718567, "S_mean": 0.236114501953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.328860282897949, "grad_norm": 1.148813247680664, "tokens_seen": 96673792, "tokens_per_sec": 20938.252956239572, "vram_gb": 5.521900544, "lambda": 0.3214435235907634, "lambda_min": 0.07575351744890213, "lambda_max": 0.7130343914031982, "S_mean": 0.23148600260416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.4963157176971436, "grad_norm": 1.103117823600769, "tokens_seen": 96837632, "tokens_per_sec": 20938.211416517468, "vram_gb": 5.521900544, "lambda": 0.321516215801239, "lambda_min": 0.0757477805018425, "lambda_max": 0.7131857872009277, "S_mean": 0.23333740234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.263985633850098, "grad_norm": 1.118350625038147, "tokens_seen": 97001472, "tokens_per_sec": 20938.2204264793, "vram_gb": 5.521900544, "lambda": 0.3215939151123166, "lambda_min": 0.07573508471250534, "lambda_max": 0.7134483456611633, "S_mean": 0.23473103841145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.236951112747192, "grad_norm": 1.2045280933380127, "tokens_seen": 97165312, "tokens_per_sec": 20938.213499055793, "vram_gb": 5.521900544, "lambda": 0.32163970296581584, "lambda_min": 0.07572296261787415, "lambda_max": 0.7135576009750366, "S_mean": 0.24397786458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.3460516929626465, "grad_norm": 1.086990237236023, "tokens_seen": 97329152, "tokens_per_sec": 20938.460278639235, "vram_gb": 5.521900544, "lambda": 0.32171495134631795, "lambda_min": 0.07570944726467133, "lambda_max": 0.7137963771820068, "S_mean": 0.24125162760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.301627159118652, "grad_norm": 1.1643012762069702, "tokens_seen": 97492992, "tokens_per_sec": 20938.509720353915, "vram_gb": 5.521900544, "lambda": 0.32180316373705864, "lambda_min": 0.07570232450962067, "lambda_max": 0.7140558362007141, "S_mean": 0.23675537109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.215116262435913, "grad_norm": 1.2696120738983154, "tokens_seen": 97656832, "tokens_per_sec": 20938.40527522433, "vram_gb": 5.521900544, "lambda": 0.321872066706419, "lambda_min": 0.07569749653339386, "lambda_max": 0.7142311334609985, "S_mean": 0.24013264973958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.202595233917236, "grad_norm": 1.538216233253479, "tokens_seen": 97820672, "tokens_per_sec": 20938.472821534993, "vram_gb": 5.521900544, "lambda": 0.321935310959816, "lambda_min": 0.07568348199129105, "lambda_max": 0.7143896222114563, "S_mean": 0.23834228515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.296751976013184, "grad_norm": 1.0792603492736816, "tokens_seen": 97984512, "tokens_per_sec": 20938.815226271898, "vram_gb": 5.521900544, "lambda": 0.3220043744271, "lambda_min": 0.07566777616739273, "lambda_max": 0.7147283554077148, "S_mean": 0.23685201009114584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.2947118282318115, "grad_norm": 1.54154372215271, "tokens_seen": 98148352, "tokens_per_sec": 20939.03424108186, "vram_gb": 5.521900544, "lambda": 0.32208347382644814, "lambda_min": 0.07566583901643753, "lambda_max": 0.7150307297706604, "S_mean": 0.23305257161458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.407615661621094, "val_ppl": 82.07353893301419, "lambda": 0.3221415774896741, "lambda_min": 0.07567994296550751, "lambda_max": 0.7152900695800781, "lambda_per_layer": [0.18680250644683838, 0.13508518040180206, 0.15476864576339722, 0.15647175908088684, 0.15856488049030304, 0.14215217530727386, 0.1373746544122696, 0.07567994296550751, 0.11839891970157623, 0.12244366854429245, 0.22932282090187073, 0.2641337215900421, 0.4371626079082489, 0.5036436915397644, 0.48382195830345154, 0.5751225352287292, 0.7152900695800781, 0.4705618619918823, 0.54891037940979, 0.5351443290710449, 0.6508606672286987, 0.3135398030281067, 0.3504140377044678, 0.26572704315185547], "S_mean": 0.2645263671875, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4696.141266584396, "nan_failures": 0} diff --git a/s_only_s3/config.json b/s_only_s3/config.json new file mode 100644 index 0000000000000000000000000000000000000000..dc42ca3627dacfa07c3eae1d02cc2d35bff9aadd --- /dev/null +++ b/s_only_s3/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 3, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s3", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 3001346, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/s_only_s3/final.pt b/s_only_s3/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..3432187f0bd1671e200a391eb6038618740e1696 --- /dev/null +++ b/s_only_s3/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3cdb25ea470c8127c5180c810b6814ce60714d9be03a5821cf53b6d48079beab +size 255945415 diff --git a/s_only_s3/metrics.jsonl b/s_only_s3/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..44939adf3d4dc141591ac94b2130978d249bd3f7 --- /dev/null +++ b/s_only_s3/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 3, "config": {"variant": "s_only", "seed": 3, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s3", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 3001346, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.89471435546875, "grad_norm": 14.873425483703613, "tokens_seen": 8192, "tokens_per_sec": 8439.456964459447, "vram_gb": 5.017291776, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.3315633138020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.87869644165039, "grad_norm": 1.967416763305664, "tokens_seen": 172032, "tokens_per_sec": 19656.461136309346, "vram_gb": 5.521900544, "lambda": 0.12694220369060835, "lambda_min": 0.12689700722694397, "lambda_max": 0.1269797384738922, "S_mean": 0.5487416585286459, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.122970581054688, "grad_norm": 1.7698146104812622, "tokens_seen": 335872, "tokens_per_sec": 20282.72488997363, "vram_gb": 5.521900544, "lambda": 0.126886830975612, "lambda_min": 0.12683071196079254, "lambda_max": 0.1269809603691101, "S_mean": 0.5449625651041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.169635772705078, "grad_norm": 1.5788322687149048, "tokens_seen": 499712, "tokens_per_sec": 20613.645471021344, "vram_gb": 5.521900544, "lambda": 0.12679918544987837, "lambda_min": 0.12670688331127167, "lambda_max": 0.12700751423835754, "S_mean": 0.5191141764322916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.532025337219238, "grad_norm": 0.9401926398277283, "tokens_seen": 663552, "tokens_per_sec": 20750.150908686024, "vram_gb": 5.521900544, "lambda": 0.12672995527585348, "lambda_min": 0.12655213475227356, "lambda_max": 0.12715083360671997, "S_mean": 0.3672129313151042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.122751235961914, "grad_norm": 0.6423443555831909, "tokens_seen": 827392, "tokens_per_sec": 20828.698294083515, "vram_gb": 5.521900544, "lambda": 0.1266972292214632, "lambda_min": 0.1264309287071228, "lambda_max": 0.127455934882164, "S_mean": 0.26263427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.194922685623169, "grad_norm": 0.6788739562034607, "tokens_seen": 991232, "tokens_per_sec": 20896.760311148955, "vram_gb": 5.521900544, "lambda": 0.12658714999755225, "lambda_min": 0.12615905702114105, "lambda_max": 0.12789706885814667, "S_mean": 0.2829182942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.9264326095581055, "grad_norm": 0.7697112560272217, "tokens_seen": 1155072, "tokens_per_sec": 20936.572965724125, "vram_gb": 5.521900544, "lambda": 0.12656180560588837, "lambda_min": 0.12599341571331024, "lambda_max": 0.12823869287967682, "S_mean": 0.2559000651041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.8878092765808105, "grad_norm": 1.0362284183502197, "tokens_seen": 1318912, "tokens_per_sec": 20966.34777035746, "vram_gb": 5.521900544, "lambda": 0.12656703094641367, "lambda_min": 0.12591195106506348, "lambda_max": 0.12838679552078247, "S_mean": 0.3067220052083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.762887954711914, "grad_norm": 0.7811712622642517, "tokens_seen": 1482752, "tokens_per_sec": 20985.67340406836, "vram_gb": 5.521900544, "lambda": 0.12656066628793874, "lambda_min": 0.12559977173805237, "lambda_max": 0.12863868474960327, "S_mean": 0.3252665201822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.68584680557251, "grad_norm": 0.6372997760772705, "tokens_seen": 1646592, "tokens_per_sec": 21014.232982469483, "vram_gb": 5.521900544, "lambda": 0.12658241080741087, "lambda_min": 0.125446617603302, "lambda_max": 0.12891443073749542, "S_mean": 0.2751973470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.5929365158081055, "grad_norm": 0.5508769154548645, "tokens_seen": 1810432, "tokens_per_sec": 21022.89116951117, "vram_gb": 5.521900544, "lambda": 0.1266044502456983, "lambda_min": 0.12537190318107605, "lambda_max": 0.1290661245584488, "S_mean": 0.291900634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.62945818901062, "grad_norm": 0.5230473875999451, "tokens_seen": 1974272, "tokens_per_sec": 21036.376581322555, "vram_gb": 5.521900544, "lambda": 0.1266349603732427, "lambda_min": 0.1252930462360382, "lambda_max": 0.12915004789829254, "S_mean": 0.26641845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.621845960617065, "grad_norm": 0.4841929078102112, "tokens_seen": 2138112, "tokens_per_sec": 21043.364315660932, "vram_gb": 5.521900544, "lambda": 0.12663273823757967, "lambda_min": 0.12505954504013062, "lambda_max": 0.12929180264472961, "S_mean": 0.2976175944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.448667764663696, "grad_norm": 0.6178326606750488, "tokens_seen": 2301952, "tokens_per_sec": 21054.71162588675, "vram_gb": 5.521900544, "lambda": 0.1266415569310387, "lambda_min": 0.12484163790941238, "lambda_max": 0.12954574823379517, "S_mean": 0.3052164713541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.332726001739502, "grad_norm": 0.5125993490219116, "tokens_seen": 2465792, "tokens_per_sec": 21053.360058515107, "vram_gb": 5.521900544, "lambda": 0.12665006828804812, "lambda_min": 0.1247558444738388, "lambda_max": 0.12993110716342926, "S_mean": 0.3263448079427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.429057359695435, "grad_norm": 0.5630685687065125, "tokens_seen": 2629632, "tokens_per_sec": 21047.068459727758, "vram_gb": 5.521900544, "lambda": 0.12663793377578259, "lambda_min": 0.12451237440109253, "lambda_max": 0.13015924394130707, "S_mean": 0.3396504720052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.48973274230957, "grad_norm": 0.6092301607131958, "tokens_seen": 2793472, "tokens_per_sec": 21023.690960679618, "vram_gb": 5.521900544, "lambda": 0.1266579208895564, "lambda_min": 0.12438330799341202, "lambda_max": 0.13042491674423218, "S_mean": 0.3083902994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.27322793006897, "grad_norm": 0.5584782958030701, "tokens_seen": 2957312, "tokens_per_sec": 21002.23325351704, "vram_gb": 5.521900544, "lambda": 0.1267132411400477, "lambda_min": 0.1244204044342041, "lambda_max": 0.13076213002204895, "S_mean": 0.3275960286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.313457012176514, "grad_norm": 0.571232259273529, "tokens_seen": 3121152, "tokens_per_sec": 20973.226637560514, "vram_gb": 5.521900544, "lambda": 0.12670761936654648, "lambda_min": 0.12433203309774399, "lambda_max": 0.13099902868270874, "S_mean": 0.324798583984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.367921352386475, "grad_norm": 0.5920793414115906, "tokens_seen": 3284992, "tokens_per_sec": 20953.198376665423, "vram_gb": 5.521900544, "lambda": 0.12676518689841032, "lambda_min": 0.12427101284265518, "lambda_max": 0.13152176141738892, "S_mean": 0.2726949055989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.288817405700684, "grad_norm": 0.46997299790382385, "tokens_seen": 3448832, "tokens_per_sec": 20937.154167022163, "vram_gb": 5.521900544, "lambda": 0.1267999398211638, "lambda_min": 0.12414056062698364, "lambda_max": 0.13183827698230743, "S_mean": 0.3342081705729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.229104280471802, "grad_norm": 0.5585354566574097, "tokens_seen": 3612672, "tokens_per_sec": 20876.09357745661, "vram_gb": 5.521900544, "lambda": 0.12688300355027118, "lambda_min": 0.12405820935964584, "lambda_max": 0.13209274411201477, "S_mean": 0.32763671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.362649917602539, "grad_norm": 0.5547759532928467, "tokens_seen": 3776512, "tokens_per_sec": 20840.06869757274, "vram_gb": 5.521900544, "lambda": 0.12695795483887196, "lambda_min": 0.1240253746509552, "lambda_max": 0.13249336183071136, "S_mean": 0.3298746744791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.211521863937378, "grad_norm": 0.5119973421096802, "tokens_seen": 3940352, "tokens_per_sec": 20812.952201453325, "vram_gb": 5.521900544, "lambda": 0.12707082461565733, "lambda_min": 0.12391414493322372, "lambda_max": 0.13269871473312378, "S_mean": 0.323089599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.24647855758667, "grad_norm": 0.5159915685653687, "tokens_seen": 4104192, "tokens_per_sec": 20792.236486472513, "vram_gb": 5.521900544, "lambda": 0.12718711979687214, "lambda_min": 0.1239493191242218, "lambda_max": 0.13319741189479828, "S_mean": 0.302154541015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.274489641189575, "grad_norm": 0.5817859172821045, "tokens_seen": 4268032, "tokens_per_sec": 20763.46189329614, "vram_gb": 5.521900544, "lambda": 0.12737652597328028, "lambda_min": 0.12392856180667877, "lambda_max": 0.13392376899719238, "S_mean": 0.2804463704427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.1697680950164795, "grad_norm": 0.5248720049858093, "tokens_seen": 4431872, "tokens_per_sec": 20745.176935950207, "vram_gb": 5.521900544, "lambda": 0.12749819147090116, "lambda_min": 0.12412895262241364, "lambda_max": 0.13425122201442719, "S_mean": 0.3323262532552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.208997964859009, "grad_norm": 0.5683438777923584, "tokens_seen": 4595712, "tokens_per_sec": 20718.568341705773, "vram_gb": 5.521900544, "lambda": 0.12764237727969885, "lambda_min": 0.12428677827119827, "lambda_max": 0.13462147116661072, "S_mean": 0.3320109049479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.201117038726807, "grad_norm": 0.5471563935279846, "tokens_seen": 4759552, "tokens_per_sec": 20709.136776622272, "vram_gb": 5.521900544, "lambda": 0.127825283134977, "lambda_min": 0.12433050572872162, "lambda_max": 0.13504336774349213, "S_mean": 0.3380940755208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.101030349731445, "grad_norm": 0.4785258173942566, "tokens_seen": 4923392, "tokens_per_sec": 20704.092364570937, "vram_gb": 5.521900544, "lambda": 0.12795953080058098, "lambda_min": 0.12441529333591461, "lambda_max": 0.13538485765457153, "S_mean": 0.3061930338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.014770746231079, "grad_norm": 0.5218915343284607, "tokens_seen": 5087232, "tokens_per_sec": 20706.77814746354, "vram_gb": 5.521900544, "lambda": 0.12812368975331387, "lambda_min": 0.12459596246480942, "lambda_max": 0.13578739762306213, "S_mean": 0.2999776204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.104251384735107, "grad_norm": 0.5437524318695068, "tokens_seen": 5251072, "tokens_per_sec": 20716.726458884114, "vram_gb": 5.521900544, "lambda": 0.12832175133128962, "lambda_min": 0.1245947927236557, "lambda_max": 0.13630715012550354, "S_mean": 0.3264007568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 5.902468681335449, "grad_norm": 0.541124165058136, "tokens_seen": 5414912, "tokens_per_sec": 20733.95505915465, "vram_gb": 5.521900544, "lambda": 0.12851136705527702, "lambda_min": 0.1247156485915184, "lambda_max": 0.13678991794586182, "S_mean": 0.3277486165364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.017203092575073, "grad_norm": 0.5036903023719788, "tokens_seen": 5578752, "tokens_per_sec": 20745.968682161387, "vram_gb": 5.521900544, "lambda": 0.12868034715453783, "lambda_min": 0.12457018345594406, "lambda_max": 0.13742287456989288, "S_mean": 0.3160654703776042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 6.075088024139404, "grad_norm": 0.5575960278511047, "tokens_seen": 5742592, "tokens_per_sec": 20760.498686939114, "vram_gb": 5.521900544, "lambda": 0.12891405262053013, "lambda_min": 0.12447946518659592, "lambda_max": 0.13775889575481415, "S_mean": 0.319427490234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.882572174072266, "grad_norm": 0.5440836548805237, "tokens_seen": 5906432, "tokens_per_sec": 20770.492653615467, "vram_gb": 5.521900544, "lambda": 0.12920820309470096, "lambda_min": 0.12437883764505386, "lambda_max": 0.13831251859664917, "S_mean": 0.302276611328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.04641580581665, "grad_norm": 0.5525895953178406, "tokens_seen": 6070272, "tokens_per_sec": 20773.319348012228, "vram_gb": 5.521900544, "lambda": 0.12944251050551733, "lambda_min": 0.12424033135175705, "lambda_max": 0.13883540034294128, "S_mean": 0.343017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 5.940531253814697, "grad_norm": 0.5287520289421082, "tokens_seen": 6234112, "tokens_per_sec": 20780.0473536597, "vram_gb": 5.521900544, "lambda": 0.12973964431633553, "lambda_min": 0.12406960129737854, "lambda_max": 0.13919761776924133, "S_mean": 0.2986704508463542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.021007537841797, "grad_norm": 0.5903657674789429, "tokens_seen": 6397952, "tokens_per_sec": 20790.515465534896, "vram_gb": 5.521900544, "lambda": 0.13004410670449337, "lambda_min": 0.12400545924901962, "lambda_max": 0.1396215409040451, "S_mean": 0.304473876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.941293239593506, "grad_norm": 0.5116471648216248, "tokens_seen": 6561792, "tokens_per_sec": 20799.781093943475, "vram_gb": 5.521900544, "lambda": 0.13040621299296618, "lambda_min": 0.12393811345100403, "lambda_max": 0.1396331936120987, "S_mean": 0.3509012858072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.8642802238464355, "grad_norm": 0.5448551774024963, "tokens_seen": 6725632, "tokens_per_sec": 20804.583975394526, "vram_gb": 5.521900544, "lambda": 0.1307692745079597, "lambda_min": 0.12389138340950012, "lambda_max": 0.1399000734090805, "S_mean": 0.34918212890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.895479679107666, "grad_norm": 0.5840333700180054, "tokens_seen": 6889472, "tokens_per_sec": 20815.750283930756, "vram_gb": 5.521900544, "lambda": 0.13110253680497408, "lambda_min": 0.12359918653964996, "lambda_max": 0.14071138203144073, "S_mean": 0.32366943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.84473180770874, "grad_norm": 0.5053789019584656, "tokens_seen": 7053312, "tokens_per_sec": 20823.8007706217, "vram_gb": 5.521900544, "lambda": 0.13148421545823416, "lambda_min": 0.12337245047092438, "lambda_max": 0.14214877784252167, "S_mean": 0.3038736979166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.997357606887817, "grad_norm": 0.6274808049201965, "tokens_seen": 7217152, "tokens_per_sec": 20832.76379786233, "vram_gb": 5.521900544, "lambda": 0.13183280701438585, "lambda_min": 0.12309284508228302, "lambda_max": 0.14357933402061462, "S_mean": 0.3261006673177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.784270286560059, "grad_norm": 0.5749346613883972, "tokens_seen": 7380992, "tokens_per_sec": 20839.276899870874, "vram_gb": 5.521900544, "lambda": 0.13218222682674727, "lambda_min": 0.12283246219158173, "lambda_max": 0.1447502076625824, "S_mean": 0.323760986328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.8090431690216064, "grad_norm": 0.5166630744934082, "tokens_seen": 7544832, "tokens_per_sec": 20846.058745286366, "vram_gb": 5.521900544, "lambda": 0.13257871506114802, "lambda_min": 0.122612364590168, "lambda_max": 0.146077960729599, "S_mean": 0.3061116536458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.649023771286011, "grad_norm": 0.5314252376556396, "tokens_seen": 7708672, "tokens_per_sec": 20851.731739459476, "vram_gb": 5.521900544, "lambda": 0.1329437584305803, "lambda_min": 0.12240764498710632, "lambda_max": 0.14729535579681396, "S_mean": 0.3030293782552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.809210538864136, "grad_norm": 0.5222557783126831, "tokens_seen": 7872512, "tokens_per_sec": 20859.007954405257, "vram_gb": 5.521900544, "lambda": 0.13336021018524966, "lambda_min": 0.12225860357284546, "lambda_max": 0.14865221083164215, "S_mean": 0.3108927408854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.739057540893555, "grad_norm": 0.5214248895645142, "tokens_seen": 8036352, "tokens_per_sec": 20866.167657467973, "vram_gb": 5.521900544, "lambda": 0.133741723994414, "lambda_min": 0.1219150498509407, "lambda_max": 0.1501310169696808, "S_mean": 0.2765096028645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.907568693161011, "grad_norm": 0.5759567618370056, "tokens_seen": 8200192, "tokens_per_sec": 20872.965750673575, "vram_gb": 5.521900544, "lambda": 0.13417456454286972, "lambda_min": 0.1216602623462677, "lambda_max": 0.15131016075611115, "S_mean": 0.272979736328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.851807737350464, "val_ppl": 347.86265674583245, "lambda": 0.1341928942129016, "lambda_min": 0.12164147198200226, "lambda_max": 0.15137247741222382, "lambda_per_layer": [0.1442771553993225, 0.13054969906806946, 0.1337905079126358, 0.1334729939699173, 0.1295071691274643, 0.12918992340564728, 0.1257997304201126, 0.12269220501184464, 0.12164147198200226, 0.12265937030315399, 0.13222216069698334, 0.13731437921524048, 0.14419342577457428, 0.15137247741222382, 0.14140965044498444, 0.1480349451303482, 0.14023564755916595, 0.13174961507320404, 0.1408623903989792, 0.12893714010715485, 0.1271377056837082, 0.13731716573238373, 0.1324840784072876, 0.13377845287322998], "S_mean": 0.2833455403645833, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.733345985412598, "grad_norm": 0.5205583572387695, "tokens_seen": 8364032, "tokens_per_sec": 20804.964262105892, "vram_gb": 5.521900544, "lambda": 0.13449585065245628, "lambda_min": 0.12133421748876572, "lambda_max": 0.15269146859645844, "S_mean": 0.285675048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.65240478515625, "grad_norm": 0.548300564289093, "tokens_seen": 8527872, "tokens_per_sec": 20812.104355663392, "vram_gb": 5.521900544, "lambda": 0.13490383668492237, "lambda_min": 0.12115403264760971, "lambda_max": 0.15416203439235687, "S_mean": 0.2901713053385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.654061317443848, "grad_norm": 0.6143186092376709, "tokens_seen": 8691712, "tokens_per_sec": 20816.45200456522, "vram_gb": 5.521900544, "lambda": 0.13530297266940275, "lambda_min": 0.12091616541147232, "lambda_max": 0.15549030900001526, "S_mean": 0.3097635904947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.704384803771973, "grad_norm": 0.52418053150177, "tokens_seen": 8855552, "tokens_per_sec": 20825.48204418371, "vram_gb": 5.521900544, "lambda": 0.1357328516120712, "lambda_min": 0.12082110345363617, "lambda_max": 0.1569274514913559, "S_mean": 0.2852783203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.663500547409058, "grad_norm": 0.5643969178199768, "tokens_seen": 9019392, "tokens_per_sec": 20829.339081062986, "vram_gb": 5.521900544, "lambda": 0.13609287949899831, "lambda_min": 0.12070342153310776, "lambda_max": 0.15831851959228516, "S_mean": 0.2891337076822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.773752927780151, "grad_norm": 0.6327715516090393, "tokens_seen": 9183232, "tokens_per_sec": 20836.833097059047, "vram_gb": 5.521900544, "lambda": 0.13648841002335152, "lambda_min": 0.12047573924064636, "lambda_max": 0.15970438718795776, "S_mean": 0.2758331298828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.660648584365845, "grad_norm": 0.5710611939430237, "tokens_seen": 9347072, "tokens_per_sec": 20838.873144516503, "vram_gb": 5.521900544, "lambda": 0.1368508137141665, "lambda_min": 0.11996310204267502, "lambda_max": 0.1609874963760376, "S_mean": 0.2816975911458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.704771041870117, "grad_norm": 0.5775783658027649, "tokens_seen": 9510912, "tokens_per_sec": 20845.168584466483, "vram_gb": 5.521900544, "lambda": 0.1373081263154745, "lambda_min": 0.11971330642700195, "lambda_max": 0.16243773698806763, "S_mean": 0.2853190104166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.629918336868286, "grad_norm": 0.538812518119812, "tokens_seen": 9674752, "tokens_per_sec": 20849.320390792516, "vram_gb": 5.521900544, "lambda": 0.13778687144319215, "lambda_min": 0.1197805255651474, "lambda_max": 0.16363605856895447, "S_mean": 0.2781168619791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.702510833740234, "grad_norm": 0.6216137409210205, "tokens_seen": 9838592, "tokens_per_sec": 20854.69321130432, "vram_gb": 5.521900544, "lambda": 0.13828484155237675, "lambda_min": 0.11971075087785721, "lambda_max": 0.1649802029132843, "S_mean": 0.2867533365885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.511151075363159, "grad_norm": 0.5459255576133728, "tokens_seen": 10002432, "tokens_per_sec": 20861.235808360492, "vram_gb": 5.521900544, "lambda": 0.13870205885420242, "lambda_min": 0.11921975016593933, "lambda_max": 0.16630078852176666, "S_mean": 0.2830708821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.561056137084961, "grad_norm": 0.5611238479614258, "tokens_seen": 10166272, "tokens_per_sec": 20866.395077658257, "vram_gb": 5.521900544, "lambda": 0.1391298739860455, "lambda_min": 0.11884063482284546, "lambda_max": 0.16773977875709534, "S_mean": 0.2906087239583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.734717130661011, "grad_norm": 0.5579114556312561, "tokens_seen": 10330112, "tokens_per_sec": 20868.278453072326, "vram_gb": 5.521900544, "lambda": 0.13961359423895678, "lambda_min": 0.11891057342290878, "lambda_max": 0.16905201971530914, "S_mean": 0.2762552897135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.568979740142822, "grad_norm": 0.5806013345718384, "tokens_seen": 10493952, "tokens_per_sec": 20870.53095869948, "vram_gb": 5.521900544, "lambda": 0.1400772612541914, "lambda_min": 0.11890701204538345, "lambda_max": 0.17037659883499146, "S_mean": 0.2851460774739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.661125183105469, "grad_norm": 0.5276055335998535, "tokens_seen": 10657792, "tokens_per_sec": 20874.452781572847, "vram_gb": 5.521900544, "lambda": 0.14053657744079828, "lambda_min": 0.11868979781866074, "lambda_max": 0.1718146800994873, "S_mean": 0.283050537109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.5687689781188965, "grad_norm": 0.5832739472389221, "tokens_seen": 10821632, "tokens_per_sec": 20879.23262151738, "vram_gb": 5.521900544, "lambda": 0.14098914340138435, "lambda_min": 0.11854378134012222, "lambda_max": 0.17314043641090393, "S_mean": 0.283447265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.635994911193848, "grad_norm": 0.5793654322624207, "tokens_seen": 10985472, "tokens_per_sec": 20882.14913521591, "vram_gb": 5.521900544, "lambda": 0.14152920028815666, "lambda_min": 0.11836733669042587, "lambda_max": 0.17446544766426086, "S_mean": 0.2949422200520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.478960275650024, "grad_norm": 0.597126305103302, "tokens_seen": 11149312, "tokens_per_sec": 20886.26964988486, "vram_gb": 5.521900544, "lambda": 0.14196714044859013, "lambda_min": 0.11814242601394653, "lambda_max": 0.1756330132484436, "S_mean": 0.2832845052083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.421781778335571, "grad_norm": 0.5461142659187317, "tokens_seen": 11313152, "tokens_per_sec": 20891.17142381489, "vram_gb": 5.521900544, "lambda": 0.14235272475828728, "lambda_min": 0.11792489886283875, "lambda_max": 0.1767663210630417, "S_mean": 0.2861429850260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.623910427093506, "grad_norm": 0.6342341899871826, "tokens_seen": 11476992, "tokens_per_sec": 20895.023152220096, "vram_gb": 5.521900544, "lambda": 0.1428036680445075, "lambda_min": 0.11765088886022568, "lambda_max": 0.17811264097690582, "S_mean": 0.3124287923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.484175443649292, "grad_norm": 0.6180492043495178, "tokens_seen": 11640832, "tokens_per_sec": 20898.194133260786, "vram_gb": 5.521900544, "lambda": 0.14324054711808762, "lambda_min": 0.11726468056440353, "lambda_max": 0.17966288328170776, "S_mean": 0.2597249348958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.374016761779785, "grad_norm": 0.5664172172546387, "tokens_seen": 11804672, "tokens_per_sec": 20902.12164683013, "vram_gb": 5.521900544, "lambda": 0.14371431898325682, "lambda_min": 0.11682926118373871, "lambda_max": 0.1810348480939865, "S_mean": 0.2844034830729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.54964280128479, "grad_norm": 0.6081245541572571, "tokens_seen": 11968512, "tokens_per_sec": 20905.593786576053, "vram_gb": 5.521900544, "lambda": 0.14421408313016096, "lambda_min": 0.11649308353662491, "lambda_max": 0.18258036673069, "S_mean": 0.3003641764322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.399353742599487, "grad_norm": 0.6018102169036865, "tokens_seen": 12132352, "tokens_per_sec": 20908.658027075388, "vram_gb": 5.521900544, "lambda": 0.14469032486279806, "lambda_min": 0.11622810363769531, "lambda_max": 0.18399226665496826, "S_mean": 0.2791239420572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.5121612548828125, "grad_norm": 0.5766735076904297, "tokens_seen": 12296192, "tokens_per_sec": 20912.608777766676, "vram_gb": 5.521900544, "lambda": 0.14517768317212662, "lambda_min": 0.11609788984060287, "lambda_max": 0.18527106940746307, "S_mean": 0.269500732421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.505599737167358, "grad_norm": 0.5664892196655273, "tokens_seen": 12460032, "tokens_per_sec": 20914.334037985984, "vram_gb": 5.521900544, "lambda": 0.14564614246288934, "lambda_min": 0.11567798256874084, "lambda_max": 0.18653427064418793, "S_mean": 0.2778116861979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.364004373550415, "grad_norm": 0.5914851427078247, "tokens_seen": 12623872, "tokens_per_sec": 20915.72471726186, "vram_gb": 5.521900544, "lambda": 0.1461265878751874, "lambda_min": 0.11554351449012756, "lambda_max": 0.1881113499403, "S_mean": 0.2925313313802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.423076629638672, "grad_norm": 0.5632637143135071, "tokens_seen": 12787712, "tokens_per_sec": 20918.457162341605, "vram_gb": 5.521900544, "lambda": 0.14658704648415247, "lambda_min": 0.11532333493232727, "lambda_max": 0.1894414871931076, "S_mean": 0.2932332356770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.4918763637542725, "grad_norm": 0.5722853541374207, "tokens_seen": 12951552, "tokens_per_sec": 20922.921127294045, "vram_gb": 5.521900544, "lambda": 0.14701714956512055, "lambda_min": 0.11485482007265091, "lambda_max": 0.19046635925769806, "S_mean": 0.2950032552083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.273558855056763, "grad_norm": 0.5836999416351318, "tokens_seen": 13115392, "tokens_per_sec": 20926.33225648985, "vram_gb": 5.521900544, "lambda": 0.14750135131180286, "lambda_min": 0.11462987959384918, "lambda_max": 0.19194070994853973, "S_mean": 0.2871602376302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.494199514389038, "grad_norm": 0.5885147452354431, "tokens_seen": 13279232, "tokens_per_sec": 20929.528784042177, "vram_gb": 5.521900544, "lambda": 0.1479597290356954, "lambda_min": 0.11458659172058105, "lambda_max": 0.19340196251869202, "S_mean": 0.2817840576171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.359287261962891, "grad_norm": 0.5941639542579651, "tokens_seen": 13443072, "tokens_per_sec": 20933.943966490096, "vram_gb": 5.521900544, "lambda": 0.14843208498011032, "lambda_min": 0.11450497061014175, "lambda_max": 0.19480745494365692, "S_mean": 0.2838694254557292, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.3763344287872314, "grad_norm": 0.6061047911643982, "tokens_seen": 13606912, "tokens_per_sec": 20937.935269997753, "vram_gb": 5.521900544, "lambda": 0.14888587904473147, "lambda_min": 0.11428148299455643, "lambda_max": 0.195611834526062, "S_mean": 0.2811482747395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.3819544315338135, "grad_norm": 0.5585935711860657, "tokens_seen": 13770752, "tokens_per_sec": 20941.220157782544, "vram_gb": 5.521900544, "lambda": 0.14940758794546127, "lambda_min": 0.1136951744556427, "lambda_max": 0.19710032641887665, "S_mean": 0.249267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.5410027503967285, "grad_norm": 0.5969176888465881, "tokens_seen": 13934592, "tokens_per_sec": 20945.444780674803, "vram_gb": 5.521900544, "lambda": 0.14982634441306195, "lambda_min": 0.11310777068138123, "lambda_max": 0.19846290349960327, "S_mean": 0.27825927734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.397405385971069, "grad_norm": 0.6019227504730225, "tokens_seen": 14098432, "tokens_per_sec": 20946.205218531206, "vram_gb": 5.521900544, "lambda": 0.1503677119811376, "lambda_min": 0.11278344690799713, "lambda_max": 0.19995330274105072, "S_mean": 0.2822062174479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.341301441192627, "grad_norm": 0.6501367688179016, "tokens_seen": 14262272, "tokens_per_sec": 20948.746046640936, "vram_gb": 5.521900544, "lambda": 0.1508328371370832, "lambda_min": 0.11236214637756348, "lambda_max": 0.20143917202949524, "S_mean": 0.2825215657552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.273922920227051, "grad_norm": 0.5841834545135498, "tokens_seen": 14426112, "tokens_per_sec": 20951.293249014012, "vram_gb": 5.521900544, "lambda": 0.1513590533286333, "lambda_min": 0.1123533546924591, "lambda_max": 0.20313513278961182, "S_mean": 0.2673136393229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.4289329051971436, "grad_norm": 0.6240056753158569, "tokens_seen": 14589952, "tokens_per_sec": 20952.543991093713, "vram_gb": 5.521900544, "lambda": 0.15195063843081394, "lambda_min": 0.11209484189748764, "lambda_max": 0.20450842380523682, "S_mean": 0.2571919759114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.341057777404785, "grad_norm": 0.598645031452179, "tokens_seen": 14753792, "tokens_per_sec": 20954.585419713396, "vram_gb": 5.521900544, "lambda": 0.1524457804237803, "lambda_min": 0.11204885691404343, "lambda_max": 0.2058270275592804, "S_mean": 0.279510498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.29297661781311, "grad_norm": 0.6109009981155396, "tokens_seen": 14917632, "tokens_per_sec": 20957.831317089254, "vram_gb": 5.521900544, "lambda": 0.15293488713602224, "lambda_min": 0.11148258298635483, "lambda_max": 0.2070973515510559, "S_mean": 0.2895100911458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.454669952392578, "grad_norm": 0.6335082054138184, "tokens_seen": 15081472, "tokens_per_sec": 20960.226528648862, "vram_gb": 5.521900544, "lambda": 0.15345919442673525, "lambda_min": 0.1115838885307312, "lambda_max": 0.2086401879787445, "S_mean": 0.2757975260416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.413090705871582, "grad_norm": 0.6256300806999207, "tokens_seen": 15245312, "tokens_per_sec": 20962.838503017272, "vram_gb": 5.521900544, "lambda": 0.15391629996399084, "lambda_min": 0.11148729175329208, "lambda_max": 0.2100018709897995, "S_mean": 0.2928568522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.307293891906738, "grad_norm": 0.6011050343513489, "tokens_seen": 15409152, "tokens_per_sec": 20964.516459371185, "vram_gb": 5.521900544, "lambda": 0.15444200237592062, "lambda_min": 0.11135659366846085, "lambda_max": 0.21141572296619415, "S_mean": 0.2863667805989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.218522787094116, "grad_norm": 0.5958831906318665, "tokens_seen": 15572992, "tokens_per_sec": 20967.254777796916, "vram_gb": 5.521900544, "lambda": 0.15496338810771704, "lambda_min": 0.11117827147245407, "lambda_max": 0.21274292469024658, "S_mean": 0.26641845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.306761741638184, "grad_norm": 0.5737004280090332, "tokens_seen": 15736832, "tokens_per_sec": 20966.403500756187, "vram_gb": 5.521900544, "lambda": 0.15550452874352536, "lambda_min": 0.11107036471366882, "lambda_max": 0.21422556042671204, "S_mean": 0.287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.411927223205566, "grad_norm": 0.6004611253738403, "tokens_seen": 15900672, "tokens_per_sec": 20969.286337879632, "vram_gb": 5.521900544, "lambda": 0.15604656096547842, "lambda_min": 0.11085949838161469, "lambda_max": 0.21547725796699524, "S_mean": 0.24723307291666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.3566155433654785, "grad_norm": 0.582560122013092, "tokens_seen": 16064512, "tokens_per_sec": 20970.13436264453, "vram_gb": 5.521900544, "lambda": 0.15654522522042194, "lambda_min": 0.11079084873199463, "lambda_max": 0.21649208664894104, "S_mean": 0.2675679524739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.104630470275879, "grad_norm": 0.559812068939209, "tokens_seen": 16228352, "tokens_per_sec": 20973.034929012763, "vram_gb": 5.521900544, "lambda": 0.1571055787305037, "lambda_min": 0.1106601133942604, "lambda_max": 0.21788965165615082, "S_mean": 0.2617289225260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.347286701202393, "grad_norm": 0.5581214427947998, "tokens_seen": 16392192, "tokens_per_sec": 20974.557980454563, "vram_gb": 5.521900544, "lambda": 0.157640913190941, "lambda_min": 0.11066383868455887, "lambda_max": 0.2192024290561676, "S_mean": 0.2577412923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.364232206344605, "val_ppl": 213.62715012697345, "lambda": 0.1576664817209045, "lambda_min": 0.11066995561122894, "lambda_max": 0.21925519406795502, "lambda_per_layer": [0.15504488348960876, 0.13356609642505646, 0.13640998303890228, 0.13518381118774414, 0.13086983561515808, 0.1292319893836975, 0.12453564256429672, 0.11333116143941879, 0.11066995561122894, 0.11511290818452835, 0.14608128368854523, 0.17610949277877808, 0.2119019329547882, 0.21925519406795502, 0.1884484738111496, 0.1990799903869629, 0.18781819939613342, 0.15823198854923248, 0.19717161357402802, 0.17278006672859192, 0.1487387716770172, 0.17416581511497498, 0.1656484305858612, 0.15460804104804993], "S_mean": 0.2519327799479167, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.396695137023926, "grad_norm": 0.6053665280342102, "tokens_seen": 16556032, "tokens_per_sec": 20938.464361221693, "vram_gb": 5.521900544, "lambda": 0.15812867445250353, "lambda_min": 0.11088363826274872, "lambda_max": 0.2202443778514862, "S_mean": 0.284149169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.404196739196777, "grad_norm": 0.6037024259567261, "tokens_seen": 16719872, "tokens_per_sec": 20941.44633733119, "vram_gb": 5.521900544, "lambda": 0.15867000973472992, "lambda_min": 0.11086257547140121, "lambda_max": 0.22138658165931702, "S_mean": 0.2668253580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.308573246002197, "grad_norm": 0.6349154710769653, "tokens_seen": 16883712, "tokens_per_sec": 20944.002722037545, "vram_gb": 5.521900544, "lambda": 0.1591886511693398, "lambda_min": 0.1106320470571518, "lambda_max": 0.22284086048603058, "S_mean": 0.2764383951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.222055196762085, "grad_norm": 0.578401505947113, "tokens_seen": 17047552, "tokens_per_sec": 20946.35786427038, "vram_gb": 5.521900544, "lambda": 0.15965720762809119, "lambda_min": 0.11030131578445435, "lambda_max": 0.22404822707176208, "S_mean": 0.2845255533854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.2930357456207275, "grad_norm": 0.5825595259666443, "tokens_seen": 17211392, "tokens_per_sec": 20947.76812582175, "vram_gb": 5.521900544, "lambda": 0.16022760265817246, "lambda_min": 0.11003538966178894, "lambda_max": 0.22559644281864166, "S_mean": 0.2664388020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.347137451171875, "grad_norm": 0.615993082523346, "tokens_seen": 17375232, "tokens_per_sec": 20950.860906231912, "vram_gb": 5.521900544, "lambda": 0.16081504120181003, "lambda_min": 0.10982585698366165, "lambda_max": 0.22695878148078918, "S_mean": 0.28472900390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.144278526306152, "grad_norm": 0.5958202481269836, "tokens_seen": 17539072, "tokens_per_sec": 20951.282128855237, "vram_gb": 5.521900544, "lambda": 0.16131479510416588, "lambda_min": 0.10968266427516937, "lambda_max": 0.22816510498523712, "S_mean": 0.2820638020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.124269008636475, "grad_norm": 0.6178785562515259, "tokens_seen": 17702912, "tokens_per_sec": 20952.919791266017, "vram_gb": 5.521900544, "lambda": 0.1618762823442618, "lambda_min": 0.10949885845184326, "lambda_max": 0.22930242121219635, "S_mean": 0.2718404134114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.155910968780518, "grad_norm": 0.6280312538146973, "tokens_seen": 17866752, "tokens_per_sec": 20955.44656116696, "vram_gb": 5.521900544, "lambda": 0.16239843952159086, "lambda_min": 0.10959195345640182, "lambda_max": 0.23038652539253235, "S_mean": 0.2713216145833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.354752540588379, "grad_norm": 0.6213621497154236, "tokens_seen": 18030592, "tokens_per_sec": 20957.962985305585, "vram_gb": 5.521900544, "lambda": 0.16292800102382898, "lambda_min": 0.10960932821035385, "lambda_max": 0.2313471883535385, "S_mean": 0.2795511881510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.282423973083496, "grad_norm": 0.6308499574661255, "tokens_seen": 18194432, "tokens_per_sec": 20960.47812585358, "vram_gb": 5.521900544, "lambda": 0.1634970015535752, "lambda_min": 0.10944529622793198, "lambda_max": 0.23253312706947327, "S_mean": 0.270599365234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.278931140899658, "grad_norm": 0.7005257606506348, "tokens_seen": 18358272, "tokens_per_sec": 20962.569402092213, "vram_gb": 5.521900544, "lambda": 0.1640439551944534, "lambda_min": 0.10935086756944656, "lambda_max": 0.23396268486976624, "S_mean": 0.2731119791666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.1492228507995605, "grad_norm": 0.6585346460342407, "tokens_seen": 18522112, "tokens_per_sec": 20964.122336571276, "vram_gb": 5.521900544, "lambda": 0.16461799448976913, "lambda_min": 0.10903332382440567, "lambda_max": 0.23534950613975525, "S_mean": 0.262054443359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.225472450256348, "grad_norm": 1.04896080493927, "tokens_seen": 18685952, "tokens_per_sec": 20966.71776102339, "vram_gb": 5.521900544, "lambda": 0.1651757045959433, "lambda_min": 0.10900752991437912, "lambda_max": 0.23663438856601715, "S_mean": 0.3025105794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.278556823730469, "grad_norm": 0.6366398334503174, "tokens_seen": 18849792, "tokens_per_sec": 20968.828256878704, "vram_gb": 5.521900544, "lambda": 0.16579562425613403, "lambda_min": 0.10905640572309494, "lambda_max": 0.2379588633775711, "S_mean": 0.2827555338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.2192301750183105, "grad_norm": 0.6779561042785645, "tokens_seen": 19013632, "tokens_per_sec": 20970.94443492655, "vram_gb": 5.521900544, "lambda": 0.16640225580583015, "lambda_min": 0.10916245728731155, "lambda_max": 0.23905058205127716, "S_mean": 0.2530008951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.002171754837036, "grad_norm": 0.6573506593704224, "tokens_seen": 19177472, "tokens_per_sec": 20971.44900698958, "vram_gb": 5.521900544, "lambda": 0.1668911144758264, "lambda_min": 0.10907451808452606, "lambda_max": 0.24016305804252625, "S_mean": 0.2662251790364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 5.222645998001099, "grad_norm": 0.6089110970497131, "tokens_seen": 19341312, "tokens_per_sec": 20972.96653771995, "vram_gb": 5.521900544, "lambda": 0.16745880153030157, "lambda_min": 0.10886089503765106, "lambda_max": 0.24173764884471893, "S_mean": 0.261077880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.147760391235352, "grad_norm": 0.6868835687637329, "tokens_seen": 19505152, "tokens_per_sec": 20974.786798930374, "vram_gb": 5.521900544, "lambda": 0.1680245685080687, "lambda_min": 0.10867395251989365, "lambda_max": 0.24343006312847137, "S_mean": 0.29815673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.117280960083008, "grad_norm": 0.6288378238677979, "tokens_seen": 19668992, "tokens_per_sec": 20976.870090556073, "vram_gb": 5.521900544, "lambda": 0.16856745351105928, "lambda_min": 0.10872254520654678, "lambda_max": 0.24490998685359955, "S_mean": 0.27606201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.1471357345581055, "grad_norm": 0.7192018032073975, "tokens_seen": 19832832, "tokens_per_sec": 20978.91874648856, "vram_gb": 5.521900544, "lambda": 0.16916084848344326, "lambda_min": 0.1086249127984047, "lambda_max": 0.24660581350326538, "S_mean": 0.2815958658854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.160322904586792, "grad_norm": 0.6057460904121399, "tokens_seen": 19996672, "tokens_per_sec": 20980.21837208584, "vram_gb": 5.521900544, "lambda": 0.16971150071670613, "lambda_min": 0.10834511369466782, "lambda_max": 0.24822944402694702, "S_mean": 0.2647501627604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.112886428833008, "grad_norm": 0.616172730922699, "tokens_seen": 20160512, "tokens_per_sec": 20981.085336727803, "vram_gb": 5.521900544, "lambda": 0.1702629942446947, "lambda_min": 0.10808255523443222, "lambda_max": 0.24983316659927368, "S_mean": 0.2585550944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.142064571380615, "grad_norm": 0.6451044678688049, "tokens_seen": 20324352, "tokens_per_sec": 20982.411514411295, "vram_gb": 5.521900544, "lambda": 0.17080736408631006, "lambda_min": 0.1077190563082695, "lambda_max": 0.25103411078453064, "S_mean": 0.27093505859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.133033037185669, "grad_norm": 0.6214342713356018, "tokens_seen": 20488192, "tokens_per_sec": 20984.875716810995, "vram_gb": 5.521900544, "lambda": 0.17129923589527607, "lambda_min": 0.10739344358444214, "lambda_max": 0.25210267305374146, "S_mean": 0.2799072265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.23378849029541, "grad_norm": 0.5903533101081848, "tokens_seen": 20652032, "tokens_per_sec": 20984.470706776556, "vram_gb": 5.521900544, "lambda": 0.17186312874158224, "lambda_min": 0.10710456222295761, "lambda_max": 0.25321587920188904, "S_mean": 0.2617390950520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.093428373336792, "grad_norm": 0.6455091834068298, "tokens_seen": 20815872, "tokens_per_sec": 20986.500578044437, "vram_gb": 5.521900544, "lambda": 0.17238754530747732, "lambda_min": 0.10677044838666916, "lambda_max": 0.25412091612815857, "S_mean": 0.2762451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.206760883331299, "grad_norm": 0.7082830667495728, "tokens_seen": 20979712, "tokens_per_sec": 20986.512227909, "vram_gb": 5.521900544, "lambda": 0.17288817061732212, "lambda_min": 0.10654311627149582, "lambda_max": 0.2552517056465149, "S_mean": 0.2779337565104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.053753137588501, "grad_norm": 0.6480036377906799, "tokens_seen": 21143552, "tokens_per_sec": 20987.081217159986, "vram_gb": 5.521900544, "lambda": 0.17347303529580435, "lambda_min": 0.10634572803974152, "lambda_max": 0.25659260153770447, "S_mean": 0.2658589680989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.113956928253174, "grad_norm": 0.615437388420105, "tokens_seen": 21307392, "tokens_per_sec": 20987.913310501805, "vram_gb": 5.521900544, "lambda": 0.17405547574162483, "lambda_min": 0.1061759740114212, "lambda_max": 0.2579973638057709, "S_mean": 0.26727294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.038482666015625, "grad_norm": 0.6298850178718567, "tokens_seen": 21471232, "tokens_per_sec": 20989.481042813088, "vram_gb": 5.521900544, "lambda": 0.174584886059165, "lambda_min": 0.10606463998556137, "lambda_max": 0.25927650928497314, "S_mean": 0.2526448567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.000420808792114, "grad_norm": 0.7497439384460449, "tokens_seen": 21635072, "tokens_per_sec": 20990.92255292779, "vram_gb": 5.521900544, "lambda": 0.17514473665505648, "lambda_min": 0.1057172641158104, "lambda_max": 0.26034873723983765, "S_mean": 0.2661844889322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.096368312835693, "grad_norm": 0.6397035717964172, "tokens_seen": 21798912, "tokens_per_sec": 20992.12785451632, "vram_gb": 5.521900544, "lambda": 0.17571115183333555, "lambda_min": 0.10539137572050095, "lambda_max": 0.2616519629955292, "S_mean": 0.2794596354166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 5.086507797241211, "grad_norm": 0.6537367105484009, "tokens_seen": 21962752, "tokens_per_sec": 20992.728161501316, "vram_gb": 5.521900544, "lambda": 0.17630939868589243, "lambda_min": 0.10533981770277023, "lambda_max": 0.26277732849121094, "S_mean": 0.274017333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.049978971481323, "grad_norm": 0.6381493210792542, "tokens_seen": 22126592, "tokens_per_sec": 20993.49778526695, "vram_gb": 5.521900544, "lambda": 0.17686768559118113, "lambda_min": 0.10509301722049713, "lambda_max": 0.26429301500320435, "S_mean": 0.26580810546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.156187057495117, "grad_norm": 0.7848343849182129, "tokens_seen": 22290432, "tokens_per_sec": 20994.697258599426, "vram_gb": 5.521900544, "lambda": 0.17745353250453869, "lambda_min": 0.10491789132356644, "lambda_max": 0.2654985189437866, "S_mean": 0.29425048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.113936424255371, "grad_norm": 0.672985851764679, "tokens_seen": 22454272, "tokens_per_sec": 20995.911265433035, "vram_gb": 5.521900544, "lambda": 0.17796417605131865, "lambda_min": 0.10488267987966537, "lambda_max": 0.2664863169193268, "S_mean": 0.2594706217447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.130937576293945, "grad_norm": 0.7052266597747803, "tokens_seen": 22618112, "tokens_per_sec": 20996.824292461562, "vram_gb": 5.521900544, "lambda": 0.1784825442979733, "lambda_min": 0.10492854565382004, "lambda_max": 0.2677050828933716, "S_mean": 0.2726643880208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.062442779541016, "grad_norm": 0.6800606846809387, "tokens_seen": 22781952, "tokens_per_sec": 20998.189655757647, "vram_gb": 5.521900544, "lambda": 0.17888430102417865, "lambda_min": 0.10495451092720032, "lambda_max": 0.26879316568374634, "S_mean": 0.2621866861979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.0143914222717285, "grad_norm": 0.7050468921661377, "tokens_seen": 22945792, "tokens_per_sec": 20999.461599134574, "vram_gb": 5.521900544, "lambda": 0.17934417507300773, "lambda_min": 0.1047237366437912, "lambda_max": 0.26986631751060486, "S_mean": 0.263275146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 5.163415193557739, "grad_norm": 0.7595208287239075, "tokens_seen": 23109632, "tokens_per_sec": 21001.920809909083, "vram_gb": 5.521900544, "lambda": 0.1799729916577538, "lambda_min": 0.1046978309750557, "lambda_max": 0.27107906341552734, "S_mean": 0.2621561686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 5.120229244232178, "grad_norm": 0.6727816462516785, "tokens_seen": 23273472, "tokens_per_sec": 21003.020941487302, "vram_gb": 5.521900544, "lambda": 0.18065636170407137, "lambda_min": 0.1046651303768158, "lambda_max": 0.27296367287635803, "S_mean": 0.2703653971354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 4.952731370925903, "grad_norm": 0.7214335799217224, "tokens_seen": 23437312, "tokens_per_sec": 21004.423838385923, "vram_gb": 5.521900544, "lambda": 0.18117472529411316, "lambda_min": 0.10454563796520233, "lambda_max": 0.2740393579006195, "S_mean": 0.2901407877604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.0321128368377686, "grad_norm": 0.6849149465560913, "tokens_seen": 23601152, "tokens_per_sec": 21005.860573745886, "vram_gb": 5.521900544, "lambda": 0.18174382330228886, "lambda_min": 0.10444716364145279, "lambda_max": 0.27542826533317566, "S_mean": 0.24269612630208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 4.890350103378296, "grad_norm": 0.6637507081031799, "tokens_seen": 23764992, "tokens_per_sec": 21007.037768241746, "vram_gb": 5.521900544, "lambda": 0.1822758965815107, "lambda_min": 0.10437972098588943, "lambda_max": 0.27674514055252075, "S_mean": 0.258026123046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 4.988467693328857, "grad_norm": 0.6722322106361389, "tokens_seen": 23928832, "tokens_per_sec": 21008.227378818337, "vram_gb": 5.521900544, "lambda": 0.1827599204455813, "lambda_min": 0.10414287447929382, "lambda_max": 0.2778562605381012, "S_mean": 0.2697855631510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 5.027454853057861, "grad_norm": 0.7287182807922363, "tokens_seen": 24092672, "tokens_per_sec": 21010.365400720828, "vram_gb": 5.521900544, "lambda": 0.18328631669282913, "lambda_min": 0.10394164174795151, "lambda_max": 0.2789146602153778, "S_mean": 0.2672119140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 4.879022121429443, "grad_norm": 0.6929059028625488, "tokens_seen": 24256512, "tokens_per_sec": 21010.96636450462, "vram_gb": 5.521900544, "lambda": 0.18391878250986338, "lambda_min": 0.10373827069997787, "lambda_max": 0.280369371175766, "S_mean": 0.27545166015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 4.784458637237549, "grad_norm": 0.8859987258911133, "tokens_seen": 24420352, "tokens_per_sec": 21011.783943248985, "vram_gb": 5.521900544, "lambda": 0.18440715440859398, "lambda_min": 0.10357777774333954, "lambda_max": 0.28121641278266907, "S_mean": 0.2847696940104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 5.070035457611084, "grad_norm": 0.742292046546936, "tokens_seen": 24584192, "tokens_per_sec": 21013.010392358145, "vram_gb": 5.521900544, "lambda": 0.18498189436892667, "lambda_min": 0.1033317819237709, "lambda_max": 0.28262221813201904, "S_mean": 0.26519775390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.043226671218872, "val_ppl": 154.96924396824403, "lambda": 0.18500845972448587, "lambda_min": 0.1033305674791336, "lambda_max": 0.2826831638813019, "lambda_per_layer": [0.158490389585495, 0.1355859935283661, 0.1391768753528595, 0.13649076223373413, 0.13295519351959229, 0.1299135982990265, 0.1243165135383606, 0.10357396304607391, 0.1033305674791336, 0.10543862730264664, 0.1559707522392273, 0.21076735854148865, 0.2765936553478241, 0.2826831638813019, 0.23982809484004974, 0.24762575328350067, 0.25354793667793274, 0.1945163905620575, 0.2616857588291168, 0.2517688274383545, 0.18346290290355682, 0.22626930475234985, 0.2093755155801773, 0.17683513462543488], "S_mean": 0.267425537109375, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 5.1425769329071045, "grad_norm": 0.7511439919471741, "tokens_seen": 24748032, "tokens_per_sec": 20989.760352109704, "vram_gb": 5.521900544, "lambda": 0.1855556114266316, "lambda_min": 0.1033334732055664, "lambda_max": 0.2840513586997986, "S_mean": 0.2872212727864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.8465659618377686, "grad_norm": 0.6731403470039368, "tokens_seen": 24911872, "tokens_per_sec": 20989.849818400584, "vram_gb": 5.521900544, "lambda": 0.18607309367507696, "lambda_min": 0.10320452600717545, "lambda_max": 0.28512799739837646, "S_mean": 0.2718607584635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 4.937549114227295, "grad_norm": 0.6930672526359558, "tokens_seen": 25075712, "tokens_per_sec": 20990.305032029835, "vram_gb": 5.521900544, "lambda": 0.18656674989809593, "lambda_min": 0.10323455184698105, "lambda_max": 0.2856719493865967, "S_mean": 0.2773030598958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 4.880101203918457, "grad_norm": 0.7069908976554871, "tokens_seen": 25239552, "tokens_per_sec": 20991.604011245, "vram_gb": 5.521900544, "lambda": 0.18712459163119396, "lambda_min": 0.10318117588758469, "lambda_max": 0.2863365709781647, "S_mean": 0.2804972330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.0333521366119385, "grad_norm": 0.7893011569976807, "tokens_seen": 25403392, "tokens_per_sec": 20992.903164492513, "vram_gb": 5.521900544, "lambda": 0.18768051204582056, "lambda_min": 0.10286848992109299, "lambda_max": 0.2875852584838867, "S_mean": 0.2678934733072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.7989866733551025, "grad_norm": 0.7388856410980225, "tokens_seen": 25567232, "tokens_per_sec": 20993.938365841754, "vram_gb": 5.521900544, "lambda": 0.18817588003973165, "lambda_min": 0.10276787728071213, "lambda_max": 0.2891986668109894, "S_mean": 0.2850138346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.927104711532593, "grad_norm": 0.6775622367858887, "tokens_seen": 25731072, "tokens_per_sec": 20995.277753619208, "vram_gb": 5.521900544, "lambda": 0.1887569141884645, "lambda_min": 0.10257004201412201, "lambda_max": 0.29026681184768677, "S_mean": 0.273223876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 4.871013164520264, "grad_norm": 0.7469791173934937, "tokens_seen": 25894912, "tokens_per_sec": 20996.901963549666, "vram_gb": 5.521900544, "lambda": 0.18937193602323532, "lambda_min": 0.1025637686252594, "lambda_max": 0.29117655754089355, "S_mean": 0.2716878255208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.901205778121948, "grad_norm": 0.7187898755073547, "tokens_seen": 26058752, "tokens_per_sec": 20997.933514795473, "vram_gb": 5.521900544, "lambda": 0.1899965868021051, "lambda_min": 0.10237909108400345, "lambda_max": 0.292312353849411, "S_mean": 0.2724202473958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.879162073135376, "grad_norm": 0.7106219530105591, "tokens_seen": 26222592, "tokens_per_sec": 20999.1823386578, "vram_gb": 5.521900544, "lambda": 0.19056051845351854, "lambda_min": 0.10213244706392288, "lambda_max": 0.293355792760849, "S_mean": 0.27947998046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 4.9825334548950195, "grad_norm": 0.8306810855865479, "tokens_seen": 26386432, "tokens_per_sec": 21000.129896537863, "vram_gb": 5.521900544, "lambda": 0.19111564382910728, "lambda_min": 0.10176488012075424, "lambda_max": 0.294455885887146, "S_mean": 0.27362060546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.954991102218628, "grad_norm": 0.6906482577323914, "tokens_seen": 26550272, "tokens_per_sec": 21001.257262058036, "vram_gb": 5.521900544, "lambda": 0.1916476171463728, "lambda_min": 0.10184193402528763, "lambda_max": 0.2950613498687744, "S_mean": 0.2681783040364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 5.024619817733765, "grad_norm": 0.7241029143333435, "tokens_seen": 26714112, "tokens_per_sec": 21001.344362542797, "vram_gb": 5.521900544, "lambda": 0.19216732742885748, "lambda_min": 0.10168042778968811, "lambda_max": 0.29553383588790894, "S_mean": 0.2770182291666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.835100889205933, "grad_norm": 0.6781482100486755, "tokens_seen": 26877952, "tokens_per_sec": 21002.18919346072, "vram_gb": 5.521900544, "lambda": 0.19270867357651392, "lambda_min": 0.10168737173080444, "lambda_max": 0.29680323600769043, "S_mean": 0.2646382649739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 5.092538356781006, "grad_norm": 0.6989898681640625, "tokens_seen": 27041792, "tokens_per_sec": 21003.576106052096, "vram_gb": 5.521900544, "lambda": 0.193344515748322, "lambda_min": 0.10160038620233536, "lambda_max": 0.29800158739089966, "S_mean": 0.2610270182291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.930326461791992, "grad_norm": 0.6932319402694702, "tokens_seen": 27205632, "tokens_per_sec": 21004.086660791214, "vram_gb": 5.521900544, "lambda": 0.19388321538766226, "lambda_min": 0.1014213040471077, "lambda_max": 0.2985977828502655, "S_mean": 0.2649434407552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.897740602493286, "grad_norm": 0.6770643591880798, "tokens_seen": 27369472, "tokens_per_sec": 21005.496712536955, "vram_gb": 5.521900544, "lambda": 0.19441270641982555, "lambda_min": 0.10134487599134445, "lambda_max": 0.29987040162086487, "S_mean": 0.2763163248697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.948593854904175, "grad_norm": 0.9038335680961609, "tokens_seen": 27533312, "tokens_per_sec": 21006.99087073119, "vram_gb": 5.521900544, "lambda": 0.19506384525448084, "lambda_min": 0.10127782821655273, "lambda_max": 0.30114495754241943, "S_mean": 0.2622273763020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 5.075263738632202, "grad_norm": 0.7480315566062927, "tokens_seen": 27697152, "tokens_per_sec": 21006.897428747583, "vram_gb": 5.521900544, "lambda": 0.19559775634358326, "lambda_min": 0.10125848650932312, "lambda_max": 0.30215218663215637, "S_mean": 0.2656758626302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 5.007988452911377, "grad_norm": 0.8585464358329773, "tokens_seen": 27860992, "tokens_per_sec": 21006.071015834837, "vram_gb": 5.521900544, "lambda": 0.19604836435367665, "lambda_min": 0.10101383924484253, "lambda_max": 0.30316808819770813, "S_mean": 0.2584330240885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.925112009048462, "grad_norm": 0.7231469750404358, "tokens_seen": 28024832, "tokens_per_sec": 21006.484346593945, "vram_gb": 5.521900544, "lambda": 0.19656820936749378, "lambda_min": 0.10096877813339233, "lambda_max": 0.30421528220176697, "S_mean": 0.2609151204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.73972225189209, "grad_norm": 0.6841897368431091, "tokens_seen": 28188672, "tokens_per_sec": 21007.713654771956, "vram_gb": 5.521900544, "lambda": 0.19712105486541986, "lambda_min": 0.10073485970497131, "lambda_max": 0.3053162693977356, "S_mean": 0.2679443359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.955582141876221, "grad_norm": 0.7368140816688538, "tokens_seen": 28352512, "tokens_per_sec": 21008.69724982266, "vram_gb": 5.521900544, "lambda": 0.19768475896368423, "lambda_min": 0.10072765499353409, "lambda_max": 0.3062001168727875, "S_mean": 0.24805704752604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 4.966207504272461, "grad_norm": 0.7454609870910645, "tokens_seen": 28516352, "tokens_per_sec": 21010.02561671941, "vram_gb": 5.521900544, "lambda": 0.19824143840620914, "lambda_min": 0.10067722946405411, "lambda_max": 0.30718326568603516, "S_mean": 0.2649027506510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 5.084865093231201, "grad_norm": 0.7390366792678833, "tokens_seen": 28680192, "tokens_per_sec": 21011.33819370354, "vram_gb": 5.521900544, "lambda": 0.19879716355353594, "lambda_min": 0.10059169679880142, "lambda_max": 0.3083210289478302, "S_mean": 0.260345458984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.843907356262207, "grad_norm": 0.775422215461731, "tokens_seen": 28844032, "tokens_per_sec": 21013.085956409217, "vram_gb": 5.521900544, "lambda": 0.1993343730767568, "lambda_min": 0.10059849917888641, "lambda_max": 0.3091542422771454, "S_mean": 0.2611592610677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 4.863922595977783, "grad_norm": 0.7440983057022095, "tokens_seen": 29007872, "tokens_per_sec": 21014.242060037457, "vram_gb": 5.521900544, "lambda": 0.19994602942218384, "lambda_min": 0.10048889368772507, "lambda_max": 0.31007060408592224, "S_mean": 0.2664896647135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.912541151046753, "grad_norm": 0.7620807886123657, "tokens_seen": 29171712, "tokens_per_sec": 21015.91738145049, "vram_gb": 5.521900544, "lambda": 0.2004567307109634, "lambda_min": 0.10015667229890823, "lambda_max": 0.3115242123603821, "S_mean": 0.270843505859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 4.926395654678345, "grad_norm": 0.7135230302810669, "tokens_seen": 29335552, "tokens_per_sec": 21017.286840440425, "vram_gb": 5.521900544, "lambda": 0.2010104317838947, "lambda_min": 0.09998242557048798, "lambda_max": 0.3128066062927246, "S_mean": 0.2579142252604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.922255039215088, "grad_norm": 0.7479787468910217, "tokens_seen": 29499392, "tokens_per_sec": 21018.416005909385, "vram_gb": 5.521900544, "lambda": 0.20156620411823192, "lambda_min": 0.09980155527591705, "lambda_max": 0.31368231773376465, "S_mean": 0.2585347493489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.906698703765869, "grad_norm": 0.7354891300201416, "tokens_seen": 29663232, "tokens_per_sec": 21019.833507135587, "vram_gb": 5.521900544, "lambda": 0.20216343582918248, "lambda_min": 0.09972264617681503, "lambda_max": 0.3147609233856201, "S_mean": 0.2625528971354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.807898283004761, "grad_norm": 0.6925397515296936, "tokens_seen": 29827072, "tokens_per_sec": 21020.67412407426, "vram_gb": 5.521900544, "lambda": 0.20265825962026915, "lambda_min": 0.09950452297925949, "lambda_max": 0.3155074119567871, "S_mean": 0.2620340983072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 5.037571430206299, "grad_norm": 0.779010534286499, "tokens_seen": 29990912, "tokens_per_sec": 21015.04978925201, "vram_gb": 5.521900544, "lambda": 0.20320954297979674, "lambda_min": 0.09914329648017883, "lambda_max": 0.31675100326538086, "S_mean": 0.2623392740885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.858129501342773, "grad_norm": 0.7564388513565063, "tokens_seen": 30154752, "tokens_per_sec": 21016.00864617302, "vram_gb": 5.521900544, "lambda": 0.20378255347410837, "lambda_min": 0.0989656075835228, "lambda_max": 0.31788599491119385, "S_mean": 0.2723795572916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.998244285583496, "grad_norm": 0.7034912109375, "tokens_seen": 30318592, "tokens_per_sec": 21016.839675502666, "vram_gb": 5.521900544, "lambda": 0.2043424757818381, "lambda_min": 0.09902243316173553, "lambda_max": 0.3186141550540924, "S_mean": 0.2616678873697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.97203516960144, "grad_norm": 0.8398833870887756, "tokens_seen": 30482432, "tokens_per_sec": 21016.99324230328, "vram_gb": 5.521900544, "lambda": 0.2048528160278996, "lambda_min": 0.0988251343369484, "lambda_max": 0.3195910155773163, "S_mean": 0.287841796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.88805079460144, "grad_norm": 0.8421798944473267, "tokens_seen": 30646272, "tokens_per_sec": 21018.532390357297, "vram_gb": 5.521900544, "lambda": 0.2053167400881648, "lambda_min": 0.09869100153446198, "lambda_max": 0.31985077261924744, "S_mean": 0.270782470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.816965103149414, "grad_norm": 0.7746244668960571, "tokens_seen": 30810112, "tokens_per_sec": 21019.003191659627, "vram_gb": 5.521900544, "lambda": 0.2059400804961721, "lambda_min": 0.0985988974571228, "lambda_max": 0.32139527797698975, "S_mean": 0.2729899088541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.76816201210022, "grad_norm": 0.7697417140007019, "tokens_seen": 30973952, "tokens_per_sec": 21020.395892818735, "vram_gb": 5.521900544, "lambda": 0.20650208772470555, "lambda_min": 0.0986367017030716, "lambda_max": 0.32279855012893677, "S_mean": 0.2740275065104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.912390470504761, "grad_norm": 0.7613922357559204, "tokens_seen": 31137792, "tokens_per_sec": 21021.415450718327, "vram_gb": 5.521900544, "lambda": 0.20702394905189672, "lambda_min": 0.09842292219400406, "lambda_max": 0.32451221346855164, "S_mean": 0.2765401204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 5.024991035461426, "grad_norm": 0.7973601222038269, "tokens_seen": 31301632, "tokens_per_sec": 21022.50489513778, "vram_gb": 5.521900544, "lambda": 0.20754016314943632, "lambda_min": 0.0983695238828659, "lambda_max": 0.32601356506347656, "S_mean": 0.264556884765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.918344736099243, "grad_norm": 0.7820979952812195, "tokens_seen": 31465472, "tokens_per_sec": 21023.52326496608, "vram_gb": 5.521900544, "lambda": 0.20805131799230972, "lambda_min": 0.0981973335146904, "lambda_max": 0.3269815742969513, "S_mean": 0.2744038899739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.874493837356567, "grad_norm": 0.7566414475440979, "tokens_seen": 31629312, "tokens_per_sec": 21024.144198711434, "vram_gb": 5.521900544, "lambda": 0.20862161523352066, "lambda_min": 0.09814134240150452, "lambda_max": 0.3279365003108978, "S_mean": 0.27142333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.925187349319458, "grad_norm": 0.7536030411720276, "tokens_seen": 31793152, "tokens_per_sec": 21025.087212843504, "vram_gb": 5.521900544, "lambda": 0.20912469395746788, "lambda_min": 0.09826193749904633, "lambda_max": 0.32920199632644653, "S_mean": 0.2874247233072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.918302536010742, "grad_norm": 0.7959495782852173, "tokens_seen": 31956992, "tokens_per_sec": 21024.917560392496, "vram_gb": 5.521900544, "lambda": 0.20965508837252855, "lambda_min": 0.09839122742414474, "lambda_max": 0.33023443818092346, "S_mean": 0.2654012044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.881539344787598, "grad_norm": 0.6870635747909546, "tokens_seen": 32120832, "tokens_per_sec": 21026.218565687213, "vram_gb": 5.521900544, "lambda": 0.21024226179967323, "lambda_min": 0.09815286099910736, "lambda_max": 0.3313073515892029, "S_mean": 0.2737528483072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.755919694900513, "grad_norm": 0.8057840466499329, "tokens_seen": 32284672, "tokens_per_sec": 21027.206267735706, "vram_gb": 5.521900544, "lambda": 0.21078395719329515, "lambda_min": 0.09812484681606293, "lambda_max": 0.33329296112060547, "S_mean": 0.26702880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.7669055461883545, "grad_norm": 0.8184844255447388, "tokens_seen": 32448512, "tokens_per_sec": 21028.455875233874, "vram_gb": 5.521900544, "lambda": 0.21136238146573305, "lambda_min": 0.09796775132417679, "lambda_max": 0.3349038064479828, "S_mean": 0.2629191080729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.831359148025513, "grad_norm": 0.7655795216560364, "tokens_seen": 32612352, "tokens_per_sec": 21029.91675044521, "vram_gb": 5.521900544, "lambda": 0.2118435949087143, "lambda_min": 0.09789153188467026, "lambda_max": 0.3364841938018799, "S_mean": 0.268829345703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.773610591888428, "grad_norm": 0.7557048201560974, "tokens_seen": 32776192, "tokens_per_sec": 21030.862817073375, "vram_gb": 5.521900544, "lambda": 0.21241079146663347, "lambda_min": 0.09789398312568665, "lambda_max": 0.33855608105659485, "S_mean": 0.2842508951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.896922183036804, "val_ppl": 133.8770957316514, "lambda": 0.21244950778782368, "lambda_min": 0.09790080785751343, "lambda_max": 0.3386763632297516, "lambda_per_layer": [0.16091659665107727, 0.13442422449588776, 0.14114908874034882, 0.1380763202905655, 0.13814043998718262, 0.1316453069448471, 0.1248784139752388, 0.10145863890647888, 0.09790080785751343, 0.09991089254617691, 0.16393408179283142, 0.23374126851558685, 0.3303314447402954, 0.3348487615585327, 0.287630558013916, 0.29373955726623535, 0.31892409920692444, 0.23120947182178497, 0.3321573734283447, 0.3386763632297516, 0.22399720549583435, 0.287789911031723, 0.2534710168838501, 0.1998363435268402], "S_mean": 0.279296875, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.740594863891602, "grad_norm": 0.802487850189209, "tokens_seen": 32940032, "tokens_per_sec": 21013.400700356186, "vram_gb": 5.521900544, "lambda": 0.21301633026450872, "lambda_min": 0.0977453663945198, "lambda_max": 0.3404097259044647, "S_mean": 0.279296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 5.09944486618042, "grad_norm": 0.8261299729347229, "tokens_seen": 33103872, "tokens_per_sec": 21013.207399689763, "vram_gb": 5.521900544, "lambda": 0.21350487073262533, "lambda_min": 0.09758588671684265, "lambda_max": 0.34230196475982666, "S_mean": 0.2660013834635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.754518985748291, "grad_norm": 0.7675819396972656, "tokens_seen": 33267712, "tokens_per_sec": 21014.48832698172, "vram_gb": 5.521900544, "lambda": 0.2140956592435638, "lambda_min": 0.09754815697669983, "lambda_max": 0.34414294362068176, "S_mean": 0.2598470052083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.911961555480957, "grad_norm": 0.8317925333976746, "tokens_seen": 33431552, "tokens_per_sec": 21014.73299752293, "vram_gb": 5.521900544, "lambda": 0.2146022000039617, "lambda_min": 0.09742237627506256, "lambda_max": 0.3456374406814575, "S_mean": 0.295013427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.829641819000244, "grad_norm": 0.7514477372169495, "tokens_seen": 33595392, "tokens_per_sec": 21015.50408607396, "vram_gb": 5.521900544, "lambda": 0.21505094785243273, "lambda_min": 0.0975145548582077, "lambda_max": 0.347146600484848, "S_mean": 0.2725118001302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.800987243652344, "grad_norm": 0.7681284546852112, "tokens_seen": 33759232, "tokens_per_sec": 21016.21281498361, "vram_gb": 5.521900544, "lambda": 0.2155224603290359, "lambda_min": 0.09737806767225266, "lambda_max": 0.34853395819664, "S_mean": 0.2758382161458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.834137201309204, "grad_norm": 0.7273730635643005, "tokens_seen": 33923072, "tokens_per_sec": 21017.31292456283, "vram_gb": 5.521900544, "lambda": 0.2160450452938676, "lambda_min": 0.09736976772546768, "lambda_max": 0.3497462868690491, "S_mean": 0.2733662923177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.7734315395355225, "grad_norm": 0.786567211151123, "tokens_seen": 34086912, "tokens_per_sec": 21018.480247751075, "vram_gb": 5.521900544, "lambda": 0.21655108965933323, "lambda_min": 0.09743757545948029, "lambda_max": 0.35133206844329834, "S_mean": 0.27703857421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.7291882038116455, "grad_norm": 0.7086758017539978, "tokens_seen": 34250752, "tokens_per_sec": 21018.474458855108, "vram_gb": 5.521900544, "lambda": 0.21710350923240185, "lambda_min": 0.0972919687628746, "lambda_max": 0.3529735803604126, "S_mean": 0.2819315592447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.838404893875122, "grad_norm": 0.8304197192192078, "tokens_seen": 34414592, "tokens_per_sec": 21019.33029019035, "vram_gb": 5.521900544, "lambda": 0.21774428865561882, "lambda_min": 0.09718997031450272, "lambda_max": 0.3544078469276428, "S_mean": 0.2994181315104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.68624472618103, "grad_norm": 0.7486091256141663, "tokens_seen": 34578432, "tokens_per_sec": 21019.357957755463, "vram_gb": 5.521900544, "lambda": 0.21827208964775005, "lambda_min": 0.09718892723321915, "lambda_max": 0.3561747968196869, "S_mean": 0.2952168782552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.856031179428101, "grad_norm": 0.7709900140762329, "tokens_seen": 34742272, "tokens_per_sec": 21020.052935934895, "vram_gb": 5.521900544, "lambda": 0.2188006949921449, "lambda_min": 0.09701884537935257, "lambda_max": 0.35788241028785706, "S_mean": 0.3052266438802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.896902561187744, "grad_norm": 0.7933716177940369, "tokens_seen": 34906112, "tokens_per_sec": 21020.90390433321, "vram_gb": 5.521900544, "lambda": 0.2194276244069139, "lambda_min": 0.09694035351276398, "lambda_max": 0.3597451150417328, "S_mean": 0.2687276204427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.804476737976074, "grad_norm": 0.7860704064369202, "tokens_seen": 35069952, "tokens_per_sec": 21021.466830788864, "vram_gb": 5.521900544, "lambda": 0.21989780416091284, "lambda_min": 0.09662730246782303, "lambda_max": 0.3610741198062897, "S_mean": 0.2629191080729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.726681232452393, "grad_norm": 0.7433906197547913, "tokens_seen": 35233792, "tokens_per_sec": 21022.604987063824, "vram_gb": 5.521900544, "lambda": 0.22049069652954736, "lambda_min": 0.0964163988828659, "lambda_max": 0.3627825379371643, "S_mean": 0.2675984700520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.766697645187378, "grad_norm": 0.8839358687400818, "tokens_seen": 35397632, "tokens_per_sec": 21023.27140698975, "vram_gb": 5.521900544, "lambda": 0.22094896590958038, "lambda_min": 0.09618555754423141, "lambda_max": 0.36479872465133667, "S_mean": 0.28387451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.882661581039429, "grad_norm": 0.8162819147109985, "tokens_seen": 35561472, "tokens_per_sec": 21023.67814510078, "vram_gb": 5.521900544, "lambda": 0.22141566158582768, "lambda_min": 0.09597882628440857, "lambda_max": 0.3662351369857788, "S_mean": 0.28582763671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.899986982345581, "grad_norm": 0.7706770300865173, "tokens_seen": 35725312, "tokens_per_sec": 21024.213418951178, "vram_gb": 5.521900544, "lambda": 0.22185522566239038, "lambda_min": 0.09582631289958954, "lambda_max": 0.3677804172039032, "S_mean": 0.2769775390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.7773613929748535, "grad_norm": 0.7612950801849365, "tokens_seen": 35889152, "tokens_per_sec": 21024.15873109726, "vram_gb": 5.521900544, "lambda": 0.22237294539809227, "lambda_min": 0.09574347734451294, "lambda_max": 0.3694215416908264, "S_mean": 0.2722676595052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.692696571350098, "grad_norm": 0.8226363658905029, "tokens_seen": 36052992, "tokens_per_sec": 21025.15624510927, "vram_gb": 5.521900544, "lambda": 0.22294518227378526, "lambda_min": 0.09578999876976013, "lambda_max": 0.3709404766559601, "S_mean": 0.2557881673177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.99003529548645, "grad_norm": 0.7798570990562439, "tokens_seen": 36216832, "tokens_per_sec": 21025.727810825407, "vram_gb": 5.521900544, "lambda": 0.22349098231643438, "lambda_min": 0.09593973308801651, "lambda_max": 0.3725341558456421, "S_mean": 0.2709452311197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.918449640274048, "grad_norm": 0.8807084560394287, "tokens_seen": 36380672, "tokens_per_sec": 21026.319825006434, "vram_gb": 5.521900544, "lambda": 0.22402157603452602, "lambda_min": 0.09586876630783081, "lambda_max": 0.37437984347343445, "S_mean": 0.2890421549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.801571369171143, "grad_norm": 0.7954322695732117, "tokens_seen": 36544512, "tokens_per_sec": 21027.139058893037, "vram_gb": 5.521900544, "lambda": 0.22449471708387136, "lambda_min": 0.09574607014656067, "lambda_max": 0.3760893642902374, "S_mean": 0.2629241943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.97577691078186, "grad_norm": 0.7723257541656494, "tokens_seen": 36708352, "tokens_per_sec": 21027.514244704005, "vram_gb": 5.521900544, "lambda": 0.22501828521490097, "lambda_min": 0.09590078145265579, "lambda_max": 0.3778124153614044, "S_mean": 0.270111083984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.912002086639404, "grad_norm": 0.8692653179168701, "tokens_seen": 36872192, "tokens_per_sec": 21028.25002456318, "vram_gb": 5.521900544, "lambda": 0.22554088259736696, "lambda_min": 0.09603232145309448, "lambda_max": 0.3797376751899719, "S_mean": 0.274749755859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.759528636932373, "grad_norm": 0.8289530873298645, "tokens_seen": 37036032, "tokens_per_sec": 21028.861680556543, "vram_gb": 5.521900544, "lambda": 0.2259930499518911, "lambda_min": 0.0959472730755806, "lambda_max": 0.3814705014228821, "S_mean": 0.266357421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.793140888214111, "grad_norm": 0.7725118398666382, "tokens_seen": 37199872, "tokens_per_sec": 21029.86504565698, "vram_gb": 5.521900544, "lambda": 0.22653298390408358, "lambda_min": 0.09595953673124313, "lambda_max": 0.3831512928009033, "S_mean": 0.2803243001302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.951211214065552, "grad_norm": 0.824103832244873, "tokens_seen": 37363712, "tokens_per_sec": 21029.80570057114, "vram_gb": 5.521900544, "lambda": 0.22706285491585732, "lambda_min": 0.09590714424848557, "lambda_max": 0.38465389609336853, "S_mean": 0.273590087890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.8109800815582275, "grad_norm": 0.8169675469398499, "tokens_seen": 37527552, "tokens_per_sec": 21030.205087749102, "vram_gb": 5.521900544, "lambda": 0.22760037519037724, "lambda_min": 0.09587863832712173, "lambda_max": 0.3865208923816681, "S_mean": 0.2698974609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.799084663391113, "grad_norm": 0.7781444787979126, "tokens_seen": 37691392, "tokens_per_sec": 21030.711631006234, "vram_gb": 5.521900544, "lambda": 0.22808931364367405, "lambda_min": 0.09570669382810593, "lambda_max": 0.3877638280391693, "S_mean": 0.2741495768229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.768272399902344, "grad_norm": 0.8571690917015076, "tokens_seen": 37855232, "tokens_per_sec": 21031.727090316337, "vram_gb": 5.521900544, "lambda": 0.2284925344089667, "lambda_min": 0.09545541554689407, "lambda_max": 0.3892964720726013, "S_mean": 0.2693888346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.736549615859985, "grad_norm": 0.7487624883651733, "tokens_seen": 38019072, "tokens_per_sec": 21032.030448768033, "vram_gb": 5.521900544, "lambda": 0.22888869885355234, "lambda_min": 0.09556873887777328, "lambda_max": 0.390502005815506, "S_mean": 0.2573343912760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.966325998306274, "grad_norm": 0.8727924227714539, "tokens_seen": 38182912, "tokens_per_sec": 21032.66965480111, "vram_gb": 5.521900544, "lambda": 0.22943979346503815, "lambda_min": 0.0955880805850029, "lambda_max": 0.39191538095474243, "S_mean": 0.265838623046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.627027273178101, "grad_norm": 0.8072529435157776, "tokens_seen": 38346752, "tokens_per_sec": 21033.79508421514, "vram_gb": 5.521900544, "lambda": 0.23000638404240212, "lambda_min": 0.09538833051919937, "lambda_max": 0.39367297291755676, "S_mean": 0.2643534342447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.636422157287598, "grad_norm": 0.8183226585388184, "tokens_seen": 38510592, "tokens_per_sec": 21033.63511372535, "vram_gb": 5.521900544, "lambda": 0.2304636680831512, "lambda_min": 0.09521970897912979, "lambda_max": 0.39483097195625305, "S_mean": 0.2534891764322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.8948891162872314, "grad_norm": 0.8695769309997559, "tokens_seen": 38674432, "tokens_per_sec": 21034.481238744764, "vram_gb": 5.521900544, "lambda": 0.23095131572335958, "lambda_min": 0.09511806070804596, "lambda_max": 0.39613595604896545, "S_mean": 0.260498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.6820783615112305, "grad_norm": 1.0483264923095703, "tokens_seen": 38838272, "tokens_per_sec": 21035.181617612914, "vram_gb": 5.521900544, "lambda": 0.23144968940565983, "lambda_min": 0.0950552374124527, "lambda_max": 0.3977392315864563, "S_mean": 0.2626851399739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.817355155944824, "grad_norm": 0.7843008637428284, "tokens_seen": 39002112, "tokens_per_sec": 21036.171000562354, "vram_gb": 5.521900544, "lambda": 0.23195957640806833, "lambda_min": 0.09503918886184692, "lambda_max": 0.39943844079971313, "S_mean": 0.2552490234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.754519939422607, "grad_norm": 0.8422973155975342, "tokens_seen": 39165952, "tokens_per_sec": 21036.894272819092, "vram_gb": 5.521900544, "lambda": 0.23244882809619108, "lambda_min": 0.09490048140287399, "lambda_max": 0.4009540379047394, "S_mean": 0.2732137044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.7621238231658936, "grad_norm": 0.7545923590660095, "tokens_seen": 39329792, "tokens_per_sec": 21037.72790656114, "vram_gb": 5.521900544, "lambda": 0.2329943006237348, "lambda_min": 0.09503442794084549, "lambda_max": 0.4025585353374481, "S_mean": 0.2544199625651042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.618552923202515, "grad_norm": 0.8372272849082947, "tokens_seen": 39493632, "tokens_per_sec": 21037.95312841219, "vram_gb": 5.521900544, "lambda": 0.23355562736590704, "lambda_min": 0.09496930241584778, "lambda_max": 0.404010534286499, "S_mean": 0.2756245930989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.731689929962158, "grad_norm": 0.8093173503875732, "tokens_seen": 39657472, "tokens_per_sec": 21038.766776434593, "vram_gb": 5.521900544, "lambda": 0.23402741147826114, "lambda_min": 0.09469414502382278, "lambda_max": 0.4053550064563751, "S_mean": 0.2549540201822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.7833404541015625, "grad_norm": 0.8556994795799255, "tokens_seen": 39821312, "tokens_per_sec": 21039.227551175358, "vram_gb": 5.521900544, "lambda": 0.23454075337698063, "lambda_min": 0.09440737962722778, "lambda_max": 0.4068988859653473, "S_mean": 0.2636820475260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.804652452468872, "grad_norm": 0.8061690330505371, "tokens_seen": 39985152, "tokens_per_sec": 21039.95348168145, "vram_gb": 5.521900544, "lambda": 0.23501304009308419, "lambda_min": 0.09414033591747284, "lambda_max": 0.40830206871032715, "S_mean": 0.271942138671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.581320762634277, "grad_norm": 0.7904144525527954, "tokens_seen": 40148992, "tokens_per_sec": 21040.66843672804, "vram_gb": 5.521900544, "lambda": 0.2354985506584247, "lambda_min": 0.09420947730541229, "lambda_max": 0.40972089767456055, "S_mean": 0.265380859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.80402398109436, "grad_norm": 0.8424675464630127, "tokens_seen": 40312832, "tokens_per_sec": 21041.277263408952, "vram_gb": 5.521900544, "lambda": 0.2359691954528292, "lambda_min": 0.09403418004512787, "lambda_max": 0.4108230471611023, "S_mean": 0.25238037109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.665336608886719, "grad_norm": 0.7953026294708252, "tokens_seen": 40476672, "tokens_per_sec": 21041.53198804592, "vram_gb": 5.521900544, "lambda": 0.23641565317908922, "lambda_min": 0.0940188616514206, "lambda_max": 0.411930114030838, "S_mean": 0.2732950846354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.733632564544678, "grad_norm": 0.8002325892448425, "tokens_seen": 40640512, "tokens_per_sec": 21041.394892052176, "vram_gb": 5.521900544, "lambda": 0.23693146090954542, "lambda_min": 0.09408469498157501, "lambda_max": 0.41357657313346863, "S_mean": 0.276214599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.689616918563843, "grad_norm": 0.7951480746269226, "tokens_seen": 40804352, "tokens_per_sec": 21042.365421816845, "vram_gb": 5.521900544, "lambda": 0.2375046011681358, "lambda_min": 0.09418909251689911, "lambda_max": 0.415270060300827, "S_mean": 0.2600809733072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.5804877281188965, "grad_norm": 0.8359171152114868, "tokens_seen": 40968192, "tokens_per_sec": 21042.12301305353, "vram_gb": 5.521900544, "lambda": 0.2379760518670082, "lambda_min": 0.09414009749889374, "lambda_max": 0.4167027771472931, "S_mean": 0.2662556966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.776581811904907, "val_ppl": 118.69792398949255, "lambda": 0.23799752164632082, "lambda_min": 0.09412743151187897, "lambda_max": 0.41679322719573975, "lambda_per_layer": [0.16321676969528198, 0.13516069948673248, 0.14297187328338623, 0.14000442624092102, 0.1449582874774933, 0.13265427947044373, 0.12478825449943542, 0.10213418304920197, 0.09412743151187897, 0.09530051797628403, 0.1690896600484848, 0.25126320123672485, 0.3732658326625824, 0.38078218698501587, 0.3273325562477112, 0.33544862270355225, 0.38412943482398987, 0.27031078934669495, 0.4053773581981659, 0.41679322719573975, 0.2603858411312103, 0.3481224775314331, 0.2933778464794159, 0.22094476222991943], "S_mean": 0.2683614095052083, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.758223295211792, "grad_norm": 0.7931774258613586, "tokens_seen": 41132032, "tokens_per_sec": 21027.69361830564, "vram_gb": 5.521900544, "lambda": 0.23840442020446062, "lambda_min": 0.09388114511966705, "lambda_max": 0.4181128740310669, "S_mean": 0.2532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.628645896911621, "grad_norm": 0.7676258087158203, "tokens_seen": 41295872, "tokens_per_sec": 21026.934428274053, "vram_gb": 5.521900544, "lambda": 0.23876803802947202, "lambda_min": 0.09380607306957245, "lambda_max": 0.4195610582828522, "S_mean": 0.2547403971354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.661813497543335, "grad_norm": 0.9243238568305969, "tokens_seen": 41459712, "tokens_per_sec": 21027.24150693294, "vram_gb": 5.521900544, "lambda": 0.23923390079289675, "lambda_min": 0.0936778262257576, "lambda_max": 0.42135560512542725, "S_mean": 0.276397705078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.718124151229858, "grad_norm": 0.8627300262451172, "tokens_seen": 41623552, "tokens_per_sec": 21028.265129582593, "vram_gb": 5.521900544, "lambda": 0.23975531694789728, "lambda_min": 0.09367729723453522, "lambda_max": 0.42273205518722534, "S_mean": 0.26727294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.6142578125, "grad_norm": 0.8502688407897949, "tokens_seen": 41787392, "tokens_per_sec": 21028.877455080205, "vram_gb": 5.521900544, "lambda": 0.2402453419441978, "lambda_min": 0.09376775473356247, "lambda_max": 0.4244368076324463, "S_mean": 0.2753804524739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.752795219421387, "grad_norm": 0.7910818457603455, "tokens_seen": 41951232, "tokens_per_sec": 21029.27331567404, "vram_gb": 5.521900544, "lambda": 0.24069552713384232, "lambda_min": 0.09365512430667877, "lambda_max": 0.4255581498146057, "S_mean": 0.2550150553385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.666713237762451, "grad_norm": 0.7849843502044678, "tokens_seen": 42115072, "tokens_per_sec": 21029.611885343318, "vram_gb": 5.521900544, "lambda": 0.24114000828315815, "lambda_min": 0.0936681255698204, "lambda_max": 0.42694780230522156, "S_mean": 0.269775390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.774755954742432, "grad_norm": 0.7716372013092041, "tokens_seen": 42278912, "tokens_per_sec": 21030.335470471735, "vram_gb": 5.521900544, "lambda": 0.24160239907602468, "lambda_min": 0.09354870766401291, "lambda_max": 0.4284243583679199, "S_mean": 0.2672932942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.505686283111572, "grad_norm": 0.7917166948318481, "tokens_seen": 42442752, "tokens_per_sec": 21029.546865532968, "vram_gb": 5.521900544, "lambda": 0.2420852491632104, "lambda_min": 0.0934353694319725, "lambda_max": 0.430102676153183, "S_mean": 0.275848388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.541905403137207, "grad_norm": 0.8718421459197998, "tokens_seen": 42606592, "tokens_per_sec": 21029.24426614456, "vram_gb": 5.521900544, "lambda": 0.24252531367043653, "lambda_min": 0.09343115985393524, "lambda_max": 0.43128353357315063, "S_mean": 0.289581298828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.742332220077515, "grad_norm": 0.8227806687355042, "tokens_seen": 42770432, "tokens_per_sec": 21030.05533607839, "vram_gb": 5.521900544, "lambda": 0.2429589688157042, "lambda_min": 0.093509241938591, "lambda_max": 0.4327113628387451, "S_mean": 0.2673848470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.5570080280303955, "grad_norm": 0.8023940920829773, "tokens_seen": 42934272, "tokens_per_sec": 21030.331161091242, "vram_gb": 5.521900544, "lambda": 0.24342031901081404, "lambda_min": 0.09335048496723175, "lambda_max": 0.43405482172966003, "S_mean": 0.2729085286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.717175006866455, "grad_norm": 0.847523033618927, "tokens_seen": 43098112, "tokens_per_sec": 21030.826965791926, "vram_gb": 5.521900544, "lambda": 0.2438338895638784, "lambda_min": 0.09331095218658447, "lambda_max": 0.4356357455253601, "S_mean": 0.2626444498697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.554510593414307, "grad_norm": 0.7797451615333557, "tokens_seen": 43261952, "tokens_per_sec": 21031.250577728188, "vram_gb": 5.521900544, "lambda": 0.2442765754337112, "lambda_min": 0.09322581440210342, "lambda_max": 0.43704307079315186, "S_mean": 0.26202392578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.764066457748413, "grad_norm": 0.9350046515464783, "tokens_seen": 43425792, "tokens_per_sec": 21031.577078148068, "vram_gb": 5.521900544, "lambda": 0.24483568531771502, "lambda_min": 0.09327851980924606, "lambda_max": 0.4389689862728119, "S_mean": 0.2531636555989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.592982292175293, "grad_norm": 0.84874027967453, "tokens_seen": 43589632, "tokens_per_sec": 21032.021891275406, "vram_gb": 5.521900544, "lambda": 0.2453006493548552, "lambda_min": 0.09309244155883789, "lambda_max": 0.44038891792297363, "S_mean": 0.253265380859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.57968282699585, "grad_norm": 0.8309482932090759, "tokens_seen": 43753472, "tokens_per_sec": 21031.990140864298, "vram_gb": 5.521900544, "lambda": 0.24576422230650982, "lambda_min": 0.09299936145544052, "lambda_max": 0.44212213158607483, "S_mean": 0.2522379557291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.600452899932861, "grad_norm": 0.8583285212516785, "tokens_seen": 43917312, "tokens_per_sec": 21032.50978628528, "vram_gb": 5.521900544, "lambda": 0.24628845850626627, "lambda_min": 0.09303086996078491, "lambda_max": 0.44386202096939087, "S_mean": 0.2709757486979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.75543212890625, "grad_norm": 0.9163647294044495, "tokens_seen": 44081152, "tokens_per_sec": 21033.297462665265, "vram_gb": 5.521900544, "lambda": 0.24681076686829329, "lambda_min": 0.09315289556980133, "lambda_max": 0.4454708397388458, "S_mean": 0.2654520670572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.664042234420776, "grad_norm": 0.8073226809501648, "tokens_seen": 44244992, "tokens_per_sec": 21034.170467529788, "vram_gb": 5.521900544, "lambda": 0.24731259668866792, "lambda_min": 0.09316609054803848, "lambda_max": 0.44731175899505615, "S_mean": 0.2598063151041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.813337564468384, "grad_norm": 0.8887878060340881, "tokens_seen": 44408832, "tokens_per_sec": 21034.99915604764, "vram_gb": 5.521900544, "lambda": 0.24777440199007592, "lambda_min": 0.09314662963151932, "lambda_max": 0.44909948110580444, "S_mean": 0.262237548828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.542625904083252, "grad_norm": 0.865015983581543, "tokens_seen": 44572672, "tokens_per_sec": 21035.100052357055, "vram_gb": 5.521900544, "lambda": 0.24821137295415005, "lambda_min": 0.09296166151762009, "lambda_max": 0.45036831498146057, "S_mean": 0.24471028645833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.599366188049316, "grad_norm": 0.8186377286911011, "tokens_seen": 44736512, "tokens_per_sec": 21035.595658419606, "vram_gb": 5.521900544, "lambda": 0.24866293898473182, "lambda_min": 0.09291879832744598, "lambda_max": 0.45179206132888794, "S_mean": 0.2696736653645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.581624507904053, "grad_norm": 0.8200629353523254, "tokens_seen": 44900352, "tokens_per_sec": 21036.277552744243, "vram_gb": 5.521900544, "lambda": 0.2491338743517796, "lambda_min": 0.0928652212023735, "lambda_max": 0.4532322883605957, "S_mean": 0.2511698404947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.777557373046875, "grad_norm": 0.8685892820358276, "tokens_seen": 45064192, "tokens_per_sec": 21036.250927718636, "vram_gb": 5.521900544, "lambda": 0.24959931491563717, "lambda_min": 0.09272399544715881, "lambda_max": 0.45486781001091003, "S_mean": 0.25799560546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.790093660354614, "grad_norm": 0.8638061881065369, "tokens_seen": 45228032, "tokens_per_sec": 21037.002773817065, "vram_gb": 5.521900544, "lambda": 0.250055651490887, "lambda_min": 0.09277379512786865, "lambda_max": 0.45670464634895325, "S_mean": 0.2524210611979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.680955171585083, "grad_norm": 0.8793543577194214, "tokens_seen": 45391872, "tokens_per_sec": 21037.36273743885, "vram_gb": 5.521900544, "lambda": 0.2504699506486456, "lambda_min": 0.09272982180118561, "lambda_max": 0.4582241177558899, "S_mean": 0.2713826497395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.716999053955078, "grad_norm": 1.0158946514129639, "tokens_seen": 45555712, "tokens_per_sec": 21037.712498792993, "vram_gb": 5.521900544, "lambda": 0.25087631524850923, "lambda_min": 0.09263976663351059, "lambda_max": 0.45977509021759033, "S_mean": 0.2805074055989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.514210224151611, "grad_norm": 0.8156459331512451, "tokens_seen": 45719552, "tokens_per_sec": 21037.853157438574, "vram_gb": 5.521900544, "lambda": 0.25127558192859095, "lambda_min": 0.09254162758588791, "lambda_max": 0.4609583616256714, "S_mean": 0.2732137044270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.722473621368408, "grad_norm": 0.9033275246620178, "tokens_seen": 45883392, "tokens_per_sec": 21038.13197393633, "vram_gb": 5.521900544, "lambda": 0.25180986647804576, "lambda_min": 0.09257092326879501, "lambda_max": 0.46306151151657104, "S_mean": 0.26446533203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.607032775878906, "grad_norm": 0.8908831477165222, "tokens_seen": 46047232, "tokens_per_sec": 21038.820484108935, "vram_gb": 5.521900544, "lambda": 0.2523468093325694, "lambda_min": 0.09276817739009857, "lambda_max": 0.46442002058029175, "S_mean": 0.270599365234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.605616807937622, "grad_norm": 0.9271634221076965, "tokens_seen": 46211072, "tokens_per_sec": 21038.714876335067, "vram_gb": 5.521900544, "lambda": 0.25270708308865625, "lambda_min": 0.09285041689872742, "lambda_max": 0.4655011296272278, "S_mean": 0.26385498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.653379440307617, "grad_norm": 0.8974596261978149, "tokens_seen": 46374912, "tokens_per_sec": 21038.94290580082, "vram_gb": 5.521900544, "lambda": 0.2530415067449212, "lambda_min": 0.09289620071649551, "lambda_max": 0.46713554859161377, "S_mean": 0.2750956217447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.747410774230957, "grad_norm": 0.8935057520866394, "tokens_seen": 46538752, "tokens_per_sec": 21039.771614701443, "vram_gb": 5.521900544, "lambda": 0.2534307176247239, "lambda_min": 0.09295506030321121, "lambda_max": 0.468828409910202, "S_mean": 0.29150390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.649128675460815, "grad_norm": 0.9528172016143799, "tokens_seen": 46702592, "tokens_per_sec": 21039.918377221496, "vram_gb": 5.521900544, "lambda": 0.2538331837082903, "lambda_min": 0.09281875938177109, "lambda_max": 0.46992331743240356, "S_mean": 0.2694091796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.608666896820068, "grad_norm": 0.7736632227897644, "tokens_seen": 46866432, "tokens_per_sec": 21040.894736340553, "vram_gb": 5.521900544, "lambda": 0.254263852412502, "lambda_min": 0.09282014518976212, "lambda_max": 0.4710884094238281, "S_mean": 0.262908935546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.6445631980896, "grad_norm": 0.8756723999977112, "tokens_seen": 47030272, "tokens_per_sec": 21041.48927615441, "vram_gb": 5.521900544, "lambda": 0.25476418311397236, "lambda_min": 0.09275040030479431, "lambda_max": 0.47262537479400635, "S_mean": 0.2688700358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.559837341308594, "grad_norm": 0.8285808563232422, "tokens_seen": 47194112, "tokens_per_sec": 21042.238325816616, "vram_gb": 5.521900544, "lambda": 0.25514222991963226, "lambda_min": 0.09279830753803253, "lambda_max": 0.47349199652671814, "S_mean": 0.2640380859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.662419319152832, "grad_norm": 0.8966439366340637, "tokens_seen": 47357952, "tokens_per_sec": 21042.324588120555, "vram_gb": 5.521900544, "lambda": 0.2555828085169196, "lambda_min": 0.09297021478414536, "lambda_max": 0.47443732619285583, "S_mean": 0.2675577799479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.643711090087891, "grad_norm": 0.8512613773345947, "tokens_seen": 47521792, "tokens_per_sec": 21042.749387451633, "vram_gb": 5.521900544, "lambda": 0.2559935025249918, "lambda_min": 0.09288959950208664, "lambda_max": 0.4757453501224518, "S_mean": 0.23664347330729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.468030214309692, "grad_norm": 0.9286496639251709, "tokens_seen": 47685632, "tokens_per_sec": 21042.686503688823, "vram_gb": 5.521900544, "lambda": 0.25641677249222994, "lambda_min": 0.09283082187175751, "lambda_max": 0.47752252221107483, "S_mean": 0.2571309407552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.563791036605835, "grad_norm": 0.8543190360069275, "tokens_seen": 47849472, "tokens_per_sec": 21042.382449431367, "vram_gb": 5.521900544, "lambda": 0.25685216176013154, "lambda_min": 0.09283070266246796, "lambda_max": 0.4790138304233551, "S_mean": 0.28582763671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.500014781951904, "grad_norm": 0.8264367580413818, "tokens_seen": 48013312, "tokens_per_sec": 21042.618741658243, "vram_gb": 5.521900544, "lambda": 0.2572278743609786, "lambda_min": 0.09271933138370514, "lambda_max": 0.4810546636581421, "S_mean": 0.2735087076822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.491957187652588, "grad_norm": 0.8796514272689819, "tokens_seen": 48177152, "tokens_per_sec": 21042.350533245848, "vram_gb": 5.521900544, "lambda": 0.2576815774664283, "lambda_min": 0.09247351437807083, "lambda_max": 0.4827669858932495, "S_mean": 0.259552001953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.539499521255493, "grad_norm": 0.854308545589447, "tokens_seen": 48340992, "tokens_per_sec": 21041.131800158022, "vram_gb": 5.521900544, "lambda": 0.25807982807358104, "lambda_min": 0.0925205871462822, "lambda_max": 0.48384273052215576, "S_mean": 0.270172119140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.532897710800171, "grad_norm": 0.8307509422302246, "tokens_seen": 48504832, "tokens_per_sec": 21041.70176907609, "vram_gb": 5.521900544, "lambda": 0.2584884262954195, "lambda_min": 0.09259748458862305, "lambda_max": 0.4847261607646942, "S_mean": 0.256683349609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.687970161437988, "grad_norm": 0.8870266079902649, "tokens_seen": 48668672, "tokens_per_sec": 21041.981757276302, "vram_gb": 5.521900544, "lambda": 0.2589837433770299, "lambda_min": 0.09254541248083115, "lambda_max": 0.486403226852417, "S_mean": 0.2557474772135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.625685691833496, "grad_norm": 0.8363768458366394, "tokens_seen": 48832512, "tokens_per_sec": 21042.67017033676, "vram_gb": 5.521900544, "lambda": 0.2593919973199566, "lambda_min": 0.09252472966909409, "lambda_max": 0.4873284697532654, "S_mean": 0.24620564778645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.462693214416504, "grad_norm": 0.8507562279701233, "tokens_seen": 48996352, "tokens_per_sec": 21043.10227781313, "vram_gb": 5.521900544, "lambda": 0.25977535483737785, "lambda_min": 0.09248875826597214, "lambda_max": 0.48849570751190186, "S_mean": 0.2670491536458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.6888744831085205, "grad_norm": 0.8757658004760742, "tokens_seen": 49160192, "tokens_per_sec": 21043.916459654436, "vram_gb": 5.521900544, "lambda": 0.26010638816903037, "lambda_min": 0.09262928366661072, "lambda_max": 0.48967868089675903, "S_mean": 0.2645467122395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.7250913858413695, "val_ppl": 112.74080081625259, "lambda": 0.2601229700570305, "lambda_min": 0.09264342486858368, "lambda_max": 0.4897277355194092, "lambda_per_layer": [0.16726626455783844, 0.13501350581645966, 0.14327697455883026, 0.1417389214038849, 0.1508839875459671, 0.13372933864593506, 0.12444067746400833, 0.10254089534282684, 0.09264342486858368, 0.09316189587116241, 0.1727731078863144, 0.26320236921310425, 0.40440359711647034, 0.4177343547344208, 0.3588219881057739, 0.3660106062889099, 0.44004520773887634, 0.3059471845626831, 0.471109002828598, 0.4897277355194092, 0.29524123668670654, 0.40345466136932373, 0.32514259219169617, 0.2446417510509491], "S_mean": 0.2715962727864583, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.773894309997559, "grad_norm": 0.8440953493118286, "tokens_seen": 49324032, "tokens_per_sec": 21032.037629986913, "vram_gb": 5.521900544, "lambda": 0.2605509363735716, "lambda_min": 0.0926990658044815, "lambda_max": 0.49073123931884766, "S_mean": 0.247772216796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.548025369644165, "grad_norm": 0.855217695236206, "tokens_seen": 49487872, "tokens_per_sec": 21032.520896539856, "vram_gb": 5.521900544, "lambda": 0.26097840815782547, "lambda_min": 0.0927053764462471, "lambda_max": 0.4919288158416748, "S_mean": 0.2502237955729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.49699854850769, "grad_norm": 0.8928739428520203, "tokens_seen": 49651712, "tokens_per_sec": 21032.271523511998, "vram_gb": 5.521900544, "lambda": 0.2614562710126241, "lambda_min": 0.09280890971422195, "lambda_max": 0.4934781491756439, "S_mean": 0.2496337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.530906677246094, "grad_norm": 0.8153320550918579, "tokens_seen": 49815552, "tokens_per_sec": 21032.583366408366, "vram_gb": 5.521900544, "lambda": 0.2618637339522441, "lambda_min": 0.09262388944625854, "lambda_max": 0.4948164224624634, "S_mean": 0.2605692545572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.664382219314575, "grad_norm": 0.8655329346656799, "tokens_seen": 49979392, "tokens_per_sec": 21032.430953482646, "vram_gb": 5.521900544, "lambda": 0.2621789959569772, "lambda_min": 0.09248875826597214, "lambda_max": 0.49623796343803406, "S_mean": 0.26251220703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.685860633850098, "grad_norm": 0.8927393555641174, "tokens_seen": 50143232, "tokens_per_sec": 21032.648204676338, "vram_gb": 5.521900544, "lambda": 0.26250711486985284, "lambda_min": 0.09249141812324524, "lambda_max": 0.4976010322570801, "S_mean": 0.263763427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.6607372760772705, "grad_norm": 0.896308183670044, "tokens_seen": 50307072, "tokens_per_sec": 21033.16860054855, "vram_gb": 5.521900544, "lambda": 0.2628472674017151, "lambda_min": 0.09246177971363068, "lambda_max": 0.49879172444343567, "S_mean": 0.2710367838541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.525060653686523, "grad_norm": 0.9351654052734375, "tokens_seen": 50470912, "tokens_per_sec": 21032.819655715884, "vram_gb": 5.521900544, "lambda": 0.26328982319682837, "lambda_min": 0.09248684346675873, "lambda_max": 0.5001287460327148, "S_mean": 0.2689615885416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.613104820251465, "grad_norm": 0.993419349193573, "tokens_seen": 50634752, "tokens_per_sec": 21033.255732594036, "vram_gb": 5.521900544, "lambda": 0.2636699276044965, "lambda_min": 0.09263169020414352, "lambda_max": 0.5008428692817688, "S_mean": 0.2536519368489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.556222677230835, "grad_norm": 1.1011381149291992, "tokens_seen": 50798592, "tokens_per_sec": 21033.156020413073, "vram_gb": 5.521900544, "lambda": 0.2640429263313611, "lambda_min": 0.0925602987408638, "lambda_max": 0.5022401213645935, "S_mean": 0.2747904459635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.5884175300598145, "grad_norm": 0.9176135063171387, "tokens_seen": 50962432, "tokens_per_sec": 21033.744610572514, "vram_gb": 5.521900544, "lambda": 0.26443176344037056, "lambda_min": 0.09251515567302704, "lambda_max": 0.503361701965332, "S_mean": 0.2688700358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.676203727722168, "grad_norm": 0.9138933420181274, "tokens_seen": 51126272, "tokens_per_sec": 21034.133622551428, "vram_gb": 5.521900544, "lambda": 0.26479389363278943, "lambda_min": 0.09246604144573212, "lambda_max": 0.5045300126075745, "S_mean": 0.2688090006510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.556408166885376, "grad_norm": 0.8624396324157715, "tokens_seen": 51290112, "tokens_per_sec": 21033.982413045534, "vram_gb": 5.521900544, "lambda": 0.2651548186937968, "lambda_min": 0.092494897544384, "lambda_max": 0.5057997703552246, "S_mean": 0.2745361328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.546761989593506, "grad_norm": 0.8890540599822998, "tokens_seen": 51453952, "tokens_per_sec": 21034.579825344823, "vram_gb": 5.521900544, "lambda": 0.2655941757063071, "lambda_min": 0.09247703105211258, "lambda_max": 0.5071558952331543, "S_mean": 0.2694193522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.680283784866333, "grad_norm": 0.833590030670166, "tokens_seen": 51617792, "tokens_per_sec": 21035.60468508613, "vram_gb": 5.521900544, "lambda": 0.2659634094064434, "lambda_min": 0.09242450445890427, "lambda_max": 0.5084326267242432, "S_mean": 0.262786865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.5117926597595215, "grad_norm": 0.8419501185417175, "tokens_seen": 51781632, "tokens_per_sec": 21036.697228992958, "vram_gb": 5.521900544, "lambda": 0.26632695893446606, "lambda_min": 0.09233503043651581, "lambda_max": 0.5099352598190308, "S_mean": 0.2632344563802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.6228625774383545, "grad_norm": 0.9313546419143677, "tokens_seen": 51945472, "tokens_per_sec": 21037.347883683782, "vram_gb": 5.521900544, "lambda": 0.2666599468017618, "lambda_min": 0.0922355055809021, "lambda_max": 0.5111128091812134, "S_mean": 0.27423095703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.7460620403289795, "grad_norm": 0.9646821618080139, "tokens_seen": 52109312, "tokens_per_sec": 21037.351977613118, "vram_gb": 5.521900544, "lambda": 0.2670133359109362, "lambda_min": 0.09233015030622482, "lambda_max": 0.5124703645706177, "S_mean": 0.2577921549479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.3176658153533936, "grad_norm": 0.8521251082420349, "tokens_seen": 52273152, "tokens_per_sec": 21037.1547197159, "vram_gb": 5.521900544, "lambda": 0.26738848350942135, "lambda_min": 0.09232377260923386, "lambda_max": 0.5137554407119751, "S_mean": 0.2746378580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.527121067047119, "grad_norm": 0.8515654802322388, "tokens_seen": 52436992, "tokens_per_sec": 21037.083255247155, "vram_gb": 5.521900544, "lambda": 0.2677568967143695, "lambda_min": 0.09222764521837234, "lambda_max": 0.5145857334136963, "S_mean": 0.2573343912760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.479568243026733, "grad_norm": 0.8883317708969116, "tokens_seen": 52600832, "tokens_per_sec": 21037.248170472612, "vram_gb": 5.521900544, "lambda": 0.2680551962306102, "lambda_min": 0.09221414476633072, "lambda_max": 0.5159772038459778, "S_mean": 0.2737223307291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.651502370834351, "grad_norm": 0.8514911532402039, "tokens_seen": 52764672, "tokens_per_sec": 21037.030811603458, "vram_gb": 5.521900544, "lambda": 0.268474033412834, "lambda_min": 0.09205092489719391, "lambda_max": 0.5174956321716309, "S_mean": 0.2545674641927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.517601490020752, "grad_norm": 0.9952943325042725, "tokens_seen": 52928512, "tokens_per_sec": 21036.659978909473, "vram_gb": 5.521900544, "lambda": 0.26887696143239737, "lambda_min": 0.09198138117790222, "lambda_max": 0.5187028050422668, "S_mean": 0.2720743815104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.52277398109436, "grad_norm": 0.928138017654419, "tokens_seen": 53092352, "tokens_per_sec": 21035.40016075161, "vram_gb": 5.521900544, "lambda": 0.26924001953254145, "lambda_min": 0.09189669042825699, "lambda_max": 0.5200451612472534, "S_mean": 0.26544189453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.486505508422852, "grad_norm": 0.8610766530036926, "tokens_seen": 53256192, "tokens_per_sec": 21034.28626997169, "vram_gb": 5.521900544, "lambda": 0.2696080633128683, "lambda_min": 0.09177498519420624, "lambda_max": 0.5216051936149597, "S_mean": 0.2590128580729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.605986833572388, "grad_norm": 0.8839123845100403, "tokens_seen": 53420032, "tokens_per_sec": 21034.36080235518, "vram_gb": 5.521900544, "lambda": 0.2699444650982817, "lambda_min": 0.0917230173945427, "lambda_max": 0.5229187607765198, "S_mean": 0.2674153645833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.524769067764282, "grad_norm": 0.8913730978965759, "tokens_seen": 53583872, "tokens_per_sec": 21034.027688616334, "vram_gb": 5.521900544, "lambda": 0.2702836294968923, "lambda_min": 0.09179528802633286, "lambda_max": 0.5239332318305969, "S_mean": 0.2787272135416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.673655986785889, "grad_norm": 0.8841941356658936, "tokens_seen": 53747712, "tokens_per_sec": 21033.689144940196, "vram_gb": 5.521900544, "lambda": 0.2706401239459713, "lambda_min": 0.09189727902412415, "lambda_max": 0.524814784526825, "S_mean": 0.2622172037760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.609796762466431, "grad_norm": 0.8993076682090759, "tokens_seen": 53911552, "tokens_per_sec": 21033.691478752655, "vram_gb": 5.521900544, "lambda": 0.27100987049440545, "lambda_min": 0.09190574288368225, "lambda_max": 0.5258599519729614, "S_mean": 0.2656758626302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.497374057769775, "grad_norm": 0.8698403835296631, "tokens_seen": 54075392, "tokens_per_sec": 21034.168411267412, "vram_gb": 5.521900544, "lambda": 0.27128772468616563, "lambda_min": 0.09183661639690399, "lambda_max": 0.5269415974617004, "S_mean": 0.270782470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.521292686462402, "grad_norm": 0.9873735308647156, "tokens_seen": 54239232, "tokens_per_sec": 21033.82115245869, "vram_gb": 5.521900544, "lambda": 0.2716782350714008, "lambda_min": 0.09178129583597183, "lambda_max": 0.5285060405731201, "S_mean": 0.2729695638020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.402324914932251, "grad_norm": 0.9735987782478333, "tokens_seen": 54403072, "tokens_per_sec": 21034.135669797208, "vram_gb": 5.521900544, "lambda": 0.27207538361350697, "lambda_min": 0.09179725497961044, "lambda_max": 0.530090868473053, "S_mean": 0.2744954427083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.4613425731658936, "grad_norm": 0.9009419083595276, "tokens_seen": 54566912, "tokens_per_sec": 21032.680274396782, "vram_gb": 5.521900544, "lambda": 0.27247627017398673, "lambda_min": 0.09183592349290848, "lambda_max": 0.5316176414489746, "S_mean": 0.24908447265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.476571559906006, "grad_norm": 0.9308190941810608, "tokens_seen": 54730752, "tokens_per_sec": 21032.897216668876, "vram_gb": 5.521900544, "lambda": 0.2727378197014332, "lambda_min": 0.09185393154621124, "lambda_max": 0.5323746204376221, "S_mean": 0.2806803385416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.780751943588257, "grad_norm": 0.9922778606414795, "tokens_seen": 54894592, "tokens_per_sec": 21032.801900269566, "vram_gb": 5.521900544, "lambda": 0.2730424103016655, "lambda_min": 0.09175224602222443, "lambda_max": 0.5333440899848938, "S_mean": 0.2652587890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.43993616104126, "grad_norm": 0.9243339896202087, "tokens_seen": 55058432, "tokens_per_sec": 21032.659135485002, "vram_gb": 5.521900544, "lambda": 0.27336607687175274, "lambda_min": 0.09174185991287231, "lambda_max": 0.5345258712768555, "S_mean": 0.2665201822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.489140272140503, "grad_norm": 0.9249644875526428, "tokens_seen": 55222272, "tokens_per_sec": 21032.450308894648, "vram_gb": 5.521900544, "lambda": 0.27376812044531107, "lambda_min": 0.09183207154273987, "lambda_max": 0.5358415842056274, "S_mean": 0.2689107259114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.573495149612427, "grad_norm": 0.9415983557701111, "tokens_seen": 55386112, "tokens_per_sec": 21032.562506979542, "vram_gb": 5.521900544, "lambda": 0.27410754933953285, "lambda_min": 0.09184230118989944, "lambda_max": 0.5368621945381165, "S_mean": 0.2663675944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.533492803573608, "grad_norm": 0.8975340723991394, "tokens_seen": 55549952, "tokens_per_sec": 21032.58983953016, "vram_gb": 5.521900544, "lambda": 0.2744017243385315, "lambda_min": 0.09175954759120941, "lambda_max": 0.5380839705467224, "S_mean": 0.2661031087239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.673030853271484, "grad_norm": 0.9512096643447876, "tokens_seen": 55713792, "tokens_per_sec": 21032.274485099464, "vram_gb": 5.521900544, "lambda": 0.27474741513530415, "lambda_min": 0.09180877357721329, "lambda_max": 0.539279580116272, "S_mean": 0.2659708658854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.584173679351807, "grad_norm": 0.967951774597168, "tokens_seen": 55877632, "tokens_per_sec": 21032.949850765588, "vram_gb": 5.521900544, "lambda": 0.2750964475174745, "lambda_min": 0.09184092283248901, "lambda_max": 0.540391206741333, "S_mean": 0.257293701171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.562900543212891, "grad_norm": 1.0664502382278442, "tokens_seen": 56041472, "tokens_per_sec": 21032.921067112376, "vram_gb": 5.521900544, "lambda": 0.2754027871415019, "lambda_min": 0.09180139005184174, "lambda_max": 0.5409904718399048, "S_mean": 0.2647298177083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.571906566619873, "grad_norm": 0.9711575508117676, "tokens_seen": 56205312, "tokens_per_sec": 21033.230607803678, "vram_gb": 5.521900544, "lambda": 0.2757884655147791, "lambda_min": 0.09192189574241638, "lambda_max": 0.5421853065490723, "S_mean": 0.24299112955729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.567875385284424, "grad_norm": 0.9853853583335876, "tokens_seen": 56369152, "tokens_per_sec": 21032.448902558306, "vram_gb": 5.521900544, "lambda": 0.27608213294297457, "lambda_min": 0.09181822836399078, "lambda_max": 0.5434945821762085, "S_mean": 0.2567545572916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.56226658821106, "grad_norm": 0.9006636142730713, "tokens_seen": 56532992, "tokens_per_sec": 21032.03829900205, "vram_gb": 5.521900544, "lambda": 0.2763896118849516, "lambda_min": 0.09173239767551422, "lambda_max": 0.5447704195976257, "S_mean": 0.262725830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.498486042022705, "grad_norm": 0.9514129161834717, "tokens_seen": 56696832, "tokens_per_sec": 21032.130586652907, "vram_gb": 5.521900544, "lambda": 0.2767505416025718, "lambda_min": 0.09167823940515518, "lambda_max": 0.54608154296875, "S_mean": 0.2536417643229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.381098031997681, "grad_norm": 0.903671145439148, "tokens_seen": 56860672, "tokens_per_sec": 21032.368096767532, "vram_gb": 5.521900544, "lambda": 0.2771001846219103, "lambda_min": 0.09160537272691727, "lambda_max": 0.5471560955047607, "S_mean": 0.2624308268229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.424048185348511, "grad_norm": 0.9878528714179993, "tokens_seen": 57024512, "tokens_per_sec": 21032.013068437544, "vram_gb": 5.521900544, "lambda": 0.2774897202228506, "lambda_min": 0.09163633733987808, "lambda_max": 0.5482819080352783, "S_mean": 0.2667744954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.630136013031006, "grad_norm": 0.9856352210044861, "tokens_seen": 57188352, "tokens_per_sec": 21031.935707811917, "vram_gb": 5.521900544, "lambda": 0.2777792022873958, "lambda_min": 0.09175366163253784, "lambda_max": 0.5491344928741455, "S_mean": 0.2587483723958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.39670729637146, "grad_norm": 0.8561208248138428, "tokens_seen": 57352192, "tokens_per_sec": 21031.94584292275, "vram_gb": 5.521900544, "lambda": 0.2780758024503787, "lambda_min": 0.09161346405744553, "lambda_max": 0.5501004457473755, "S_mean": 0.2691141764322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.6173667192459105, "val_ppl": 101.22712142772463, "lambda": 0.2780893075590332, "lambda_min": 0.09160814434289932, "lambda_max": 0.5501437187194824, "lambda_per_layer": [0.1707226037979126, 0.13548322021961212, 0.1447616070508957, 0.14348956942558289, 0.15537050366401672, 0.13427965342998505, 0.1246691569685936, 0.10422879457473755, 0.09160814434289932, 0.09208142012357712, 0.1752491295337677, 0.2717970013618469, 0.425601601600647, 0.44444334506988525, 0.38127782940864563, 0.3898204565048218, 0.48431748151779175, 0.33654528856277466, 0.5270248651504517, 0.5501437187194824, 0.32360315322875977, 0.44974178075790405, 0.35219457745552063, 0.2656884789466858], "S_mean": 0.270233154296875, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.5065598487854, "grad_norm": 0.935077965259552, "tokens_seen": 57516032, "tokens_per_sec": 21021.019151154385, "vram_gb": 5.521900544, "lambda": 0.2783716522778074, "lambda_min": 0.09158752858638763, "lambda_max": 0.5508743524551392, "S_mean": 0.255126953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.485753536224365, "grad_norm": 0.9637565612792969, "tokens_seen": 57679872, "tokens_per_sec": 21020.760426051405, "vram_gb": 5.521900544, "lambda": 0.2786837083597978, "lambda_min": 0.09160979092121124, "lambda_max": 0.5517879724502563, "S_mean": 0.2528788248697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.538292169570923, "grad_norm": 1.007568359375, "tokens_seen": 57843712, "tokens_per_sec": 21020.981923713534, "vram_gb": 5.521900544, "lambda": 0.2789633385837078, "lambda_min": 0.09163211286067963, "lambda_max": 0.5528479814529419, "S_mean": 0.24849446614583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.487907886505127, "grad_norm": 0.9748916625976562, "tokens_seen": 58007552, "tokens_per_sec": 21021.15308766943, "vram_gb": 5.521900544, "lambda": 0.2792740287259221, "lambda_min": 0.0916474461555481, "lambda_max": 0.554160475730896, "S_mean": 0.266632080078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.471431016921997, "grad_norm": 0.8813769221305847, "tokens_seen": 58171392, "tokens_per_sec": 21021.00036987185, "vram_gb": 5.521900544, "lambda": 0.27957747876644135, "lambda_min": 0.09158911556005478, "lambda_max": 0.5552202463150024, "S_mean": 0.2612101236979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.619876384735107, "grad_norm": 0.9628017544746399, "tokens_seen": 58335232, "tokens_per_sec": 21020.799274368354, "vram_gb": 5.521900544, "lambda": 0.27987515553832054, "lambda_min": 0.0915282815694809, "lambda_max": 0.556187093257904, "S_mean": 0.2603352864583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.516741991043091, "grad_norm": 1.0314091444015503, "tokens_seen": 58499072, "tokens_per_sec": 21021.187945721173, "vram_gb": 5.521900544, "lambda": 0.28020206031699973, "lambda_min": 0.09171005338430405, "lambda_max": 0.557509183883667, "S_mean": 0.265960693359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.683722734451294, "grad_norm": 1.0237736701965332, "tokens_seen": 58662912, "tokens_per_sec": 21021.360427444313, "vram_gb": 5.521900544, "lambda": 0.2805076325312257, "lambda_min": 0.09172097593545914, "lambda_max": 0.5585517287254333, "S_mean": 0.265899658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.502719163894653, "grad_norm": 0.9516106843948364, "tokens_seen": 58826752, "tokens_per_sec": 21021.491360214513, "vram_gb": 5.521900544, "lambda": 0.28077773936092854, "lambda_min": 0.09170611947774887, "lambda_max": 0.559495210647583, "S_mean": 0.2543843587239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.412096261978149, "grad_norm": 0.8965070247650146, "tokens_seen": 58990592, "tokens_per_sec": 21021.628302436024, "vram_gb": 5.521900544, "lambda": 0.2811049235363801, "lambda_min": 0.09161464124917984, "lambda_max": 0.5608908534049988, "S_mean": 0.24242146809895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.414718389511108, "grad_norm": 0.9715361595153809, "tokens_seen": 59154432, "tokens_per_sec": 21021.941781196176, "vram_gb": 5.521900544, "lambda": 0.28141277842223644, "lambda_min": 0.09158028662204742, "lambda_max": 0.5620594024658203, "S_mean": 0.250244140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.5760931968688965, "grad_norm": 0.9134202599525452, "tokens_seen": 59318272, "tokens_per_sec": 21022.404294612876, "vram_gb": 5.521900544, "lambda": 0.2817480228841305, "lambda_min": 0.09150134772062302, "lambda_max": 0.5628634095191956, "S_mean": 0.2558390299479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.399798393249512, "grad_norm": 0.9667413234710693, "tokens_seen": 59482112, "tokens_per_sec": 21022.699386433498, "vram_gb": 5.521900544, "lambda": 0.28202537385125953, "lambda_min": 0.09140054136514664, "lambda_max": 0.5640283226966858, "S_mean": 0.2552083333333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.386069059371948, "grad_norm": 0.961380124092102, "tokens_seen": 59645952, "tokens_per_sec": 21023.099929574993, "vram_gb": 5.521900544, "lambda": 0.28232967139532167, "lambda_min": 0.09138966351747513, "lambda_max": 0.565031111240387, "S_mean": 0.2658793131510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.62733006477356, "grad_norm": 0.9077422022819519, "tokens_seen": 59809792, "tokens_per_sec": 21022.91879163571, "vram_gb": 5.521900544, "lambda": 0.2825979304810365, "lambda_min": 0.09136847406625748, "lambda_max": 0.5658652782440186, "S_mean": 0.24638875325520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.624338150024414, "grad_norm": 0.9833446741104126, "tokens_seen": 59973632, "tokens_per_sec": 21023.174451829207, "vram_gb": 5.521900544, "lambda": 0.2828828416143854, "lambda_min": 0.09134778380393982, "lambda_max": 0.5668538808822632, "S_mean": 0.2538553873697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.434554815292358, "grad_norm": 0.9238514304161072, "tokens_seen": 60137472, "tokens_per_sec": 21023.199093886913, "vram_gb": 5.521900544, "lambda": 0.2832132950425148, "lambda_min": 0.09139608591794968, "lambda_max": 0.568015456199646, "S_mean": 0.2594502766927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.4038074016571045, "grad_norm": 0.9408513903617859, "tokens_seen": 60301312, "tokens_per_sec": 21023.624507624183, "vram_gb": 5.521900544, "lambda": 0.28358427466203767, "lambda_min": 0.09141092747449875, "lambda_max": 0.5693471431732178, "S_mean": 0.2567850748697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.454869747161865, "grad_norm": 0.9251437187194824, "tokens_seen": 60465152, "tokens_per_sec": 21023.711508170014, "vram_gb": 5.521900544, "lambda": 0.2839121436700225, "lambda_min": 0.09139497578144073, "lambda_max": 0.5704137086868286, "S_mean": 0.261322021484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.57977557182312, "grad_norm": 0.9304789304733276, "tokens_seen": 60628992, "tokens_per_sec": 21023.796021252387, "vram_gb": 5.521900544, "lambda": 0.2841806523501873, "lambda_min": 0.0913894847035408, "lambda_max": 0.5714114904403687, "S_mean": 0.251495361328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.447073459625244, "grad_norm": 0.9318394064903259, "tokens_seen": 60792832, "tokens_per_sec": 21023.97218719762, "vram_gb": 5.521900544, "lambda": 0.2844470785930753, "lambda_min": 0.09135227650403976, "lambda_max": 0.5722958445549011, "S_mean": 0.265411376953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.5167272090911865, "grad_norm": 0.962448239326477, "tokens_seen": 60956672, "tokens_per_sec": 21024.0916401905, "vram_gb": 5.521900544, "lambda": 0.28473740505675477, "lambda_min": 0.09133028239011765, "lambda_max": 0.5733338594436646, "S_mean": 0.267333984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.589054346084595, "grad_norm": 0.9418764114379883, "tokens_seen": 61120512, "tokens_per_sec": 21024.160526017404, "vram_gb": 5.521900544, "lambda": 0.285005207794408, "lambda_min": 0.09122227877378464, "lambda_max": 0.5744230151176453, "S_mean": 0.2619120279947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.504487991333008, "grad_norm": 1.0156426429748535, "tokens_seen": 61284352, "tokens_per_sec": 21023.973868570887, "vram_gb": 5.521900544, "lambda": 0.28527701180428267, "lambda_min": 0.0911211222410202, "lambda_max": 0.5755219459533691, "S_mean": 0.24922688802083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.589372158050537, "grad_norm": 1.0111929178237915, "tokens_seen": 61448192, "tokens_per_sec": 21023.79056613546, "vram_gb": 5.521900544, "lambda": 0.2855298410480221, "lambda_min": 0.09103728085756302, "lambda_max": 0.5762296319007874, "S_mean": 0.246734619140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.402729749679565, "grad_norm": 0.9179485440254211, "tokens_seen": 61612032, "tokens_per_sec": 21023.50026176744, "vram_gb": 5.521900544, "lambda": 0.28579395450651646, "lambda_min": 0.09109906852245331, "lambda_max": 0.5770165324211121, "S_mean": 0.2726338704427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.4905924797058105, "grad_norm": 0.9497737288475037, "tokens_seen": 61775872, "tokens_per_sec": 21023.99312798307, "vram_gb": 5.521900544, "lambda": 0.28603280025223893, "lambda_min": 0.09109962731599808, "lambda_max": 0.5778113007545471, "S_mean": 0.2642110188802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.5267274379730225, "grad_norm": 0.8913556337356567, "tokens_seen": 61939712, "tokens_per_sec": 21024.048826840837, "vram_gb": 5.521900544, "lambda": 0.28629320207983255, "lambda_min": 0.09116990119218826, "lambda_max": 0.5788277983665466, "S_mean": 0.2712809244791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.460933208465576, "grad_norm": 1.0374566316604614, "tokens_seen": 62103552, "tokens_per_sec": 21023.857312626318, "vram_gb": 5.521900544, "lambda": 0.28661202763517696, "lambda_min": 0.09124245494604111, "lambda_max": 0.5794229507446289, "S_mean": 0.2816874186197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.40445876121521, "grad_norm": 0.9943028092384338, "tokens_seen": 62267392, "tokens_per_sec": 21023.76815463222, "vram_gb": 5.521900544, "lambda": 0.2869257063915332, "lambda_min": 0.09123100340366364, "lambda_max": 0.5801938772201538, "S_mean": 0.2512105305989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.291643381118774, "grad_norm": 0.9701923727989197, "tokens_seen": 62431232, "tokens_per_sec": 21023.275636414215, "vram_gb": 5.521900544, "lambda": 0.2871959653372566, "lambda_min": 0.09127852320671082, "lambda_max": 0.5809803009033203, "S_mean": 0.2586669921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.502832889556885, "grad_norm": 0.9524524211883545, "tokens_seen": 62595072, "tokens_per_sec": 21023.536461742668, "vram_gb": 5.521900544, "lambda": 0.2874716476847728, "lambda_min": 0.09120381623506546, "lambda_max": 0.5817151069641113, "S_mean": 0.2667643229166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.431029558181763, "grad_norm": 1.22605562210083, "tokens_seen": 62758912, "tokens_per_sec": 21023.24520657603, "vram_gb": 5.521900544, "lambda": 0.2877583159133792, "lambda_min": 0.0911426916718483, "lambda_max": 0.582538902759552, "S_mean": 0.2665812174479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.5166099071502686, "grad_norm": 1.0094918012619019, "tokens_seen": 62922752, "tokens_per_sec": 21022.794699132704, "vram_gb": 5.521900544, "lambda": 0.2880103134860595, "lambda_min": 0.09102867543697357, "lambda_max": 0.58344966173172, "S_mean": 0.2612202962239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.457244396209717, "grad_norm": 0.9604601263999939, "tokens_seen": 63086592, "tokens_per_sec": 21022.712882942596, "vram_gb": 5.521900544, "lambda": 0.28828644069532555, "lambda_min": 0.09096255153417587, "lambda_max": 0.5841870307922363, "S_mean": 0.2621561686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.347984075546265, "grad_norm": 1.0723313093185425, "tokens_seen": 63250432, "tokens_per_sec": 21022.714765270877, "vram_gb": 5.521900544, "lambda": 0.2885421725610892, "lambda_min": 0.09087634831666946, "lambda_max": 0.584913969039917, "S_mean": 0.2706197102864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.331559658050537, "grad_norm": 0.9579970836639404, "tokens_seen": 63414272, "tokens_per_sec": 21022.931413950344, "vram_gb": 5.521900544, "lambda": 0.2887974229330818, "lambda_min": 0.09074029326438904, "lambda_max": 0.585888147354126, "S_mean": 0.2617085774739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.435171604156494, "grad_norm": 1.0379043817520142, "tokens_seen": 63578112, "tokens_per_sec": 21023.281260564476, "vram_gb": 5.521900544, "lambda": 0.28910736832767725, "lambda_min": 0.09067851305007935, "lambda_max": 0.5869147181510925, "S_mean": 0.2736714680989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.450227975845337, "grad_norm": 0.9551507234573364, "tokens_seen": 63741952, "tokens_per_sec": 21023.46033795008, "vram_gb": 5.521900544, "lambda": 0.28933737830569345, "lambda_min": 0.09056136757135391, "lambda_max": 0.5875272750854492, "S_mean": 0.2530415852864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.449274778366089, "grad_norm": 1.0359318256378174, "tokens_seen": 63905792, "tokens_per_sec": 21023.94989665821, "vram_gb": 5.521900544, "lambda": 0.2895875150958697, "lambda_min": 0.09057621657848358, "lambda_max": 0.5881872177124023, "S_mean": 0.2638041178385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.5421833992004395, "grad_norm": 0.9497877359390259, "tokens_seen": 64069632, "tokens_per_sec": 21024.318051013688, "vram_gb": 5.521900544, "lambda": 0.28985273869087297, "lambda_min": 0.09061659127473831, "lambda_max": 0.5889913439750671, "S_mean": 0.275909423828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.485810279846191, "grad_norm": 1.0020495653152466, "tokens_seen": 64233472, "tokens_per_sec": 21024.292566800104, "vram_gb": 5.521900544, "lambda": 0.29011591865370673, "lambda_min": 0.09055858850479126, "lambda_max": 0.5899495482444763, "S_mean": 0.255523681640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.238908052444458, "grad_norm": 0.8869267106056213, "tokens_seen": 64397312, "tokens_per_sec": 21024.806676367378, "vram_gb": 5.521900544, "lambda": 0.290333600093921, "lambda_min": 0.09054415673017502, "lambda_max": 0.5907346606254578, "S_mean": 0.2596232096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.5332112312316895, "grad_norm": 1.0006664991378784, "tokens_seen": 64561152, "tokens_per_sec": 21024.991694575205, "vram_gb": 5.521900544, "lambda": 0.29056544322520494, "lambda_min": 0.09047803282737732, "lambda_max": 0.591246485710144, "S_mean": 0.260589599609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.533342599868774, "grad_norm": 1.0991710424423218, "tokens_seen": 64724992, "tokens_per_sec": 21025.11246281948, "vram_gb": 5.521900544, "lambda": 0.29079249625404674, "lambda_min": 0.09039842337369919, "lambda_max": 0.5918396711349487, "S_mean": 0.2697041829427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.29754114151001, "grad_norm": 0.9132459759712219, "tokens_seen": 64888832, "tokens_per_sec": 21025.04717269792, "vram_gb": 5.521900544, "lambda": 0.29100404276202124, "lambda_min": 0.0903833732008934, "lambda_max": 0.5923349261283875, "S_mean": 0.2818806966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.481905937194824, "grad_norm": 0.9732475876808167, "tokens_seen": 65052672, "tokens_per_sec": 21025.25073500367, "vram_gb": 5.521900544, "lambda": 0.2912351780881484, "lambda_min": 0.0903785303235054, "lambda_max": 0.5924442410469055, "S_mean": 0.24462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.517768859863281, "grad_norm": 0.9653536081314087, "tokens_seen": 65216512, "tokens_per_sec": 21025.38884396773, "vram_gb": 5.521900544, "lambda": 0.29150691100706655, "lambda_min": 0.09043588489294052, "lambda_max": 0.5929366946220398, "S_mean": 0.254852294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.401296377182007, "grad_norm": 0.9715009331703186, "tokens_seen": 65380352, "tokens_per_sec": 21025.438739860645, "vram_gb": 5.521900544, "lambda": 0.2917752868185441, "lambda_min": 0.09053141623735428, "lambda_max": 0.5937773585319519, "S_mean": 0.2595621744791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.469074964523315, "grad_norm": 0.9305746555328369, "tokens_seen": 65544192, "tokens_per_sec": 21025.717959913767, "vram_gb": 5.521900544, "lambda": 0.2919822692250212, "lambda_min": 0.09060278534889221, "lambda_max": 0.5945996046066284, "S_mean": 0.26934814453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.500423741340637, "val_ppl": 90.05528336315164, "lambda": 0.29199721198529005, "lambda_min": 0.09060706943273544, "lambda_max": 0.5946558713912964, "lambda_per_layer": [0.17484018206596375, 0.13597223162651062, 0.1455107182264328, 0.14512866735458374, 0.15885226428508759, 0.1349833756685257, 0.12483926117420197, 0.1054796651005745, 0.09144964069128036, 0.09060706943273544, 0.17766757309436798, 0.2773631513118744, 0.43880346417427063, 0.4617454707622528, 0.3979707956314087, 0.4052197337150574, 0.5196920037269592, 0.36258307099342346, 0.5728812217712402, 0.5946558713912964, 0.347057044506073, 0.4840003550052643, 0.377704381942749, 0.28292587399482727], "S_mean": 0.2773844401041667, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.518914222717285, "grad_norm": 1.0208721160888672, "tokens_seen": 65708032, "tokens_per_sec": 21016.730360840324, "vram_gb": 5.521900544, "lambda": 0.29226516280323267, "lambda_min": 0.09068422019481659, "lambda_max": 0.5956084132194519, "S_mean": 0.2574462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.528888940811157, "grad_norm": 0.9856957197189331, "tokens_seen": 65871872, "tokens_per_sec": 21016.199881061108, "vram_gb": 5.521900544, "lambda": 0.2925489963963628, "lambda_min": 0.09065958112478256, "lambda_max": 0.596464991569519, "S_mean": 0.2783406575520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.4814629554748535, "grad_norm": 0.96944260597229, "tokens_seen": 66035712, "tokens_per_sec": 21015.873278069583, "vram_gb": 5.521900544, "lambda": 0.29277190876503784, "lambda_min": 0.09068544209003448, "lambda_max": 0.5971652865409851, "S_mean": 0.267486572265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.411101579666138, "grad_norm": 0.9995148181915283, "tokens_seen": 66199552, "tokens_per_sec": 21015.482868660456, "vram_gb": 5.521900544, "lambda": 0.29306782130151987, "lambda_min": 0.09072926640510559, "lambda_max": 0.5982979536056519, "S_mean": 0.26708984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.2945685386657715, "grad_norm": 0.9276056289672852, "tokens_seen": 66363392, "tokens_per_sec": 21015.74650530389, "vram_gb": 5.521900544, "lambda": 0.29327696810166043, "lambda_min": 0.09073511511087418, "lambda_max": 0.5989863276481628, "S_mean": 0.26544189453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.712631702423096, "grad_norm": 1.0365290641784668, "tokens_seen": 66527232, "tokens_per_sec": 21015.322935821376, "vram_gb": 5.521900544, "lambda": 0.29354423998544615, "lambda_min": 0.09081156551837921, "lambda_max": 0.5996749401092529, "S_mean": 0.263641357421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.417947769165039, "grad_norm": 1.193984866142273, "tokens_seen": 66691072, "tokens_per_sec": 21015.700080251616, "vram_gb": 5.521900544, "lambda": 0.29382862616330385, "lambda_min": 0.0907662883400917, "lambda_max": 0.6006635427474976, "S_mean": 0.2527872721354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.392273187637329, "grad_norm": 0.9521836042404175, "tokens_seen": 66854912, "tokens_per_sec": 21015.88431197856, "vram_gb": 5.521900544, "lambda": 0.2940215989947319, "lambda_min": 0.0906861424446106, "lambda_max": 0.6015435457229614, "S_mean": 0.2678629557291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.496668338775635, "grad_norm": 0.9913126826286316, "tokens_seen": 67018752, "tokens_per_sec": 21016.249280068205, "vram_gb": 5.521900544, "lambda": 0.2941976059228182, "lambda_min": 0.09065910428762436, "lambda_max": 0.602264404296875, "S_mean": 0.2692667643229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.44029974937439, "grad_norm": 0.9975454211235046, "tokens_seen": 67182592, "tokens_per_sec": 21015.842384521544, "vram_gb": 5.521900544, "lambda": 0.2943640798330307, "lambda_min": 0.0906791090965271, "lambda_max": 0.6027725338935852, "S_mean": 0.2644856770833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.4605913162231445, "grad_norm": 1.0278511047363281, "tokens_seen": 67346432, "tokens_per_sec": 21015.329560126054, "vram_gb": 5.521900544, "lambda": 0.2945721506451567, "lambda_min": 0.09076625108718872, "lambda_max": 0.6036444306373596, "S_mean": 0.259918212890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.597192287445068, "grad_norm": 1.0522239208221436, "tokens_seen": 67510272, "tokens_per_sec": 21015.565000740666, "vram_gb": 5.521900544, "lambda": 0.2948270433892806, "lambda_min": 0.09084460139274597, "lambda_max": 0.6043859124183655, "S_mean": 0.2586161295572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.425204753875732, "grad_norm": 0.9959383606910706, "tokens_seen": 67674112, "tokens_per_sec": 21016.01494382457, "vram_gb": 5.521900544, "lambda": 0.2950559767584006, "lambda_min": 0.09089823812246323, "lambda_max": 0.6050735116004944, "S_mean": 0.2682698567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.373362302780151, "grad_norm": 0.9694532752037048, "tokens_seen": 67837952, "tokens_per_sec": 21013.93897576559, "vram_gb": 5.521900544, "lambda": 0.2953093309576313, "lambda_min": 0.09100117534399033, "lambda_max": 0.6059529185295105, "S_mean": 0.2639668782552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.412765026092529, "grad_norm": 1.336053729057312, "tokens_seen": 68001792, "tokens_per_sec": 21013.994189008044, "vram_gb": 5.521900544, "lambda": 0.29555914519975585, "lambda_min": 0.09104341268539429, "lambda_max": 0.6069188714027405, "S_mean": 0.26055908203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.388865232467651, "grad_norm": 1.1357307434082031, "tokens_seen": 68165632, "tokens_per_sec": 21014.339750690033, "vram_gb": 5.521900544, "lambda": 0.29578693583607674, "lambda_min": 0.09105585515499115, "lambda_max": 0.6076141595840454, "S_mean": 0.2620646158854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.457464933395386, "grad_norm": 1.0585036277770996, "tokens_seen": 68329472, "tokens_per_sec": 21013.818494069783, "vram_gb": 5.521900544, "lambda": 0.29600506493200857, "lambda_min": 0.0911419466137886, "lambda_max": 0.6083207130432129, "S_mean": 0.27301025390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.319464206695557, "grad_norm": 1.0229592323303223, "tokens_seen": 68493312, "tokens_per_sec": 21013.761929953595, "vram_gb": 5.521900544, "lambda": 0.2962373737245798, "lambda_min": 0.09111331403255463, "lambda_max": 0.6090831160545349, "S_mean": 0.2567850748697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.420138359069824, "grad_norm": 1.065114974975586, "tokens_seen": 68657152, "tokens_per_sec": 21013.744374828275, "vram_gb": 5.521900544, "lambda": 0.2964725581308206, "lambda_min": 0.09110656380653381, "lambda_max": 0.6100472211837769, "S_mean": 0.3013509114583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.498245716094971, "grad_norm": 1.0581544637680054, "tokens_seen": 68820992, "tokens_per_sec": 21013.811060735996, "vram_gb": 5.521900544, "lambda": 0.29670541019489366, "lambda_min": 0.09118397533893585, "lambda_max": 0.6109119653701782, "S_mean": 0.2670186360677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.477935314178467, "grad_norm": 1.0740123987197876, "tokens_seen": 68984832, "tokens_per_sec": 21013.797788164884, "vram_gb": 5.521900544, "lambda": 0.29693524818867445, "lambda_min": 0.09117749333381653, "lambda_max": 0.611889660358429, "S_mean": 0.253326416015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.415880918502808, "grad_norm": 1.0196079015731812, "tokens_seen": 69148672, "tokens_per_sec": 21013.943469029604, "vram_gb": 5.521900544, "lambda": 0.29714473709464073, "lambda_min": 0.09117542952299118, "lambda_max": 0.6128400564193726, "S_mean": 0.2755330403645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.514618396759033, "grad_norm": 1.0341252088546753, "tokens_seen": 69312512, "tokens_per_sec": 21014.240132043222, "vram_gb": 5.521900544, "lambda": 0.2973694521933794, "lambda_min": 0.09122611582279205, "lambda_max": 0.6135873198509216, "S_mean": 0.282470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.457834243774414, "grad_norm": 1.0574759244918823, "tokens_seen": 69476352, "tokens_per_sec": 21014.146955116983, "vram_gb": 5.521900544, "lambda": 0.2975805504247546, "lambda_min": 0.0912156030535698, "lambda_max": 0.6140815019607544, "S_mean": 0.2641703287760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.4153571128845215, "grad_norm": 1.063583493232727, "tokens_seen": 69640192, "tokens_per_sec": 21014.165776675916, "vram_gb": 5.521900544, "lambda": 0.29778565373271704, "lambda_min": 0.09118632227182388, "lambda_max": 0.6149315237998962, "S_mean": 0.2551981608072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.423595428466797, "grad_norm": 1.0642298460006714, "tokens_seen": 69804032, "tokens_per_sec": 21014.196218771434, "vram_gb": 5.521900544, "lambda": 0.29796665472288925, "lambda_min": 0.09110649675130844, "lambda_max": 0.6152523756027222, "S_mean": 0.2667948404947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.415833950042725, "grad_norm": 0.9794681668281555, "tokens_seen": 69967872, "tokens_per_sec": 21014.502971153463, "vram_gb": 5.521900544, "lambda": 0.2981470062707861, "lambda_min": 0.09100941568613052, "lambda_max": 0.6155548095703125, "S_mean": 0.2693074544270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.3839430809021, "grad_norm": 1.0690433979034424, "tokens_seen": 70131712, "tokens_per_sec": 21014.59440068448, "vram_gb": 5.521900544, "lambda": 0.2983068637549877, "lambda_min": 0.09095170348882675, "lambda_max": 0.6161403656005859, "S_mean": 0.2515767415364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.496198654174805, "grad_norm": 1.0817712545394897, "tokens_seen": 70295552, "tokens_per_sec": 21014.820748201735, "vram_gb": 5.521900544, "lambda": 0.2984922140215834, "lambda_min": 0.09097287058830261, "lambda_max": 0.6169152855873108, "S_mean": 0.2585245768229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.31842303276062, "grad_norm": 1.016312599182129, "tokens_seen": 70459392, "tokens_per_sec": 21014.899287076747, "vram_gb": 5.521900544, "lambda": 0.29867590374002856, "lambda_min": 0.09102045744657516, "lambda_max": 0.6175551414489746, "S_mean": 0.2604268391927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.42510461807251, "grad_norm": 1.0610592365264893, "tokens_seen": 70623232, "tokens_per_sec": 21015.045231494045, "vram_gb": 5.521900544, "lambda": 0.2988306765134136, "lambda_min": 0.09104983508586884, "lambda_max": 0.6178475022315979, "S_mean": 0.2713216145833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.436126708984375, "grad_norm": 1.0448484420776367, "tokens_seen": 70787072, "tokens_per_sec": 21015.41694472715, "vram_gb": 5.521900544, "lambda": 0.2990329597766201, "lambda_min": 0.09107194095849991, "lambda_max": 0.6187139749526978, "S_mean": 0.25897216796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.447984457015991, "grad_norm": 1.1046651601791382, "tokens_seen": 70950912, "tokens_per_sec": 21015.28897793173, "vram_gb": 5.521900544, "lambda": 0.2992394467194875, "lambda_min": 0.0910196378827095, "lambda_max": 0.6194894313812256, "S_mean": 0.2561238606770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.4798805713653564, "grad_norm": 1.045493721961975, "tokens_seen": 71114752, "tokens_per_sec": 21015.509003297724, "vram_gb": 5.521900544, "lambda": 0.29942688532173634, "lambda_min": 0.09098242968320847, "lambda_max": 0.6201854348182678, "S_mean": 0.2531331380208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.446868181228638, "grad_norm": 1.12211275100708, "tokens_seen": 71278592, "tokens_per_sec": 21015.634073019366, "vram_gb": 5.521900544, "lambda": 0.2996521483485897, "lambda_min": 0.09103423357009888, "lambda_max": 0.6210977435112, "S_mean": 0.2555338541666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.453229188919067, "grad_norm": 0.9899631142616272, "tokens_seen": 71442432, "tokens_per_sec": 21015.952857174143, "vram_gb": 5.521900544, "lambda": 0.299875491609176, "lambda_min": 0.09098111838102341, "lambda_max": 0.6217678785324097, "S_mean": 0.253753662109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.349034786224365, "grad_norm": 1.0645633935928345, "tokens_seen": 71606272, "tokens_per_sec": 21015.818098133204, "vram_gb": 5.521900544, "lambda": 0.30011247967680293, "lambda_min": 0.09097946435213089, "lambda_max": 0.6221939325332642, "S_mean": 0.2662760416666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.440057754516602, "grad_norm": 1.0785197019577026, "tokens_seen": 71770112, "tokens_per_sec": 21016.027163062936, "vram_gb": 5.521900544, "lambda": 0.30033106977740925, "lambda_min": 0.09092424064874649, "lambda_max": 0.6228283047676086, "S_mean": 0.2569376627604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.297578811645508, "grad_norm": 0.999235987663269, "tokens_seen": 71933952, "tokens_per_sec": 21016.024150140085, "vram_gb": 5.521900544, "lambda": 0.3005097374940912, "lambda_min": 0.09095786511898041, "lambda_max": 0.6233134865760803, "S_mean": 0.2702128092447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.467806816101074, "grad_norm": 1.0595163106918335, "tokens_seen": 72097792, "tokens_per_sec": 21016.24614075974, "vram_gb": 5.521900544, "lambda": 0.30064757478733856, "lambda_min": 0.09092407673597336, "lambda_max": 0.6239071488380432, "S_mean": 0.2763671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.4907591342926025, "grad_norm": 1.2539145946502686, "tokens_seen": 72261632, "tokens_per_sec": 21016.440625339314, "vram_gb": 5.521900544, "lambda": 0.3008455897991856, "lambda_min": 0.09097372740507126, "lambda_max": 0.6243095993995667, "S_mean": 0.2547505696614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.422304630279541, "grad_norm": 1.0338521003723145, "tokens_seen": 72425472, "tokens_per_sec": 21016.448525764165, "vram_gb": 5.521900544, "lambda": 0.30098267178982496, "lambda_min": 0.09092693775892258, "lambda_max": 0.6249551773071289, "S_mean": 0.270050048828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.357058525085449, "grad_norm": 1.0383259057998657, "tokens_seen": 72589312, "tokens_per_sec": 21016.603546292987, "vram_gb": 5.521900544, "lambda": 0.301154020242393, "lambda_min": 0.09085007011890411, "lambda_max": 0.6255798935890198, "S_mean": 0.2663777669270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.560805082321167, "grad_norm": 1.0867360830307007, "tokens_seen": 72753152, "tokens_per_sec": 21016.650017923905, "vram_gb": 5.521900544, "lambda": 0.30128964968025684, "lambda_min": 0.09086133539676666, "lambda_max": 0.6262954473495483, "S_mean": 0.2614847819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.404013156890869, "grad_norm": 1.0651506185531616, "tokens_seen": 72916992, "tokens_per_sec": 21016.971050284566, "vram_gb": 5.521900544, "lambda": 0.3014502314229806, "lambda_min": 0.0908547043800354, "lambda_max": 0.6269681453704834, "S_mean": 0.251983642578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.423370361328125, "grad_norm": 1.1146165132522583, "tokens_seen": 73080832, "tokens_per_sec": 21016.80526151913, "vram_gb": 5.521900544, "lambda": 0.30163124793519575, "lambda_min": 0.0908055454492569, "lambda_max": 0.6273400187492371, "S_mean": 0.259857177734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.2796502113342285, "grad_norm": 1.0609861612319946, "tokens_seen": 73244672, "tokens_per_sec": 21017.02648502196, "vram_gb": 5.521900544, "lambda": 0.3018195467690627, "lambda_min": 0.09078320860862732, "lambda_max": 0.6277734041213989, "S_mean": 0.2573445638020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.450437784194946, "grad_norm": 1.056638479232788, "tokens_seen": 73408512, "tokens_per_sec": 21017.135185245148, "vram_gb": 5.521900544, "lambda": 0.3020295671497782, "lambda_min": 0.09081091731786728, "lambda_max": 0.628571629524231, "S_mean": 0.2624104817708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.441889047622681, "grad_norm": 1.1060032844543457, "tokens_seen": 73572352, "tokens_per_sec": 21017.562609770932, "vram_gb": 5.521900544, "lambda": 0.30221302186449367, "lambda_min": 0.09081564098596573, "lambda_max": 0.6291868090629578, "S_mean": 0.2604777018229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.535930156707764, "grad_norm": 1.013267159461975, "tokens_seen": 73736192, "tokens_per_sec": 21017.961602974785, "vram_gb": 5.521900544, "lambda": 0.3023571229229371, "lambda_min": 0.0907822772860527, "lambda_max": 0.6297630071640015, "S_mean": 0.2642415364583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.472919893264771, "val_ppl": 87.61216807526922, "lambda": 0.3023621380950014, "lambda_min": 0.0907779410481453, "lambda_max": 0.629792332649231, "lambda_per_layer": [0.17940296232700348, 0.13610713183879852, 0.14606554806232452, 0.146504744887352, 0.16157791018486023, 0.1355917602777481, 0.12487729638814926, 0.10631246864795685, 0.0914873406291008, 0.0907779410481453, 0.17883576452732086, 0.2818849980831146, 0.4463350474834442, 0.4721808433532715, 0.40860074758529663, 0.41843679547309875, 0.5453203320503235, 0.3816527724266052, 0.6054390072822571, 0.629792332649231, 0.3653874099254608, 0.5133309960365295, 0.39472582936286926, 0.2960633337497711], "S_mean": 0.2566630045572917, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.359071493148804, "grad_norm": 0.9856134653091431, "tokens_seen": 73900032, "tokens_per_sec": 21009.70170615016, "vram_gb": 5.521900544, "lambda": 0.30248314794152975, "lambda_min": 0.09073380380868912, "lambda_max": 0.6302327513694763, "S_mean": 0.2649943033854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.451601505279541, "grad_norm": 1.0875041484832764, "tokens_seen": 74063872, "tokens_per_sec": 21009.649692339775, "vram_gb": 5.521900544, "lambda": 0.30262507932881516, "lambda_min": 0.09073201566934586, "lambda_max": 0.6303890943527222, "S_mean": 0.2573750813802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.234619617462158, "grad_norm": 0.9932016730308533, "tokens_seen": 74227712, "tokens_per_sec": 21009.676969902506, "vram_gb": 5.521900544, "lambda": 0.3027652424449722, "lambda_min": 0.09075749665498734, "lambda_max": 0.6309807300567627, "S_mean": 0.2615458170572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.4063215255737305, "grad_norm": 1.0175071954727173, "tokens_seen": 74391552, "tokens_per_sec": 21009.988561329483, "vram_gb": 5.521900544, "lambda": 0.3029285604134202, "lambda_min": 0.09076560288667679, "lambda_max": 0.6315534710884094, "S_mean": 0.25323486328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.511996269226074, "grad_norm": 1.1415444612503052, "tokens_seen": 74555392, "tokens_per_sec": 21010.054552477028, "vram_gb": 5.521900544, "lambda": 0.30311941479643184, "lambda_min": 0.09078901261091232, "lambda_max": 0.6322629451751709, "S_mean": 0.2560831705729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.466564178466797, "grad_norm": 1.0588829517364502, "tokens_seen": 74719232, "tokens_per_sec": 21009.827627637304, "vram_gb": 5.521900544, "lambda": 0.30330307222902775, "lambda_min": 0.09080127626657486, "lambda_max": 0.6328588128089905, "S_mean": 0.2646077473958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.246944904327393, "grad_norm": 1.0413317680358887, "tokens_seen": 74883072, "tokens_per_sec": 21009.766591743188, "vram_gb": 5.521900544, "lambda": 0.3034737982476751, "lambda_min": 0.09079450368881226, "lambda_max": 0.6336432695388794, "S_mean": 0.2537943522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.539880990982056, "grad_norm": 1.2424495220184326, "tokens_seen": 75046912, "tokens_per_sec": 21009.697206852376, "vram_gb": 5.521900544, "lambda": 0.3036585608497262, "lambda_min": 0.0908125564455986, "lambda_max": 0.6342403888702393, "S_mean": 0.2676188151041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.324896574020386, "grad_norm": 1.0434187650680542, "tokens_seen": 75210752, "tokens_per_sec": 21009.853702448738, "vram_gb": 5.521900544, "lambda": 0.30384047018984955, "lambda_min": 0.09079497307538986, "lambda_max": 0.6347796320915222, "S_mean": 0.2695515950520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.438380241394043, "grad_norm": 1.0869625806808472, "tokens_seen": 75374592, "tokens_per_sec": 21009.581718948502, "vram_gb": 5.521900544, "lambda": 0.3040421061838667, "lambda_min": 0.09072273224592209, "lambda_max": 0.6353920698165894, "S_mean": 0.251129150390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.383284091949463, "grad_norm": 1.1489222049713135, "tokens_seen": 75538432, "tokens_per_sec": 21009.769528556717, "vram_gb": 5.521900544, "lambda": 0.3042104837174217, "lambda_min": 0.09067549556493759, "lambda_max": 0.6358035206794739, "S_mean": 0.23979695638020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.423556089401245, "grad_norm": 1.1090202331542969, "tokens_seen": 75702272, "tokens_per_sec": 21009.944354468094, "vram_gb": 5.521900544, "lambda": 0.3043612226222952, "lambda_min": 0.0906355157494545, "lambda_max": 0.6361938714981079, "S_mean": 0.2534993489583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.442939043045044, "grad_norm": 1.0801146030426025, "tokens_seen": 75866112, "tokens_per_sec": 21009.9711508792, "vram_gb": 5.521900544, "lambda": 0.304512751288712, "lambda_min": 0.09061886370182037, "lambda_max": 0.6367466449737549, "S_mean": 0.2586568196614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.405155658721924, "grad_norm": 0.996555507183075, "tokens_seen": 76029952, "tokens_per_sec": 21010.484093086638, "vram_gb": 5.521900544, "lambda": 0.3046166331817706, "lambda_min": 0.09056023508310318, "lambda_max": 0.6371285319328308, "S_mean": 0.2732340494791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.311236143112183, "grad_norm": 1.198142647743225, "tokens_seen": 76193792, "tokens_per_sec": 21010.429454649344, "vram_gb": 5.521900544, "lambda": 0.30475779840101797, "lambda_min": 0.0905095785856247, "lambda_max": 0.637569785118103, "S_mean": 0.25616455078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.535049676895142, "grad_norm": 1.0942150354385376, "tokens_seen": 76357632, "tokens_per_sec": 21010.878434271857, "vram_gb": 5.521900544, "lambda": 0.3049195023874442, "lambda_min": 0.09049422293901443, "lambda_max": 0.6379281282424927, "S_mean": 0.24788411458333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.532470703125, "grad_norm": 1.0733445882797241, "tokens_seen": 76521472, "tokens_per_sec": 21010.921182896753, "vram_gb": 5.521900544, "lambda": 0.30508960566173, "lambda_min": 0.09053972363471985, "lambda_max": 0.6382052302360535, "S_mean": 0.25372314453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.3559136390686035, "grad_norm": 1.065471887588501, "tokens_seen": 76685312, "tokens_per_sec": 21011.27292963912, "vram_gb": 5.521900544, "lambda": 0.3052930148939292, "lambda_min": 0.09057337045669556, "lambda_max": 0.6387513279914856, "S_mean": 0.263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.436573028564453, "grad_norm": 1.0626764297485352, "tokens_seen": 76849152, "tokens_per_sec": 21011.562353941103, "vram_gb": 5.521900544, "lambda": 0.30548496078699827, "lambda_min": 0.09062838554382324, "lambda_max": 0.6392920017242432, "S_mean": 0.2574462890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.4015398025512695, "grad_norm": 1.1321676969528198, "tokens_seen": 77012992, "tokens_per_sec": 21011.917104366326, "vram_gb": 5.521900544, "lambda": 0.3056222964078188, "lambda_min": 0.09062869846820831, "lambda_max": 0.6396175026893616, "S_mean": 0.2517598470052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.396230697631836, "grad_norm": 1.11697256565094, "tokens_seen": 77176832, "tokens_per_sec": 21012.13306732912, "vram_gb": 5.521900544, "lambda": 0.30575343190381926, "lambda_min": 0.09053885191679001, "lambda_max": 0.6400315761566162, "S_mean": 0.256591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.2596964836120605, "grad_norm": 1.015498399734497, "tokens_seen": 77340672, "tokens_per_sec": 21012.274860219186, "vram_gb": 5.521900544, "lambda": 0.3058700455973546, "lambda_min": 0.09054822474718094, "lambda_max": 0.6404430270195007, "S_mean": 0.2828776041666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.4924235343933105, "grad_norm": 1.0289589166641235, "tokens_seen": 77504512, "tokens_per_sec": 21012.612857094733, "vram_gb": 5.521900544, "lambda": 0.3059997158125043, "lambda_min": 0.09053412824869156, "lambda_max": 0.6407247185707092, "S_mean": 0.2561442057291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.549541711807251, "grad_norm": 1.1535471677780151, "tokens_seen": 77668352, "tokens_per_sec": 21012.72410733036, "vram_gb": 5.521900544, "lambda": 0.3061444725220402, "lambda_min": 0.09055489301681519, "lambda_max": 0.6410741209983826, "S_mean": 0.2522074381510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.289504528045654, "grad_norm": 0.9588766098022461, "tokens_seen": 77832192, "tokens_per_sec": 21013.061082126693, "vram_gb": 5.521900544, "lambda": 0.306277137560149, "lambda_min": 0.09059252589941025, "lambda_max": 0.6416912078857422, "S_mean": 0.2578938802083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.413730621337891, "grad_norm": 1.239914059638977, "tokens_seen": 77996032, "tokens_per_sec": 21013.182337492635, "vram_gb": 5.521900544, "lambda": 0.3063956005498767, "lambda_min": 0.0905374065041542, "lambda_max": 0.641960859298706, "S_mean": 0.264892578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.357296705245972, "grad_norm": 1.1002476215362549, "tokens_seen": 78159872, "tokens_per_sec": 21013.29654655295, "vram_gb": 5.521900544, "lambda": 0.3064803583547473, "lambda_min": 0.09049995988607407, "lambda_max": 0.6423109173774719, "S_mean": 0.2613525390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.292205095291138, "grad_norm": 1.0067883729934692, "tokens_seen": 78323712, "tokens_per_sec": 21013.486399842182, "vram_gb": 5.521900544, "lambda": 0.30660199870665866, "lambda_min": 0.0905127078294754, "lambda_max": 0.6429423689842224, "S_mean": 0.2628173828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.394137859344482, "grad_norm": 1.0970748662948608, "tokens_seen": 78487552, "tokens_per_sec": 21013.621587236776, "vram_gb": 5.521900544, "lambda": 0.3067548271889488, "lambda_min": 0.09051883220672607, "lambda_max": 0.6434029340744019, "S_mean": 0.2520548502604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.591259479522705, "grad_norm": 1.0629637241363525, "tokens_seen": 78651392, "tokens_per_sec": 21013.956550776144, "vram_gb": 5.521900544, "lambda": 0.306904342956841, "lambda_min": 0.09053710103034973, "lambda_max": 0.6437731981277466, "S_mean": 0.2607828776041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.424809217453003, "grad_norm": 1.081278920173645, "tokens_seen": 78815232, "tokens_per_sec": 21013.840944553685, "vram_gb": 5.521900544, "lambda": 0.307048549875617, "lambda_min": 0.09049452096223831, "lambda_max": 0.6445052027702332, "S_mean": 0.2649739583333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.24094295501709, "grad_norm": 1.0974979400634766, "tokens_seen": 78979072, "tokens_per_sec": 21014.220970367125, "vram_gb": 5.521900544, "lambda": 0.3071735060463349, "lambda_min": 0.09047943353652954, "lambda_max": 0.6449143886566162, "S_mean": 0.2715962727864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.495712757110596, "grad_norm": 1.079502820968628, "tokens_seen": 79142912, "tokens_per_sec": 21014.164835604806, "vram_gb": 5.521900544, "lambda": 0.3072992271433274, "lambda_min": 0.09048933535814285, "lambda_max": 0.6453267931938171, "S_mean": 0.26312255859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.412468910217285, "grad_norm": 1.205015778541565, "tokens_seen": 79306752, "tokens_per_sec": 21014.60429330312, "vram_gb": 5.521900544, "lambda": 0.3074504438166817, "lambda_min": 0.09051419049501419, "lambda_max": 0.6458309292793274, "S_mean": 0.25897216796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.253758907318115, "grad_norm": 0.9830899834632874, "tokens_seen": 79470592, "tokens_per_sec": 21014.789019846638, "vram_gb": 5.521900544, "lambda": 0.3075818444291751, "lambda_min": 0.09050243347883224, "lambda_max": 0.6463115215301514, "S_mean": 0.2632039388020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.347456693649292, "grad_norm": 1.0621256828308105, "tokens_seen": 79634432, "tokens_per_sec": 21014.862078734463, "vram_gb": 5.521900544, "lambda": 0.30769234523177147, "lambda_min": 0.09047295898199081, "lambda_max": 0.6468275189399719, "S_mean": 0.2628987630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.326702833175659, "grad_norm": 1.0793362855911255, "tokens_seen": 79798272, "tokens_per_sec": 21014.96980571042, "vram_gb": 5.521900544, "lambda": 0.307798162723581, "lambda_min": 0.09049861878156662, "lambda_max": 0.6473613381385803, "S_mean": 0.25201416015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.395983695983887, "grad_norm": 1.1031831502914429, "tokens_seen": 79962112, "tokens_per_sec": 21014.748006974718, "vram_gb": 5.521900544, "lambda": 0.3079418533792098, "lambda_min": 0.09054816514253616, "lambda_max": 0.6476631164550781, "S_mean": 0.24766031901041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.32209038734436, "grad_norm": 0.9932079315185547, "tokens_seen": 80125952, "tokens_per_sec": 21014.1823247716, "vram_gb": 5.521900544, "lambda": 0.30808891107638675, "lambda_min": 0.09058065712451935, "lambda_max": 0.648209810256958, "S_mean": 0.263427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.3388872146606445, "grad_norm": 1.164433479309082, "tokens_seen": 80289792, "tokens_per_sec": 21014.112390906907, "vram_gb": 5.521900544, "lambda": 0.30822421175738174, "lambda_min": 0.09055330604314804, "lambda_max": 0.6486760973930359, "S_mean": 0.2584431966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.3095502853393555, "grad_norm": 1.0202113389968872, "tokens_seen": 80453632, "tokens_per_sec": 21014.116826214184, "vram_gb": 5.521900544, "lambda": 0.30835101163635653, "lambda_min": 0.09046956151723862, "lambda_max": 0.6489689946174622, "S_mean": 0.2639973958333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.337835073471069, "grad_norm": 1.0145165920257568, "tokens_seen": 80617472, "tokens_per_sec": 21013.740657917602, "vram_gb": 5.521900544, "lambda": 0.3084780365849535, "lambda_min": 0.09046155959367752, "lambda_max": 0.6493476629257202, "S_mean": 0.2621256510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.324390649795532, "grad_norm": 1.1227831840515137, "tokens_seen": 80781312, "tokens_per_sec": 21013.467900119238, "vram_gb": 5.521900544, "lambda": 0.30861766388018924, "lambda_min": 0.09046155959367752, "lambda_max": 0.6498310565948486, "S_mean": 0.2591756184895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.249878406524658, "grad_norm": 1.2997547388076782, "tokens_seen": 80945152, "tokens_per_sec": 21013.30518197213, "vram_gb": 5.521900544, "lambda": 0.3087437491243084, "lambda_min": 0.09048393368721008, "lambda_max": 0.6501836180686951, "S_mean": 0.2732035319010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.338538646697998, "grad_norm": 1.005505919456482, "tokens_seen": 81108992, "tokens_per_sec": 21013.50351337865, "vram_gb": 5.521900544, "lambda": 0.3088924406717221, "lambda_min": 0.09052668511867523, "lambda_max": 0.6505834460258484, "S_mean": 0.264923095703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.3346850872039795, "grad_norm": 1.3130378723144531, "tokens_seen": 81272832, "tokens_per_sec": 21013.88678709366, "vram_gb": 5.521900544, "lambda": 0.3090210898468892, "lambda_min": 0.09055572003126144, "lambda_max": 0.6509690284729004, "S_mean": 0.262115478515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.308052062988281, "grad_norm": 1.199553370475769, "tokens_seen": 81436672, "tokens_per_sec": 21014.046049351276, "vram_gb": 5.521900544, "lambda": 0.30910401574025553, "lambda_min": 0.09058305621147156, "lambda_max": 0.6512352228164673, "S_mean": 0.26068115234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.479057312011719, "grad_norm": 1.266324758529663, "tokens_seen": 81600512, "tokens_per_sec": 21014.55199555983, "vram_gb": 5.521900544, "lambda": 0.3092257206638654, "lambda_min": 0.09064317494630814, "lambda_max": 0.6515396237373352, "S_mean": 0.26239013671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.260561943054199, "grad_norm": 1.026857852935791, "tokens_seen": 81764352, "tokens_per_sec": 21014.692132519704, "vram_gb": 5.521900544, "lambda": 0.3093541075165073, "lambda_min": 0.09065032750368118, "lambda_max": 0.6519721150398254, "S_mean": 0.2569580078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.401309490203857, "grad_norm": 1.1736228466033936, "tokens_seen": 81928192, "tokens_per_sec": 21015.262875512384, "vram_gb": 5.521900544, "lambda": 0.30948216157654923, "lambda_min": 0.09063389897346497, "lambda_max": 0.6523592472076416, "S_mean": 0.2505391438802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.460418272018432, "val_ppl": 86.52369196794216, "lambda": 0.3094880363593499, "lambda_min": 0.0906360074877739, "lambda_max": 0.6523798704147339, "lambda_per_layer": [0.18442118167877197, 0.1360664814710617, 0.14700819551944733, 0.14770014584064484, 0.16342230141162872, 0.1359744518995285, 0.12500809133052826, 0.10720503330230713, 0.09147883206605911, 0.0906360074877739, 0.1798393279314041, 0.28368476033210754, 0.45143836736679077, 0.47971564531326294, 0.4160868227481842, 0.4262176752090454, 0.5640860199928284, 0.39370521903038025, 0.6269516944885254, 0.6523798704147339, 0.37844082713127136, 0.5335033535957336, 0.40608835220336914, 0.3066542148590088], "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.248322010040283, "grad_norm": 1.08829665184021, "tokens_seen": 82092032, "tokens_per_sec": 21008.057517640184, "vram_gb": 5.521900544, "lambda": 0.30958569298187893, "lambda_min": 0.09064339846372604, "lambda_max": 0.6526376605033875, "S_mean": 0.2772623697916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.369068384170532, "grad_norm": 1.2672309875488281, "tokens_seen": 82255872, "tokens_per_sec": 21008.19363310288, "vram_gb": 5.521900544, "lambda": 0.3096873393903176, "lambda_min": 0.09063556790351868, "lambda_max": 0.6529971957206726, "S_mean": 0.2667643229166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.252246379852295, "grad_norm": 1.1552975177764893, "tokens_seen": 82419712, "tokens_per_sec": 21008.578622393263, "vram_gb": 5.521900544, "lambda": 0.30981788008163375, "lambda_min": 0.09062237292528152, "lambda_max": 0.6533038020133972, "S_mean": 0.2514750162760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.190907955169678, "grad_norm": 1.001346230506897, "tokens_seen": 82583552, "tokens_per_sec": 21008.27954364075, "vram_gb": 5.521900544, "lambda": 0.3099231819311778, "lambda_min": 0.09061155468225479, "lambda_max": 0.6536694765090942, "S_mean": 0.24921671549479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.494091987609863, "grad_norm": 1.1181973218917847, "tokens_seen": 82747392, "tokens_per_sec": 21008.425819368837, "vram_gb": 5.521900544, "lambda": 0.3100415092582504, "lambda_min": 0.09059114754199982, "lambda_max": 0.6541681289672852, "S_mean": 0.250885009765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.397896766662598, "grad_norm": 1.1354132890701294, "tokens_seen": 82911232, "tokens_per_sec": 21008.481226845786, "vram_gb": 5.521900544, "lambda": 0.3101574620231986, "lambda_min": 0.09055840224027634, "lambda_max": 0.6544907093048096, "S_mean": 0.24224853515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.288987398147583, "grad_norm": 1.0862609148025513, "tokens_seen": 83075072, "tokens_per_sec": 21008.899793664394, "vram_gb": 5.521900544, "lambda": 0.3102905598158638, "lambda_min": 0.09053466469049454, "lambda_max": 0.6548537015914917, "S_mean": 0.2540690104166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.418850660324097, "grad_norm": 1.1580233573913574, "tokens_seen": 83238912, "tokens_per_sec": 21009.01582630868, "vram_gb": 5.521900544, "lambda": 0.3103799354285002, "lambda_min": 0.09054554253816605, "lambda_max": 0.6552268862724304, "S_mean": 0.2554524739583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.518718242645264, "grad_norm": 1.150994896888733, "tokens_seen": 83402752, "tokens_per_sec": 21009.322704936, "vram_gb": 5.521900544, "lambda": 0.31047465559095144, "lambda_min": 0.09055940806865692, "lambda_max": 0.6556981801986694, "S_mean": 0.2513936360677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.48742413520813, "grad_norm": 1.1720036268234253, "tokens_seen": 83566592, "tokens_per_sec": 21009.521101200644, "vram_gb": 5.521900544, "lambda": 0.3105418436850111, "lambda_min": 0.09060794115066528, "lambda_max": 0.6559842228889465, "S_mean": 0.2604166666666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.389825105667114, "grad_norm": 1.1887497901916504, "tokens_seen": 83730432, "tokens_per_sec": 21009.764078477743, "vram_gb": 5.521900544, "lambda": 0.3106403707837065, "lambda_min": 0.09061169624328613, "lambda_max": 0.6563870906829834, "S_mean": 0.262298583984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.260148763656616, "grad_norm": 1.1947226524353027, "tokens_seen": 83894272, "tokens_per_sec": 21009.91837130366, "vram_gb": 5.521900544, "lambda": 0.31079629343003035, "lambda_min": 0.09061965346336365, "lambda_max": 0.656865119934082, "S_mean": 0.25775146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.385068416595459, "grad_norm": 1.1582461595535278, "tokens_seen": 84058112, "tokens_per_sec": 21009.945511494698, "vram_gb": 5.521900544, "lambda": 0.3109354094291727, "lambda_min": 0.0906534418463707, "lambda_max": 0.6573477387428284, "S_mean": 0.2584431966145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.4094626903533936, "grad_norm": 1.330570101737976, "tokens_seen": 84221952, "tokens_per_sec": 21010.27489382301, "vram_gb": 5.521900544, "lambda": 0.3110424932092428, "lambda_min": 0.09067873656749725, "lambda_max": 0.6578167676925659, "S_mean": 0.253631591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.3012659549713135, "grad_norm": 1.260948657989502, "tokens_seen": 84385792, "tokens_per_sec": 21010.57445076721, "vram_gb": 5.521900544, "lambda": 0.3111277852828304, "lambda_min": 0.0907062366604805, "lambda_max": 0.6582003831863403, "S_mean": 0.2575480143229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.430909872055054, "grad_norm": 1.067106008529663, "tokens_seen": 84549632, "tokens_per_sec": 21010.733693285907, "vram_gb": 5.521900544, "lambda": 0.3112040860578418, "lambda_min": 0.09070228040218353, "lambda_max": 0.6585326194763184, "S_mean": 0.252197265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.258019685745239, "grad_norm": 1.0224477052688599, "tokens_seen": 84713472, "tokens_per_sec": 21010.468699829264, "vram_gb": 5.521900544, "lambda": 0.3113103797659278, "lambda_min": 0.09070995450019836, "lambda_max": 0.6587232947349548, "S_mean": 0.2582194010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.290006637573242, "grad_norm": 1.078150987625122, "tokens_seen": 84877312, "tokens_per_sec": 21010.4085434663, "vram_gb": 5.521900544, "lambda": 0.31141173746436834, "lambda_min": 0.09071364998817444, "lambda_max": 0.6587328314781189, "S_mean": 0.244110107421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.444975137710571, "grad_norm": 1.1123543977737427, "tokens_seen": 85041152, "tokens_per_sec": 21009.86029908047, "vram_gb": 5.521900544, "lambda": 0.3114861762151122, "lambda_min": 0.09073902666568756, "lambda_max": 0.6589876413345337, "S_mean": 0.2611897786458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.392656564712524, "grad_norm": 1.0964356660842896, "tokens_seen": 85204992, "tokens_per_sec": 21009.9416634936, "vram_gb": 5.521900544, "lambda": 0.31158564829578, "lambda_min": 0.09074384719133377, "lambda_max": 0.659325897693634, "S_mean": 0.2626749674479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.237177610397339, "grad_norm": 1.1693003177642822, "tokens_seen": 85368832, "tokens_per_sec": 21010.043803346052, "vram_gb": 5.521900544, "lambda": 0.3116895779967308, "lambda_min": 0.09072744846343994, "lambda_max": 0.6595057249069214, "S_mean": 0.2641194661458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.240571022033691, "grad_norm": 1.1431149244308472, "tokens_seen": 85532672, "tokens_per_sec": 21009.896155895636, "vram_gb": 5.521900544, "lambda": 0.3118029907345772, "lambda_min": 0.09069281816482544, "lambda_max": 0.6599375009536743, "S_mean": 0.2620849609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.45021653175354, "grad_norm": 1.2155388593673706, "tokens_seen": 85696512, "tokens_per_sec": 21009.971038974825, "vram_gb": 5.521900544, "lambda": 0.3119182111695409, "lambda_min": 0.0907241553068161, "lambda_max": 0.6603829264640808, "S_mean": 0.2655232747395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.560302734375, "grad_norm": 1.1517012119293213, "tokens_seen": 85860352, "tokens_per_sec": 21009.673233994483, "vram_gb": 5.521900544, "lambda": 0.31201620027422905, "lambda_min": 0.09074706584215164, "lambda_max": 0.660710871219635, "S_mean": 0.24944051106770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.389276504516602, "grad_norm": 1.180543303489685, "tokens_seen": 86024192, "tokens_per_sec": 21009.553330682145, "vram_gb": 5.521900544, "lambda": 0.3121182657778263, "lambda_min": 0.09074477851390839, "lambda_max": 0.6609765291213989, "S_mean": 0.253692626953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.302953481674194, "grad_norm": 1.0260127782821655, "tokens_seen": 86188032, "tokens_per_sec": 21009.639476620858, "vram_gb": 5.521900544, "lambda": 0.3122436326617996, "lambda_min": 0.09075695276260376, "lambda_max": 0.6614069938659668, "S_mean": 0.24812825520833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.282255411148071, "grad_norm": 1.081708312034607, "tokens_seen": 86351872, "tokens_per_sec": 21009.38083159472, "vram_gb": 5.521900544, "lambda": 0.312337056423227, "lambda_min": 0.09079595655202866, "lambda_max": 0.6616925597190857, "S_mean": 0.27532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.2242560386657715, "grad_norm": 1.142867088317871, "tokens_seen": 86515712, "tokens_per_sec": 21009.134890911162, "vram_gb": 5.521900544, "lambda": 0.3124554868166645, "lambda_min": 0.09084497392177582, "lambda_max": 0.6621188521385193, "S_mean": 0.2674357096354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.4395201206207275, "grad_norm": 1.113214373588562, "tokens_seen": 86679552, "tokens_per_sec": 21008.825464573347, "vram_gb": 5.521900544, "lambda": 0.31256064772605896, "lambda_min": 0.09087136387825012, "lambda_max": 0.6624661684036255, "S_mean": 0.2571614583333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.234490156173706, "grad_norm": 1.0873477458953857, "tokens_seen": 86843392, "tokens_per_sec": 21009.390529820223, "vram_gb": 5.521900544, "lambda": 0.3126483193288247, "lambda_min": 0.09089252352714539, "lambda_max": 0.662671685218811, "S_mean": 0.2605387369791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.395570278167725, "grad_norm": 1.143418312072754, "tokens_seen": 87007232, "tokens_per_sec": 21009.58772386811, "vram_gb": 5.521900544, "lambda": 0.31275580171495676, "lambda_min": 0.09089116007089615, "lambda_max": 0.6629464030265808, "S_mean": 0.2622477213541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.245443820953369, "grad_norm": 1.0828033685684204, "tokens_seen": 87171072, "tokens_per_sec": 21009.844162197493, "vram_gb": 5.521900544, "lambda": 0.3128508732964595, "lambda_min": 0.09089461714029312, "lambda_max": 0.6633025407791138, "S_mean": 0.2958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.322468042373657, "grad_norm": 1.1329258680343628, "tokens_seen": 87334912, "tokens_per_sec": 21009.887825017096, "vram_gb": 5.521900544, "lambda": 0.3129451849187414, "lambda_min": 0.09090516716241837, "lambda_max": 0.6636455059051514, "S_mean": 0.264312744140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.386939764022827, "grad_norm": 1.1842739582061768, "tokens_seen": 87498752, "tokens_per_sec": 21009.820904734883, "vram_gb": 5.521900544, "lambda": 0.3130538597082098, "lambda_min": 0.09091970324516296, "lambda_max": 0.6640622615814209, "S_mean": 0.24125162760416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.387023687362671, "grad_norm": 1.1061763763427734, "tokens_seen": 87662592, "tokens_per_sec": 21010.049660803343, "vram_gb": 5.521900544, "lambda": 0.3131609366585811, "lambda_min": 0.09096468240022659, "lambda_max": 0.6643847227096558, "S_mean": 0.254486083984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.3107216358184814, "grad_norm": 1.120513677597046, "tokens_seen": 87826432, "tokens_per_sec": 21009.853941303227, "vram_gb": 5.521900544, "lambda": 0.31326413433998823, "lambda_min": 0.09099124372005463, "lambda_max": 0.6646866202354431, "S_mean": 0.2666015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.254167556762695, "grad_norm": 1.1249721050262451, "tokens_seen": 87990272, "tokens_per_sec": 21009.965229521156, "vram_gb": 5.521900544, "lambda": 0.31335595312217873, "lambda_min": 0.09101799875497818, "lambda_max": 0.6650918126106262, "S_mean": 0.25653076171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.343533277511597, "grad_norm": 1.2284204959869385, "tokens_seen": 88154112, "tokens_per_sec": 21009.969759270585, "vram_gb": 5.521900544, "lambda": 0.31345368331919116, "lambda_min": 0.09102614969015121, "lambda_max": 0.665422260761261, "S_mean": 0.24334716796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.341678857803345, "grad_norm": 1.2333505153656006, "tokens_seen": 88317952, "tokens_per_sec": 21010.287792882464, "vram_gb": 5.521900544, "lambda": 0.3135839095339179, "lambda_min": 0.09103601425886154, "lambda_max": 0.6658121347427368, "S_mean": 0.2678426106770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.378147840499878, "grad_norm": 1.3539855480194092, "tokens_seen": 88481792, "tokens_per_sec": 21010.25123616614, "vram_gb": 5.521900544, "lambda": 0.3136873592933019, "lambda_min": 0.0909910798072815, "lambda_max": 0.6661390662193298, "S_mean": 0.24616495768229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.309100866317749, "grad_norm": 1.0880519151687622, "tokens_seen": 88645632, "tokens_per_sec": 21010.42490026534, "vram_gb": 5.521900544, "lambda": 0.313779662673672, "lambda_min": 0.09099536389112473, "lambda_max": 0.6663738489151001, "S_mean": 0.24448649088541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.470162868499756, "grad_norm": 1.2039381265640259, "tokens_seen": 88809472, "tokens_per_sec": 21010.839631842668, "vram_gb": 5.521900544, "lambda": 0.3138764553392927, "lambda_min": 0.0910249799489975, "lambda_max": 0.6666703820228577, "S_mean": 0.2650960286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.412304401397705, "grad_norm": 1.1261215209960938, "tokens_seen": 88973312, "tokens_per_sec": 21011.178528296106, "vram_gb": 5.521900544, "lambda": 0.31396608023593825, "lambda_min": 0.0909997895359993, "lambda_max": 0.6668974757194519, "S_mean": 0.2543741861979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.4039225578308105, "grad_norm": 1.2168174982070923, "tokens_seen": 89137152, "tokens_per_sec": 21011.57502127483, "vram_gb": 5.521900544, "lambda": 0.3140347022563219, "lambda_min": 0.09097278863191605, "lambda_max": 0.6671070456504822, "S_mean": 0.25469970703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.279214143753052, "grad_norm": 1.1881283521652222, "tokens_seen": 89300992, "tokens_per_sec": 21011.430830684025, "vram_gb": 5.521900544, "lambda": 0.31412109918892384, "lambda_min": 0.09096789360046387, "lambda_max": 0.667399525642395, "S_mean": 0.2571207682291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.160165548324585, "grad_norm": 1.1707791090011597, "tokens_seen": 89464832, "tokens_per_sec": 21011.799476814722, "vram_gb": 5.521900544, "lambda": 0.314202181994915, "lambda_min": 0.09095769375562668, "lambda_max": 0.6676632165908813, "S_mean": 0.2552693684895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.432069540023804, "grad_norm": 1.2870802879333496, "tokens_seen": 89628672, "tokens_per_sec": 21011.934499526174, "vram_gb": 5.521900544, "lambda": 0.3143148592983683, "lambda_min": 0.09095528721809387, "lambda_max": 0.6679899096488953, "S_mean": 0.262451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.355221509933472, "grad_norm": 1.1561702489852905, "tokens_seen": 89792512, "tokens_per_sec": 21012.009546704558, "vram_gb": 5.521900544, "lambda": 0.31440695965041715, "lambda_min": 0.09093385934829712, "lambda_max": 0.6683142185211182, "S_mean": 0.249481201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.3278868198394775, "grad_norm": 1.2074676752090454, "tokens_seen": 89956352, "tokens_per_sec": 21011.961837239905, "vram_gb": 5.521900544, "lambda": 0.3144907004510363, "lambda_min": 0.09092086553573608, "lambda_max": 0.6686156988143921, "S_mean": 0.24286905924479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.321343421936035, "grad_norm": 1.1537706851959229, "tokens_seen": 90120192, "tokens_per_sec": 21012.08119025137, "vram_gb": 5.521900544, "lambda": 0.3145651655892531, "lambda_min": 0.09090392291545868, "lambda_max": 0.6689766645431519, "S_mean": 0.246124267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.376419711112976, "val_ppl": 79.55270127184865, "lambda": 0.3145692798619469, "lambda_min": 0.090901680290699, "lambda_max": 0.6689931154251099, "lambda_per_layer": [0.18791738152503967, 0.13598321378231049, 0.14761361479759216, 0.1484573483467102, 0.16446442902088165, 0.13632920384407043, 0.12505309283733368, 0.10756701231002808, 0.09136304259300232, 0.090901680290699, 0.18052522838115692, 0.28531453013420105, 0.4537283182144165, 0.4847291111946106, 0.42128702998161316, 0.43242186307907104, 0.5767478942871094, 0.4031948149204254, 0.643115758895874, 0.6689931154251099, 0.38779890537261963, 0.5469785332679749, 0.41481542587280273, 0.31436216831207275], "S_mean": 0.2507527669270833, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.278783798217773, "grad_norm": 1.172613501548767, "tokens_seen": 90284032, "tokens_per_sec": 21005.363601647834, "vram_gb": 5.521900544, "lambda": 0.3146486474821965, "lambda_min": 0.09088821709156036, "lambda_max": 0.6693705916404724, "S_mean": 0.246612548828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.27755069732666, "grad_norm": 1.0286906957626343, "tokens_seen": 90447872, "tokens_per_sec": 21004.974612444206, "vram_gb": 5.521900544, "lambda": 0.31471580856790143, "lambda_min": 0.09087686240673065, "lambda_max": 0.6696951985359192, "S_mean": 0.2502034505208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.408415079116821, "grad_norm": 1.1162391901016235, "tokens_seen": 90611712, "tokens_per_sec": 21005.10096410183, "vram_gb": 5.521900544, "lambda": 0.3147904959817727, "lambda_min": 0.09087686240673065, "lambda_max": 0.6698467135429382, "S_mean": 0.270416259765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.143553256988525, "grad_norm": 1.0937373638153076, "tokens_seen": 90775552, "tokens_per_sec": 21004.731674515697, "vram_gb": 5.521900544, "lambda": 0.3148776063074668, "lambda_min": 0.09087216109037399, "lambda_max": 0.6702193021774292, "S_mean": 0.2504170735677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.375275135040283, "grad_norm": 1.2702407836914062, "tokens_seen": 90939392, "tokens_per_sec": 21004.79975465728, "vram_gb": 5.521900544, "lambda": 0.314977222122252, "lambda_min": 0.09089438617229462, "lambda_max": 0.6705501079559326, "S_mean": 0.251678466796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.3086161613464355, "grad_norm": 1.022568941116333, "tokens_seen": 91103232, "tokens_per_sec": 21005.103828691415, "vram_gb": 5.521900544, "lambda": 0.3150577774892251, "lambda_min": 0.09088864922523499, "lambda_max": 0.6708915829658508, "S_mean": 0.26007080078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.327717065811157, "grad_norm": 1.1367379426956177, "tokens_seen": 91267072, "tokens_per_sec": 21005.080449730893, "vram_gb": 5.521900544, "lambda": 0.31513983476907015, "lambda_min": 0.09090350568294525, "lambda_max": 0.6711215376853943, "S_mean": 0.2611185709635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.365555286407471, "grad_norm": 1.137884259223938, "tokens_seen": 91430912, "tokens_per_sec": 21005.339064950163, "vram_gb": 5.521900544, "lambda": 0.31522727260986966, "lambda_min": 0.09090731292963028, "lambda_max": 0.6713612675666809, "S_mean": 0.24873860677083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.308859348297119, "grad_norm": 1.1793341636657715, "tokens_seen": 91594752, "tokens_per_sec": 21005.25803299444, "vram_gb": 5.521900544, "lambda": 0.31530104267100495, "lambda_min": 0.09090449661016464, "lambda_max": 0.6716160178184509, "S_mean": 0.2556864420572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.215060234069824, "grad_norm": 1.2259234189987183, "tokens_seen": 91758592, "tokens_per_sec": 21005.572766181747, "vram_gb": 5.521900544, "lambda": 0.3153944682950775, "lambda_min": 0.09089625626802444, "lambda_max": 0.6718760132789612, "S_mean": 0.24668375651041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.276715040206909, "grad_norm": 1.1623624563217163, "tokens_seen": 91922432, "tokens_per_sec": 21005.494123604498, "vram_gb": 5.521900544, "lambda": 0.31546003309388954, "lambda_min": 0.09088434278964996, "lambda_max": 0.6721566319465637, "S_mean": 0.2582804361979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.340242385864258, "grad_norm": 1.1056913137435913, "tokens_seen": 92086272, "tokens_per_sec": 21005.683885432212, "vram_gb": 5.521900544, "lambda": 0.3155176906536023, "lambda_min": 0.09089890867471695, "lambda_max": 0.6722506284713745, "S_mean": 0.24981689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.344062566757202, "grad_norm": 1.121964931488037, "tokens_seen": 92250112, "tokens_per_sec": 21005.79715898507, "vram_gb": 5.521900544, "lambda": 0.3155877587075035, "lambda_min": 0.09089583903551102, "lambda_max": 0.6725063920021057, "S_mean": 0.2605183919270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.286384105682373, "grad_norm": 1.2061535120010376, "tokens_seen": 92413952, "tokens_per_sec": 21005.56295624978, "vram_gb": 5.521900544, "lambda": 0.31567253079265356, "lambda_min": 0.09091680496931076, "lambda_max": 0.6728108525276184, "S_mean": 0.24595133463541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.282984972000122, "grad_norm": 1.2799370288848877, "tokens_seen": 92577792, "tokens_per_sec": 21005.753797906375, "vram_gb": 5.521900544, "lambda": 0.31575398209194344, "lambda_min": 0.09090029448270798, "lambda_max": 0.6731389164924622, "S_mean": 0.24754842122395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.312730312347412, "grad_norm": 1.1486560106277466, "tokens_seen": 92741632, "tokens_per_sec": 21005.731166631158, "vram_gb": 5.521900544, "lambda": 0.3158643962815404, "lambda_min": 0.09089198708534241, "lambda_max": 0.6734937429428101, "S_mean": 0.24879964192708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.36328649520874, "grad_norm": 1.1383323669433594, "tokens_seen": 92905472, "tokens_per_sec": 21006.040231579653, "vram_gb": 5.521900544, "lambda": 0.3159371443713705, "lambda_min": 0.09087160229682922, "lambda_max": 0.6737697124481201, "S_mean": 0.24571736653645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.480492830276489, "grad_norm": 1.2242822647094727, "tokens_seen": 93069312, "tokens_per_sec": 21006.255319874002, "vram_gb": 5.521900544, "lambda": 0.3159923854594429, "lambda_min": 0.09084329754114151, "lambda_max": 0.6739908456802368, "S_mean": 0.2530619303385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.310962677001953, "grad_norm": 1.21246337890625, "tokens_seen": 93233152, "tokens_per_sec": 21005.88337177079, "vram_gb": 5.521900544, "lambda": 0.3160481195275982, "lambda_min": 0.09082304686307907, "lambda_max": 0.6740941405296326, "S_mean": 0.246337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.388233661651611, "grad_norm": 1.216609239578247, "tokens_seen": 93396992, "tokens_per_sec": 21005.887486553005, "vram_gb": 5.521900544, "lambda": 0.3161244820803404, "lambda_min": 0.09080978482961655, "lambda_max": 0.6743593811988831, "S_mean": 0.2548014322916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.2950263023376465, "grad_norm": 1.1645256280899048, "tokens_seen": 93560832, "tokens_per_sec": 21005.808365615678, "vram_gb": 5.521900544, "lambda": 0.31620865128934383, "lambda_min": 0.09079728275537491, "lambda_max": 0.6745859980583191, "S_mean": 0.2570597330729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.337209463119507, "grad_norm": 1.1627212762832642, "tokens_seen": 93724672, "tokens_per_sec": 21006.14697996471, "vram_gb": 5.521900544, "lambda": 0.3162989563619097, "lambda_min": 0.09077713638544083, "lambda_max": 0.6750094890594482, "S_mean": 0.2528788248697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.3631463050842285, "grad_norm": 1.1351169347763062, "tokens_seen": 93888512, "tokens_per_sec": 21006.268227356955, "vram_gb": 5.521900544, "lambda": 0.3163876834635933, "lambda_min": 0.09077461063861847, "lambda_max": 0.6753219366073608, "S_mean": 0.25103759765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.362409830093384, "grad_norm": 1.30950129032135, "tokens_seen": 94052352, "tokens_per_sec": 21006.550373267968, "vram_gb": 5.521900544, "lambda": 0.3164501537879308, "lambda_min": 0.09077653288841248, "lambda_max": 0.6755263209342957, "S_mean": 0.2519938151041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.265178203582764, "grad_norm": 1.217068076133728, "tokens_seen": 94216192, "tokens_per_sec": 21006.748844231588, "vram_gb": 5.521900544, "lambda": 0.3165199737995863, "lambda_min": 0.09078706055879593, "lambda_max": 0.6757338643074036, "S_mean": 0.257415771484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.181073069572449, "grad_norm": 1.1469768285751343, "tokens_seen": 94380032, "tokens_per_sec": 21006.743643077934, "vram_gb": 5.521900544, "lambda": 0.316603130971392, "lambda_min": 0.09079829603433609, "lambda_max": 0.6759692430496216, "S_mean": 0.2532450358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.426462650299072, "grad_norm": 1.122361183166504, "tokens_seen": 94543872, "tokens_per_sec": 21007.036195171313, "vram_gb": 5.521900544, "lambda": 0.31666958673546713, "lambda_min": 0.09077642858028412, "lambda_max": 0.6761515736579895, "S_mean": 0.24757893880208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.3365654945373535, "grad_norm": 1.3674262762069702, "tokens_seen": 94707712, "tokens_per_sec": 21007.039338828505, "vram_gb": 5.521900544, "lambda": 0.3167252391576767, "lambda_min": 0.09077029675245285, "lambda_max": 0.6763157248497009, "S_mean": 0.24936930338541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.199811697006226, "grad_norm": 1.1276873350143433, "tokens_seen": 94871552, "tokens_per_sec": 21006.964866998664, "vram_gb": 5.521900544, "lambda": 0.3167978146423896, "lambda_min": 0.09074268490076065, "lambda_max": 0.6765865087509155, "S_mean": 0.2504984537760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.332968235015869, "grad_norm": 1.1634502410888672, "tokens_seen": 95035392, "tokens_per_sec": 21007.078776622828, "vram_gb": 5.521900544, "lambda": 0.31688616580019396, "lambda_min": 0.09070811420679092, "lambda_max": 0.6768179535865784, "S_mean": 0.2559814453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.274101734161377, "grad_norm": 1.1338293552398682, "tokens_seen": 95199232, "tokens_per_sec": 21006.988012472855, "vram_gb": 5.521900544, "lambda": 0.3169640752797325, "lambda_min": 0.09069253504276276, "lambda_max": 0.6769680976867676, "S_mean": 0.2540486653645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.337565898895264, "grad_norm": 1.1844192743301392, "tokens_seen": 95363072, "tokens_per_sec": 21007.24593460162, "vram_gb": 5.521900544, "lambda": 0.3170490333189567, "lambda_min": 0.0907098725438118, "lambda_max": 0.6772609353065491, "S_mean": 0.2526957194010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.4342803955078125, "grad_norm": 1.1962157487869263, "tokens_seen": 95526912, "tokens_per_sec": 21007.398631122498, "vram_gb": 5.521900544, "lambda": 0.31712669506669044, "lambda_min": 0.09074773639440536, "lambda_max": 0.6774417757987976, "S_mean": 0.2518208821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.209367036819458, "grad_norm": 1.3436561822891235, "tokens_seen": 95690752, "tokens_per_sec": 21007.52586555627, "vram_gb": 5.521900544, "lambda": 0.31721649784594774, "lambda_min": 0.0907704308629036, "lambda_max": 0.6777108907699585, "S_mean": 0.272216796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.355717897415161, "grad_norm": 1.1497833728790283, "tokens_seen": 95854592, "tokens_per_sec": 21007.44172547492, "vram_gb": 5.521900544, "lambda": 0.31729846944411594, "lambda_min": 0.0907956063747406, "lambda_max": 0.6780676245689392, "S_mean": 0.245452880859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.283484935760498, "grad_norm": 1.3010311126708984, "tokens_seen": 96018432, "tokens_per_sec": 21007.462948187025, "vram_gb": 5.521900544, "lambda": 0.3173710924262802, "lambda_min": 0.09079790860414505, "lambda_max": 0.6783590316772461, "S_mean": 0.25164794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.331711769104004, "grad_norm": 1.2383618354797363, "tokens_seen": 96182272, "tokens_per_sec": 21007.583566617675, "vram_gb": 5.521900544, "lambda": 0.317448900702099, "lambda_min": 0.09079591929912567, "lambda_max": 0.6785616874694824, "S_mean": 0.2563883463541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.306972980499268, "grad_norm": 1.1652560234069824, "tokens_seen": 96346112, "tokens_per_sec": 21007.82244900086, "vram_gb": 5.521900544, "lambda": 0.31751856425156194, "lambda_min": 0.09078621119260788, "lambda_max": 0.6787056922912598, "S_mean": 0.253387451171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.3687968254089355, "grad_norm": 1.2494239807128906, "tokens_seen": 96509952, "tokens_per_sec": 21008.066533182875, "vram_gb": 5.521900544, "lambda": 0.31759327184408903, "lambda_min": 0.0907721221446991, "lambda_max": 0.6789132356643677, "S_mean": 0.2568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.312378644943237, "grad_norm": 1.231900691986084, "tokens_seen": 96673792, "tokens_per_sec": 21008.30541787792, "vram_gb": 5.521900544, "lambda": 0.31768871036668617, "lambda_min": 0.09079183638095856, "lambda_max": 0.6792068481445312, "S_mean": 0.24405924479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.352414608001709, "grad_norm": 1.300783634185791, "tokens_seen": 96837632, "tokens_per_sec": 21008.328586207506, "vram_gb": 5.521900544, "lambda": 0.31775357325871784, "lambda_min": 0.09077204018831253, "lambda_max": 0.6794055104255676, "S_mean": 0.2523193359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.244914770126343, "grad_norm": 1.1544371843338013, "tokens_seen": 97001472, "tokens_per_sec": 21008.7770908141, "vram_gb": 5.521900544, "lambda": 0.31781917562087375, "lambda_min": 0.0907636359333992, "lambda_max": 0.6796598434448242, "S_mean": 0.2618408203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.266395092010498, "grad_norm": 1.156166434288025, "tokens_seen": 97165312, "tokens_per_sec": 21009.015606959678, "vram_gb": 5.521900544, "lambda": 0.3178585882609089, "lambda_min": 0.0907645896077156, "lambda_max": 0.6798047423362732, "S_mean": 0.2564290364583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.463529109954834, "grad_norm": 1.3000210523605347, "tokens_seen": 97329152, "tokens_per_sec": 21009.07845623822, "vram_gb": 5.521900544, "lambda": 0.31791513971984386, "lambda_min": 0.09077305346727371, "lambda_max": 0.6801166534423828, "S_mean": 0.246337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.284616470336914, "grad_norm": 1.3212281465530396, "tokens_seen": 97492992, "tokens_per_sec": 21008.935319433906, "vram_gb": 5.521900544, "lambda": 0.3179887297252814, "lambda_min": 0.09077386558055878, "lambda_max": 0.6804585456848145, "S_mean": 0.25286865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.262445688247681, "grad_norm": 1.225072979927063, "tokens_seen": 97656832, "tokens_per_sec": 21008.95036305537, "vram_gb": 5.521900544, "lambda": 0.3180524256701271, "lambda_min": 0.090766042470932, "lambda_max": 0.6807023882865906, "S_mean": 0.2566426595052083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.407498359680176, "grad_norm": 1.3287405967712402, "tokens_seen": 97820672, "tokens_per_sec": 21008.996263176716, "vram_gb": 5.521900544, "lambda": 0.3181242533028126, "lambda_min": 0.09076463431119919, "lambda_max": 0.6809176206588745, "S_mean": 0.25762939453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.236544132232666, "grad_norm": 1.181047797203064, "tokens_seen": 97984512, "tokens_per_sec": 21009.223941204884, "vram_gb": 5.521900544, "lambda": 0.31818261711547774, "lambda_min": 0.0907859057188034, "lambda_max": 0.6811374425888062, "S_mean": 0.2542521158854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.29194450378418, "grad_norm": 1.2470948696136475, "tokens_seen": 98148352, "tokens_per_sec": 21009.523967589274, "vram_gb": 5.521900544, "lambda": 0.3182733968521158, "lambda_min": 0.09080163389444351, "lambda_max": 0.681446373462677, "S_mean": 0.2543843587239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.3564903974533085, "val_ppl": 77.98296438017636, "lambda": 0.3183447327464819, "lambda_min": 0.0908360481262207, "lambda_max": 0.6816989183425903, "lambda_per_layer": [0.19080153107643127, 0.1360117346048355, 0.1479468196630478, 0.14901793003082275, 0.16528959572315216, 0.13653431832790375, 0.1251608431339264, 0.10797806829214096, 0.09133448451757431, 0.0908360481262207, 0.18115243315696716, 0.28648582100868225, 0.4550967812538147, 0.48778659105300903, 0.42501765489578247, 0.4365147352218628, 0.5860754251480103, 0.4096337854862213, 0.6549112796783447, 0.6816989183425903, 0.394685834646225, 0.5585580468177795, 0.42126813530921936, 0.320476770401001], "S_mean": 0.2521158854166667, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4680.379953622818, "nan_failures": 0} diff --git a/s_only_s4/config.json b/s_only_s4/config.json new file mode 100644 index 0000000000000000000000000000000000000000..046436317b79b141bf6aaeeb929ea92715bb9832 --- /dev/null +++ b/s_only_s4/config.json @@ -0,0 +1,21 @@ +{ + "variant": "s_only", + "seed": 4, + "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s4", + "max_steps": 12000, + "warmup": 200, + "lr_max": 0.0006, + "lr_min": 6e-05, + "wd": 0.1, + "batch_size": 16, + "grad_accum": 2, + "block_size": 256, + "n_layer": 24, + "n_head": 6, + "n_embd": 384, + "vocab_size": 50257, + "data_seed": 4001349, + "eval_interval": 1000, + "eval_batches": 20, + "log_interval": 20 +} \ No newline at end of file diff --git a/s_only_s4/final.pt b/s_only_s4/final.pt new file mode 100644 index 0000000000000000000000000000000000000000..6c88e99ebc457ec4cfddb779f653c18ad0cac78e --- /dev/null +++ b/s_only_s4/final.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22b4d5cdababf49e3481af62fef555097c9aa7e5389076eda38566c75554793e +size 255945415 diff --git a/s_only_s4/metrics.jsonl b/s_only_s4/metrics.jsonl new file mode 100644 index 0000000000000000000000000000000000000000..b17026b51128c1b7b04cb1d48b524dfdebc30fce --- /dev/null +++ b/s_only_s4/metrics.jsonl @@ -0,0 +1,614 @@ +{"event": "start", "params": 63966360, "variant": "s_only", "seed": 4, "config": {"variant": "s_only", "seed": 4, "out": "C:\\Users\\Bhorn\\Desktop\\SMATest\\runs\\s_only_s4", "max_steps": 12000, "warmup": 200, "lr_max": 0.0006, "lr_min": 6e-05, "wd": 0.1, "batch_size": 16, "grad_accum": 2, "block_size": 256, "n_layer": 24, "n_head": 6, "n_embd": 384, "vocab_size": 50257, "data_seed": 4001349, "eval_interval": 1000, "eval_batches": 20, "log_interval": 20}} +{"event": "step", "step": 0, "lr": 2.9999999999999997e-06, "train_loss": 10.874931335449219, "grad_norm": 13.181059837341309, "tokens_seen": 8192, "tokens_per_sec": 8230.804034945251, "vram_gb": 5.017291776, "lambda": 0.12692801654338837, "lambda_min": 0.12692801654338837, "lambda_max": 0.12692801654338837, "S_mean": 0.31914520263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 20, "lr": 6.299999999999999e-05, "train_loss": 9.863513946533203, "grad_norm": 1.9721418619155884, "tokens_seen": 172032, "tokens_per_sec": 19663.39909354626, "vram_gb": 5.521900544, "lambda": 0.1269354640195767, "lambda_min": 0.1269039362668991, "lambda_max": 0.12698042392730713, "S_mean": 0.5094706217447916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 40, "lr": 0.00012299999999999998, "train_loss": 9.104570388793945, "grad_norm": 1.7537769079208374, "tokens_seen": 335872, "tokens_per_sec": 20319.874044092772, "vram_gb": 5.521900544, "lambda": 0.1268776822835207, "lambda_min": 0.12681475281715393, "lambda_max": 0.12701737880706787, "S_mean": 0.5075785319010416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 60, "lr": 0.00018299999999999998, "train_loss": 8.239202499389648, "grad_norm": 1.5072894096374512, "tokens_seen": 499712, "tokens_per_sec": 20640.196422395173, "vram_gb": 5.521900544, "lambda": 0.12680154169599214, "lambda_min": 0.12670445442199707, "lambda_max": 0.12704412639141083, "S_mean": 0.5070953369140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 80, "lr": 0.000243, "train_loss": 7.562503814697266, "grad_norm": 0.776163637638092, "tokens_seen": 663552, "tokens_per_sec": 20769.70747537252, "vram_gb": 5.521900544, "lambda": 0.12670300155878067, "lambda_min": 0.1265764683485031, "lambda_max": 0.12708814442157745, "S_mean": 0.4029541015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 100, "lr": 0.000303, "train_loss": 7.005550861358643, "grad_norm": 0.6774674654006958, "tokens_seen": 827392, "tokens_per_sec": 20857.82021021424, "vram_gb": 5.521900544, "lambda": 0.12663045898079872, "lambda_min": 0.126322403550148, "lambda_max": 0.12728330492973328, "S_mean": 0.2835286458333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 120, "lr": 0.000363, "train_loss": 7.1905083656311035, "grad_norm": 0.5908095836639404, "tokens_seen": 991232, "tokens_per_sec": 20907.302147605562, "vram_gb": 5.521900544, "lambda": 0.12653953147431216, "lambda_min": 0.1260509490966797, "lambda_max": 0.12759846448898315, "S_mean": 0.308746337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 140, "lr": 0.000423, "train_loss": 6.822659492492676, "grad_norm": 0.6042094230651855, "tokens_seen": 1155072, "tokens_per_sec": 20943.343620905493, "vram_gb": 5.521900544, "lambda": 0.12645861444373926, "lambda_min": 0.1258380264043808, "lambda_max": 0.1279047578573227, "S_mean": 0.2834574381510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 160, "lr": 0.000483, "train_loss": 6.7416090965271, "grad_norm": 0.7147442102432251, "tokens_seen": 1318912, "tokens_per_sec": 20976.874947127595, "vram_gb": 5.521900544, "lambda": 0.1264731672902902, "lambda_min": 0.12562131881713867, "lambda_max": 0.1280893087387085, "S_mean": 0.2725423177083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 180, "lr": 0.000543, "train_loss": 6.763713836669922, "grad_norm": 0.6084068417549133, "tokens_seen": 1482752, "tokens_per_sec": 20998.409966016687, "vram_gb": 5.521900544, "lambda": 0.12646111225088438, "lambda_min": 0.12524163722991943, "lambda_max": 0.12832258641719818, "S_mean": 0.2669677734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 200, "lr": 0.0006, "train_loss": 6.724217891693115, "grad_norm": 0.9838268756866455, "tokens_seen": 1646592, "tokens_per_sec": 21024.32551995575, "vram_gb": 5.521900544, "lambda": 0.126350407799085, "lambda_min": 0.12482544034719467, "lambda_max": 0.12845878303050995, "S_mean": 0.2735799153645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 220, "lr": 0.0005999961723830909, "train_loss": 6.736434698104858, "grad_norm": 0.6177918910980225, "tokens_seen": 1810432, "tokens_per_sec": 21019.84092021762, "vram_gb": 5.521900544, "lambda": 0.1263274255519112, "lambda_min": 0.12443643808364868, "lambda_max": 0.12850189208984375, "S_mean": 0.2620951334635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 240, "lr": 0.000599984689640887, "train_loss": 6.541828632354736, "grad_norm": 0.6629209518432617, "tokens_seen": 1974272, "tokens_per_sec": 21037.019229580867, "vram_gb": 5.521900544, "lambda": 0.1262963181361556, "lambda_min": 0.1241091936826706, "lambda_max": 0.1286354959011078, "S_mean": 0.2534281412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 260, "lr": 0.0005999655520989551, "train_loss": 6.6100733280181885, "grad_norm": 0.46206581592559814, "tokens_seen": 2138112, "tokens_per_sec": 21048.119772551665, "vram_gb": 5.521900544, "lambda": 0.12624366860836744, "lambda_min": 0.12384624034166336, "lambda_max": 0.12868188321590424, "S_mean": 0.260223388671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 280, "lr": 0.0005999387602998968, "train_loss": 6.673372983932495, "grad_norm": 0.5557544231414795, "tokens_seen": 2301952, "tokens_per_sec": 21061.97791188317, "vram_gb": 5.521900544, "lambda": 0.12626296592255434, "lambda_min": 0.12377690523862839, "lambda_max": 0.12894709408283234, "S_mean": 0.2996826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 300, "lr": 0.0005999043150033322, "train_loss": 6.3614747524261475, "grad_norm": 0.5485736727714539, "tokens_seen": 2465792, "tokens_per_sec": 21072.29122312571, "vram_gb": 5.521900544, "lambda": 0.12622429430484772, "lambda_min": 0.1235758513212204, "lambda_max": 0.1290895789861679, "S_mean": 0.2876383463541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 320, "lr": 0.000599862217185879, "train_loss": 6.408588886260986, "grad_norm": 0.5674452781677246, "tokens_seen": 2629632, "tokens_per_sec": 21083.98266841816, "vram_gb": 5.521900544, "lambda": 0.1262331005806724, "lambda_min": 0.12337509542703629, "lambda_max": 0.1293649822473526, "S_mean": 0.24433390299479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 340, "lr": 0.0005998124680411251, "train_loss": 6.4346537590026855, "grad_norm": 0.5201497673988342, "tokens_seen": 2793472, "tokens_per_sec": 21082.75451263786, "vram_gb": 5.521900544, "lambda": 0.1262821014970541, "lambda_min": 0.12330054491758347, "lambda_max": 0.12952739000320435, "S_mean": 0.271331787109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 360, "lr": 0.0005997550689795936, "train_loss": 6.4866437911987305, "grad_norm": 0.5233914256095886, "tokens_seen": 2957312, "tokens_per_sec": 21089.296930710807, "vram_gb": 5.521900544, "lambda": 0.12626108154654503, "lambda_min": 0.12315033376216888, "lambda_max": 0.1298583745956421, "S_mean": 0.26727294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 380, "lr": 0.0005996900216287041, "train_loss": 6.407009840011597, "grad_norm": 0.5762526988983154, "tokens_seen": 3121152, "tokens_per_sec": 21086.888919091474, "vram_gb": 5.521900544, "lambda": 0.12636485882103443, "lambda_min": 0.12323135882616043, "lambda_max": 0.13002993166446686, "S_mean": 0.2756449381510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 400, "lr": 0.0005996173278327257, "train_loss": 6.303015947341919, "grad_norm": 0.5291386246681213, "tokens_seen": 3284992, "tokens_per_sec": 21085.590885363345, "vram_gb": 5.521900544, "lambda": 0.12637504500647387, "lambda_min": 0.12325804680585861, "lambda_max": 0.13017134368419647, "S_mean": 0.29644775390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 420, "lr": 0.0005995369896527251, "train_loss": 6.182068824768066, "grad_norm": 0.5565021634101868, "tokens_seen": 3448832, "tokens_per_sec": 21088.728503773982, "vram_gb": 5.521900544, "lambda": 0.1264394416163365, "lambda_min": 0.12306222319602966, "lambda_max": 0.1303117871284485, "S_mean": 0.295257568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 440, "lr": 0.0005994490093665081, "train_loss": 6.279193162918091, "grad_norm": 0.5605332255363464, "tokens_seen": 3612672, "tokens_per_sec": 21094.618360987643, "vram_gb": 5.521900544, "lambda": 0.1264825069035093, "lambda_min": 0.12288903445005417, "lambda_max": 0.130867138504982, "S_mean": 0.2643330891927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 460, "lr": 0.0005993533894685548, "train_loss": 6.311230897903442, "grad_norm": 0.5606299042701721, "tokens_seen": 3776512, "tokens_per_sec": 21097.121634367966, "vram_gb": 5.521900544, "lambda": 0.1265180641785264, "lambda_min": 0.12300688028335571, "lambda_max": 0.1312960684299469, "S_mean": 0.2906697591145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 480, "lr": 0.0005992501326699493, "train_loss": 6.298585414886475, "grad_norm": 0.5291255712509155, "tokens_seen": 3940352, "tokens_per_sec": 21102.537565906525, "vram_gb": 5.521900544, "lambda": 0.12657696443299452, "lambda_min": 0.12300307303667068, "lambda_max": 0.1316841095685959, "S_mean": 0.2840677897135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 500, "lr": 0.0005991392418983024, "train_loss": 6.234930515289307, "grad_norm": 0.5405012369155884, "tokens_seen": 4104192, "tokens_per_sec": 21104.140488377765, "vram_gb": 5.521900544, "lambda": 0.1267233146354556, "lambda_min": 0.12294013798236847, "lambda_max": 0.13227421045303345, "S_mean": 0.3298746744791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 520, "lr": 0.0005990207202976691, "train_loss": 6.271438121795654, "grad_norm": 0.4991741180419922, "tokens_seen": 4268032, "tokens_per_sec": 21107.701935164816, "vram_gb": 5.521900544, "lambda": 0.12685844643662372, "lambda_min": 0.12297752499580383, "lambda_max": 0.1326889842748642, "S_mean": 0.3149617513020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 540, "lr": 0.0005988945712284585, "train_loss": 6.089621543884277, "grad_norm": 0.5573171377182007, "tokens_seen": 4431872, "tokens_per_sec": 21110.76950458963, "vram_gb": 5.521900544, "lambda": 0.1269174808015426, "lambda_min": 0.12284719944000244, "lambda_max": 0.1330401748418808, "S_mean": 0.3163248697916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 560, "lr": 0.00059876079826734, "train_loss": 6.231960773468018, "grad_norm": 0.5997297763824463, "tokens_seen": 4595712, "tokens_per_sec": 21112.02769528827, "vram_gb": 5.521900544, "lambda": 0.1270720042909185, "lambda_min": 0.12287294864654541, "lambda_max": 0.13338804244995117, "S_mean": 0.325042724609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 580, "lr": 0.0005986194052071403, "train_loss": 6.0924232006073, "grad_norm": 0.5210584998130798, "tokens_seen": 4759552, "tokens_per_sec": 21113.319220051224, "vram_gb": 5.521900544, "lambda": 0.12728003381441036, "lambda_min": 0.1231641173362732, "lambda_max": 0.1337713897228241, "S_mean": 0.3301289876302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 600, "lr": 0.0005984703960567369, "train_loss": 6.036342144012451, "grad_norm": 0.5927815437316895, "tokens_seen": 4923392, "tokens_per_sec": 21109.966709140626, "vram_gb": 5.521900544, "lambda": 0.12754694713900486, "lambda_min": 0.1235513836145401, "lambda_max": 0.13410066068172455, "S_mean": 0.2986246744791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 620, "lr": 0.0005983137750409445, "train_loss": 6.14092755317688, "grad_norm": 0.46911337971687317, "tokens_seen": 5087232, "tokens_per_sec": 21111.108567130144, "vram_gb": 5.521900544, "lambda": 0.12779868704577288, "lambda_min": 0.12397482991218567, "lambda_max": 0.134490504860878, "S_mean": 0.3021748860677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 640, "lr": 0.0005981495466003945, "train_loss": 6.237729072570801, "grad_norm": 0.523864209651947, "tokens_seen": 5251072, "tokens_per_sec": 21112.22336005095, "vram_gb": 5.521900544, "lambda": 0.12804412438223758, "lambda_min": 0.12403222173452377, "lambda_max": 0.13479453325271606, "S_mean": 0.3312174479166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 660, "lr": 0.0005979777153914092, "train_loss": 6.078505992889404, "grad_norm": 0.5611025094985962, "tokens_seen": 5414912, "tokens_per_sec": 21115.505226597215, "vram_gb": 5.521900544, "lambda": 0.12834056839346886, "lambda_min": 0.12376205623149872, "lambda_max": 0.13551047444343567, "S_mean": 0.2962849934895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 680, "lr": 0.0005977982862858707, "train_loss": 6.080739736557007, "grad_norm": 0.6557668447494507, "tokens_seen": 5578752, "tokens_per_sec": 21117.13805389315, "vram_gb": 5.521900544, "lambda": 0.12852566440900168, "lambda_min": 0.12347426265478134, "lambda_max": 0.1359674483537674, "S_mean": 0.3180745442708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 700, "lr": 0.0005976112643710818, "train_loss": 5.992506504058838, "grad_norm": 0.5202039480209351, "tokens_seen": 5742592, "tokens_per_sec": 21123.6396978893, "vram_gb": 5.521900544, "lambda": 0.1287958218405644, "lambda_min": 0.12339532375335693, "lambda_max": 0.13641825318336487, "S_mean": 0.3517557779947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 720, "lr": 0.0005974166549496221, "train_loss": 5.951037406921387, "grad_norm": 0.5940263867378235, "tokens_seen": 5906432, "tokens_per_sec": 21123.12388787531, "vram_gb": 5.521900544, "lambda": 0.1290908350298802, "lambda_min": 0.12337538599967957, "lambda_max": 0.13692152500152588, "S_mean": 0.3120015462239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 740, "lr": 0.0005972144635391974, "train_loss": 6.0542662143707275, "grad_norm": 0.5399299263954163, "tokens_seen": 6070272, "tokens_per_sec": 21124.88876403485, "vram_gb": 5.521900544, "lambda": 0.12948724099745354, "lambda_min": 0.12323739379644394, "lambda_max": 0.1373680979013443, "S_mean": 0.3111572265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 760, "lr": 0.0005970046958724839, "train_loss": 6.008872032165527, "grad_norm": 0.49785199761390686, "tokens_seen": 6234112, "tokens_per_sec": 21126.86169469564, "vram_gb": 5.521900544, "lambda": 0.1299094914769133, "lambda_min": 0.1232507973909378, "lambda_max": 0.1376415640115738, "S_mean": 0.293365478515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 780, "lr": 0.000596787357896965, "train_loss": 6.138638257980347, "grad_norm": 0.5798051357269287, "tokens_seen": 6397952, "tokens_per_sec": 21127.558293421498, "vram_gb": 5.521900544, "lambda": 0.13032151044656834, "lambda_min": 0.12335047870874405, "lambda_max": 0.13807614147663116, "S_mean": 0.2980550130208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 800, "lr": 0.000596562455774763, "train_loss": 5.8521482944488525, "grad_norm": 0.55014967918396, "tokens_seen": 6561792, "tokens_per_sec": 21125.225730398422, "vram_gb": 5.521900544, "lambda": 0.13065565843135118, "lambda_min": 0.12282044440507889, "lambda_max": 0.13829965889453888, "S_mean": 0.3040771484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 820, "lr": 0.0005963299958824643, "train_loss": 5.923781156539917, "grad_norm": 0.5074601173400879, "tokens_seen": 6725632, "tokens_per_sec": 21125.13987848398, "vram_gb": 5.521900544, "lambda": 0.13110509204367796, "lambda_min": 0.12217173725366592, "lambda_max": 0.13889814913272858, "S_mean": 0.2598317464192708, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 840, "lr": 0.0005960899848109387, "train_loss": 5.871917247772217, "grad_norm": 0.593573272228241, "tokens_seen": 6889472, "tokens_per_sec": 21127.518539802702, "vram_gb": 5.521900544, "lambda": 0.1314980968212088, "lambda_min": 0.1218932718038559, "lambda_max": 0.13996894657611847, "S_mean": 0.2626749674479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 860, "lr": 0.0005958424293651523, "train_loss": 5.9278974533081055, "grad_norm": 0.589074432849884, "tokens_seen": 7053312, "tokens_per_sec": 21125.201528812497, "vram_gb": 5.521900544, "lambda": 0.13195150438696146, "lambda_min": 0.12167124450206757, "lambda_max": 0.1411718875169754, "S_mean": 0.2664794921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 880, "lr": 0.0005955873365639749, "train_loss": 5.786293983459473, "grad_norm": 0.5281029343605042, "tokens_seen": 7217152, "tokens_per_sec": 21127.82335764043, "vram_gb": 5.521900544, "lambda": 0.13244060271730027, "lambda_min": 0.12143852561712265, "lambda_max": 0.14249607920646667, "S_mean": 0.2658233642578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 900, "lr": 0.0005953247136399806, "train_loss": 5.78353476524353, "grad_norm": 0.5506352186203003, "tokens_seen": 7380992, "tokens_per_sec": 21130.405822870503, "vram_gb": 5.521900544, "lambda": 0.13282673278202614, "lambda_min": 0.12131041288375854, "lambda_max": 0.14367370307445526, "S_mean": 0.263214111328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 920, "lr": 0.0005950545680392432, "train_loss": 5.8572869300842285, "grad_norm": 0.5710182189941406, "tokens_seen": 7544832, "tokens_per_sec": 21131.7666367633, "vram_gb": 5.521900544, "lambda": 0.13325486580530801, "lambda_min": 0.12100674957036972, "lambda_max": 0.1448831558227539, "S_mean": 0.2622477213541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 940, "lr": 0.0005947769074211247, "train_loss": 5.769529819488525, "grad_norm": 0.5606385469436646, "tokens_seen": 7708672, "tokens_per_sec": 21133.666781037125, "vram_gb": 5.521900544, "lambda": 0.13370732621600231, "lambda_min": 0.12062773108482361, "lambda_max": 0.14614960551261902, "S_mean": 0.2740071614583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 960, "lr": 0.0005944917396580579, "train_loss": 5.816460609436035, "grad_norm": 0.5542579293251038, "tokens_seen": 7872512, "tokens_per_sec": 21135.21340792443, "vram_gb": 5.521900544, "lambda": 0.13413415849208832, "lambda_min": 0.12056060880422592, "lambda_max": 0.14738084375858307, "S_mean": 0.2655843098958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 980, "lr": 0.0005941990728353245, "train_loss": 5.818384408950806, "grad_norm": 0.5729849338531494, "tokens_seen": 8036352, "tokens_per_sec": 21137.589591319607, "vram_gb": 5.521900544, "lambda": 0.1346285796413819, "lambda_min": 0.12047625333070755, "lambda_max": 0.1487584412097931, "S_mean": 0.2743326822916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1000, "lr": 0.000593898915250824, "train_loss": 5.730960369110107, "grad_norm": 0.6336616277694702, "tokens_seen": 8200192, "tokens_per_sec": 21138.05033563129, "vram_gb": 5.521900544, "lambda": 0.13513978415479264, "lambda_min": 0.12011777609586716, "lambda_max": 0.14996162056922913, "S_mean": 0.23381551106770834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 1000, "val_loss": 5.789678311347961, "val_ppl": 326.9078449153868, "lambda": 0.1351624857634306, "lambda_min": 0.1200975626707077, "lambda_max": 0.15000170469284058, "lambda_per_layer": [0.14160609245300293, 0.1289348602294922, 0.1333676278591156, 0.13129359483718872, 0.1332312971353531, 0.13063547015190125, 0.1245293989777565, 0.12434970587491989, 0.1200975626707077, 0.12607350945472717, 0.13981389999389648, 0.1457996517419815, 0.14010074734687805, 0.14953166246414185, 0.15000170469284058, 0.14811991155147552, 0.1405547708272934, 0.12838433682918549, 0.13437345623970032, 0.1361389011144638, 0.13593819737434387, 0.1387849748134613, 0.12891633808612823, 0.13332198560237885], "S_mean": 0.2776285807291667, "G_mean": null, "G_std": null, "tokens_seen": 8200192} +{"event": "step", "step": 1020, "lr": 0.0005935912754148397, "train_loss": 5.781391620635986, "grad_norm": 0.5728614926338196, "tokens_seen": 8364032, "tokens_per_sec": 21063.705543918615, "vram_gb": 5.521900544, "lambda": 0.1356215492511789, "lambda_min": 0.11983601748943329, "lambda_max": 0.15108565986156464, "S_mean": 0.2599283854166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1040, "lr": 0.0005932761620497971, "train_loss": 5.7327892780303955, "grad_norm": 0.5494298338890076, "tokens_seen": 8527872, "tokens_per_sec": 21063.704966873993, "vram_gb": 5.521900544, "lambda": 0.13606574945151806, "lambda_min": 0.11940615624189377, "lambda_max": 0.15239916741847992, "S_mean": 0.2657979329427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1060, "lr": 0.0005929535840900167, "train_loss": 5.622752904891968, "grad_norm": 0.524607241153717, "tokens_seen": 8691712, "tokens_per_sec": 21061.53155288743, "vram_gb": 5.521900544, "lambda": 0.1365930556009213, "lambda_min": 0.11899746209383011, "lambda_max": 0.15355753898620605, "S_mean": 0.2699686686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1080, "lr": 0.0005926235506814601, "train_loss": 5.670794486999512, "grad_norm": 0.5820996761322021, "tokens_seen": 8855552, "tokens_per_sec": 21065.09041327316, "vram_gb": 5.521900544, "lambda": 0.1370550518234571, "lambda_min": 0.11889150738716125, "lambda_max": 0.15509489178657532, "S_mean": 0.304718017578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1100, "lr": 0.0005922860711814717, "train_loss": 5.83411431312561, "grad_norm": 0.5514994859695435, "tokens_seen": 9019392, "tokens_per_sec": 21065.712257942752, "vram_gb": 5.521900544, "lambda": 0.13754963502287865, "lambda_min": 0.11856541782617569, "lambda_max": 0.15644241869449615, "S_mean": 0.2668965657552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1120, "lr": 0.0005919411551585122, "train_loss": 5.769472122192383, "grad_norm": 0.522611141204834, "tokens_seen": 9183232, "tokens_per_sec": 21066.744210400022, "vram_gb": 5.521900544, "lambda": 0.13799921981990337, "lambda_min": 0.11826539039611816, "lambda_max": 0.1577564775943756, "S_mean": 0.279296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1140, "lr": 0.0005915888123918887, "train_loss": 5.592040538787842, "grad_norm": 0.5369085669517517, "tokens_seen": 9347072, "tokens_per_sec": 21068.147139327553, "vram_gb": 5.521900544, "lambda": 0.13847187533974648, "lambda_min": 0.11823872476816177, "lambda_max": 0.15898868441581726, "S_mean": 0.2523549397786458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1160, "lr": 0.0005912290528714757, "train_loss": 5.626990079879761, "grad_norm": 0.5198366641998291, "tokens_seen": 9510912, "tokens_per_sec": 21070.645643912405, "vram_gb": 5.521900544, "lambda": 0.13897408482929072, "lambda_min": 0.11827030032873154, "lambda_max": 0.16030289232730865, "S_mean": 0.272552490234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1180, "lr": 0.0005908618867974339, "train_loss": 5.718325853347778, "grad_norm": 0.5658671259880066, "tokens_seen": 9674752, "tokens_per_sec": 21072.55425968856, "vram_gb": 5.521900544, "lambda": 0.13945387521137795, "lambda_min": 0.1179129108786583, "lambda_max": 0.16160358488559723, "S_mean": 0.24741109212239584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1200, "lr": 0.0005904873245799193, "train_loss": 5.683991432189941, "grad_norm": 0.5383244156837463, "tokens_seen": 9838592, "tokens_per_sec": 21072.280842615735, "vram_gb": 5.521900544, "lambda": 0.1399195995181799, "lambda_min": 0.11792190372943878, "lambda_max": 0.16285139322280884, "S_mean": 0.2515970865885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1220, "lr": 0.0005901053768387885, "train_loss": 5.901339054107666, "grad_norm": 0.5668198466300964, "tokens_seen": 10002432, "tokens_per_sec": 21075.483840386674, "vram_gb": 5.521900544, "lambda": 0.14036059007048607, "lambda_min": 0.11795959621667862, "lambda_max": 0.16397391259670258, "S_mean": 0.24723307291666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1240, "lr": 0.0005897160544032983, "train_loss": 5.550371408462524, "grad_norm": 0.5177686214447021, "tokens_seen": 10166272, "tokens_per_sec": 21077.124088129767, "vram_gb": 5.521900544, "lambda": 0.14083771035075188, "lambda_min": 0.11782773584127426, "lambda_max": 0.16539601981639862, "S_mean": 0.2616373697916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1260, "lr": 0.0005893193683117977, "train_loss": 5.586817741394043, "grad_norm": 0.5611817836761475, "tokens_seen": 10330112, "tokens_per_sec": 21077.819186713983, "vram_gb": 5.521900544, "lambda": 0.14135889802128077, "lambda_min": 0.11778751760721207, "lambda_max": 0.1668216437101364, "S_mean": 0.2662455240885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1280, "lr": 0.0005889153298114155, "train_loss": 5.453607559204102, "grad_norm": 0.5111786723136902, "tokens_seen": 10493952, "tokens_per_sec": 21079.94019865511, "vram_gb": 5.521900544, "lambda": 0.14184506703168154, "lambda_min": 0.11788401752710342, "lambda_max": 0.16811852157115936, "S_mean": 0.2640279134114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1300, "lr": 0.0005885039503577414, "train_loss": 5.768213272094727, "grad_norm": 0.5614323019981384, "tokens_seen": 10657792, "tokens_per_sec": 21081.836889743932, "vram_gb": 5.521900544, "lambda": 0.14232502474139133, "lambda_min": 0.11769114434719086, "lambda_max": 0.16962504386901855, "S_mean": 0.2602742513020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1320, "lr": 0.0005880852416145011, "train_loss": 5.535966157913208, "grad_norm": 0.5890491008758545, "tokens_seen": 10821632, "tokens_per_sec": 21085.512180458543, "vram_gb": 5.521900544, "lambda": 0.14278402210523686, "lambda_min": 0.11772466450929642, "lambda_max": 0.17100390791893005, "S_mean": 0.2826131184895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1340, "lr": 0.0005876592154532252, "train_loss": 5.4998719692230225, "grad_norm": 0.5436090230941772, "tokens_seen": 10985472, "tokens_per_sec": 21085.990984085394, "vram_gb": 5.521900544, "lambda": 0.14332549987981716, "lambda_min": 0.11765535920858383, "lambda_max": 0.17241951823234558, "S_mean": 0.2793680826822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1360, "lr": 0.0005872258839529134, "train_loss": 5.619993448257446, "grad_norm": 0.5940786004066467, "tokens_seen": 11149312, "tokens_per_sec": 21087.236694411036, "vram_gb": 5.521900544, "lambda": 0.14378527738153934, "lambda_min": 0.11727353930473328, "lambda_max": 0.17385832965373993, "S_mean": 0.2926127115885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1380, "lr": 0.0005867852593996914, "train_loss": 5.556062698364258, "grad_norm": 0.572330117225647, "tokens_seen": 11313152, "tokens_per_sec": 21088.42999710117, "vram_gb": 5.521900544, "lambda": 0.14423525892198086, "lambda_min": 0.1165895015001297, "lambda_max": 0.17507176101207733, "S_mean": 0.2756703694661458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1400, "lr": 0.0005863373542864629, "train_loss": 5.446352958679199, "grad_norm": 0.5920472741127014, "tokens_seen": 11476992, "tokens_per_sec": 21088.692898185833, "vram_gb": 5.521900544, "lambda": 0.14473018236458302, "lambda_min": 0.1162000373005867, "lambda_max": 0.17637234926223755, "S_mean": 0.2953389485677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1420, "lr": 0.0005858821813125552, "train_loss": 5.557538986206055, "grad_norm": 0.5504327416419983, "tokens_seen": 11640832, "tokens_per_sec": 21088.62150086291, "vram_gb": 5.521900544, "lambda": 0.14516903273761272, "lambda_min": 0.11582780629396439, "lambda_max": 0.17762641608715057, "S_mean": 0.2674662272135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1440, "lr": 0.0005854197533833591, "train_loss": 5.504576921463013, "grad_norm": 0.6466546058654785, "tokens_seen": 11804672, "tokens_per_sec": 21090.11230876198, "vram_gb": 5.521900544, "lambda": 0.14563736350586018, "lambda_min": 0.11541218310594559, "lambda_max": 0.17894724011421204, "S_mean": 0.2703959147135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1460, "lr": 0.0005849500836099632, "train_loss": 5.428011655807495, "grad_norm": 0.5944636464118958, "tokens_seen": 11968512, "tokens_per_sec": 21091.426210588837, "vram_gb": 5.521900544, "lambda": 0.1461489892875155, "lambda_min": 0.11496131867170334, "lambda_max": 0.18033985793590546, "S_mean": 0.2539469401041667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1480, "lr": 0.0005844731853087822, "train_loss": 5.468031167984009, "grad_norm": 0.5951301455497742, "tokens_seen": 12132352, "tokens_per_sec": 21092.442793701484, "vram_gb": 5.521900544, "lambda": 0.14668714224050441, "lambda_min": 0.11504196375608444, "lambda_max": 0.1815178543329239, "S_mean": 0.24660746256510416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1500, "lr": 0.0005839890720011789, "train_loss": 5.508108377456665, "grad_norm": 0.6068283319473267, "tokens_seen": 12296192, "tokens_per_sec": 21094.643459143303, "vram_gb": 5.521900544, "lambda": 0.14718262509753308, "lambda_min": 0.11497072130441666, "lambda_max": 0.18286225199699402, "S_mean": 0.2585245768229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1520, "lr": 0.0005834977574130815, "train_loss": 5.345278978347778, "grad_norm": 0.5818976163864136, "tokens_seen": 12460032, "tokens_per_sec": 21095.688241801665, "vram_gb": 5.521900544, "lambda": 0.14758962796380123, "lambda_min": 0.1148151382803917, "lambda_max": 0.18423013389110565, "S_mean": 0.2606099446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1540, "lr": 0.0005829992554745938, "train_loss": 5.4387171268463135, "grad_norm": 0.6032196879386902, "tokens_seen": 12623872, "tokens_per_sec": 21097.890778240155, "vram_gb": 5.521900544, "lambda": 0.14806531059245268, "lambda_min": 0.11455389112234116, "lambda_max": 0.18562354147434235, "S_mean": 0.2542826334635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1560, "lr": 0.0005824935803196007, "train_loss": 5.502437353134155, "grad_norm": 0.5602533221244812, "tokens_seen": 12787712, "tokens_per_sec": 21098.583595487766, "vram_gb": 5.521900544, "lambda": 0.14854030528416237, "lambda_min": 0.11426207423210144, "lambda_max": 0.18709996342658997, "S_mean": 0.2745869954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1580, "lr": 0.0005819807462853669, "train_loss": 5.58038330078125, "grad_norm": 0.5827456712722778, "tokens_seen": 12951552, "tokens_per_sec": 21099.74813581439, "vram_gb": 5.521900544, "lambda": 0.14898411153505245, "lambda_min": 0.11432217061519623, "lambda_max": 0.1883617341518402, "S_mean": 0.2582804361979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1600, "lr": 0.0005814607679121314, "train_loss": 5.267961740493774, "grad_norm": 0.5594846606254578, "tokens_seen": 13115392, "tokens_per_sec": 21101.46072943681, "vram_gb": 5.521900544, "lambda": 0.1495009878029426, "lambda_min": 0.11455360800027847, "lambda_max": 0.18965840339660645, "S_mean": 0.2592264811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1620, "lr": 0.0005809336599426941, "train_loss": 5.404059171676636, "grad_norm": 0.5897700190544128, "tokens_seen": 13279232, "tokens_per_sec": 21103.728080784073, "vram_gb": 5.521900544, "lambda": 0.14998728719850382, "lambda_min": 0.114540234208107, "lambda_max": 0.19102129340171814, "S_mean": 0.2568868001302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1640, "lr": 0.0005803994373219986, "train_loss": 5.431464195251465, "grad_norm": 0.6048803329467773, "tokens_seen": 13443072, "tokens_per_sec": 21104.672753792423, "vram_gb": 5.521900544, "lambda": 0.15047285022834936, "lambda_min": 0.1145818829536438, "lambda_max": 0.19272677600383759, "S_mean": 0.2694193522135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1660, "lr": 0.0005798581151967081, "train_loss": 5.3782031536102295, "grad_norm": 0.5496515035629272, "tokens_seen": 13606912, "tokens_per_sec": 21106.204553829233, "vram_gb": 5.521900544, "lambda": 0.15097498365988335, "lambda_min": 0.1143377348780632, "lambda_max": 0.19407464563846588, "S_mean": 0.271728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1680, "lr": 0.0005793097089147762, "train_loss": 5.26152777671814, "grad_norm": 0.565549910068512, "tokens_seen": 13770752, "tokens_per_sec": 21108.081669613428, "vram_gb": 5.521900544, "lambda": 0.15139164589345455, "lambda_min": 0.11378560215234756, "lambda_max": 0.19512876868247986, "S_mean": 0.2633158365885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1700, "lr": 0.0005787542340250114, "train_loss": 5.331394195556641, "grad_norm": 0.5718653202056885, "tokens_seen": 13934592, "tokens_per_sec": 21108.95509932173, "vram_gb": 5.521900544, "lambda": 0.15190307578692833, "lambda_min": 0.11366188526153564, "lambda_max": 0.19632630050182343, "S_mean": 0.3008524576822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1720, "lr": 0.0005781917062766366, "train_loss": 5.428118467330933, "grad_norm": 0.6220008134841919, "tokens_seen": 14098432, "tokens_per_sec": 21109.752939374674, "vram_gb": 5.521900544, "lambda": 0.1523720740030209, "lambda_min": 0.11340371519327164, "lambda_max": 0.19732601940631866, "S_mean": 0.3119099934895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1740, "lr": 0.000577622141618842, "train_loss": 5.503091096878052, "grad_norm": 0.5995718836784363, "tokens_seen": 14262272, "tokens_per_sec": 21110.00700502762, "vram_gb": 5.521900544, "lambda": 0.15287526790052652, "lambda_min": 0.11310901492834091, "lambda_max": 0.19862572848796844, "S_mean": 0.2876688639322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1760, "lr": 0.0005770455562003342, "train_loss": 5.3175787925720215, "grad_norm": 0.5749948620796204, "tokens_seen": 14426112, "tokens_per_sec": 21109.989643008958, "vram_gb": 5.521900544, "lambda": 0.15336126772065958, "lambda_min": 0.11275997757911682, "lambda_max": 0.19978031516075134, "S_mean": 0.29034423828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1780, "lr": 0.0005764619663688763, "train_loss": 5.4728474617004395, "grad_norm": 0.604274332523346, "tokens_seen": 14589952, "tokens_per_sec": 21111.26951075128, "vram_gb": 5.521900544, "lambda": 0.15387929572413364, "lambda_min": 0.11247840523719788, "lambda_max": 0.20119591057300568, "S_mean": 0.2602437337239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1800, "lr": 0.0005758713886708266, "train_loss": 5.223930835723877, "grad_norm": 0.5990628004074097, "tokens_seen": 14753792, "tokens_per_sec": 21110.13636204383, "vram_gb": 5.521900544, "lambda": 0.15444793365895748, "lambda_min": 0.11227605491876602, "lambda_max": 0.20277440547943115, "S_mean": 0.2803243001302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1820, "lr": 0.0005752738398506675, "train_loss": 5.311127185821533, "grad_norm": 0.6011694669723511, "tokens_seen": 14917632, "tokens_per_sec": 21112.185699842063, "vram_gb": 5.521900544, "lambda": 0.1549294845511516, "lambda_min": 0.11194150149822235, "lambda_max": 0.20400401949882507, "S_mean": 0.2685648600260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1840, "lr": 0.0005746693368505325, "train_loss": 5.363290786743164, "grad_norm": 0.6293551325798035, "tokens_seen": 15081472, "tokens_per_sec": 21111.946516804812, "vram_gb": 5.521900544, "lambda": 0.15544309156636396, "lambda_min": 0.11165229231119156, "lambda_max": 0.20532947778701782, "S_mean": 0.2798563639322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1860, "lr": 0.0005740578968097244, "train_loss": 5.561949968338013, "grad_norm": 0.5928006768226624, "tokens_seen": 15245312, "tokens_per_sec": 21113.503145402487, "vram_gb": 5.521900544, "lambda": 0.15590902976691723, "lambda_min": 0.11152613162994385, "lambda_max": 0.2069709450006485, "S_mean": 0.2645975748697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1880, "lr": 0.0005734395370642301, "train_loss": 5.424478054046631, "grad_norm": 0.6050539612770081, "tokens_seen": 15409152, "tokens_per_sec": 21114.03732527344, "vram_gb": 5.521900544, "lambda": 0.15646408963948488, "lambda_min": 0.11149010807275772, "lambda_max": 0.20834624767303467, "S_mean": 0.2801513671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1900, "lr": 0.0005728142751462292, "train_loss": 5.333287715911865, "grad_norm": 0.6013861894607544, "tokens_seen": 15572992, "tokens_per_sec": 21114.95803222517, "vram_gb": 5.521900544, "lambda": 0.15700544199595848, "lambda_min": 0.11167726665735245, "lambda_max": 0.20973095297813416, "S_mean": 0.2801310221354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1920, "lr": 0.0005721821287835963, "train_loss": 5.334294319152832, "grad_norm": 0.6091041564941406, "tokens_seen": 15736832, "tokens_per_sec": 21116.252064343145, "vram_gb": 5.521900544, "lambda": 0.15751903286824623, "lambda_min": 0.1118374690413475, "lambda_max": 0.21094690263271332, "S_mean": 0.2778828938802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1940, "lr": 0.0005715431158993987, "train_loss": 5.338111877441406, "grad_norm": 0.5793552398681641, "tokens_seen": 15900672, "tokens_per_sec": 21117.195237161213, "vram_gb": 5.521900544, "lambda": 0.1580360767741998, "lambda_min": 0.11173415929079056, "lambda_max": 0.2123790830373764, "S_mean": 0.2968343098958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1960, "lr": 0.0005708972546113887, "train_loss": 5.463583469390869, "grad_norm": 0.6039956212043762, "tokens_seen": 16064512, "tokens_per_sec": 21117.648583360733, "vram_gb": 5.521900544, "lambda": 0.15865825271854797, "lambda_min": 0.11207108199596405, "lambda_max": 0.21362802386283875, "S_mean": 0.2727508544921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 1980, "lr": 0.000570244563231489, "train_loss": 5.230794906616211, "grad_norm": 0.5605623722076416, "tokens_seen": 16228352, "tokens_per_sec": 21117.673630987796, "vram_gb": 5.521900544, "lambda": 0.15921222263326248, "lambda_min": 0.11191201955080032, "lambda_max": 0.214780792593956, "S_mean": 0.277557373046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2000, "lr": 0.0005695850602652743, "train_loss": 5.290179252624512, "grad_norm": 0.6402096748352051, "tokens_seen": 16392192, "tokens_per_sec": 21118.5920976279, "vram_gb": 5.521900544, "lambda": 0.15977114625275135, "lambda_min": 0.11183570325374603, "lambda_max": 0.2164643406867981, "S_mean": 0.2775675455729167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 2000, "val_loss": 5.342695522308349, "val_ppl": 209.0755191423914, "lambda": 0.15979463669161, "lambda_min": 0.11181716620922089, "lambda_max": 0.21653106808662415, "lambda_per_layer": [0.1537914127111435, 0.13165126740932465, 0.1362735778093338, 0.13255737721920013, 0.1357840895652771, 0.1314612329006195, 0.12412847578525543, 0.11746393889188766, 0.11181716620922089, 0.1240415945649147, 0.16792435944080353, 0.1966891586780548, 0.178921177983284, 0.2160184383392334, 0.2148108184337616, 0.21653106808662415, 0.197066530585289, 0.14738930761814117, 0.1679842621088028, 0.18554535508155823, 0.17172996699810028, 0.17844755947589874, 0.14467447996139526, 0.1523686647415161], "S_mean": 0.2573649088541667, "G_mean": null, "G_std": null, "tokens_seen": 16392192} +{"event": "step", "step": 2020, "lr": 0.0005689187644114459, "train_loss": 5.252659320831299, "grad_norm": 0.6935187578201294, "tokens_seen": 16556032, "tokens_per_sec": 21080.011847752423, "vram_gb": 5.521900544, "lambda": 0.1603714128335317, "lambda_min": 0.11171213537454605, "lambda_max": 0.21806229650974274, "S_mean": 0.2781778971354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2040, "lr": 0.0005682456945613022, "train_loss": 5.421512126922607, "grad_norm": 0.693669855594635, "tokens_seen": 16719872, "tokens_per_sec": 21080.575030785098, "vram_gb": 5.521900544, "lambda": 0.16094677615910769, "lambda_min": 0.1117616668343544, "lambda_max": 0.21982021629810333, "S_mean": 0.26470947265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2060, "lr": 0.0005675658697982029, "train_loss": 5.339954376220703, "grad_norm": 0.654189944267273, "tokens_seen": 16883712, "tokens_per_sec": 21081.272689559268, "vram_gb": 5.521900544, "lambda": 0.1614952559272448, "lambda_min": 0.1116068959236145, "lambda_max": 0.22138096392154694, "S_mean": 0.2833353678385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2080, "lr": 0.0005668793093970275, "train_loss": 5.240072727203369, "grad_norm": 0.616296112537384, "tokens_seen": 17047552, "tokens_per_sec": 21082.27713488967, "vram_gb": 5.521900544, "lambda": 0.16203524420658746, "lambda_min": 0.11139357835054398, "lambda_max": 0.22283796966075897, "S_mean": 0.2575480143229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2100, "lr": 0.0005661860328236294, "train_loss": 5.165584087371826, "grad_norm": 0.6904970407485962, "tokens_seen": 17211392, "tokens_per_sec": 21083.503283809532, "vram_gb": 5.521900544, "lambda": 0.1625828609491388, "lambda_min": 0.11127009242773056, "lambda_max": 0.22438427805900574, "S_mean": 0.256134033203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2120, "lr": 0.0005654860597342838, "train_loss": 5.256476879119873, "grad_norm": 0.6252650022506714, "tokens_seen": 17375232, "tokens_per_sec": 21084.168136117434, "vram_gb": 5.521900544, "lambda": 0.16315384022891521, "lambda_min": 0.11136505752801895, "lambda_max": 0.22568243741989136, "S_mean": 0.260101318359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2140, "lr": 0.0005647794099751301, "train_loss": 5.221043348312378, "grad_norm": 0.6121689081192017, "tokens_seen": 17539072, "tokens_per_sec": 21084.047288709244, "vram_gb": 5.521900544, "lambda": 0.16369951236993074, "lambda_min": 0.11135558784008026, "lambda_max": 0.22723817825317383, "S_mean": 0.251373291015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2160, "lr": 0.0005640661035816095, "train_loss": 5.186765432357788, "grad_norm": 0.5904375910758972, "tokens_seen": 17702912, "tokens_per_sec": 21085.6666080995, "vram_gb": 5.521900544, "lambda": 0.16433575997749963, "lambda_min": 0.11116311699151993, "lambda_max": 0.22864817082881927, "S_mean": 0.2802327473958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2180, "lr": 0.0005633461607778969, "train_loss": 5.32067084312439, "grad_norm": 0.6023889183998108, "tokens_seen": 17866752, "tokens_per_sec": 21086.348189830118, "vram_gb": 5.521900544, "lambda": 0.1649023313075304, "lambda_min": 0.11077786237001419, "lambda_max": 0.23011288046836853, "S_mean": 0.2749125162760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2200, "lr": 0.0005626196019763276, "train_loss": 5.227927207946777, "grad_norm": 0.6403812766075134, "tokens_seen": 18030592, "tokens_per_sec": 21087.36858520729, "vram_gb": 5.521900544, "lambda": 0.16543951289107403, "lambda_min": 0.11045661568641663, "lambda_max": 0.23175469040870667, "S_mean": 0.24781290690104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2220, "lr": 0.0005618864477768182, "train_loss": 5.178889274597168, "grad_norm": 0.5813051462173462, "tokens_seen": 18194432, "tokens_per_sec": 21087.86585151092, "vram_gb": 5.521900544, "lambda": 0.16608269046992064, "lambda_min": 0.11012560874223709, "lambda_max": 0.2336413711309433, "S_mean": 0.2681070963541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2240, "lr": 0.0005611467189662829, "train_loss": 5.101381301879883, "grad_norm": 0.6179369688034058, "tokens_seen": 18358272, "tokens_per_sec": 21088.033905494485, "vram_gb": 5.521900544, "lambda": 0.1666701715439558, "lambda_min": 0.1098765879869461, "lambda_max": 0.23535475134849548, "S_mean": 0.268280029296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2260, "lr": 0.0005604004365180439, "train_loss": 5.290397644042969, "grad_norm": 0.7453876733779907, "tokens_seen": 18522112, "tokens_per_sec": 21088.16382417761, "vram_gb": 5.521900544, "lambda": 0.16722732999672493, "lambda_min": 0.10949599742889404, "lambda_max": 0.23688830435276031, "S_mean": 0.2680460611979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2280, "lr": 0.000559647621591237, "train_loss": 5.181470632553101, "grad_norm": 0.6297639012336731, "tokens_seen": 18685952, "tokens_per_sec": 21088.596926225277, "vram_gb": 5.521900544, "lambda": 0.16780183495332798, "lambda_min": 0.10903089493513107, "lambda_max": 0.23834958672523499, "S_mean": 0.2772420247395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2300, "lr": 0.0005588882955302111, "train_loss": 5.1393842697143555, "grad_norm": 0.5969367623329163, "tokens_seen": 18849792, "tokens_per_sec": 21088.879196000416, "vram_gb": 5.521900544, "lambda": 0.16832317505031824, "lambda_min": 0.10880667716264725, "lambda_max": 0.23999741673469543, "S_mean": 0.2882181803385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2320, "lr": 0.0005581224798639242, "train_loss": 5.039039134979248, "grad_norm": 0.6792570352554321, "tokens_seen": 19013632, "tokens_per_sec": 21088.73250438767, "vram_gb": 5.521900544, "lambda": 0.1689060271407167, "lambda_min": 0.1085510179400444, "lambda_max": 0.2415943294763565, "S_mean": 0.2616678873697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2340, "lr": 0.0005573501963053314, "train_loss": 5.198476552963257, "grad_norm": 0.6203964352607727, "tokens_seen": 19177472, "tokens_per_sec": 21088.186649663483, "vram_gb": 5.521900544, "lambda": 0.16950232225159803, "lambda_min": 0.10829398036003113, "lambda_max": 0.2432418018579483, "S_mean": 0.2772725423177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2360, "lr": 0.0005565714667507708, "train_loss": 4.949087381362915, "grad_norm": 0.6673406362533569, "tokens_seen": 19341312, "tokens_per_sec": 21079.71840232436, "vram_gb": 5.521900544, "lambda": 0.16999093784640232, "lambda_min": 0.10793409496545792, "lambda_max": 0.24510374665260315, "S_mean": 0.2838643391927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2380, "lr": 0.0005557863132793418, "train_loss": 5.132278680801392, "grad_norm": 0.6709757447242737, "tokens_seen": 19505152, "tokens_per_sec": 21080.49937745347, "vram_gb": 5.521900544, "lambda": 0.17052102275192738, "lambda_min": 0.1075994223356247, "lambda_max": 0.24665425717830658, "S_mean": 0.2696431477864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2400, "lr": 0.000554994758152279, "train_loss": 5.114001750946045, "grad_norm": 0.6598201394081116, "tokens_seen": 19668992, "tokens_per_sec": 21080.834767272314, "vram_gb": 5.521900544, "lambda": 0.17110408221681914, "lambda_min": 0.10744086652994156, "lambda_max": 0.2480347901582718, "S_mean": 0.25604248046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2420, "lr": 0.0005541968238123216, "train_loss": 5.0623650550842285, "grad_norm": 0.649265706539154, "tokens_seen": 19832832, "tokens_per_sec": 21081.450726277424, "vram_gb": 5.521900544, "lambda": 0.17163931609441838, "lambda_min": 0.10730667412281036, "lambda_max": 0.24923557043075562, "S_mean": 0.2717183430989583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2440, "lr": 0.0005533925328830771, "train_loss": 5.115721702575684, "grad_norm": 0.6455134749412537, "tokens_seen": 19996672, "tokens_per_sec": 21082.607283881014, "vram_gb": 5.521900544, "lambda": 0.17223171424120665, "lambda_min": 0.10699455440044403, "lambda_max": 0.251211553812027, "S_mean": 0.2555135091145833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2460, "lr": 0.000552581908168379, "train_loss": 5.079483509063721, "grad_norm": 0.6419891119003296, "tokens_seen": 20160512, "tokens_per_sec": 21083.48869406191, "vram_gb": 5.521900544, "lambda": 0.1728110620751977, "lambda_min": 0.10656371712684631, "lambda_max": 0.25291353464126587, "S_mean": 0.2585347493489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2480, "lr": 0.000551764972651641, "train_loss": 5.217102766036987, "grad_norm": 0.6872730851173401, "tokens_seen": 20324352, "tokens_per_sec": 21085.108426200415, "vram_gb": 5.521900544, "lambda": 0.17339391137162843, "lambda_min": 0.10625143349170685, "lambda_max": 0.2544524073600769, "S_mean": 0.2569681803385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2500, "lr": 0.0005509417494952056, "train_loss": 5.137790203094482, "grad_norm": 0.6373742818832397, "tokens_seen": 20488192, "tokens_per_sec": 21085.755150423927, "vram_gb": 5.521900544, "lambda": 0.17406232127298912, "lambda_min": 0.10628421604633331, "lambda_max": 0.25606846809387207, "S_mean": 0.2594807942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2520, "lr": 0.0005501122620396866, "train_loss": 5.023809194564819, "grad_norm": 0.6929673552513123, "tokens_seen": 20652032, "tokens_per_sec": 21086.749349610054, "vram_gb": 5.521900544, "lambda": 0.17463181478281817, "lambda_min": 0.10605381429195404, "lambda_max": 0.25735756754875183, "S_mean": 0.2560323079427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2540, "lr": 0.0005492765338033077, "train_loss": 5.040412902832031, "grad_norm": 0.6359744668006897, "tokens_seen": 20815872, "tokens_per_sec": 21086.794819122853, "vram_gb": 5.521900544, "lambda": 0.1752179025982817, "lambda_min": 0.10543093830347061, "lambda_max": 0.258774071931839, "S_mean": 0.267974853515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2560, "lr": 0.0005484345884812357, "train_loss": 5.229405879974365, "grad_norm": 0.6119130849838257, "tokens_seen": 20979712, "tokens_per_sec": 21087.958504966202, "vram_gb": 5.521900544, "lambda": 0.1758183343335986, "lambda_min": 0.10498537123203278, "lambda_max": 0.2603902816772461, "S_mean": 0.2526753743489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2580, "lr": 0.000547586449944909, "train_loss": 5.074173450469971, "grad_norm": 0.6258500814437866, "tokens_seen": 21143552, "tokens_per_sec": 21090.543233159984, "vram_gb": 5.521900544, "lambda": 0.1764581153790156, "lambda_min": 0.10476352274417877, "lambda_max": 0.2622814476490021, "S_mean": 0.2657470703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2600, "lr": 0.00054673214224136, "train_loss": 5.0489137172698975, "grad_norm": 0.6097601056098938, "tokens_seen": 21307392, "tokens_per_sec": 21093.284329511334, "vram_gb": 5.521900544, "lambda": 0.17703604511916637, "lambda_min": 0.10445184260606766, "lambda_max": 0.2639867067337036, "S_mean": 0.263519287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2620, "lr": 0.0005458716895925343, "train_loss": 5.101990699768066, "grad_norm": 0.6997770667076111, "tokens_seen": 21471232, "tokens_per_sec": 21095.953507147446, "vram_gb": 5.521900544, "lambda": 0.1775686483209332, "lambda_min": 0.10406539589166641, "lambda_max": 0.2654350697994232, "S_mean": 0.2533671061197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2640, "lr": 0.0005450051163946026, "train_loss": 5.11267614364624, "grad_norm": 0.6851661205291748, "tokens_seen": 21635072, "tokens_per_sec": 21098.30748473049, "vram_gb": 5.521900544, "lambda": 0.17812064538399378, "lambda_min": 0.10380200296640396, "lambda_max": 0.2670615017414093, "S_mean": 0.2520243326822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2660, "lr": 0.0005441324472172707, "train_loss": 5.210331916809082, "grad_norm": 0.6297528147697449, "tokens_seen": 21798912, "tokens_per_sec": 21101.148136476062, "vram_gb": 5.521900544, "lambda": 0.17868295001486936, "lambda_min": 0.10371195524930954, "lambda_max": 0.268706351518631, "S_mean": 0.2681681315104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2680, "lr": 0.0005432537068030813, "train_loss": 4.933150291442871, "grad_norm": 0.680510938167572, "tokens_seen": 21962752, "tokens_per_sec": 21103.48022688338, "vram_gb": 5.521900544, "lambda": 0.17930863921840987, "lambda_min": 0.10341425985097885, "lambda_max": 0.27038782835006714, "S_mean": 0.2622884114583333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2700, "lr": 0.0005423689200667135, "train_loss": 5.19373893737793, "grad_norm": 0.6936460733413696, "tokens_seen": 22126592, "tokens_per_sec": 21105.81070782315, "vram_gb": 5.521900544, "lambda": 0.17990991535286108, "lambda_min": 0.1030459776520729, "lambda_max": 0.2718811631202698, "S_mean": 0.24495442708333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2720, "lr": 0.0005414781120942758, "train_loss": 5.154819488525391, "grad_norm": 0.6467520594596863, "tokens_seen": 22290432, "tokens_per_sec": 21106.50979118748, "vram_gb": 5.521900544, "lambda": 0.1804751018062234, "lambda_min": 0.10260435193777084, "lambda_max": 0.2733657658100128, "S_mean": 0.2715962727864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2740, "lr": 0.0005405813081425956, "train_loss": 5.1408116817474365, "grad_norm": 0.7017590999603271, "tokens_seen": 22454272, "tokens_per_sec": 21106.906687787865, "vram_gb": 5.521900544, "lambda": 0.18107955250889063, "lambda_min": 0.10230895131826401, "lambda_max": 0.27510225772857666, "S_mean": 0.2655131022135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2760, "lr": 0.0005396785336385022, "train_loss": 5.048671245574951, "grad_norm": 0.6961805820465088, "tokens_seen": 22618112, "tokens_per_sec": 21107.406313552747, "vram_gb": 5.521900544, "lambda": 0.18168274872004986, "lambda_min": 0.10198860615491867, "lambda_max": 0.27671387791633606, "S_mean": 0.2540995279947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2780, "lr": 0.0005387698141781065, "train_loss": 5.134580850601196, "grad_norm": 0.752636194229126, "tokens_seen": 22781952, "tokens_per_sec": 21107.54176255776, "vram_gb": 5.521900544, "lambda": 0.18222266001005968, "lambda_min": 0.10177145898342133, "lambda_max": 0.2784138321876526, "S_mean": 0.27618408203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2800, "lr": 0.0005378551755260746, "train_loss": 5.0341477394104, "grad_norm": 0.6572780609130859, "tokens_seen": 22945792, "tokens_per_sec": 21107.98743447049, "vram_gb": 5.521900544, "lambda": 0.18276780874778828, "lambda_min": 0.10128551721572876, "lambda_max": 0.27996450662612915, "S_mean": 0.27716064453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2820, "lr": 0.0005369346436148984, "train_loss": 4.955930471420288, "grad_norm": 0.6808498501777649, "tokens_seen": 23109632, "tokens_per_sec": 21108.933255383243, "vram_gb": 5.521900544, "lambda": 0.1833235820134481, "lambda_min": 0.10097222030162811, "lambda_max": 0.28143903613090515, "S_mean": 0.2719828287760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2840, "lr": 0.0005360082445441596, "train_loss": 4.943510293960571, "grad_norm": 0.6989557147026062, "tokens_seen": 23273472, "tokens_per_sec": 21110.02718338488, "vram_gb": 5.521900544, "lambda": 0.1838997658342123, "lambda_min": 0.10049226880073547, "lambda_max": 0.2826805114746094, "S_mean": 0.2823079427083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2860, "lr": 0.0005350760045797893, "train_loss": 5.049582242965698, "grad_norm": 0.6788214445114136, "tokens_seen": 23437312, "tokens_per_sec": 21111.374653867744, "vram_gb": 5.521900544, "lambda": 0.18451861975093684, "lambda_min": 0.10006155073642731, "lambda_max": 0.28395241498947144, "S_mean": 0.25811767578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2880, "lr": 0.0005341379501533244, "train_loss": 5.032736778259277, "grad_norm": 0.6826159954071045, "tokens_seen": 23601152, "tokens_per_sec": 21113.053800645812, "vram_gb": 5.521900544, "lambda": 0.18510191577176252, "lambda_min": 0.09957440942525864, "lambda_max": 0.2850366532802582, "S_mean": 0.2530008951822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2900, "lr": 0.0005331941078611573, "train_loss": 5.032248258590698, "grad_norm": 0.6776204109191895, "tokens_seen": 23764992, "tokens_per_sec": 21114.701110938247, "vram_gb": 5.521900544, "lambda": 0.18572899606078863, "lambda_min": 0.09934163093566895, "lambda_max": 0.2869310677051544, "S_mean": 0.2744140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2920, "lr": 0.0005322445044637823, "train_loss": 5.08170485496521, "grad_norm": 0.6375691294670105, "tokens_seen": 23928832, "tokens_per_sec": 21116.320072696017, "vram_gb": 5.521900544, "lambda": 0.18633599765598774, "lambda_min": 0.09929368644952774, "lambda_max": 0.2884159982204437, "S_mean": 0.2666829427083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2940, "lr": 0.0005312891668850364, "train_loss": 4.989362478256226, "grad_norm": 0.6710571050643921, "tokens_seen": 24092672, "tokens_per_sec": 21117.91068277396, "vram_gb": 5.521900544, "lambda": 0.18699054171641669, "lambda_min": 0.0993034690618515, "lambda_max": 0.290202796459198, "S_mean": 0.2673746744791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2960, "lr": 0.0005303281222113367, "train_loss": 5.007481098175049, "grad_norm": 0.6932963132858276, "tokens_seen": 24256512, "tokens_per_sec": 21118.358416089737, "vram_gb": 5.521900544, "lambda": 0.1876399659862121, "lambda_min": 0.09909281134605408, "lambda_max": 0.29159632325172424, "S_mean": 0.2614847819010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 2980, "lr": 0.0005293613976909113, "train_loss": 5.003688812255859, "grad_norm": 0.7495265007019043, "tokens_seen": 24420352, "tokens_per_sec": 21117.41094638799, "vram_gb": 5.521900544, "lambda": 0.1881393746783336, "lambda_min": 0.09852970391511917, "lambda_max": 0.29318588972091675, "S_mean": 0.271209716796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3000, "lr": 0.0005283890207330281, "train_loss": 4.978726625442505, "grad_norm": 0.6688151359558105, "tokens_seen": 24584192, "tokens_per_sec": 21117.455233394456, "vram_gb": 5.521900544, "lambda": 0.18870916217565536, "lambda_min": 0.09821147471666336, "lambda_max": 0.29486143589019775, "S_mean": 0.2656453450520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 3000, "val_loss": 5.00460991859436, "val_ppl": 149.098911100385, "lambda": 0.18874848602960506, "lambda_min": 0.09818661212921143, "lambda_max": 0.2949604094028473, "lambda_per_layer": [0.15711571276187897, 0.1347751021385193, 0.13902448117733002, 0.13364902138710022, 0.13941164314746857, 0.1338009387254715, 0.12442333251237869, 0.1122720018029213, 0.09818661212921143, 0.12429241091012955, 0.18498384952545166, 0.2429323047399521, 0.21503980457782745, 0.2897709906101227, 0.2826204299926758, 0.2949604094028473, 0.27163493633270264, 0.17714059352874756, 0.21724000573158264, 0.25932252407073975, 0.22628214955329895, 0.23454393446445465, 0.1618727743625641, 0.17466770112514496], "S_mean": 0.264801025390625, "G_mean": null, "G_std": null, "tokens_seen": 24584192} +{"event": "step", "step": 3020, "lr": 0.0005274110189072165, "train_loss": 4.911332130432129, "grad_norm": 0.6993024945259094, "tokens_seen": 24748032, "tokens_per_sec": 21092.357733648612, "vram_gb": 5.521900544, "lambda": 0.18933993360648552, "lambda_min": 0.09796740114688873, "lambda_max": 0.29641062021255493, "S_mean": 0.2533365885416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3040, "lr": 0.0005264274199424862, "train_loss": 4.856982946395874, "grad_norm": 0.6954782605171204, "tokens_seen": 24911872, "tokens_per_sec": 21089.70486418132, "vram_gb": 5.521900544, "lambda": 0.18986332695931196, "lambda_min": 0.09779882431030273, "lambda_max": 0.2976178228855133, "S_mean": 0.2529296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3060, "lr": 0.0005254382517265413, "train_loss": 5.139976739883423, "grad_norm": 0.7626746296882629, "tokens_seen": 25075712, "tokens_per_sec": 21087.43576423618, "vram_gb": 5.521900544, "lambda": 0.19043198755631843, "lambda_min": 0.09756934642791748, "lambda_max": 0.299089640378952, "S_mean": 0.24599202473958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3080, "lr": 0.000524443542304989, "train_loss": 5.015273809432983, "grad_norm": 0.7039650082588196, "tokens_seen": 25239552, "tokens_per_sec": 21085.963592470984, "vram_gb": 5.521900544, "lambda": 0.19102840746442476, "lambda_min": 0.0970916599035263, "lambda_max": 0.3003815710544586, "S_mean": 0.2821553548177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3100, "lr": 0.000523443319880545, "train_loss": 5.042913436889648, "grad_norm": 0.7280545830726624, "tokens_seen": 25403392, "tokens_per_sec": 21083.296300826838, "vram_gb": 5.521900544, "lambda": 0.19160251008967558, "lambda_min": 0.09658325463533401, "lambda_max": 0.30162885785102844, "S_mean": 0.2607218424479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3120, "lr": 0.0005224376128122334, "train_loss": 4.92411994934082, "grad_norm": 0.6772509217262268, "tokens_seen": 25567232, "tokens_per_sec": 21082.534927938934, "vram_gb": 5.521900544, "lambda": 0.19221791283537945, "lambda_min": 0.09626476466655731, "lambda_max": 0.30319252610206604, "S_mean": 0.2527974446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3140, "lr": 0.0005214264496145829, "train_loss": 4.919338226318359, "grad_norm": 0.7190325856208801, "tokens_seen": 25731072, "tokens_per_sec": 21082.10665898934, "vram_gb": 5.521900544, "lambda": 0.1928134939322869, "lambda_min": 0.095911905169487, "lambda_max": 0.30479660630226135, "S_mean": 0.2605082194010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3160, "lr": 0.0005204098589568184, "train_loss": 5.031998872756958, "grad_norm": 0.6920601725578308, "tokens_seen": 25894912, "tokens_per_sec": 21081.75402920748, "vram_gb": 5.521900544, "lambda": 0.19343890187640986, "lambda_min": 0.09540003538131714, "lambda_max": 0.30617281794548035, "S_mean": 0.2669779459635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3180, "lr": 0.0005193878696620481, "train_loss": 4.927802085876465, "grad_norm": 0.7403512597084045, "tokens_seen": 26058752, "tokens_per_sec": 21080.513252776585, "vram_gb": 5.521900544, "lambda": 0.1939571142817537, "lambda_min": 0.09512367099523544, "lambda_max": 0.30777671933174133, "S_mean": 0.24338785807291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3200, "lr": 0.0005183605107064458, "train_loss": 4.937707185745239, "grad_norm": 0.7143543362617493, "tokens_seen": 26222592, "tokens_per_sec": 21080.03974425784, "vram_gb": 5.521900544, "lambda": 0.19447297789156437, "lambda_min": 0.09506773948669434, "lambda_max": 0.3089403808116913, "S_mean": 0.2535909016927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3220, "lr": 0.0005173278112184304, "train_loss": 5.115887880325317, "grad_norm": 0.7840239405632019, "tokens_seen": 26386432, "tokens_per_sec": 21079.27045184995, "vram_gb": 5.521900544, "lambda": 0.19506021651128927, "lambda_min": 0.09511084854602814, "lambda_max": 0.3105158805847168, "S_mean": 0.2546488444010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3240, "lr": 0.0005162898004778389, "train_loss": 4.960320234298706, "grad_norm": 0.7243950366973877, "tokens_seen": 26550272, "tokens_per_sec": 21079.48940420267, "vram_gb": 5.521900544, "lambda": 0.19565477998306355, "lambda_min": 0.09491568058729172, "lambda_max": 0.3121793270111084, "S_mean": 0.25042724609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3260, "lr": 0.000515246507915097, "train_loss": 4.965903997421265, "grad_norm": 0.696716845035553, "tokens_seen": 26714112, "tokens_per_sec": 21079.09486400042, "vram_gb": 5.521900544, "lambda": 0.19619773235172033, "lambda_min": 0.09472773224115372, "lambda_max": 0.31346720457077026, "S_mean": 0.249481201171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3280, "lr": 0.0005141979631103845, "train_loss": 4.859595775604248, "grad_norm": 0.7365702986717224, "tokens_seen": 26877952, "tokens_per_sec": 21078.543933517507, "vram_gb": 5.521900544, "lambda": 0.19677575336148342, "lambda_min": 0.09450791031122208, "lambda_max": 0.31504741311073303, "S_mean": 0.265716552734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3300, "lr": 0.0005131441957927962, "train_loss": 4.9473717212677, "grad_norm": 0.7109087109565735, "tokens_seen": 27041792, "tokens_per_sec": 21077.054865514245, "vram_gb": 5.521900544, "lambda": 0.19731864022711912, "lambda_min": 0.09432325512170792, "lambda_max": 0.3163870573043823, "S_mean": 0.275787353515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3320, "lr": 0.0005120852358394997, "train_loss": 4.8484132289886475, "grad_norm": 0.6944897174835205, "tokens_seen": 27205632, "tokens_per_sec": 21076.034339047095, "vram_gb": 5.521900544, "lambda": 0.1978702029834191, "lambda_min": 0.094337597489357, "lambda_max": 0.3177197575569153, "S_mean": 0.2592569986979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3340, "lr": 0.0005110211132748877, "train_loss": 4.955989122390747, "grad_norm": 0.7205293774604797, "tokens_seen": 27369472, "tokens_per_sec": 21076.02006518124, "vram_gb": 5.521900544, "lambda": 0.19837298958251873, "lambda_min": 0.09432189911603928, "lambda_max": 0.3191564381122589, "S_mean": 0.24479166666666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3360, "lr": 0.0005099518582697269, "train_loss": 4.919289588928223, "grad_norm": 0.7658808827400208, "tokens_seen": 27533312, "tokens_per_sec": 21076.382906927607, "vram_gb": 5.521900544, "lambda": 0.1989029347896576, "lambda_min": 0.09421434998512268, "lambda_max": 0.32077673077583313, "S_mean": 0.2723592122395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3380, "lr": 0.000508877501140303, "train_loss": 4.816269636154175, "grad_norm": 0.6663738489151001, "tokens_seen": 27697152, "tokens_per_sec": 21076.15528868304, "vram_gb": 5.521900544, "lambda": 0.1994598532716433, "lambda_min": 0.09415221214294434, "lambda_max": 0.3223787844181061, "S_mean": 0.248748779296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3400, "lr": 0.0005077980723475604, "train_loss": 4.8375468254089355, "grad_norm": 0.7494646906852722, "tokens_seen": 27860992, "tokens_per_sec": 21075.481246347837, "vram_gb": 5.521900544, "lambda": 0.19993178887913624, "lambda_min": 0.09385443478822708, "lambda_max": 0.3238155245780945, "S_mean": 0.2527567545572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3420, "lr": 0.0005067136024962391, "train_loss": 4.85534930229187, "grad_norm": 0.6824119091033936, "tokens_seen": 28024832, "tokens_per_sec": 21075.47674141374, "vram_gb": 5.521900544, "lambda": 0.20046847872436047, "lambda_min": 0.09371668100357056, "lambda_max": 0.32520002126693726, "S_mean": 0.2457275390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3440, "lr": 0.0005056241223340067, "train_loss": 4.781128168106079, "grad_norm": 0.6575406789779663, "tokens_seen": 28188672, "tokens_per_sec": 21075.59633218305, "vram_gb": 5.521900544, "lambda": 0.20100337794671455, "lambda_min": 0.09340102225542068, "lambda_max": 0.3267427682876587, "S_mean": 0.26910400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3460, "lr": 0.0005045296627505872, "train_loss": 4.8055644035339355, "grad_norm": 0.697329044342041, "tokens_seen": 28352512, "tokens_per_sec": 21075.632469034357, "vram_gb": 5.521900544, "lambda": 0.20160925357292095, "lambda_min": 0.09341689199209213, "lambda_max": 0.3284183144569397, "S_mean": 0.262481689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3480, "lr": 0.0005034302547768841, "train_loss": 5.032242774963379, "grad_norm": 0.7251992225646973, "tokens_seen": 28516352, "tokens_per_sec": 21076.08736636222, "vram_gb": 5.521900544, "lambda": 0.20220927459498247, "lambda_min": 0.09339086711406708, "lambda_max": 0.32976335287094116, "S_mean": 0.262176513671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3500, "lr": 0.0005023259295841014, "train_loss": 4.9472267627716064, "grad_norm": 0.6834469437599182, "tokens_seen": 28680192, "tokens_per_sec": 21075.273889177723, "vram_gb": 5.521900544, "lambda": 0.2027640532081326, "lambda_min": 0.09324988722801208, "lambda_max": 0.3310371935367584, "S_mean": 0.2615763346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3520, "lr": 0.0005012167184828599, "train_loss": 4.7838287353515625, "grad_norm": 0.7386937737464905, "tokens_seen": 28844032, "tokens_per_sec": 21074.685395093817, "vram_gb": 5.521900544, "lambda": 0.2032819570352634, "lambda_min": 0.09302269667387009, "lambda_max": 0.33277320861816406, "S_mean": 0.2649739583333333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3540, "lr": 0.0005001026529223088, "train_loss": 5.047935485839844, "grad_norm": 0.7419058680534363, "tokens_seen": 29007872, "tokens_per_sec": 21074.357718116487, "vram_gb": 5.521900544, "lambda": 0.203931444324553, "lambda_min": 0.09310545027256012, "lambda_max": 0.3342442214488983, "S_mean": 0.2543131510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3560, "lr": 0.0004989837644892347, "train_loss": 4.929050445556641, "grad_norm": 0.7114051580429077, "tokens_seen": 29171712, "tokens_per_sec": 21074.204587978373, "vram_gb": 5.521900544, "lambda": 0.20451344860096773, "lambda_min": 0.09292924404144287, "lambda_max": 0.33551496267318726, "S_mean": 0.2638448079427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3580, "lr": 0.0004978600849071656, "train_loss": 5.163308143615723, "grad_norm": 0.7461475133895874, "tokens_seen": 29335552, "tokens_per_sec": 21074.055827063225, "vram_gb": 5.521900544, "lambda": 0.20512024126946926, "lambda_min": 0.09278450906276703, "lambda_max": 0.3371707499027252, "S_mean": 0.2669881184895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3600, "lr": 0.0004967316460354718, "train_loss": 4.909942388534546, "grad_norm": 0.7252492308616638, "tokens_seen": 29499392, "tokens_per_sec": 21073.712470542858, "vram_gb": 5.521900544, "lambda": 0.20564792398363352, "lambda_min": 0.09251102805137634, "lambda_max": 0.33862215280532837, "S_mean": 0.2557779947916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3620, "lr": 0.000495598479868462, "train_loss": 4.7674560546875, "grad_norm": 0.7517778277397156, "tokens_seen": 29663232, "tokens_per_sec": 21074.21480180539, "vram_gb": 5.521900544, "lambda": 0.20622948867579302, "lambda_min": 0.09226067364215851, "lambda_max": 0.3400430977344513, "S_mean": 0.256103515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3640, "lr": 0.000494460618534477, "train_loss": 4.901851654052734, "grad_norm": 0.7441741228103638, "tokens_seen": 29827072, "tokens_per_sec": 21071.7790595043, "vram_gb": 5.521900544, "lambda": 0.2067316916460792, "lambda_min": 0.09191528707742691, "lambda_max": 0.34132450819015503, "S_mean": 0.2530619303385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3660, "lr": 0.0004933180942949779, "train_loss": 4.836073875427246, "grad_norm": 0.9264237284660339, "tokens_seen": 29990912, "tokens_per_sec": 21066.272848555236, "vram_gb": 5.521900544, "lambda": 0.20736028160899878, "lambda_min": 0.0914834588766098, "lambda_max": 0.3432711958885193, "S_mean": 0.2800394694010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3680, "lr": 0.0004921709395436325, "train_loss": 4.9077537059783936, "grad_norm": 0.7345581650733948, "tokens_seen": 30154752, "tokens_per_sec": 21060.86383507243, "vram_gb": 5.521900544, "lambda": 0.20791446417570114, "lambda_min": 0.09104796499013901, "lambda_max": 0.3447501063346863, "S_mean": 0.273193359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3700, "lr": 0.0004910191868053954, "train_loss": 4.8940348625183105, "grad_norm": 0.7523168325424194, "tokens_seen": 30318592, "tokens_per_sec": 21055.037124131246, "vram_gb": 5.521900544, "lambda": 0.20850591889272133, "lambda_min": 0.09090951085090637, "lambda_max": 0.34596705436706543, "S_mean": 0.2532958984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3720, "lr": 0.0004898628687355871, "train_loss": 4.799189567565918, "grad_norm": 0.7029179334640503, "tokens_seen": 30482432, "tokens_per_sec": 21047.58559855948, "vram_gb": 5.521900544, "lambda": 0.20908387936651707, "lambda_min": 0.09065547585487366, "lambda_max": 0.3474772572517395, "S_mean": 0.28173828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3740, "lr": 0.0004887020181189677, "train_loss": 4.6354875564575195, "grad_norm": 0.6919098496437073, "tokens_seen": 30646272, "tokens_per_sec": 21044.302146977167, "vram_gb": 5.521900544, "lambda": 0.2096655803422133, "lambda_min": 0.09057760238647461, "lambda_max": 0.3488856554031372, "S_mean": 0.2776285807291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3760, "lr": 0.0004875366678688071, "train_loss": 4.795724391937256, "grad_norm": 0.7384501099586487, "tokens_seen": 30810112, "tokens_per_sec": 21038.413903284425, "vram_gb": 5.521900544, "lambda": 0.21020421975602707, "lambda_min": 0.09059660136699677, "lambda_max": 0.3499046564102173, "S_mean": 0.26275634765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3780, "lr": 0.00048636685102595214, "train_loss": 4.792783737182617, "grad_norm": 0.8162615895271301, "tokens_seen": 30973952, "tokens_per_sec": 21030.32788533645, "vram_gb": 5.521900544, "lambda": 0.21075239218771458, "lambda_min": 0.09037768840789795, "lambda_max": 0.35155898332595825, "S_mean": 0.2570393880208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3800, "lr": 0.00048519260075788974, "train_loss": 4.861346244812012, "grad_norm": 0.7484921813011169, "tokens_seen": 31137792, "tokens_per_sec": 21024.892618346017, "vram_gb": 5.521900544, "lambda": 0.2112573158616821, "lambda_min": 0.09008225798606873, "lambda_max": 0.3526338040828705, "S_mean": 0.252197265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3820, "lr": 0.00048401395035780644, "train_loss": 4.820066928863525, "grad_norm": 0.6979297399520874, "tokens_seen": 31301632, "tokens_per_sec": 21018.26368201123, "vram_gb": 5.521900544, "lambda": 0.21174217698474726, "lambda_min": 0.09002593159675598, "lambda_max": 0.3539329171180725, "S_mean": 0.2776590983072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3840, "lr": 0.00048283093324364443, "train_loss": 4.62475323677063, "grad_norm": 0.7035164833068848, "tokens_seen": 31465472, "tokens_per_sec": 21009.829448783676, "vram_gb": 5.521900544, "lambda": 0.21226417956252894, "lambda_min": 0.08989107608795166, "lambda_max": 0.3553246259689331, "S_mean": 0.24571736653645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3860, "lr": 0.0004816435829571542, "train_loss": 4.920701742172241, "grad_norm": 0.7437624931335449, "tokens_seen": 31629312, "tokens_per_sec": 21005.461193787633, "vram_gb": 5.521900544, "lambda": 0.2128626083334287, "lambda_min": 0.08961892127990723, "lambda_max": 0.3569409251213074, "S_mean": 0.2712605794270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3880, "lr": 0.0004804519331629432, "train_loss": 4.858447074890137, "grad_norm": 0.7255768775939941, "tokens_seen": 31793152, "tokens_per_sec": 21001.788546786935, "vram_gb": 5.521900544, "lambda": 0.213404826199015, "lambda_min": 0.08943396061658859, "lambda_max": 0.35837215185165405, "S_mean": 0.2412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3900, "lr": 0.0004792560176475221, "train_loss": 4.87313437461853, "grad_norm": 0.7689458727836609, "tokens_seen": 31956992, "tokens_per_sec": 20999.09224805598, "vram_gb": 5.521900544, "lambda": 0.21396617560336986, "lambda_min": 0.08936115354299545, "lambda_max": 0.3597755432128906, "S_mean": 0.2625223795572917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3920, "lr": 0.00047805587031834595, "train_loss": 4.873087406158447, "grad_norm": 0.7878900170326233, "tokens_seen": 32120832, "tokens_per_sec": 20998.7933363372, "vram_gb": 5.521900544, "lambda": 0.2144981399178505, "lambda_min": 0.08900479972362518, "lambda_max": 0.3614019453525543, "S_mean": 0.27166748046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3940, "lr": 0.0004768515252028534, "train_loss": 4.7589545249938965, "grad_norm": 0.7594813108444214, "tokens_seen": 32284672, "tokens_per_sec": 20999.03134885359, "vram_gb": 5.521900544, "lambda": 0.21507881892224154, "lambda_min": 0.08862005919218063, "lambda_max": 0.3632943630218506, "S_mean": 0.2609965006510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3960, "lr": 0.0004756430164475016, "train_loss": 4.9632017612457275, "grad_norm": 0.9494003057479858, "tokens_seen": 32448512, "tokens_per_sec": 20999.79650984535, "vram_gb": 5.521900544, "lambda": 0.21565426296244064, "lambda_min": 0.08865821361541748, "lambda_max": 0.36452755331993103, "S_mean": 0.289031982421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 3980, "lr": 0.0004744303783167983, "train_loss": 4.815706253051758, "grad_norm": 0.8210580348968506, "tokens_seen": 32612352, "tokens_per_sec": 21000.042092010313, "vram_gb": 5.521900544, "lambda": 0.21620505085835853, "lambda_min": 0.08845318853855133, "lambda_max": 0.36578914523124695, "S_mean": 0.2514139811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4000, "lr": 0.0004732136451923304, "train_loss": 4.730746746063232, "grad_norm": 0.72492915391922, "tokens_seen": 32776192, "tokens_per_sec": 21000.703091905423, "vram_gb": 5.521900544, "lambda": 0.21682525301973024, "lambda_min": 0.0885234922170639, "lambda_max": 0.36739271879196167, "S_mean": 0.2519124348958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 4000, "val_loss": 4.904535269737243, "val_ppl": 134.9002032301777, "lambda": 0.2168537542844812, "lambda_min": 0.08851739764213562, "lambda_max": 0.3675021231174469, "lambda_per_layer": [0.16064444184303284, 0.1362314373254776, 0.14106854796409607, 0.1356101632118225, 0.14406749606132507, 0.1361757069826126, 0.12376898527145386, 0.11101100593805313, 0.08851739764213562, 0.12448830902576447, 0.19594134390354156, 0.2767564356327057, 0.24918493628501892, 0.3568788766860962, 0.34675332903862, 0.3675021231174469, 0.3498421609401703, 0.21255214512348175, 0.27106305956840515, 0.3369450867176056, 0.2838505208492279, 0.2878391444683075, 0.18160706758499146, 0.1861903816461563], "S_mean": 0.2528584798177083, "G_mean": null, "G_std": null, "tokens_seen": 32776192} +{"event": "step", "step": 4020, "lr": 0.00047199285157178874, "train_loss": 4.665363311767578, "grad_norm": 0.7320865392684937, "tokens_seen": 32940032, "tokens_per_sec": 20981.29005497685, "vram_gb": 5.521900544, "lambda": 0.21743608793864647, "lambda_min": 0.08839277923107147, "lambda_max": 0.3693768084049225, "S_mean": 0.2622477213541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4040, "lr": 0.0004707680320679905, "train_loss": 4.78705358505249, "grad_norm": 0.7089172005653381, "tokens_seen": 33103872, "tokens_per_sec": 20981.585866640886, "vram_gb": 5.521900544, "lambda": 0.21796236063043276, "lambda_min": 0.08839593082666397, "lambda_max": 0.370684951543808, "S_mean": 0.2550150553385417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4060, "lr": 0.0004695392214078974, "train_loss": 4.783505439758301, "grad_norm": 0.7380269765853882, "tokens_seen": 33267712, "tokens_per_sec": 20982.445835397026, "vram_gb": 5.521900544, "lambda": 0.21855009098847708, "lambda_min": 0.08833839744329453, "lambda_max": 0.3724956810474396, "S_mean": 0.27294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4080, "lr": 0.00046830645443163136, "train_loss": 4.808498382568359, "grad_norm": 0.823894202709198, "tokens_seen": 33431552, "tokens_per_sec": 20983.112835633812, "vram_gb": 5.521900544, "lambda": 0.21906473767012358, "lambda_min": 0.08804074674844742, "lambda_max": 0.3739355504512787, "S_mean": 0.2557576497395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4100, "lr": 0.0004670697660914867, "train_loss": 4.858686923980713, "grad_norm": 0.7741719484329224, "tokens_seen": 33595392, "tokens_per_sec": 20983.463798757988, "vram_gb": 5.521900544, "lambda": 0.2196924618134896, "lambda_min": 0.08800018578767776, "lambda_max": 0.3755638897418976, "S_mean": 0.2663675944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4120, "lr": 0.000465829191450939, "train_loss": 4.818402051925659, "grad_norm": 0.7746566534042358, "tokens_seen": 33759232, "tokens_per_sec": 20983.55818254885, "vram_gb": 5.521900544, "lambda": 0.22030672958741584, "lambda_min": 0.08784819394350052, "lambda_max": 0.3771976828575134, "S_mean": 0.24477132161458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4140, "lr": 0.0004645847656836509, "train_loss": 4.832482814788818, "grad_norm": 0.7418181896209717, "tokens_seen": 33923072, "tokens_per_sec": 20983.691901061273, "vram_gb": 5.521900544, "lambda": 0.22091180055091778, "lambda_min": 0.08769197016954422, "lambda_max": 0.37911751866340637, "S_mean": 0.24875895182291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4160, "lr": 0.00046333652407247504, "train_loss": 4.93041205406189, "grad_norm": 0.9914360046386719, "tokens_seen": 34086912, "tokens_per_sec": 20984.005820419618, "vram_gb": 5.521900544, "lambda": 0.2215104770536224, "lambda_min": 0.08752234280109406, "lambda_max": 0.3806181848049164, "S_mean": 0.2742513020833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4180, "lr": 0.0004620845020084538, "train_loss": 4.8146162033081055, "grad_norm": 0.7392452359199524, "tokens_seen": 34250752, "tokens_per_sec": 20984.32882265992, "vram_gb": 5.521900544, "lambda": 0.22200161932657161, "lambda_min": 0.0872272402048111, "lambda_max": 0.38204050064086914, "S_mean": 0.23673502604166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4200, "lr": 0.00046082873498981547, "train_loss": 4.8227574825286865, "grad_norm": 0.7859405279159546, "tokens_seen": 34414592, "tokens_per_sec": 20983.725508663676, "vram_gb": 5.521900544, "lambda": 0.2225548354908824, "lambda_min": 0.08702220767736435, "lambda_max": 0.3832806646823883, "S_mean": 0.2609965006510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4220, "lr": 0.00045956925862096785, "train_loss": 4.770753383636475, "grad_norm": 0.7372036576271057, "tokens_seen": 34578432, "tokens_per_sec": 20983.919124443673, "vram_gb": 5.521900544, "lambda": 0.22313200837622085, "lambda_min": 0.08684711158275604, "lambda_max": 0.38464805483818054, "S_mean": 0.24888102213541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4240, "lr": 0.00045830610861148923, "train_loss": 4.675665378570557, "grad_norm": 0.7712346911430359, "tokens_seen": 34742272, "tokens_per_sec": 20984.454403069314, "vram_gb": 5.521900544, "lambda": 0.22368351370096207, "lambda_min": 0.08685725182294846, "lambda_max": 0.3860545754432678, "S_mean": 0.2545267740885417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4260, "lr": 0.00045703932077511525, "train_loss": 4.77797269821167, "grad_norm": 0.8720619678497314, "tokens_seen": 34906112, "tokens_per_sec": 20984.92643388159, "vram_gb": 5.521900544, "lambda": 0.2243651651466886, "lambda_min": 0.08712109178304672, "lambda_max": 0.3874223232269287, "S_mean": 0.23998006184895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4280, "lr": 0.00045576893102872417, "train_loss": 4.884914398193359, "grad_norm": 0.8302434682846069, "tokens_seen": 35069952, "tokens_per_sec": 20985.53190684115, "vram_gb": 5.521900544, "lambda": 0.22488130753238997, "lambda_min": 0.08704306185245514, "lambda_max": 0.38856789469718933, "S_mean": 0.247528076171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4300, "lr": 0.0004544949753913179, "train_loss": 4.904812574386597, "grad_norm": 0.7966921329498291, "tokens_seen": 35233792, "tokens_per_sec": 20985.709723813718, "vram_gb": 5.521900544, "lambda": 0.22540627575169006, "lambda_min": 0.08706936985254288, "lambda_max": 0.3898426294326782, "S_mean": 0.24408976236979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4320, "lr": 0.0004532174899830013, "train_loss": 4.718127727508545, "grad_norm": 0.885789692401886, "tokens_seen": 35397632, "tokens_per_sec": 20986.7997443111, "vram_gb": 5.521900544, "lambda": 0.22602208610624075, "lambda_min": 0.0871044471859932, "lambda_max": 0.3910902440547943, "S_mean": 0.24382527669270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4340, "lr": 0.00045193651102395753, "train_loss": 4.953683614730835, "grad_norm": 0.7270557284355164, "tokens_seen": 35561472, "tokens_per_sec": 20987.106238406886, "vram_gb": 5.521900544, "lambda": 0.2265478310485681, "lambda_min": 0.08685030043125153, "lambda_max": 0.3925219774246216, "S_mean": 0.249298095703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4360, "lr": 0.0004506520748334217, "train_loss": 4.746351003646851, "grad_norm": 0.7758455872535706, "tokens_seen": 35725312, "tokens_per_sec": 20987.970653656554, "vram_gb": 5.521900544, "lambda": 0.22713151387870312, "lambda_min": 0.08689168095588684, "lambda_max": 0.3942837119102478, "S_mean": 0.25665283203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4380, "lr": 0.00044936421782865074, "train_loss": 4.452578067779541, "grad_norm": 0.73456871509552, "tokens_seen": 35889152, "tokens_per_sec": 20988.224761380636, "vram_gb": 5.521900544, "lambda": 0.22765629583348831, "lambda_min": 0.08677352219820023, "lambda_max": 0.39580389857292175, "S_mean": 0.24561564127604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4400, "lr": 0.0004480729765238909, "train_loss": 4.915068626403809, "grad_norm": 0.7669040560722351, "tokens_seen": 36052992, "tokens_per_sec": 20988.536937613128, "vram_gb": 5.521900544, "lambda": 0.22817335526148477, "lambda_min": 0.0867970660328865, "lambda_max": 0.39720991253852844, "S_mean": 0.24323527018229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4420, "lr": 0.0004467783875293425, "train_loss": 4.753709316253662, "grad_norm": 0.7818626761436462, "tokens_seen": 36216832, "tokens_per_sec": 20988.966901050684, "vram_gb": 5.521900544, "lambda": 0.22865010052919388, "lambda_min": 0.08654754608869553, "lambda_max": 0.3986794948577881, "S_mean": 0.2593485514322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4440, "lr": 0.00044548048755012207, "train_loss": 4.911590099334717, "grad_norm": 0.8263241648674011, "tokens_seen": 36380672, "tokens_per_sec": 20988.16800002638, "vram_gb": 5.521900544, "lambda": 0.22913893653700748, "lambda_min": 0.08648178726434708, "lambda_max": 0.4002920687198639, "S_mean": 0.2649129231770833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4460, "lr": 0.0004441793133852216, "train_loss": 4.621618747711182, "grad_norm": 0.7971652746200562, "tokens_seen": 36544512, "tokens_per_sec": 20988.951437764197, "vram_gb": 5.521900544, "lambda": 0.22961131514360508, "lambda_min": 0.08638656884431839, "lambda_max": 0.40119028091430664, "S_mean": 0.25286865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4480, "lr": 0.00044287490192646483, "train_loss": 4.764202117919922, "grad_norm": 0.729503333568573, "tokens_seen": 36708352, "tokens_per_sec": 20988.678946736116, "vram_gb": 5.521900544, "lambda": 0.23012209497392178, "lambda_min": 0.08614487946033478, "lambda_max": 0.40280812978744507, "S_mean": 0.2610270182291667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4500, "lr": 0.00044156729015746205, "train_loss": 4.853525400161743, "grad_norm": 0.7659574747085571, "tokens_seen": 36872192, "tokens_per_sec": 20989.091857572887, "vram_gb": 5.521900544, "lambda": 0.2305930865307649, "lambda_min": 0.08590622246265411, "lambda_max": 0.4037622809410095, "S_mean": 0.2539265950520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4520, "lr": 0.00044025651515256067, "train_loss": 4.553828716278076, "grad_norm": 0.8211483359336853, "tokens_seen": 37036032, "tokens_per_sec": 20989.50462523386, "vram_gb": 5.521900544, "lambda": 0.23102025563518205, "lambda_min": 0.0857619121670723, "lambda_max": 0.40451759099960327, "S_mean": 0.2568359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4540, "lr": 0.0004389426140757946, "train_loss": 4.655303955078125, "grad_norm": 0.7319908142089844, "tokens_seen": 37199872, "tokens_per_sec": 20989.554786876342, "vram_gb": 5.521900544, "lambda": 0.231504092613856, "lambda_min": 0.08562915027141571, "lambda_max": 0.4057779014110565, "S_mean": 0.2534688313802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4560, "lr": 0.00043762562417983024, "train_loss": 4.651759386062622, "grad_norm": 0.7435259222984314, "tokens_seen": 37363712, "tokens_per_sec": 20990.10477606453, "vram_gb": 5.521900544, "lambda": 0.23208291052530208, "lambda_min": 0.08540035039186478, "lambda_max": 0.40697911381721497, "S_mean": 0.2527974446614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4580, "lr": 0.0004363055828049108, "train_loss": 4.7152159214019775, "grad_norm": 0.778947651386261, "tokens_seen": 37527552, "tokens_per_sec": 20990.714682208305, "vram_gb": 5.521900544, "lambda": 0.2325651136537393, "lambda_min": 0.08527194708585739, "lambda_max": 0.40844985842704773, "S_mean": 0.2640279134114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4600, "lr": 0.00043498252737779653, "train_loss": 4.6074113845825195, "grad_norm": 0.7982891201972961, "tokens_seen": 37691392, "tokens_per_sec": 20991.598956481586, "vram_gb": 5.521900544, "lambda": 0.23304426887383065, "lambda_min": 0.08507049828767776, "lambda_max": 0.4097978174686432, "S_mean": 0.2540791829427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4620, "lr": 0.0004336564954107049, "train_loss": 4.706508636474609, "grad_norm": 0.7768170237541199, "tokens_seen": 37855232, "tokens_per_sec": 20992.501084627853, "vram_gb": 5.521900544, "lambda": 0.23353508673608303, "lambda_min": 0.08468334376811981, "lambda_max": 0.41137295961380005, "S_mean": 0.2604573567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4640, "lr": 0.00043232752450024606, "train_loss": 4.756089925765991, "grad_norm": 0.8121845722198486, "tokens_seen": 38019072, "tokens_per_sec": 20993.203665010424, "vram_gb": 5.521900544, "lambda": 0.23411860689520836, "lambda_min": 0.0844510942697525, "lambda_max": 0.4134259521961212, "S_mean": 0.246734619140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4660, "lr": 0.00043099565232635736, "train_loss": 4.693630933761597, "grad_norm": 0.7509578466415405, "tokens_seen": 38182912, "tokens_per_sec": 20993.796243424884, "vram_gb": 5.521900544, "lambda": 0.2345980784545342, "lambda_min": 0.08408661931753159, "lambda_max": 0.41459670662879944, "S_mean": 0.24503580729166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4680, "lr": 0.0004296609166512347, "train_loss": 4.636187553405762, "grad_norm": 0.8352367877960205, "tokens_seen": 38346752, "tokens_per_sec": 20994.854187286823, "vram_gb": 5.521900544, "lambda": 0.23515196641286215, "lambda_min": 0.08391930162906647, "lambda_max": 0.41597557067871094, "S_mean": 0.2767130533854167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4700, "lr": 0.00042832335531826194, "train_loss": 4.668982982635498, "grad_norm": 0.7586252093315125, "tokens_seen": 38510592, "tokens_per_sec": 20995.365933984704, "vram_gb": 5.521900544, "lambda": 0.23574292287230492, "lambda_min": 0.08382298052310944, "lambda_max": 0.4172360599040985, "S_mean": 0.24509684244791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4720, "lr": 0.00042698300625093827, "train_loss": 4.691728830337524, "grad_norm": 0.8486685752868652, "tokens_seen": 38674432, "tokens_per_sec": 20995.79678253618, "vram_gb": 5.521900544, "lambda": 0.23628123849630356, "lambda_min": 0.08377042412757874, "lambda_max": 0.41819602251052856, "S_mean": 0.24460856119791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4740, "lr": 0.0004256399074518026, "train_loss": 4.748567342758179, "grad_norm": 0.7982181906700134, "tokens_seen": 38838272, "tokens_per_sec": 20996.406922839033, "vram_gb": 5.521900544, "lambda": 0.2367440415546298, "lambda_min": 0.08365470916032791, "lambda_max": 0.4192596971988678, "S_mean": 0.2688700358072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4760, "lr": 0.00042429409700135625, "train_loss": 4.8019232749938965, "grad_norm": 0.8140795230865479, "tokens_seen": 39002112, "tokens_per_sec": 20996.89099995621, "vram_gb": 5.521900544, "lambda": 0.23729485180228949, "lambda_min": 0.0835786983370781, "lambda_max": 0.42054998874664307, "S_mean": 0.257171630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4780, "lr": 0.0004229456130569832, "train_loss": 4.660472869873047, "grad_norm": 0.8048344254493713, "tokens_seen": 39165952, "tokens_per_sec": 20997.182455353817, "vram_gb": 5.521900544, "lambda": 0.23778675279269615, "lambda_min": 0.08350322395563126, "lambda_max": 0.42200228571891785, "S_mean": 0.276763916015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4800, "lr": 0.00042159449385186805, "train_loss": 4.834096193313599, "grad_norm": 0.8016808032989502, "tokens_seen": 39329792, "tokens_per_sec": 20996.643489661237, "vram_gb": 5.521900544, "lambda": 0.23825581992665926, "lambda_min": 0.08338262885808945, "lambda_max": 0.42334219813346863, "S_mean": 0.2542826334635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4820, "lr": 0.0004202407776939127, "train_loss": 4.660287141799927, "grad_norm": 0.7970667481422424, "tokens_seen": 39493632, "tokens_per_sec": 20996.36152046047, "vram_gb": 5.521900544, "lambda": 0.23877115019907555, "lambda_min": 0.08348007500171661, "lambda_max": 0.4246319532394409, "S_mean": 0.260711669921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4840, "lr": 0.00041888450296464927, "train_loss": 4.573675632476807, "grad_norm": 0.8512887954711914, "tokens_seen": 39657472, "tokens_per_sec": 20995.832272377593, "vram_gb": 5.521900544, "lambda": 0.2393283365915219, "lambda_min": 0.08341620117425919, "lambda_max": 0.42583537101745605, "S_mean": 0.256591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4860, "lr": 0.0004175257081181524, "train_loss": 4.6443750858306885, "grad_norm": 0.7893083691596985, "tokens_seen": 39821312, "tokens_per_sec": 20996.17373115887, "vram_gb": 5.521900544, "lambda": 0.2398453870167335, "lambda_min": 0.08345115929841995, "lambda_max": 0.42703813314437866, "S_mean": 0.2626546223958333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4880, "lr": 0.0004161644316799491, "train_loss": 4.637110710144043, "grad_norm": 0.7462030053138733, "tokens_seen": 39985152, "tokens_per_sec": 20996.872299689414, "vram_gb": 5.521900544, "lambda": 0.24035916198045015, "lambda_min": 0.08342889696359634, "lambda_max": 0.42792990803718567, "S_mean": 0.267578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4900, "lr": 0.00041480071224592614, "train_loss": 4.699953079223633, "grad_norm": 0.7902789115905762, "tokens_seen": 40148992, "tokens_per_sec": 20997.303953323324, "vram_gb": 5.521900544, "lambda": 0.24080803555746874, "lambda_min": 0.08323920518159866, "lambda_max": 0.42928385734558105, "S_mean": 0.2596842447916667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4920, "lr": 0.00041343458848123576, "train_loss": 4.671994686126709, "grad_norm": 0.8023773431777954, "tokens_seen": 40312832, "tokens_per_sec": 20997.86570942013, "vram_gb": 5.521900544, "lambda": 0.2412382693340381, "lambda_min": 0.08312465250492096, "lambda_max": 0.4306276738643646, "S_mean": 0.2641194661458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4940, "lr": 0.0004120660991191996, "train_loss": 4.64241623878479, "grad_norm": 0.7922612428665161, "tokens_seen": 40476672, "tokens_per_sec": 20998.05751468646, "vram_gb": 5.521900544, "lambda": 0.24167581057796875, "lambda_min": 0.08307361602783203, "lambda_max": 0.431959331035614, "S_mean": 0.2620137532552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4960, "lr": 0.00041069528296021053, "train_loss": 4.702622890472412, "grad_norm": 0.8026744723320007, "tokens_seen": 40640512, "tokens_per_sec": 20998.6552194327, "vram_gb": 5.521900544, "lambda": 0.2422363112370173, "lambda_min": 0.0829429179430008, "lambda_max": 0.4332636296749115, "S_mean": 0.25927734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 4980, "lr": 0.0004093221788706323, "train_loss": 4.631278991699219, "grad_norm": 0.7689884305000305, "tokens_seen": 40804352, "tokens_per_sec": 20994.05062962243, "vram_gb": 5.521900544, "lambda": 0.24277615236739317, "lambda_min": 0.08278759568929672, "lambda_max": 0.4343682825565338, "S_mean": 0.250244140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5000, "lr": 0.0004079468257816976, "train_loss": 4.796159029006958, "grad_norm": 0.8580016493797302, "tokens_seen": 40968192, "tokens_per_sec": 20993.29138311294, "vram_gb": 5.521900544, "lambda": 0.2432463833441337, "lambda_min": 0.0826098695397377, "lambda_max": 0.4355867803096771, "S_mean": 0.24446614583333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 5000, "val_loss": 4.764658093452454, "val_ppl": 117.29100786780445, "lambda": 0.24326783635963997, "lambda_min": 0.08260264992713928, "lambda_max": 0.4356656074523926, "lambda_per_layer": [0.1644657552242279, 0.13582465052604675, 0.1418795883655548, 0.13857600092887878, 0.14940288662910461, 0.1381380259990692, 0.12384593486785889, 0.11181984096765518, 0.08260264992713928, 0.1260332614183426, 0.2046521157026291, 0.30297747254371643, 0.2781093716621399, 0.4155898690223694, 0.40573301911354065, 0.4356656074523926, 0.42222434282302856, 0.2473238706588745, 0.3214934468269348, 0.4113389551639557, 0.34074875712394714, 0.33664238452911377, 0.20031297206878662, 0.20302729308605194], "S_mean": 0.254486083984375, "G_mean": null, "G_std": null, "tokens_seen": 40968192} +{"event": "step", "step": 5020, "lr": 0.00040656926268840463, "train_loss": 4.746846437454224, "grad_norm": 0.7749696373939514, "tokens_seen": 41132032, "tokens_per_sec": 20976.698020338088, "vram_gb": 5.521900544, "lambda": 0.2437811428681016, "lambda_min": 0.08257928490638733, "lambda_max": 0.43673890829086304, "S_mean": 0.2578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5040, "lr": 0.00040518952864841104, "train_loss": 4.823336601257324, "grad_norm": 0.7823236584663391, "tokens_seen": 41295872, "tokens_per_sec": 20974.640835706847, "vram_gb": 5.521900544, "lambda": 0.24429051919529834, "lambda_min": 0.08265912532806396, "lambda_max": 0.4381469190120697, "S_mean": 0.24977620442708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5060, "lr": 0.00040380766278092677, "train_loss": 4.6645307540893555, "grad_norm": 0.9060154557228088, "tokens_seen": 41459712, "tokens_per_sec": 20974.950351493557, "vram_gb": 5.521900544, "lambda": 0.24483079059670368, "lambda_min": 0.08264901489019394, "lambda_max": 0.4394305646419525, "S_mean": 0.2650960286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5080, "lr": 0.0004024237042656049, "train_loss": 4.760303735733032, "grad_norm": 0.7955174446105957, "tokens_seen": 41623552, "tokens_per_sec": 20974.807601149383, "vram_gb": 5.521900544, "lambda": 0.2453694473952055, "lambda_min": 0.08269528299570084, "lambda_max": 0.4406185746192932, "S_mean": 0.23569742838541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5100, "lr": 0.00040103769234143063, "train_loss": 4.7001893520355225, "grad_norm": 0.8335199356079102, "tokens_seen": 41787392, "tokens_per_sec": 20975.293755604293, "vram_gb": 5.521900544, "lambda": 0.24584175242731968, "lambda_min": 0.08270849287509918, "lambda_max": 0.4415881931781769, "S_mean": 0.23570760091145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5120, "lr": 0.0003996496663056091, "train_loss": 4.7367026805877686, "grad_norm": 0.8139266967773438, "tokens_seen": 41951232, "tokens_per_sec": 20975.63233070168, "vram_gb": 5.521900544, "lambda": 0.24632523177812496, "lambda_min": 0.08267456293106079, "lambda_max": 0.4430226683616638, "S_mean": 0.24341837565104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5140, "lr": 0.0003982596655124508, "train_loss": 4.66167688369751, "grad_norm": 0.8249711990356445, "tokens_seen": 42115072, "tokens_per_sec": 20976.137267510327, "vram_gb": 5.521900544, "lambda": 0.24676122547437748, "lambda_min": 0.08254222571849823, "lambda_max": 0.44413211941719055, "S_mean": 0.248565673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5160, "lr": 0.000396867729372256, "train_loss": 4.758740186691284, "grad_norm": 0.8306905627250671, "tokens_seen": 42278912, "tokens_per_sec": 20976.67654013793, "vram_gb": 5.521900544, "lambda": 0.24725271513064703, "lambda_min": 0.08249625563621521, "lambda_max": 0.44553232192993164, "S_mean": 0.24537150065104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5180, "lr": 0.0003954738973501974, "train_loss": 4.563353538513184, "grad_norm": 0.7622811198234558, "tokens_seen": 42442752, "tokens_per_sec": 20977.183482180168, "vram_gb": 5.521900544, "lambda": 0.24776360609879097, "lambda_min": 0.08234363794326782, "lambda_max": 0.4470508098602295, "S_mean": 0.243743896484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5200, "lr": 0.00039407820896520095, "train_loss": 4.539409637451172, "grad_norm": 0.789727509021759, "tokens_seen": 42606592, "tokens_per_sec": 20977.392571603425, "vram_gb": 5.521900544, "lambda": 0.2482512422526876, "lambda_min": 0.08236667513847351, "lambda_max": 0.448342502117157, "S_mean": 0.2707621256510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5220, "lr": 0.00039268070378882574, "train_loss": 4.713792085647583, "grad_norm": 0.8133523464202881, "tokens_seen": 42770432, "tokens_per_sec": 20977.417508145754, "vram_gb": 5.521900544, "lambda": 0.2487087231129408, "lambda_min": 0.0824456512928009, "lambda_max": 0.44953301548957825, "S_mean": 0.2502543131510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5240, "lr": 0.0003912814214441416, "train_loss": 4.752437114715576, "grad_norm": 0.8513299226760864, "tokens_seen": 42934272, "tokens_per_sec": 20978.17264876147, "vram_gb": 5.521900544, "lambda": 0.24908532047023377, "lambda_min": 0.0823821872472763, "lambda_max": 0.45085006952285767, "S_mean": 0.2584126790364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5260, "lr": 0.0003898804016046061, "train_loss": 4.632939338684082, "grad_norm": 0.7819294333457947, "tokens_seen": 43098112, "tokens_per_sec": 20978.48994376632, "vram_gb": 5.521900544, "lambda": 0.24966076792528233, "lambda_min": 0.0823960229754448, "lambda_max": 0.45188426971435547, "S_mean": 0.24302164713541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5280, "lr": 0.0003884776839929396, "train_loss": 4.621971130371094, "grad_norm": 0.7914143800735474, "tokens_seen": 43261952, "tokens_per_sec": 20979.22620322747, "vram_gb": 5.521900544, "lambda": 0.25018677581101656, "lambda_min": 0.08237546682357788, "lambda_max": 0.45317161083221436, "S_mean": 0.24656168619791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5300, "lr": 0.0003870733083799985, "train_loss": 4.639799118041992, "grad_norm": 0.7375804781913757, "tokens_seen": 43425792, "tokens_per_sec": 20979.435649131698, "vram_gb": 5.521900544, "lambda": 0.25063683837652206, "lambda_min": 0.08218514174222946, "lambda_max": 0.4544105529785156, "S_mean": 0.2623189290364583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5320, "lr": 0.00038566731458364843, "train_loss": 4.633275508880615, "grad_norm": 0.8025828003883362, "tokens_seen": 43589632, "tokens_per_sec": 20979.543686761364, "vram_gb": 5.521900544, "lambda": 0.25106844616432983, "lambda_min": 0.08197291940450668, "lambda_max": 0.45532286167144775, "S_mean": 0.25469970703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5340, "lr": 0.0003842597424676348, "train_loss": 4.728629112243652, "grad_norm": 0.7905840277671814, "tokens_seen": 43753472, "tokens_per_sec": 20980.06245865124, "vram_gb": 5.521900544, "lambda": 0.25150423776358366, "lambda_min": 0.08193046599626541, "lambda_max": 0.456369012594223, "S_mean": 0.2571512858072917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5360, "lr": 0.00038285063194045267, "train_loss": 4.658613204956055, "grad_norm": 0.7766525149345398, "tokens_seen": 43917312, "tokens_per_sec": 20980.57348285939, "vram_gb": 5.521900544, "lambda": 0.25195325290163356, "lambda_min": 0.08180127292871475, "lambda_max": 0.4574092626571655, "S_mean": 0.244171142578125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5380, "lr": 0.00038144002295421506, "train_loss": 4.615634202957153, "grad_norm": 0.7986853122711182, "tokens_seen": 44081152, "tokens_per_sec": 20980.78693046896, "vram_gb": 5.521900544, "lambda": 0.2524067560831706, "lambda_min": 0.0815381333231926, "lambda_max": 0.4586663842201233, "S_mean": 0.2535400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5400, "lr": 0.00038002795550352083, "train_loss": 4.724681377410889, "grad_norm": 0.8187639117240906, "tokens_seen": 44244992, "tokens_per_sec": 20980.845334230853, "vram_gb": 5.521900544, "lambda": 0.2528624360760053, "lambda_min": 0.08125879615545273, "lambda_max": 0.46034306287765503, "S_mean": 0.263275146484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5420, "lr": 0.00037861446962431973, "train_loss": 4.59291934967041, "grad_norm": 0.9825116395950317, "tokens_seen": 44408832, "tokens_per_sec": 20980.91910132478, "vram_gb": 5.521900544, "lambda": 0.25325274746865034, "lambda_min": 0.0812334343791008, "lambda_max": 0.4616692066192627, "S_mean": 0.26641845703125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5440, "lr": 0.0003771996053927782, "train_loss": 4.7693164348602295, "grad_norm": 0.8301616311073303, "tokens_seen": 44572672, "tokens_per_sec": 20980.954009731475, "vram_gb": 5.521900544, "lambda": 0.25368431645135087, "lambda_min": 0.08119583874940872, "lambda_max": 0.46325987577438354, "S_mean": 0.2671915690104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5460, "lr": 0.0003757834029241427, "train_loss": 4.582128524780273, "grad_norm": 0.8019569516181946, "tokens_seen": 44736512, "tokens_per_sec": 20981.589765099772, "vram_gb": 5.521900544, "lambda": 0.2540362663567066, "lambda_min": 0.08109891414642334, "lambda_max": 0.46442145109176636, "S_mean": 0.2514444986979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5480, "lr": 0.000374365902371602, "train_loss": 4.703950643539429, "grad_norm": 0.928266167640686, "tokens_seen": 44900352, "tokens_per_sec": 20981.73728061348, "vram_gb": 5.521900544, "lambda": 0.2545130178332329, "lambda_min": 0.08122553676366806, "lambda_max": 0.4656314551830292, "S_mean": 0.2759195963541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5500, "lr": 0.00037294714392514935, "train_loss": 4.722979784011841, "grad_norm": 0.8222254514694214, "tokens_seen": 45064192, "tokens_per_sec": 20982.541181330227, "vram_gb": 5.521900544, "lambda": 0.25493588640044135, "lambda_min": 0.08120850473642349, "lambda_max": 0.46687960624694824, "S_mean": 0.2700398763020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5520, "lr": 0.00037152716781044286, "train_loss": 4.83812952041626, "grad_norm": 0.8621094226837158, "tokens_seen": 45228032, "tokens_per_sec": 20982.56262248145, "vram_gb": 5.521900544, "lambda": 0.2554251039400697, "lambda_min": 0.08104021102190018, "lambda_max": 0.46805596351623535, "S_mean": 0.24445597330729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5540, "lr": 0.00037010601428766446, "train_loss": 4.739147901535034, "grad_norm": 0.8478983044624329, "tokens_seen": 45391872, "tokens_per_sec": 20983.01233655089, "vram_gb": 5.521900544, "lambda": 0.2558932552735011, "lambda_min": 0.0809500440955162, "lambda_max": 0.46914970874786377, "S_mean": 0.2555745442708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5560, "lr": 0.00036868372365037906, "train_loss": 4.532168388366699, "grad_norm": 0.8190029859542847, "tokens_seen": 45555712, "tokens_per_sec": 20983.384194473823, "vram_gb": 5.521900544, "lambda": 0.2563313903907935, "lambda_min": 0.08098025619983673, "lambda_max": 0.46986088156700134, "S_mean": 0.2560017903645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5580, "lr": 0.00036726033622439187, "train_loss": 4.573068618774414, "grad_norm": 0.8467618227005005, "tokens_seen": 45719552, "tokens_per_sec": 20983.368465518408, "vram_gb": 5.521900544, "lambda": 0.2568667586892843, "lambda_min": 0.08091621845960617, "lambda_max": 0.471194326877594, "S_mean": 0.2620137532552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5600, "lr": 0.0003658358923666049, "train_loss": 4.676116466522217, "grad_norm": 0.8735311031341553, "tokens_seen": 45883392, "tokens_per_sec": 20983.733706453484, "vram_gb": 5.521900544, "lambda": 0.2573286993429065, "lambda_min": 0.08095908910036087, "lambda_max": 0.4726669192314148, "S_mean": 0.23534138997395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5620, "lr": 0.0003644104324638729, "train_loss": 4.4331958293914795, "grad_norm": 0.792891263961792, "tokens_seen": 46047232, "tokens_per_sec": 20983.033747784055, "vram_gb": 5.521900544, "lambda": 0.2578336897616585, "lambda_min": 0.08080572634935379, "lambda_max": 0.47414731979370117, "S_mean": 0.24616495768229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5640, "lr": 0.0003629839969318583, "train_loss": 4.687671422958374, "grad_norm": 0.8136645555496216, "tokens_seen": 46211072, "tokens_per_sec": 20982.237258855836, "vram_gb": 5.521900544, "lambda": 0.2582383931924899, "lambda_min": 0.08082480728626251, "lambda_max": 0.474720299243927, "S_mean": 0.24995930989583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5660, "lr": 0.00036155662621388534, "train_loss": 4.715968608856201, "grad_norm": 0.8428239822387695, "tokens_seen": 46374912, "tokens_per_sec": 20982.485073872966, "vram_gb": 5.521900544, "lambda": 0.2586594820022583, "lambda_min": 0.08059646189212799, "lambda_max": 0.4757223427295685, "S_mean": 0.24922688802083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5680, "lr": 0.0003601283607797931, "train_loss": 4.536926031112671, "grad_norm": 0.8900238871574402, "tokens_seen": 46538752, "tokens_per_sec": 20982.58356223395, "vram_gb": 5.521900544, "lambda": 0.25910444495578605, "lambda_min": 0.08066832274198532, "lambda_max": 0.4765118956565857, "S_mean": 0.248565673828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5700, "lr": 0.0003586992411247886, "train_loss": 4.47566032409668, "grad_norm": 0.7751615643501282, "tokens_seen": 46702592, "tokens_per_sec": 20982.876556047642, "vram_gb": 5.521900544, "lambda": 0.2595475198080142, "lambda_min": 0.0806371420621872, "lambda_max": 0.47722893953323364, "S_mean": 0.2442626953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5720, "lr": 0.0003572693077682981, "train_loss": 4.628169775009155, "grad_norm": 0.8503604531288147, "tokens_seen": 46866432, "tokens_per_sec": 20982.97341191305, "vram_gb": 5.521900544, "lambda": 0.26010451031227905, "lambda_min": 0.08060170710086823, "lambda_max": 0.47876638174057007, "S_mean": 0.237274169921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5740, "lr": 0.00035583860125281867, "train_loss": 4.667150497436523, "grad_norm": 0.8277754187583923, "tokens_seen": 47030272, "tokens_per_sec": 20982.98256944623, "vram_gb": 5.521900544, "lambda": 0.2605771590024233, "lambda_min": 0.08044729381799698, "lambda_max": 0.4803851246833801, "S_mean": 0.2514851888020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5760, "lr": 0.00035440716214276834, "train_loss": 4.572441101074219, "grad_norm": 0.8524147868156433, "tokens_seen": 47194112, "tokens_per_sec": 20982.997026115634, "vram_gb": 5.521900544, "lambda": 0.2610245772326986, "lambda_min": 0.0803670883178711, "lambda_max": 0.4813030958175659, "S_mean": 0.241058349609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5780, "lr": 0.0003529750310233364, "train_loss": 4.546912908554077, "grad_norm": 0.8617548942565918, "tokens_seen": 47357952, "tokens_per_sec": 20983.249614306078, "vram_gb": 5.521900544, "lambda": 0.2614086118216316, "lambda_min": 0.08025161921977997, "lambda_max": 0.4823994040489197, "S_mean": 0.25909423828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5800, "lr": 0.0003515422484993324, "train_loss": 4.526669502258301, "grad_norm": 0.8720699548721313, "tokens_seen": 47521792, "tokens_per_sec": 20983.934777083097, "vram_gb": 5.521900544, "lambda": 0.26191724340120953, "lambda_min": 0.08025136590003967, "lambda_max": 0.48404234647750854, "S_mean": 0.2655131022135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5820, "lr": 0.0003501088551940349, "train_loss": 4.56260871887207, "grad_norm": 0.8596431016921997, "tokens_seen": 47685632, "tokens_per_sec": 20984.2239386737, "vram_gb": 5.521900544, "lambda": 0.26234522213538486, "lambda_min": 0.08009520918130875, "lambda_max": 0.48488759994506836, "S_mean": 0.2507425944010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5840, "lr": 0.00034867489174803993, "train_loss": 4.742162704467773, "grad_norm": 0.8986932635307312, "tokens_seen": 47849472, "tokens_per_sec": 20984.394140224485, "vram_gb": 5.521900544, "lambda": 0.2627431461587548, "lambda_min": 0.07999107986688614, "lambda_max": 0.4864889979362488, "S_mean": 0.24610392252604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5860, "lr": 0.00034724039881810844, "train_loss": 4.5738325119018555, "grad_norm": 0.8324095010757446, "tokens_seen": 48013312, "tokens_per_sec": 20984.4506369024, "vram_gb": 5.521900544, "lambda": 0.2631686208769679, "lambda_min": 0.07982780039310455, "lambda_max": 0.4870586395263672, "S_mean": 0.23557535807291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5880, "lr": 0.00034580541707601367, "train_loss": 4.671162843704224, "grad_norm": 0.8742061853408813, "tokens_seen": 48177152, "tokens_per_sec": 20984.823125855364, "vram_gb": 5.521900544, "lambda": 0.263636389747262, "lambda_min": 0.07982026040554047, "lambda_max": 0.4883239269256592, "S_mean": 0.26361083984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5900, "lr": 0.0003443699872073882, "train_loss": 4.49013876914978, "grad_norm": 0.8656385540962219, "tokens_seen": 48340992, "tokens_per_sec": 20985.614142394075, "vram_gb": 5.521900544, "lambda": 0.2640724303200841, "lambda_min": 0.07980339974164963, "lambda_max": 0.48885977268218994, "S_mean": 0.248260498046875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5920, "lr": 0.00034293414991056994, "train_loss": 4.45465874671936, "grad_norm": 0.8313068151473999, "tokens_seen": 48504832, "tokens_per_sec": 20986.164122926275, "vram_gb": 5.521900544, "lambda": 0.26452096179127693, "lambda_min": 0.07980059832334518, "lambda_max": 0.4901677072048187, "S_mean": 0.24759928385416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5940, "lr": 0.00034149794589544873, "train_loss": 4.619777679443359, "grad_norm": 0.802946150302887, "tokens_seen": 48668672, "tokens_per_sec": 20986.758041835576, "vram_gb": 5.521900544, "lambda": 0.26494669169187546, "lambda_min": 0.07989920675754547, "lambda_max": 0.49141058325767517, "S_mean": 0.25799560546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5960, "lr": 0.0003400614158823117, "train_loss": 4.537591934204102, "grad_norm": 0.8013402223587036, "tokens_seen": 48832512, "tokens_per_sec": 20987.495583589418, "vram_gb": 5.521900544, "lambda": 0.2653238885104656, "lambda_min": 0.07979018986225128, "lambda_max": 0.49241286516189575, "S_mean": 0.244476318359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 5980, "lr": 0.0003386246006006892, "train_loss": 4.655392169952393, "grad_norm": 0.8705464005470276, "tokens_seen": 48996352, "tokens_per_sec": 20986.71339154162, "vram_gb": 5.521900544, "lambda": 0.26579016136626404, "lambda_min": 0.07980914413928986, "lambda_max": 0.4934166669845581, "S_mean": 0.23230997721354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6000, "lr": 0.00033718754078819913, "train_loss": 4.703929662704468, "grad_norm": 0.928422212600708, "tokens_seen": 49160192, "tokens_per_sec": 20987.061782817436, "vram_gb": 5.521900544, "lambda": 0.2661558712522189, "lambda_min": 0.07976561039686203, "lambda_max": 0.4943588972091675, "S_mean": 0.2384033203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 6000, "val_loss": 4.661808705329895, "val_ppl": 105.82731959543808, "lambda": 0.2661791785309712, "lambda_min": 0.07975689321756363, "lambda_max": 0.49442997574806213, "lambda_per_layer": [0.1680193394422531, 0.13607348501682281, 0.14240670204162598, 0.14322935044765472, 0.1555097997188568, 0.1403212547302246, 0.12398435920476913, 0.11180973052978516, 0.07975689321756363, 0.12672953307628632, 0.2100825011730194, 0.32255908846855164, 0.3005196452140808, 0.46396327018737793, 0.45609837770462036, 0.49442997574806213, 0.48486530780792236, 0.27380818128585815, 0.3697792887687683, 0.4763576090335846, 0.3913104832172394, 0.3790389895439148, 0.21929946541786194, 0.21834765374660492], "S_mean": 0.2511393229166667, "G_mean": null, "G_std": null, "tokens_seen": 49160192} +{"event": "step", "step": 6020, "lr": 0.0003357502771893932, "train_loss": 4.690326690673828, "grad_norm": 0.9255490899085999, "tokens_seen": 49324032, "tokens_per_sec": 20974.78987125899, "vram_gb": 5.521900544, "lambda": 0.26657322111229104, "lambda_min": 0.07973354309797287, "lambda_max": 0.4956223666667938, "S_mean": 0.23783365885416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6040, "lr": 0.00033431285055460057, "train_loss": 4.493700742721558, "grad_norm": 0.8981624245643616, "tokens_seen": 49487872, "tokens_per_sec": 20975.362964899923, "vram_gb": 5.521900544, "lambda": 0.26695321934918564, "lambda_min": 0.07985568046569824, "lambda_max": 0.49650251865386963, "S_mean": 0.25, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6060, "lr": 0.00033287530163877304, "train_loss": 4.556125164031982, "grad_norm": 0.7858673930168152, "tokens_seen": 49651712, "tokens_per_sec": 20975.873220223384, "vram_gb": 5.521900544, "lambda": 0.26728356288125116, "lambda_min": 0.08007064461708069, "lambda_max": 0.4970194101333618, "S_mean": 0.24643961588541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6080, "lr": 0.0003314376712003294, "train_loss": 4.742836236953735, "grad_norm": 0.8249275088310242, "tokens_seen": 49815552, "tokens_per_sec": 20976.276814409637, "vram_gb": 5.521900544, "lambda": 0.2677405836681525, "lambda_min": 0.08017592132091522, "lambda_max": 0.49803847074508667, "S_mean": 0.246856689453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6100, "lr": 0.00032999999999999994, "train_loss": 4.419336318969727, "grad_norm": 0.8465285301208496, "tokens_seen": 49979392, "tokens_per_sec": 20976.64042958005, "vram_gb": 5.521900544, "lambda": 0.26817986462265253, "lambda_min": 0.08022740483283997, "lambda_max": 0.4993172585964203, "S_mean": 0.257598876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6120, "lr": 0.0003285623287996705, "train_loss": 4.677541255950928, "grad_norm": 0.8827100992202759, "tokens_seen": 50143232, "tokens_per_sec": 20976.96130060987, "vram_gb": 5.521900544, "lambda": 0.2686259603748719, "lambda_min": 0.08023694902658463, "lambda_max": 0.5006086826324463, "S_mean": 0.2566324869791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6140, "lr": 0.00032712469836122695, "train_loss": 4.498504400253296, "grad_norm": 0.8730813264846802, "tokens_seen": 50307072, "tokens_per_sec": 20977.50555467078, "vram_gb": 5.521900544, "lambda": 0.269013164875408, "lambda_min": 0.08021804690361023, "lambda_max": 0.5023893117904663, "S_mean": 0.250274658203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6160, "lr": 0.0003256871494453993, "train_loss": 4.58861231803894, "grad_norm": 0.8444654941558838, "tokens_seen": 50470912, "tokens_per_sec": 20978.403364861, "vram_gb": 5.521900544, "lambda": 0.2694746156533559, "lambda_min": 0.08028732240200043, "lambda_max": 0.5032833814620972, "S_mean": 0.2579752604166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6180, "lr": 0.00032424972281060676, "train_loss": 4.494140625, "grad_norm": 0.9021814465522766, "tokens_seen": 50634752, "tokens_per_sec": 20978.496549514304, "vram_gb": 5.521900544, "lambda": 0.26989477034658194, "lambda_min": 0.08014049381017685, "lambda_max": 0.5042340159416199, "S_mean": 0.23980712890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6200, "lr": 0.00032281245921180076, "train_loss": 4.63109016418457, "grad_norm": 0.8893541097640991, "tokens_seen": 50798592, "tokens_per_sec": 20979.19278871303, "vram_gb": 5.521900544, "lambda": 0.27032812871038914, "lambda_min": 0.08003245294094086, "lambda_max": 0.5055146813392639, "S_mean": 0.24457804361979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6220, "lr": 0.0003213753993993108, "train_loss": 4.675162315368652, "grad_norm": 0.8854402303695679, "tokens_seen": 50962432, "tokens_per_sec": 20979.262662189023, "vram_gb": 5.521900544, "lambda": 0.27069155871868134, "lambda_min": 0.07988964766263962, "lambda_max": 0.5062984228134155, "S_mean": 0.250518798828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6240, "lr": 0.0003199385841176881, "train_loss": 4.609327554702759, "grad_norm": 0.7906394004821777, "tokens_seen": 51126272, "tokens_per_sec": 20979.876034662488, "vram_gb": 5.521900544, "lambda": 0.27111527354766923, "lambda_min": 0.07979913800954819, "lambda_max": 0.5072360038757324, "S_mean": 0.2531229654947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6260, "lr": 0.00031850205410455116, "train_loss": 4.48613977432251, "grad_norm": 0.8342742919921875, "tokens_seen": 51290112, "tokens_per_sec": 20980.219051075266, "vram_gb": 5.521900544, "lambda": 0.2715427416066329, "lambda_min": 0.07981790602207184, "lambda_max": 0.5080574750900269, "S_mean": 0.2556559244791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6280, "lr": 0.00031706585008943, "train_loss": 4.470339059829712, "grad_norm": 0.8419554829597473, "tokens_seen": 51453952, "tokens_per_sec": 20980.50901519872, "vram_gb": 5.521900544, "lambda": 0.2719097708662351, "lambda_min": 0.07976208627223969, "lambda_max": 0.5091539621353149, "S_mean": 0.24310302734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6300, "lr": 0.00031563001279261176, "train_loss": 4.572797775268555, "grad_norm": 0.9056700468063354, "tokens_seen": 51617792, "tokens_per_sec": 20981.05714383515, "vram_gb": 5.521900544, "lambda": 0.27232065610587597, "lambda_min": 0.07975716888904572, "lambda_max": 0.5105207562446594, "S_mean": 0.259246826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6320, "lr": 0.0003141945829239862, "train_loss": 4.354335308074951, "grad_norm": 0.8002041578292847, "tokens_seen": 51781632, "tokens_per_sec": 20981.44355547834, "vram_gb": 5.521900544, "lambda": 0.2726631825789809, "lambda_min": 0.07963085919618607, "lambda_max": 0.511759340763092, "S_mean": 0.2592264811197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6340, "lr": 0.0003127596011818915, "train_loss": 4.597695827484131, "grad_norm": 0.8169025182723999, "tokens_seen": 51945472, "tokens_per_sec": 20981.6452129348, "vram_gb": 5.521900544, "lambda": 0.2730106385424733, "lambda_min": 0.07949188351631165, "lambda_max": 0.5125846862792969, "S_mean": 0.2534281412760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6360, "lr": 0.00031132510825196, "train_loss": 4.45911979675293, "grad_norm": 0.8632407188415527, "tokens_seen": 52109312, "tokens_per_sec": 20981.864040524124, "vram_gb": 5.521900544, "lambda": 0.27337938764443, "lambda_min": 0.0794498398900032, "lambda_max": 0.5137372016906738, "S_mean": 0.24506632486979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6380, "lr": 0.00030989114480596503, "train_loss": 4.623958110809326, "grad_norm": 0.8360240459442139, "tokens_seen": 52273152, "tokens_per_sec": 20982.468932983134, "vram_gb": 5.521900544, "lambda": 0.27381266094744205, "lambda_min": 0.07951110601425171, "lambda_max": 0.5151202082633972, "S_mean": 0.24497477213541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6400, "lr": 0.0003084577515006675, "train_loss": 4.565737009048462, "grad_norm": 0.8626669049263, "tokens_seen": 52436992, "tokens_per_sec": 20983.092182906665, "vram_gb": 5.521900544, "lambda": 0.2741311068336169, "lambda_min": 0.07938679307699203, "lambda_max": 0.5161282420158386, "S_mean": 0.2814229329427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6420, "lr": 0.0003070249689766635, "train_loss": 4.6274824142456055, "grad_norm": 0.8443800806999207, "tokens_seen": 52600832, "tokens_per_sec": 20983.476616718843, "vram_gb": 5.521900544, "lambda": 0.27450680422286194, "lambda_min": 0.07933102548122406, "lambda_max": 0.5172008275985718, "S_mean": 0.23958333333333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6440, "lr": 0.0003055928378572316, "train_loss": 4.658819198608398, "grad_norm": 0.9876176714897156, "tokens_seen": 52764672, "tokens_per_sec": 20983.80689963437, "vram_gb": 5.521900544, "lambda": 0.27495268639177084, "lambda_min": 0.07942510396242142, "lambda_max": 0.518332839012146, "S_mean": 0.2500101725260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6460, "lr": 0.00030416139874718133, "train_loss": 4.534384250640869, "grad_norm": 0.8620323538780212, "tokens_seen": 52928512, "tokens_per_sec": 20984.08127035595, "vram_gb": 5.521900544, "lambda": 0.2753937964638074, "lambda_min": 0.07938023656606674, "lambda_max": 0.5194380879402161, "S_mean": 0.2534586588541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6480, "lr": 0.0003027306922317018, "train_loss": 4.727263450622559, "grad_norm": 0.9158629179000854, "tokens_seen": 53092352, "tokens_per_sec": 20982.970107811976, "vram_gb": 5.521900544, "lambda": 0.2757017013306419, "lambda_min": 0.07939255982637405, "lambda_max": 0.5205596089363098, "S_mean": 0.2696024576822917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6500, "lr": 0.0003013007588752113, "train_loss": 4.472965240478516, "grad_norm": 0.8924241662025452, "tokens_seen": 53256192, "tokens_per_sec": 20983.57966767164, "vram_gb": 5.521900544, "lambda": 0.27597553593417007, "lambda_min": 0.07944266498088837, "lambda_max": 0.521496593952179, "S_mean": 0.251953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6520, "lr": 0.00029987163922020677, "train_loss": 4.53828763961792, "grad_norm": 0.8399786353111267, "tokens_seen": 53420032, "tokens_per_sec": 20984.473045156134, "vram_gb": 5.521900544, "lambda": 0.27633371359358233, "lambda_min": 0.0794806256890297, "lambda_max": 0.5228180885314941, "S_mean": 0.2714640299479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6540, "lr": 0.00029844337378611465, "train_loss": 4.615608215332031, "grad_norm": 0.8551581501960754, "tokens_seen": 53583872, "tokens_per_sec": 20984.689741399758, "vram_gb": 5.521900544, "lambda": 0.2767102091262738, "lambda_min": 0.07959743589162827, "lambda_max": 0.5237019062042236, "S_mean": 0.2540181477864583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6560, "lr": 0.00029701600306814154, "train_loss": 4.3933985233306885, "grad_norm": 0.8981248140335083, "tokens_seen": 53747712, "tokens_per_sec": 20984.37408923428, "vram_gb": 5.521900544, "lambda": 0.27701200762142736, "lambda_min": 0.07959634065628052, "lambda_max": 0.5242408514022827, "S_mean": 0.25347900390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6580, "lr": 0.00029558956753612705, "train_loss": 4.697816610336304, "grad_norm": 0.9519138336181641, "tokens_seen": 53911552, "tokens_per_sec": 20984.386183995182, "vram_gb": 5.521900544, "lambda": 0.27726567722857, "lambda_min": 0.07956347614526749, "lambda_max": 0.5249168872833252, "S_mean": 0.2557474772135417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6600, "lr": 0.00029416410763339506, "train_loss": 4.5119218826293945, "grad_norm": 0.9052547812461853, "tokens_seen": 54075392, "tokens_per_sec": 20984.564273431835, "vram_gb": 5.521900544, "lambda": 0.27763359900563955, "lambda_min": 0.07943686097860336, "lambda_max": 0.5258252024650574, "S_mean": 0.2527669270833333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6620, "lr": 0.0002927396637756081, "train_loss": 4.759325265884399, "grad_norm": 0.9683821797370911, "tokens_seen": 54239232, "tokens_per_sec": 20984.881128361034, "vram_gb": 5.521900544, "lambda": 0.2779787549128135, "lambda_min": 0.07941972464323044, "lambda_max": 0.5264871120452881, "S_mean": 0.2611185709635417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6640, "lr": 0.0002913162763496208, "train_loss": 4.452909708023071, "grad_norm": 1.0019166469573975, "tokens_seen": 54403072, "tokens_per_sec": 20984.72229589293, "vram_gb": 5.521900544, "lambda": 0.2783484359582265, "lambda_min": 0.07933533191680908, "lambda_max": 0.5275429487228394, "S_mean": 0.2544352213541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6660, "lr": 0.00028989398571233543, "train_loss": 4.4635443687438965, "grad_norm": 0.8510944843292236, "tokens_seen": 54566912, "tokens_per_sec": 20984.868062035708, "vram_gb": 5.521900544, "lambda": 0.27872083615511656, "lambda_min": 0.07929326593875885, "lambda_max": 0.5284467935562134, "S_mean": 0.24776204427083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6680, "lr": 0.000288472832189557, "train_loss": 4.4571452140808105, "grad_norm": 0.8713749647140503, "tokens_seen": 54730752, "tokens_per_sec": 20985.13316561215, "vram_gb": 5.521900544, "lambda": 0.27911099636306363, "lambda_min": 0.07932402938604355, "lambda_max": 0.5293594598770142, "S_mean": 0.24492390950520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6700, "lr": 0.00028705285607485054, "train_loss": 4.616662502288818, "grad_norm": 0.8811624050140381, "tokens_seen": 54894592, "tokens_per_sec": 20985.685387487247, "vram_gb": 5.521900544, "lambda": 0.2794766615455349, "lambda_min": 0.07916899025440216, "lambda_max": 0.5304958820343018, "S_mean": 0.24405924479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6720, "lr": 0.000285634097628398, "train_loss": 4.4996442794799805, "grad_norm": 0.8716208934783936, "tokens_seen": 55058432, "tokens_per_sec": 20986.05327673633, "vram_gb": 5.521900544, "lambda": 0.27988815276573104, "lambda_min": 0.07905685901641846, "lambda_max": 0.531584620475769, "S_mean": 0.24242146809895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6740, "lr": 0.00028421659707585724, "train_loss": 4.494008302688599, "grad_norm": 0.8746498227119446, "tokens_seen": 55222272, "tokens_per_sec": 20986.319656669613, "vram_gb": 5.521900544, "lambda": 0.2803427117566268, "lambda_min": 0.07901372015476227, "lambda_max": 0.5326156616210938, "S_mean": 0.24945068359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6760, "lr": 0.00028280039460722167, "train_loss": 4.5921244621276855, "grad_norm": 0.856980562210083, "tokens_seen": 55386112, "tokens_per_sec": 20986.37863370189, "vram_gb": 5.521900544, "lambda": 0.28070089345177013, "lambda_min": 0.0790020003914833, "lambda_max": 0.5330455303192139, "S_mean": 0.24647013346354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6780, "lr": 0.0002813855303756802, "train_loss": 4.462835073471069, "grad_norm": 0.9152406454086304, "tokens_seen": 55549952, "tokens_per_sec": 20986.54908336911, "vram_gb": 5.521900544, "lambda": 0.28102158289402723, "lambda_min": 0.07893837988376617, "lambda_max": 0.5338614583015442, "S_mean": 0.24413045247395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6800, "lr": 0.00027997204449647917, "train_loss": 4.40234375, "grad_norm": 0.8495352864265442, "tokens_seen": 55713792, "tokens_per_sec": 20986.46808902185, "vram_gb": 5.521900544, "lambda": 0.28138805242876214, "lambda_min": 0.07885606586933136, "lambda_max": 0.5347735285758972, "S_mean": 0.25482177734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6820, "lr": 0.00027855997704578483, "train_loss": 4.602871894836426, "grad_norm": 0.8932948112487793, "tokens_seen": 55877632, "tokens_per_sec": 20986.337207306533, "vram_gb": 5.521900544, "lambda": 0.28170794031272334, "lambda_min": 0.07892826944589615, "lambda_max": 0.5351602435112, "S_mean": 0.245635986328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6840, "lr": 0.0002771493680595473, "train_loss": 4.429517030715942, "grad_norm": 0.8397026062011719, "tokens_seen": 56041472, "tokens_per_sec": 20986.698575585255, "vram_gb": 5.521900544, "lambda": 0.2820820923273762, "lambda_min": 0.07889968156814575, "lambda_max": 0.5362011194229126, "S_mean": 0.2515055338541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6860, "lr": 0.00027574025753236513, "train_loss": 4.6283440589904785, "grad_norm": 0.8676273822784424, "tokens_seen": 56205312, "tokens_per_sec": 20986.53968865908, "vram_gb": 5.521900544, "lambda": 0.2824389686187108, "lambda_min": 0.07894185930490494, "lambda_max": 0.5371639728546143, "S_mean": 0.252105712890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6880, "lr": 0.0002743326854163515, "train_loss": 4.530574321746826, "grad_norm": 0.9519714117050171, "tokens_seen": 56369152, "tokens_per_sec": 20986.54346753185, "vram_gb": 5.521900544, "lambda": 0.2827875229219596, "lambda_min": 0.07880976796150208, "lambda_max": 0.5380884408950806, "S_mean": 0.269287109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6900, "lr": 0.0002729266916200014, "train_loss": 4.4764084815979, "grad_norm": 0.9118508100509644, "tokens_seen": 56532992, "tokens_per_sec": 20986.861476611575, "vram_gb": 5.521900544, "lambda": 0.2830973385522763, "lambda_min": 0.07866627722978592, "lambda_max": 0.5387788414955139, "S_mean": 0.2567036946614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6920, "lr": 0.0002715223160070603, "train_loss": 4.489711761474609, "grad_norm": 0.9126458168029785, "tokens_seen": 56696832, "tokens_per_sec": 20987.292817020996, "vram_gb": 5.521900544, "lambda": 0.2834770083427429, "lambda_min": 0.07860157638788223, "lambda_max": 0.5398094654083252, "S_mean": 0.273681640625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6940, "lr": 0.00027011959839539376, "train_loss": 4.560346603393555, "grad_norm": 0.9471614956855774, "tokens_seen": 56860672, "tokens_per_sec": 20986.393037092967, "vram_gb": 5.521900544, "lambda": 0.2838409577185909, "lambda_min": 0.07860436290502548, "lambda_max": 0.5407891869544983, "S_mean": 0.2565409342447917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6960, "lr": 0.0002687185785558583, "train_loss": 4.53919243812561, "grad_norm": 0.9259656667709351, "tokens_seen": 57024512, "tokens_per_sec": 20983.805631165884, "vram_gb": 5.521900544, "lambda": 0.28418359470864135, "lambda_min": 0.07854854315519333, "lambda_max": 0.5417556762695312, "S_mean": 0.2684529622395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 6980, "lr": 0.0002673192962111742, "train_loss": 4.487853288650513, "grad_norm": 0.8344074487686157, "tokens_seen": 57188352, "tokens_per_sec": 20983.818761806066, "vram_gb": 5.521900544, "lambda": 0.2845180717607339, "lambda_min": 0.07849246263504028, "lambda_max": 0.5423339605331421, "S_mean": 0.2598673502604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7000, "lr": 0.00026592179103479893, "train_loss": 4.334140300750732, "grad_norm": 0.8767554759979248, "tokens_seen": 57352192, "tokens_per_sec": 20983.87782975399, "vram_gb": 5.521900544, "lambda": 0.2848631500576933, "lambda_min": 0.07838114351034164, "lambda_max": 0.5429320931434631, "S_mean": 0.2729085286458333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 7000, "val_loss": 4.622126722335816, "val_ppl": 101.71011144353626, "lambda": 0.28488423271725577, "lambda_min": 0.07837866991758347, "lambda_max": 0.542987585067749, "lambda_per_layer": [0.17159903049468994, 0.1365743726491928, 0.14239148795604706, 0.14746467769145966, 0.16070236265659332, 0.14207662642002106, 0.12370266020298004, 0.111729696393013, 0.07837866991758347, 0.12752382457256317, 0.212543323636055, 0.3353130519390106, 0.3150675892829895, 0.4989645481109619, 0.4927915930747986, 0.542987585067749, 0.5389178991317749, 0.29617446660995483, 0.40968137979507446, 0.534470796585083, 0.4370231032371521, 0.41439288854599, 0.23501716554164886, 0.2317327857017517], "S_mean": 0.2617289225260417, "G_mean": null, "G_std": null, "tokens_seen": 57352192} +{"event": "step", "step": 7020, "lr": 0.00026452610264980257, "train_loss": 4.6896491050720215, "grad_norm": 0.8708326816558838, "tokens_seen": 57516032, "tokens_per_sec": 20972.91445044126, "vram_gb": 5.521900544, "lambda": 0.2851637282098333, "lambda_min": 0.07843340933322906, "lambda_max": 0.5439355373382568, "S_mean": 0.2505391438802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7040, "lr": 0.00026313227062774397, "train_loss": 4.67486047744751, "grad_norm": 0.9801273345947266, "tokens_seen": 57679872, "tokens_per_sec": 20973.1999941362, "vram_gb": 5.521900544, "lambda": 0.2854777028163274, "lambda_min": 0.07853052020072937, "lambda_max": 0.5447056889533997, "S_mean": 0.2751057942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7060, "lr": 0.0002617403344875491, "train_loss": 4.571392059326172, "grad_norm": 0.8621765971183777, "tokens_seen": 57843712, "tokens_per_sec": 20973.512745630862, "vram_gb": 5.521900544, "lambda": 0.2857873331134518, "lambda_min": 0.07871470600366592, "lambda_max": 0.5457111597061157, "S_mean": 0.2504679361979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7080, "lr": 0.0002603503336943908, "train_loss": 4.543806076049805, "grad_norm": 0.872699499130249, "tokens_seen": 58007552, "tokens_per_sec": 20974.05931514394, "vram_gb": 5.521900544, "lambda": 0.28612879446397227, "lambda_min": 0.07876279205083847, "lambda_max": 0.5465506911277771, "S_mean": 0.2628987630208333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7100, "lr": 0.00025896230765856925, "train_loss": 4.563219308853149, "grad_norm": 0.8795990943908691, "tokens_seen": 58171392, "tokens_per_sec": 20974.183845097738, "vram_gb": 5.521900544, "lambda": 0.28651184526582557, "lambda_min": 0.07872931659221649, "lambda_max": 0.547269344329834, "S_mean": 0.239044189453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7120, "lr": 0.000257576295734395, "train_loss": 4.761590242385864, "grad_norm": 1.0785471200942993, "tokens_seen": 58335232, "tokens_per_sec": 20974.784427174192, "vram_gb": 5.521900544, "lambda": 0.28690661303699017, "lambda_min": 0.07870826125144958, "lambda_max": 0.5483412742614746, "S_mean": 0.2613016764322917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7140, "lr": 0.0002561923372190731, "train_loss": 4.4685540199279785, "grad_norm": 0.9166490435600281, "tokens_seen": 58499072, "tokens_per_sec": 20975.395558029606, "vram_gb": 5.521900544, "lambda": 0.28723378758877516, "lambda_min": 0.07871955633163452, "lambda_max": 0.5493991374969482, "S_mean": 0.24540201822916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7160, "lr": 0.00025481047135158884, "train_loss": 4.497730016708374, "grad_norm": 0.8576001524925232, "tokens_seen": 58662912, "tokens_per_sec": 20975.55104084158, "vram_gb": 5.521900544, "lambda": 0.2874448914080858, "lambda_min": 0.07870592176914215, "lambda_max": 0.5501471161842346, "S_mean": 0.249847412109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7180, "lr": 0.0002534307373115953, "train_loss": 4.5604095458984375, "grad_norm": 0.9854990243911743, "tokens_seen": 58826752, "tokens_per_sec": 20975.370888522935, "vram_gb": 5.521900544, "lambda": 0.28778999640295905, "lambda_min": 0.07876554131507874, "lambda_max": 0.5506067276000977, "S_mean": 0.24809773763020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7200, "lr": 0.00025205317421830235, "train_loss": 4.550976753234863, "grad_norm": 0.965362548828125, "tokens_seen": 58990592, "tokens_per_sec": 20975.418837519952, "vram_gb": 5.521900544, "lambda": 0.28814688480148715, "lambda_min": 0.07873667031526566, "lambda_max": 0.551372766494751, "S_mean": 0.2541605631510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7220, "lr": 0.0002506778211293677, "train_loss": 4.3480377197265625, "grad_norm": 0.8971816301345825, "tokens_seen": 59154432, "tokens_per_sec": 20975.551720237196, "vram_gb": 5.521900544, "lambda": 0.28844433433065814, "lambda_min": 0.07867901027202606, "lambda_max": 0.5521982908248901, "S_mean": 0.2548319498697917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7240, "lr": 0.00024930471703978936, "train_loss": 4.438292741775513, "grad_norm": 0.811331033706665, "tokens_seen": 59318272, "tokens_per_sec": 20975.461077569238, "vram_gb": 5.521900544, "lambda": 0.2887680825466911, "lambda_min": 0.07860919088125229, "lambda_max": 0.5529100298881531, "S_mean": 0.2543436686197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7260, "lr": 0.0002479339008808003, "train_loss": 4.551785469055176, "grad_norm": 0.8905462026596069, "tokens_seen": 59482112, "tokens_per_sec": 20975.78053297973, "vram_gb": 5.521900544, "lambda": 0.28912104417880374, "lambda_min": 0.07856274396181107, "lambda_max": 0.5535492897033691, "S_mean": 0.2396240234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7280, "lr": 0.0002465654115187642, "train_loss": 4.444106340408325, "grad_norm": 0.8604570031166077, "tokens_seen": 59645952, "tokens_per_sec": 20975.819598096085, "vram_gb": 5.521900544, "lambda": 0.28946252384533483, "lambda_min": 0.078690305352211, "lambda_max": 0.5545587539672852, "S_mean": 0.243865966796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7300, "lr": 0.00024519928775407386, "train_loss": 4.589764595031738, "grad_norm": 0.8801121711730957, "tokens_seen": 59809792, "tokens_per_sec": 20976.088380675257, "vram_gb": 5.521900544, "lambda": 0.28982597899933654, "lambda_min": 0.0787120908498764, "lambda_max": 0.5555797219276428, "S_mean": 0.23822021484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7320, "lr": 0.00024383556832005074, "train_loss": 4.4035725593566895, "grad_norm": 0.8591136932373047, "tokens_seen": 59973632, "tokens_per_sec": 20976.01917981566, "vram_gb": 5.521900544, "lambda": 0.29010155114034814, "lambda_min": 0.07867518812417984, "lambda_max": 0.5562386512756348, "S_mean": 0.251708984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7340, "lr": 0.00024247429188184748, "train_loss": 4.491088628768921, "grad_norm": 0.8925963044166565, "tokens_seen": 60137472, "tokens_per_sec": 20976.21574804622, "vram_gb": 5.521900544, "lambda": 0.29039802805831033, "lambda_min": 0.07863122224807739, "lambda_max": 0.5570225715637207, "S_mean": 0.24184163411458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7360, "lr": 0.0002411154970353507, "train_loss": 4.5630693435668945, "grad_norm": 0.8943066000938416, "tokens_seen": 60301312, "tokens_per_sec": 20976.267716618786, "vram_gb": 5.521900544, "lambda": 0.2906757043674588, "lambda_min": 0.0786636620759964, "lambda_max": 0.5578330755233765, "S_mean": 0.2523905436197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7380, "lr": 0.00023975922230608723, "train_loss": 4.536259174346924, "grad_norm": 0.9562311172485352, "tokens_seen": 60465152, "tokens_per_sec": 20975.429891477957, "vram_gb": 5.521900544, "lambda": 0.29100907283524674, "lambda_min": 0.0786329060792923, "lambda_max": 0.558641254901886, "S_mean": 0.24881998697916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7400, "lr": 0.00023840550614813178, "train_loss": 4.411401748657227, "grad_norm": 0.9012432098388672, "tokens_seen": 60628992, "tokens_per_sec": 20975.368864278025, "vram_gb": 5.521900544, "lambda": 0.29134827107191086, "lambda_min": 0.0786515399813652, "lambda_max": 0.5595454573631287, "S_mean": 0.24066162109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7420, "lr": 0.00023705438694301677, "train_loss": 4.47687292098999, "grad_norm": 0.9898325800895691, "tokens_seen": 60792832, "tokens_per_sec": 20975.57596089141, "vram_gb": 5.521900544, "lambda": 0.29161531447122496, "lambda_min": 0.07861287146806717, "lambda_max": 0.5603177547454834, "S_mean": 0.2561136881510417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7440, "lr": 0.00023570590299864364, "train_loss": 4.415331840515137, "grad_norm": 0.9180206060409546, "tokens_seen": 60956672, "tokens_per_sec": 20975.908712132747, "vram_gb": 5.521900544, "lambda": 0.2919024241467317, "lambda_min": 0.07856371253728867, "lambda_max": 0.560905396938324, "S_mean": 0.239410400390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7460, "lr": 0.00023436009254819732, "train_loss": 4.571638107299805, "grad_norm": 0.8958237767219543, "tokens_seen": 61120512, "tokens_per_sec": 20976.274181924393, "vram_gb": 5.521900544, "lambda": 0.2922171751658122, "lambda_min": 0.07855148613452911, "lambda_max": 0.5616620779037476, "S_mean": 0.244842529296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7480, "lr": 0.00023301699374906156, "train_loss": 4.40109658241272, "grad_norm": 0.9430983662605286, "tokens_seen": 61284352, "tokens_per_sec": 20976.66702033049, "vram_gb": 5.521900544, "lambda": 0.2925238385796547, "lambda_min": 0.07858886569738388, "lambda_max": 0.5624642372131348, "S_mean": 0.2550760904947917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7500, "lr": 0.00023167664468173798, "train_loss": 4.464381217956543, "grad_norm": 0.9017815589904785, "tokens_seen": 61448192, "tokens_per_sec": 20976.886014600135, "vram_gb": 5.521900544, "lambda": 0.292859704233706, "lambda_min": 0.07861990481615067, "lambda_max": 0.5630761384963989, "S_mean": 0.23200480143229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7520, "lr": 0.00023033908334876529, "train_loss": 4.522508144378662, "grad_norm": 0.9190294742584229, "tokens_seen": 61612032, "tokens_per_sec": 20976.832691987416, "vram_gb": 5.521900544, "lambda": 0.2931307752927144, "lambda_min": 0.07876057922840118, "lambda_max": 0.5638695359230042, "S_mean": 0.2543741861979167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7540, "lr": 0.0002290043476736426, "train_loss": 4.395350933074951, "grad_norm": 0.9724200963973999, "tokens_seen": 61775872, "tokens_per_sec": 20976.972907085135, "vram_gb": 5.521900544, "lambda": 0.2933978255217274, "lambda_min": 0.07882840186357498, "lambda_max": 0.5645718574523926, "S_mean": 0.264068603515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7560, "lr": 0.00022767247549975377, "train_loss": 4.3196165561676025, "grad_norm": 0.9307107329368591, "tokens_seen": 61939712, "tokens_per_sec": 20977.034806473854, "vram_gb": 5.521900544, "lambda": 0.29366523834566277, "lambda_min": 0.0788457989692688, "lambda_max": 0.5651707649230957, "S_mean": 0.2586466471354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7580, "lr": 0.00022634350458929501, "train_loss": 4.6086602210998535, "grad_norm": 1.0280545949935913, "tokens_seen": 62103552, "tokens_per_sec": 20977.41934556737, "vram_gb": 5.521900544, "lambda": 0.29390792331347865, "lambda_min": 0.07886409014463425, "lambda_max": 0.5655542612075806, "S_mean": 0.2636515299479167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7600, "lr": 0.0002250174726222034, "train_loss": 4.3618621826171875, "grad_norm": 0.9162599444389343, "tokens_seen": 62267392, "tokens_per_sec": 20977.65064586345, "vram_gb": 5.521900544, "lambda": 0.2941531104346116, "lambda_min": 0.0789112076163292, "lambda_max": 0.5662668943405151, "S_mean": 0.2513834635416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7620, "lr": 0.00022369441719508924, "train_loss": 4.445008277893066, "grad_norm": 0.9127629399299622, "tokens_seen": 62431232, "tokens_per_sec": 20977.684597542684, "vram_gb": 5.521900544, "lambda": 0.2944639893248677, "lambda_min": 0.07889407128095627, "lambda_max": 0.5673880577087402, "S_mean": 0.24577840169270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7640, "lr": 0.0002223743758201696, "train_loss": 4.40557599067688, "grad_norm": 0.8893635869026184, "tokens_seen": 62595072, "tokens_per_sec": 20977.337879261777, "vram_gb": 5.521900544, "lambda": 0.2947266198073824, "lambda_min": 0.07892189919948578, "lambda_max": 0.5681447386741638, "S_mean": 0.2572428385416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7660, "lr": 0.00022105738592420533, "train_loss": 4.4481470584869385, "grad_norm": 0.9156457781791687, "tokens_seen": 62758912, "tokens_per_sec": 20977.673773247305, "vram_gb": 5.521900544, "lambda": 0.2949335960050424, "lambda_min": 0.0788944736123085, "lambda_max": 0.5685145854949951, "S_mean": 0.24347941080729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7680, "lr": 0.0002197434848474393, "train_loss": 4.621950626373291, "grad_norm": 0.9489699006080627, "tokens_seen": 62922752, "tokens_per_sec": 20978.016978108804, "vram_gb": 5.521900544, "lambda": 0.295140213953952, "lambda_min": 0.07894021272659302, "lambda_max": 0.5692105293273926, "S_mean": 0.23884073893229166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7700, "lr": 0.00021843270984253792, "train_loss": 4.589292049407959, "grad_norm": 0.9395637512207031, "tokens_seen": 63086592, "tokens_per_sec": 20977.657581509644, "vram_gb": 5.521900544, "lambda": 0.2954183258116245, "lambda_min": 0.07900591194629669, "lambda_max": 0.5699811577796936, "S_mean": 0.2643229166666667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7720, "lr": 0.00021712509807353517, "train_loss": 4.570131778717041, "grad_norm": 0.9152734875679016, "tokens_seen": 63250432, "tokens_per_sec": 20977.947091640854, "vram_gb": 5.521900544, "lambda": 0.29573597727964324, "lambda_min": 0.07900708168745041, "lambda_max": 0.5708680152893066, "S_mean": 0.2708028157552083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7740, "lr": 0.00021582068661477836, "train_loss": 4.493866205215454, "grad_norm": 0.9625746011734009, "tokens_seen": 63414272, "tokens_per_sec": 20978.29289885652, "vram_gb": 5.521900544, "lambda": 0.29600651655346155, "lambda_min": 0.07904238253831863, "lambda_max": 0.5715217590332031, "S_mean": 0.25042724609375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7760, "lr": 0.00021451951244987784, "train_loss": 4.482449293136597, "grad_norm": 0.8310917019844055, "tokens_seen": 63578112, "tokens_per_sec": 20978.777666588874, "vram_gb": 5.521900544, "lambda": 0.29627326782792807, "lambda_min": 0.07908596843481064, "lambda_max": 0.5720677971839905, "S_mean": 0.2524515787760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7780, "lr": 0.00021322161247065748, "train_loss": 4.575523376464844, "grad_norm": 0.9621367454528809, "tokens_seen": 63741952, "tokens_per_sec": 20979.04854402816, "vram_gb": 5.521900544, "lambda": 0.29656230192631483, "lambda_min": 0.07914683222770691, "lambda_max": 0.5729277729988098, "S_mean": 0.2507222493489583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7800, "lr": 0.00021192702347610902, "train_loss": 4.495920658111572, "grad_norm": 0.9590712189674377, "tokens_seen": 63905792, "tokens_per_sec": 20979.186986133584, "vram_gb": 5.521900544, "lambda": 0.29684832350661355, "lambda_min": 0.07914300262928009, "lambda_max": 0.5736713409423828, "S_mean": 0.2596333821614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7820, "lr": 0.00021063578217134918, "train_loss": 4.351296901702881, "grad_norm": 0.9198001027107239, "tokens_seen": 64069632, "tokens_per_sec": 20979.22545148502, "vram_gb": 5.521900544, "lambda": 0.2970991426457961, "lambda_min": 0.07908441871404648, "lambda_max": 0.5742205381393433, "S_mean": 0.25018310546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7840, "lr": 0.0002093479251665781, "train_loss": 4.392167091369629, "grad_norm": 0.9448521733283997, "tokens_seen": 64233472, "tokens_per_sec": 20979.43265792089, "vram_gb": 5.521900544, "lambda": 0.2973560181756814, "lambda_min": 0.07902366667985916, "lambda_max": 0.5751426219940186, "S_mean": 0.2558797200520833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7860, "lr": 0.00020806348897604236, "train_loss": 4.40053915977478, "grad_norm": 0.9428156614303589, "tokens_seen": 64397312, "tokens_per_sec": 20979.422388907664, "vram_gb": 5.521900544, "lambda": 0.2976074026276668, "lambda_min": 0.07902166992425919, "lambda_max": 0.5760269165039062, "S_mean": 0.2615152994791667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7880, "lr": 0.00020678251001699866, "train_loss": 4.387638092041016, "grad_norm": 0.95381760597229, "tokens_seen": 64561152, "tokens_per_sec": 20979.62232464922, "vram_gb": 5.521900544, "lambda": 0.29785167922576267, "lambda_min": 0.07910902798175812, "lambda_max": 0.576896607875824, "S_mean": 0.2562967936197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7900, "lr": 0.00020550502460868193, "train_loss": 4.351397752761841, "grad_norm": 0.9206064939498901, "tokens_seen": 64724992, "tokens_per_sec": 20979.777810118052, "vram_gb": 5.521900544, "lambda": 0.2981217736378312, "lambda_min": 0.07908505946397781, "lambda_max": 0.5777039527893066, "S_mean": 0.2602640787760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7920, "lr": 0.00020423106897127574, "train_loss": 4.519649028778076, "grad_norm": 0.9874264597892761, "tokens_seen": 64888832, "tokens_per_sec": 20979.734404147704, "vram_gb": 5.521900544, "lambda": 0.29837618333597976, "lambda_min": 0.07908304780721664, "lambda_max": 0.5782179236412048, "S_mean": 0.24620564778645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7940, "lr": 0.00020296067922488466, "train_loss": 4.5095086097717285, "grad_norm": 0.9781835675239563, "tokens_seen": 65052672, "tokens_per_sec": 20980.230577497445, "vram_gb": 5.521900544, "lambda": 0.2986345471193393, "lambda_min": 0.0790640264749527, "lambda_max": 0.578998327255249, "S_mean": 0.24097696940104166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7960, "lr": 0.00020169389138851074, "train_loss": 4.498129844665527, "grad_norm": 0.9322009086608887, "tokens_seen": 65216512, "tokens_per_sec": 20980.526169849345, "vram_gb": 5.521900544, "lambda": 0.2988432018707196, "lambda_min": 0.07905832678079605, "lambda_max": 0.5794868469238281, "S_mean": 0.2518819173177083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 7980, "lr": 0.000200430741379032, "train_loss": 4.504613399505615, "grad_norm": 0.9253043532371521, "tokens_seen": 65380352, "tokens_per_sec": 20980.94020370298, "vram_gb": 5.521900544, "lambda": 0.299059871584177, "lambda_min": 0.07899565994739532, "lambda_max": 0.5799097418785095, "S_mean": 0.24573771158854166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8000, "lr": 0.00019917126501018445, "train_loss": 4.549946308135986, "grad_norm": 0.9918299913406372, "tokens_seen": 65544192, "tokens_per_sec": 20981.11322612031, "vram_gb": 5.521900544, "lambda": 0.2992765322948496, "lambda_min": 0.07903445512056351, "lambda_max": 0.5804718732833862, "S_mean": 0.24886067708333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 8000, "val_loss": 4.5488279342651365, "val_ppl": 94.52155788641498, "lambda": 0.29929488711059093, "lambda_min": 0.07903815805912018, "lambda_max": 0.5805047154426575, "lambda_per_layer": [0.17500407993793488, 0.1363612711429596, 0.14264880120754242, 0.15066242218017578, 0.16515377163887024, 0.14368188381195068, 0.12371639162302017, 0.11214642971754074, 0.07903815805912018, 0.1281808763742447, 0.21559582650661469, 0.3440804183483124, 0.3252883851528168, 0.5223663449287415, 0.5201734900474548, 0.5805047154426575, 0.5799003839492798, 0.31275221705436707, 0.44171425700187683, 0.5784860253334045, 0.46951889991760254, 0.4430803060531616, 0.24858935177326202, 0.24443258345127106], "S_mean": 0.2531636555989583, "G_mean": null, "G_std": null, "tokens_seen": 65544192} +{"event": "step", "step": 8020, "lr": 0.00019791549799154606, "train_loss": 4.518017530441284, "grad_norm": 0.9202138185501099, "tokens_seen": 65708032, "tokens_per_sec": 20971.561671495347, "vram_gb": 5.521900544, "lambda": 0.2995269459982713, "lambda_min": 0.07905004173517227, "lambda_max": 0.5812440514564514, "S_mean": 0.24123128255208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8040, "lr": 0.00019666347592752487, "train_loss": 4.430692911148071, "grad_norm": 1.0928962230682373, "tokens_seen": 65871872, "tokens_per_sec": 20971.942071023932, "vram_gb": 5.521900544, "lambda": 0.29977660595128935, "lambda_min": 0.07902282476425171, "lambda_max": 0.5819032788276672, "S_mean": 0.2545166015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8060, "lr": 0.00019541523431634904, "train_loss": 4.457033634185791, "grad_norm": 0.9215697646141052, "tokens_seen": 66035712, "tokens_per_sec": 20972.077231627474, "vram_gb": 5.521900544, "lambda": 0.29997926236440736, "lambda_min": 0.07900382578372955, "lambda_max": 0.5821620225906372, "S_mean": 0.2589518229166667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8080, "lr": 0.00019417080854906098, "train_loss": 4.559003829956055, "grad_norm": 0.933743417263031, "tokens_seen": 66199552, "tokens_per_sec": 20972.578149762063, "vram_gb": 5.521900544, "lambda": 0.3001612626637022, "lambda_min": 0.07899513840675354, "lambda_max": 0.5828253626823425, "S_mean": 0.2508341471354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8100, "lr": 0.00019293023390851326, "train_loss": 4.52972936630249, "grad_norm": 0.9493692517280579, "tokens_seen": 66363392, "tokens_per_sec": 20972.778138034744, "vram_gb": 5.521900544, "lambda": 0.30047966726124287, "lambda_min": 0.0790034830570221, "lambda_max": 0.5835997462272644, "S_mean": 0.24677530924479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8120, "lr": 0.00019169354556836853, "train_loss": 4.658291816711426, "grad_norm": 0.9685158729553223, "tokens_seen": 66527232, "tokens_per_sec": 20973.10724604333, "vram_gb": 5.521900544, "lambda": 0.3007390508428216, "lambda_min": 0.07904614508152008, "lambda_max": 0.5840094089508057, "S_mean": 0.239471435546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8140, "lr": 0.00019046077859210254, "train_loss": 4.3279523849487305, "grad_norm": 0.8508661389350891, "tokens_seen": 66691072, "tokens_per_sec": 20973.479177335947, "vram_gb": 5.521900544, "lambda": 0.3009115243330598, "lambda_min": 0.07901298999786377, "lambda_max": 0.5844390988349915, "S_mean": 0.259033203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8160, "lr": 0.00018923196793200936, "train_loss": 4.399469375610352, "grad_norm": 0.9368712306022644, "tokens_seen": 66854912, "tokens_per_sec": 20973.448319038846, "vram_gb": 5.521900544, "lambda": 0.3011290514841676, "lambda_min": 0.07902694493532181, "lambda_max": 0.5853543281555176, "S_mean": 0.2839558919270833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8180, "lr": 0.00018800714842821115, "train_loss": 4.492725610733032, "grad_norm": 1.0806317329406738, "tokens_seen": 67018752, "tokens_per_sec": 20973.92138972304, "vram_gb": 5.521900544, "lambda": 0.301312948577106, "lambda_min": 0.07896540313959122, "lambda_max": 0.5860658884048462, "S_mean": 0.251434326171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8200, "lr": 0.00018678635480766953, "train_loss": 4.344448804855347, "grad_norm": 0.9442764520645142, "tokens_seen": 67182592, "tokens_per_sec": 20973.74077995629, "vram_gb": 5.521900544, "lambda": 0.3015705257033308, "lambda_min": 0.07895012199878693, "lambda_max": 0.5868239998817444, "S_mean": 0.24925740559895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8220, "lr": 0.0001855696216832015, "train_loss": 4.439742088317871, "grad_norm": 1.0181611776351929, "tokens_seen": 67346432, "tokens_per_sec": 20974.09422763399, "vram_gb": 5.521900544, "lambda": 0.3017765013501048, "lambda_min": 0.07895466685295105, "lambda_max": 0.587139904499054, "S_mean": 0.24778238932291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8240, "lr": 0.0001843569835524983, "train_loss": 4.374099493026733, "grad_norm": 1.0053914785385132, "tokens_seen": 67510272, "tokens_per_sec": 20974.21105096055, "vram_gb": 5.521900544, "lambda": 0.30202646398295957, "lambda_min": 0.07892748713493347, "lambda_max": 0.588004469871521, "S_mean": 0.25299072265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8260, "lr": 0.00018314847479714653, "train_loss": 4.339751482009888, "grad_norm": 1.0040007829666138, "tokens_seen": 67674112, "tokens_per_sec": 20974.52045420686, "vram_gb": 5.521900544, "lambda": 0.30228884021441144, "lambda_min": 0.07892763614654541, "lambda_max": 0.5888601541519165, "S_mean": 0.28515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8280, "lr": 0.00018194412968165394, "train_loss": 4.333745956420898, "grad_norm": 0.9229691028594971, "tokens_seen": 67837952, "tokens_per_sec": 20974.481665195937, "vram_gb": 5.521900544, "lambda": 0.30250814743340015, "lambda_min": 0.07892230898141861, "lambda_max": 0.5897846221923828, "S_mean": 0.25762939453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8300, "lr": 0.00018074398235247785, "train_loss": 4.537668228149414, "grad_norm": 0.9214960336685181, "tokens_seen": 68001792, "tokens_per_sec": 20974.85347183406, "vram_gb": 5.521900544, "lambda": 0.30271473123381537, "lambda_min": 0.07891926169395447, "lambda_max": 0.59047931432724, "S_mean": 0.2608846028645833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8320, "lr": 0.00017954806683705664, "train_loss": 4.4012720584869385, "grad_norm": 0.9360026121139526, "tokens_seen": 68165632, "tokens_per_sec": 20975.35839499382, "vram_gb": 5.521900544, "lambda": 0.3028920032083988, "lambda_min": 0.0789206326007843, "lambda_max": 0.5910881757736206, "S_mean": 0.2592875162760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8340, "lr": 0.0001783564170428458, "train_loss": 4.450741767883301, "grad_norm": 0.941343367099762, "tokens_seen": 68329472, "tokens_per_sec": 20975.608239952584, "vram_gb": 5.521900544, "lambda": 0.30314979919542867, "lambda_min": 0.07898615300655365, "lambda_max": 0.591840922832489, "S_mean": 0.261627197265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8360, "lr": 0.00017716906675635554, "train_loss": 4.546349048614502, "grad_norm": 1.0403867959976196, "tokens_seen": 68493312, "tokens_per_sec": 20975.20313524532, "vram_gb": 5.521900544, "lambda": 0.3034237399697304, "lambda_min": 0.07891086488962173, "lambda_max": 0.5927855968475342, "S_mean": 0.264862060546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8380, "lr": 0.00017598604964219356, "train_loss": 4.368118762969971, "grad_norm": 0.9580990076065063, "tokens_seen": 68657152, "tokens_per_sec": 20975.530323590843, "vram_gb": 5.521900544, "lambda": 0.3036789372563362, "lambda_min": 0.0789031907916069, "lambda_max": 0.5935141444206238, "S_mean": 0.261444091796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8400, "lr": 0.00017480739924211012, "train_loss": 4.467247486114502, "grad_norm": 0.9638374447822571, "tokens_seen": 68820992, "tokens_per_sec": 20975.601061036425, "vram_gb": 5.521900544, "lambda": 0.30391396085421246, "lambda_min": 0.07891621440649033, "lambda_max": 0.5941497683525085, "S_mean": 0.241943359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8420, "lr": 0.0001736331489740478, "train_loss": 4.350087404251099, "grad_norm": 1.0042483806610107, "tokens_seen": 68984832, "tokens_per_sec": 20975.29978194202, "vram_gb": 5.521900544, "lambda": 0.3041386092081666, "lambda_min": 0.07896041125059128, "lambda_max": 0.5948693156242371, "S_mean": 0.24924723307291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8440, "lr": 0.0001724633321311928, "train_loss": 4.303185701370239, "grad_norm": 0.8888928890228271, "tokens_seen": 69148672, "tokens_per_sec": 20975.349310864174, "vram_gb": 5.521900544, "lambda": 0.3043541644389431, "lambda_min": 0.07897163182497025, "lambda_max": 0.5955244302749634, "S_mean": 0.24186197916666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8460, "lr": 0.00017129798188103226, "train_loss": 4.380190134048462, "grad_norm": 1.0220311880111694, "tokens_seen": 69312512, "tokens_per_sec": 20975.31373870851, "vram_gb": 5.521900544, "lambda": 0.3045568475499749, "lambda_min": 0.0789613351225853, "lambda_max": 0.5959576964378357, "S_mean": 0.24961344401041666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8480, "lr": 0.0001701371312644128, "train_loss": 4.376126050949097, "grad_norm": 0.9074544310569763, "tokens_seen": 69476352, "tokens_per_sec": 20975.521311908906, "vram_gb": 5.521900544, "lambda": 0.30478647413353127, "lambda_min": 0.07899897545576096, "lambda_max": 0.5967522859573364, "S_mean": 0.2528076171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8500, "lr": 0.00016898081319460456, "train_loss": 4.393259525299072, "grad_norm": 1.0537147521972656, "tokens_seen": 69640192, "tokens_per_sec": 20975.527743940856, "vram_gb": 5.521900544, "lambda": 0.3050056391706069, "lambda_min": 0.07905696332454681, "lambda_max": 0.5972875356674194, "S_mean": 0.2533671061197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8520, "lr": 0.00016782906045636751, "train_loss": 4.492147207260132, "grad_norm": 1.0186975002288818, "tokens_seen": 69804032, "tokens_per_sec": 20975.555913084532, "vram_gb": 5.521900544, "lambda": 0.3052216734116276, "lambda_min": 0.07910844683647156, "lambda_max": 0.5979407429695129, "S_mean": 0.24650065104166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8540, "lr": 0.000166681905705022, "train_loss": 4.555817127227783, "grad_norm": 0.9698880314826965, "tokens_seen": 69967872, "tokens_per_sec": 20975.86875031946, "vram_gb": 5.521900544, "lambda": 0.3054409930482507, "lambda_min": 0.0790756493806839, "lambda_max": 0.5985487699508667, "S_mean": 0.2509867350260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8560, "lr": 0.00016553938146552296, "train_loss": 4.355663776397705, "grad_norm": 0.8912935853004456, "tokens_seen": 70131712, "tokens_per_sec": 20976.077746961917, "vram_gb": 5.521900544, "lambda": 0.3056090821822484, "lambda_min": 0.07901111245155334, "lambda_max": 0.5989934206008911, "S_mean": 0.2547810872395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8580, "lr": 0.00016440152013153794, "train_loss": 4.463746547698975, "grad_norm": 0.9775569438934326, "tokens_seen": 70295552, "tokens_per_sec": 20976.403912736514, "vram_gb": 5.521900544, "lambda": 0.3057541136319439, "lambda_min": 0.07901277393102646, "lambda_max": 0.5988996624946594, "S_mean": 0.24988810221354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8600, "lr": 0.0001632683539645282, "train_loss": 4.28497838973999, "grad_norm": 0.9846783876419067, "tokens_seen": 70459392, "tokens_per_sec": 20976.54125302471, "vram_gb": 5.521900544, "lambda": 0.30596814459810656, "lambda_min": 0.07902112603187561, "lambda_max": 0.5996685028076172, "S_mean": 0.24566650390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8620, "lr": 0.00016213991509283438, "train_loss": 4.426886320114136, "grad_norm": 1.063116431236267, "tokens_seen": 70623232, "tokens_per_sec": 20976.793475131457, "vram_gb": 5.521900544, "lambda": 0.306221437950929, "lambda_min": 0.07901939004659653, "lambda_max": 0.6006917357444763, "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8640, "lr": 0.00016101623551076523, "train_loss": 4.490510940551758, "grad_norm": 0.985008955001831, "tokens_seen": 70787072, "tokens_per_sec": 20976.600716764766, "vram_gb": 5.521900544, "lambda": 0.3064166021843751, "lambda_min": 0.07895581424236298, "lambda_max": 0.601387619972229, "S_mean": 0.2526448567708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8660, "lr": 0.0001598973470776912, "train_loss": 4.304872512817383, "grad_norm": 1.00902259349823, "tokens_seen": 70950912, "tokens_per_sec": 20976.692728283266, "vram_gb": 5.521900544, "lambda": 0.3065540681903561, "lambda_min": 0.07892780750989914, "lambda_max": 0.6017410755157471, "S_mean": 0.2720133463541667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8680, "lr": 0.00015878328151714015, "train_loss": 4.404650688171387, "grad_norm": 0.8972557783126831, "tokens_seen": 71114752, "tokens_per_sec": 20976.883664649245, "vram_gb": 5.521900544, "lambda": 0.306756757820646, "lambda_min": 0.07889389246702194, "lambda_max": 0.6025830507278442, "S_mean": 0.2513427734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8700, "lr": 0.00015767407041589863, "train_loss": 4.260864496231079, "grad_norm": 0.97556072473526, "tokens_seen": 71278592, "tokens_per_sec": 20977.089783391406, "vram_gb": 5.521900544, "lambda": 0.3069349608073632, "lambda_min": 0.07887325435876846, "lambda_max": 0.6030782461166382, "S_mean": 0.251861572265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8720, "lr": 0.0001565697452231159, "train_loss": 4.510683059692383, "grad_norm": 0.9853975772857666, "tokens_seen": 71442432, "tokens_per_sec": 20977.467645166478, "vram_gb": 5.521900544, "lambda": 0.30718077967564267, "lambda_min": 0.07885360717773438, "lambda_max": 0.6038873195648193, "S_mean": 0.253082275390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8740, "lr": 0.00015547033724941278, "train_loss": 4.408792734146118, "grad_norm": 0.9555802345275879, "tokens_seen": 71606272, "tokens_per_sec": 20977.476927818636, "vram_gb": 5.521900544, "lambda": 0.3074283062790831, "lambda_min": 0.07878220826387405, "lambda_max": 0.604396641254425, "S_mean": 0.24237060546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8760, "lr": 0.0001543758776659931, "train_loss": 4.335913419723511, "grad_norm": 0.900704026222229, "tokens_seen": 71770112, "tokens_per_sec": 20977.63365975178, "vram_gb": 5.521900544, "lambda": 0.3076296312113603, "lambda_min": 0.0787801519036293, "lambda_max": 0.604884684085846, "S_mean": 0.250396728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8780, "lr": 0.00015328639750376085, "train_loss": 4.415759325027466, "grad_norm": 0.9849194884300232, "tokens_seen": 71933952, "tokens_per_sec": 20977.719288508193, "vram_gb": 5.521900544, "lambda": 0.30779602968444425, "lambda_min": 0.07881498336791992, "lambda_max": 0.6056069731712341, "S_mean": 0.24706013997395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8800, "lr": 0.00015220192765243957, "train_loss": 4.35111403465271, "grad_norm": 0.9948795437812805, "tokens_seen": 72097792, "tokens_per_sec": 20977.796674789504, "vram_gb": 5.521900544, "lambda": 0.3079209827507536, "lambda_min": 0.07876116782426834, "lambda_max": 0.6058730483055115, "S_mean": 0.2511494954427083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8820, "lr": 0.0001511224988596969, "train_loss": 4.292647838592529, "grad_norm": 1.0463354587554932, "tokens_seen": 72261632, "tokens_per_sec": 20977.771985300173, "vram_gb": 5.521900544, "lambda": 0.3081057832265894, "lambda_min": 0.07877764105796814, "lambda_max": 0.606468141078949, "S_mean": 0.2645161946614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8840, "lr": 0.00015004814173027298, "train_loss": 4.502118825912476, "grad_norm": 0.9980103373527527, "tokens_seen": 72425472, "tokens_per_sec": 20977.87109634071, "vram_gb": 5.521900544, "lambda": 0.30831694540878135, "lambda_min": 0.07880593091249466, "lambda_max": 0.6073061227798462, "S_mean": 0.25762939453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8860, "lr": 0.00014897888672511225, "train_loss": 4.356769561767578, "grad_norm": 0.9529381990432739, "tokens_seen": 72589312, "tokens_per_sec": 20978.02678869861, "vram_gb": 5.521900544, "lambda": 0.30854168130705756, "lambda_min": 0.0788779929280281, "lambda_max": 0.6079073548316956, "S_mean": 0.2524515787760417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8880, "lr": 0.00014791476416050025, "train_loss": 4.385635614395142, "grad_norm": 0.9862433671951294, "tokens_seen": 72753152, "tokens_per_sec": 20977.836395089835, "vram_gb": 5.521900544, "lambda": 0.3087791108215849, "lambda_min": 0.07888497412204742, "lambda_max": 0.6083694100379944, "S_mean": 0.24818929036458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8900, "lr": 0.00014685580420720364, "train_loss": 4.196193218231201, "grad_norm": 0.9962818622589111, "tokens_seen": 72916992, "tokens_per_sec": 20978.169521180327, "vram_gb": 5.521900544, "lambda": 0.308940379259487, "lambda_min": 0.07891784608364105, "lambda_max": 0.608963131904602, "S_mean": 0.2553202311197917, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8920, "lr": 0.0001458020368896154, "train_loss": 4.390070199966431, "grad_norm": 0.9393192529678345, "tokens_seen": 73080832, "tokens_per_sec": 20978.489601841688, "vram_gb": 5.521900544, "lambda": 0.30908972055961687, "lambda_min": 0.07899492979049683, "lambda_max": 0.6094479560852051, "S_mean": 0.23856608072916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8940, "lr": 0.0001447534920849029, "train_loss": 4.411184549331665, "grad_norm": 0.9483096599578857, "tokens_seen": 73244672, "tokens_per_sec": 20978.211848985466, "vram_gb": 5.521900544, "lambda": 0.3092916930715243, "lambda_min": 0.07900724560022354, "lambda_max": 0.6099867820739746, "S_mean": 0.24188232421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8960, "lr": 0.0001437101995221611, "train_loss": 4.3953857421875, "grad_norm": 0.9744001030921936, "tokens_seen": 73408512, "tokens_per_sec": 20978.65880993135, "vram_gb": 5.521900544, "lambda": 0.3094771020114422, "lambda_min": 0.07898827642202377, "lambda_max": 0.610621452331543, "S_mean": 0.2594807942708333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 8980, "lr": 0.00014267218878156958, "train_loss": 4.44512414932251, "grad_norm": 0.9886878132820129, "tokens_seen": 73572352, "tokens_per_sec": 20978.73931418917, "vram_gb": 5.521900544, "lambda": 0.30964909090350073, "lambda_min": 0.07897568494081497, "lambda_max": 0.6110621690750122, "S_mean": 0.24952189127604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9000, "lr": 0.00014163948929355415, "train_loss": 4.3814697265625, "grad_norm": 1.0418463945388794, "tokens_seen": 73736192, "tokens_per_sec": 20979.010606279935, "vram_gb": 5.521900544, "lambda": 0.3098092516884208, "lambda_min": 0.07897230982780457, "lambda_max": 0.6116237044334412, "S_mean": 0.247161865234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 9000, "val_loss": 4.462890219688416, "val_ppl": 86.73783857726096, "lambda": 0.30981984299918014, "lambda_min": 0.07897081971168518, "lambda_max": 0.6116538643836975, "lambda_per_layer": [0.1784057319164276, 0.13668283820152283, 0.14328241348266602, 0.1531038135290146, 0.16820329427719116, 0.14507538080215454, 0.12353087961673737, 0.11258038878440857, 0.07897081971168518, 0.12819844484329224, 0.21707122027873993, 0.3490646183490753, 0.33203160762786865, 0.5376014113426208, 0.5390259027481079, 0.6058290004730225, 0.6086527705192566, 0.3272797465324402, 0.4639905095100403, 0.6116538643836975, 0.4964912533760071, 0.4665437936782837, 0.25768372416496277, 0.25472280383110046], "S_mean": 0.2474365234375, "G_mean": null, "G_std": null, "tokens_seen": 73736192} +{"event": "step", "step": 9020, "lr": 0.00014061213033795194, "train_loss": 4.468338966369629, "grad_norm": 0.943628191947937, "tokens_seen": 73900032, "tokens_per_sec": 20970.47317940378, "vram_gb": 5.521900544, "lambda": 0.30994386319071054, "lambda_min": 0.07892820984125137, "lambda_max": 0.6119966506958008, "S_mean": 0.264404296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9040, "lr": 0.00013959014104318148, "train_loss": 4.410509824752808, "grad_norm": 0.9930468201637268, "tokens_seen": 74063872, "tokens_per_sec": 20970.194453438475, "vram_gb": 5.521900544, "lambda": 0.310146354449292, "lambda_min": 0.07888465374708176, "lambda_max": 0.6127352118492126, "S_mean": 0.24516805013020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9060, "lr": 0.00013857355038541703, "train_loss": 4.132365465164185, "grad_norm": 0.923859715461731, "tokens_seen": 74227712, "tokens_per_sec": 20970.32518715787, "vram_gb": 5.521900544, "lambda": 0.3103058636188507, "lambda_min": 0.07886038720607758, "lambda_max": 0.6133530139923096, "S_mean": 0.244842529296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9080, "lr": 0.0001375623871877665, "train_loss": 4.482189893722534, "grad_norm": 0.9734010696411133, "tokens_seen": 74391552, "tokens_per_sec": 20970.432517312973, "vram_gb": 5.521900544, "lambda": 0.31045226907978457, "lambda_min": 0.07887015491724014, "lambda_max": 0.6138429045677185, "S_mean": 0.250335693359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9100, "lr": 0.0001365566801194549, "train_loss": 4.433854103088379, "grad_norm": 0.9720543622970581, "tokens_seen": 74555392, "tokens_per_sec": 20970.7515738775, "vram_gb": 5.521900544, "lambda": 0.3105774127567808, "lambda_min": 0.07888941466808319, "lambda_max": 0.6141086220741272, "S_mean": 0.24965413411458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9120, "lr": 0.0001355564576950109, "train_loss": 4.287553548812866, "grad_norm": 0.9879579544067383, "tokens_seen": 74719232, "tokens_per_sec": 20970.711453167536, "vram_gb": 5.521900544, "lambda": 0.31075249674419564, "lambda_min": 0.07881470769643784, "lambda_max": 0.6144979000091553, "S_mean": 0.24885050455729166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9140, "lr": 0.00013456174827345858, "train_loss": 4.225468873977661, "grad_norm": 0.9182308912277222, "tokens_seen": 74883072, "tokens_per_sec": 20970.919484212893, "vram_gb": 5.521900544, "lambda": 0.31095711266001064, "lambda_min": 0.07875305414199829, "lambda_max": 0.6151853799819946, "S_mean": 0.23792012532552084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9160, "lr": 0.0001335725800575137, "train_loss": 4.464519500732422, "grad_norm": 0.9516516327857971, "tokens_seen": 75046912, "tokens_per_sec": 20971.04131523821, "vram_gb": 5.521900544, "lambda": 0.3111248854547739, "lambda_min": 0.078754723072052, "lambda_max": 0.6157265901565552, "S_mean": 0.2574259440104167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9180, "lr": 0.0001325889810927835, "train_loss": 4.463542461395264, "grad_norm": 0.9482018351554871, "tokens_seen": 75210752, "tokens_per_sec": 20971.426417234943, "vram_gb": 5.521900544, "lambda": 0.31127120833843946, "lambda_min": 0.07868855446577072, "lambda_max": 0.6160945892333984, "S_mean": 0.24986775716145834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9200, "lr": 0.00013161097926697187, "train_loss": 4.262861490249634, "grad_norm": 1.0014634132385254, "tokens_seen": 75374592, "tokens_per_sec": 20971.605351765695, "vram_gb": 5.521900544, "lambda": 0.3114823329572876, "lambda_min": 0.07867664843797684, "lambda_max": 0.6166484355926514, "S_mean": 0.254547119140625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9220, "lr": 0.00013063860230908857, "train_loss": 4.458808183670044, "grad_norm": 1.0332047939300537, "tokens_seen": 75538432, "tokens_per_sec": 20971.93753595626, "vram_gb": 5.521900544, "lambda": 0.31166115899880725, "lambda_min": 0.07874730229377747, "lambda_max": 0.6173765659332275, "S_mean": 0.24744160970052084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9240, "lr": 0.00012967187778866327, "train_loss": 4.47895073890686, "grad_norm": 0.9986141324043274, "tokens_seen": 75702272, "tokens_per_sec": 20972.134390296276, "vram_gb": 5.521900544, "lambda": 0.3118564309552312, "lambda_min": 0.07875091582536697, "lambda_max": 0.6179175972938538, "S_mean": 0.2530975341796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9260, "lr": 0.0001287108331149635, "train_loss": 4.352611780166626, "grad_norm": 0.9803586602210999, "tokens_seen": 75866112, "tokens_per_sec": 20972.088215978285, "vram_gb": 5.521900544, "lambda": 0.31203212061276037, "lambda_min": 0.07873755693435669, "lambda_max": 0.6185272336006165, "S_mean": 0.248809814453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9280, "lr": 0.00012775549553621773, "train_loss": 4.350796699523926, "grad_norm": 0.9937576055526733, "tokens_seen": 76029952, "tokens_per_sec": 20971.779146311826, "vram_gb": 5.521900544, "lambda": 0.3122306289151311, "lambda_min": 0.07873431593179703, "lambda_max": 0.6190860271453857, "S_mean": 0.255767822265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9300, "lr": 0.0001268058921388426, "train_loss": 4.5044355392456055, "grad_norm": 1.0898795127868652, "tokens_seen": 76193792, "tokens_per_sec": 20972.05033058093, "vram_gb": 5.521900544, "lambda": 0.31237834009031457, "lambda_min": 0.07868408411741257, "lambda_max": 0.6194592714309692, "S_mean": 0.2498779296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9320, "lr": 0.00012586204984667556, "train_loss": 4.304200649261475, "grad_norm": 0.9606972336769104, "tokens_seen": 76357632, "tokens_per_sec": 20972.267286535145, "vram_gb": 5.521900544, "lambda": 0.31255405272046727, "lambda_min": 0.07864710688591003, "lambda_max": 0.6200690269470215, "S_mean": 0.2530110677083333, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9340, "lr": 0.00012492399542021068, "train_loss": 4.499797344207764, "grad_norm": 1.0396451950073242, "tokens_seen": 76521472, "tokens_per_sec": 20972.237757740993, "vram_gb": 5.521900544, "lambda": 0.31276874151080847, "lambda_min": 0.07863041758537292, "lambda_max": 0.6207466125488281, "S_mean": 0.23582967122395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9360, "lr": 0.00012399175545584035, "train_loss": 4.306730270385742, "grad_norm": 0.979953944683075, "tokens_seen": 76685312, "tokens_per_sec": 20972.53273250553, "vram_gb": 5.521900544, "lambda": 0.3129515666514635, "lambda_min": 0.07864920049905777, "lambda_max": 0.6214670538902283, "S_mean": 0.24468485514322916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9380, "lr": 0.00012306535638510144, "train_loss": 4.451812267303467, "grad_norm": 1.0001899003982544, "tokens_seen": 76849152, "tokens_per_sec": 20972.575457529714, "vram_gb": 5.521900544, "lambda": 0.3131057446201642, "lambda_min": 0.07864744961261749, "lambda_max": 0.6220331788063049, "S_mean": 0.2504069010416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9400, "lr": 0.00012214482447392528, "train_loss": 4.2351250648498535, "grad_norm": 1.0424822568893433, "tokens_seen": 77012992, "tokens_per_sec": 20972.74063906736, "vram_gb": 5.521900544, "lambda": 0.3132893079891801, "lambda_min": 0.07861995697021484, "lambda_max": 0.6226294040679932, "S_mean": 0.2410888671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9420, "lr": 0.00012123018582189349, "train_loss": 4.2720746994018555, "grad_norm": 0.9649015665054321, "tokens_seen": 77176832, "tokens_per_sec": 20973.002085499946, "vram_gb": 5.521900544, "lambda": 0.3134178513040145, "lambda_min": 0.07866137474775314, "lambda_max": 0.6231100559234619, "S_mean": 0.24871826171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9440, "lr": 0.0001203214663614977, "train_loss": 4.3201141357421875, "grad_norm": 1.1084595918655396, "tokens_seen": 77340672, "tokens_per_sec": 20973.228549394935, "vram_gb": 5.521900544, "lambda": 0.3135824544976155, "lambda_min": 0.07867417484521866, "lambda_max": 0.6237726211547852, "S_mean": 0.24036153157552084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9460, "lr": 0.00011941869185740428, "train_loss": 4.389872074127197, "grad_norm": 0.9831513166427612, "tokens_seen": 77504512, "tokens_per_sec": 20973.594032387467, "vram_gb": 5.521900544, "lambda": 0.31373650828997296, "lambda_min": 0.0786917507648468, "lambda_max": 0.624364972114563, "S_mean": 0.24559529622395834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9480, "lr": 0.00011852188790572408, "train_loss": 4.380370140075684, "grad_norm": 0.9773229956626892, "tokens_seen": 77668352, "tokens_per_sec": 20973.700875056515, "vram_gb": 5.521900544, "lambda": 0.31384786684066057, "lambda_min": 0.07875382155179977, "lambda_max": 0.624767005443573, "S_mean": 0.246246337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9500, "lr": 0.00011763107993328652, "train_loss": 4.542376756668091, "grad_norm": 1.068457841873169, "tokens_seen": 77832192, "tokens_per_sec": 20974.148094802913, "vram_gb": 5.521900544, "lambda": 0.314007091956834, "lambda_min": 0.07874397933483124, "lambda_max": 0.6251479983329773, "S_mean": 0.24324544270833334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9520, "lr": 0.0001167462931969187, "train_loss": 4.236148357391357, "grad_norm": 1.0001221895217896, "tokens_seen": 77996032, "tokens_per_sec": 20974.216025740512, "vram_gb": 5.521900544, "lambda": 0.31414498823384446, "lambda_min": 0.0787794291973114, "lambda_max": 0.6255050301551819, "S_mean": 0.24507649739583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9540, "lr": 0.00011586755278272929, "train_loss": 4.380887269973755, "grad_norm": 0.9741703867912292, "tokens_seen": 78159872, "tokens_per_sec": 20974.551137552276, "vram_gb": 5.521900544, "lambda": 0.3142385569711526, "lambda_min": 0.07875396311283112, "lambda_max": 0.6258538961410522, "S_mean": 0.23274739583333334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9560, "lr": 0.00011499488360539726, "train_loss": 4.5587780475616455, "grad_norm": 1.0607105493545532, "tokens_seen": 78323712, "tokens_per_sec": 20974.807755093192, "vram_gb": 5.521900544, "lambda": 0.3143646822621425, "lambda_min": 0.07871019840240479, "lambda_max": 0.6264631748199463, "S_mean": 0.23551432291666666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9580, "lr": 0.00011412831040746567, "train_loss": 4.267563343048096, "grad_norm": 1.057944416999817, "tokens_seen": 78487552, "tokens_per_sec": 20974.95885943021, "vram_gb": 5.521900544, "lambda": 0.31452817873408395, "lambda_min": 0.07872365415096283, "lambda_max": 0.6270191073417664, "S_mean": 0.24305216471354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9600, "lr": 0.00011326785775863986, "train_loss": 4.4099225997924805, "grad_norm": 1.015395998954773, "tokens_seen": 78651392, "tokens_per_sec": 20975.23156754206, "vram_gb": 5.521900544, "lambda": 0.3146650704244773, "lambda_min": 0.07878250628709793, "lambda_max": 0.6275733709335327, "S_mean": 0.24405924479166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9620, "lr": 0.00011241355005509098, "train_loss": 4.361031532287598, "grad_norm": 1.1149541139602661, "tokens_seen": 78815232, "tokens_per_sec": 20975.47893667494, "vram_gb": 5.521900544, "lambda": 0.3147767983997862, "lambda_min": 0.07876866310834885, "lambda_max": 0.6277399063110352, "S_mean": 0.24302164713541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9640, "lr": 0.00011156541151876421, "train_loss": 4.524844169616699, "grad_norm": 1.230578899383545, "tokens_seen": 78979072, "tokens_per_sec": 20975.590498203575, "vram_gb": 5.521900544, "lambda": 0.3149119419977069, "lambda_min": 0.07870634645223618, "lambda_max": 0.6281344294548035, "S_mean": 0.270538330078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9660, "lr": 0.00011072346619669235, "train_loss": 4.3422839641571045, "grad_norm": 1.0200856924057007, "tokens_seen": 79142912, "tokens_per_sec": 20975.65413632524, "vram_gb": 5.521900544, "lambda": 0.31507083556304377, "lambda_min": 0.0786215290427208, "lambda_max": 0.6287824511528015, "S_mean": 0.24376424153645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9680, "lr": 0.00010988773796031338, "train_loss": 4.520211219787598, "grad_norm": 1.03693687915802, "tokens_seen": 79306752, "tokens_per_sec": 20976.039914721296, "vram_gb": 5.521900544, "lambda": 0.31520187947899103, "lambda_min": 0.07858414947986603, "lambda_max": 0.6294252872467041, "S_mean": 0.251068115234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9700, "lr": 0.00010905825050479436, "train_loss": 4.2838664054870605, "grad_norm": 0.9268338084220886, "tokens_seen": 79470592, "tokens_per_sec": 20975.84825235268, "vram_gb": 5.521900544, "lambda": 0.3153287485862772, "lambda_min": 0.07859495282173157, "lambda_max": 0.6298888325691223, "S_mean": 0.24851481119791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9720, "lr": 0.0001082350273483589, "train_loss": 4.337287664413452, "grad_norm": 1.065602421760559, "tokens_seen": 79634432, "tokens_per_sec": 20975.371172517967, "vram_gb": 5.521900544, "lambda": 0.315458486477534, "lambda_min": 0.07858816534280777, "lambda_max": 0.630127489566803, "S_mean": 0.270416259765625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9740, "lr": 0.00010741809183162102, "train_loss": 4.380239963531494, "grad_norm": 0.9678027033805847, "tokens_seen": 79798272, "tokens_per_sec": 20975.54624208815, "vram_gb": 5.521900544, "lambda": 0.3155565358077486, "lambda_min": 0.07855229079723358, "lambda_max": 0.6304115653038025, "S_mean": 0.24672444661458334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9760, "lr": 0.0001066074671169229, "train_loss": 4.39326024055481, "grad_norm": 0.9719191193580627, "tokens_seen": 79962112, "tokens_per_sec": 20975.532087562267, "vram_gb": 5.521900544, "lambda": 0.3156759003177285, "lambda_min": 0.07851642370223999, "lambda_max": 0.6307791471481323, "S_mean": 0.24556477864583334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9780, "lr": 0.00010580317618767827, "train_loss": 4.11680793762207, "grad_norm": 0.9313965439796448, "tokens_seen": 80125952, "tokens_per_sec": 20975.915554613217, "vram_gb": 5.521900544, "lambda": 0.3158117051546772, "lambda_min": 0.07847977429628372, "lambda_max": 0.6312589645385742, "S_mean": 0.24441019694010416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9800, "lr": 0.00010500524184772093, "train_loss": 4.294008493423462, "grad_norm": 0.9785882234573364, "tokens_seen": 80289792, "tokens_per_sec": 20976.032202055423, "vram_gb": 5.521900544, "lambda": 0.3159366569792231, "lambda_min": 0.07845320552587509, "lambda_max": 0.6318874359130859, "S_mean": 0.2535909016927083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9820, "lr": 0.00010421368672065815, "train_loss": 4.173714876174927, "grad_norm": 0.9999881386756897, "tokens_seen": 80453632, "tokens_per_sec": 20975.987389000336, "vram_gb": 5.521900544, "lambda": 0.3160382645825545, "lambda_min": 0.07846244424581528, "lambda_max": 0.6323105096817017, "S_mean": 0.24248758951822916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9840, "lr": 0.00010342853324922906, "train_loss": 4.511286735534668, "grad_norm": 0.9695425629615784, "tokens_seen": 80617472, "tokens_per_sec": 20976.22636899029, "vram_gb": 5.521900544, "lambda": 0.3161689784998695, "lambda_min": 0.0785033106803894, "lambda_max": 0.6327047944068909, "S_mean": 0.2699381510416667, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9860, "lr": 0.00010264980369466855, "train_loss": 4.311937570571899, "grad_norm": 1.002387523651123, "tokens_seen": 80781312, "tokens_per_sec": 20976.63846594248, "vram_gb": 5.521900544, "lambda": 0.31626544675479334, "lambda_min": 0.07848168164491653, "lambda_max": 0.632783055305481, "S_mean": 0.249908447265625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9880, "lr": 0.0001018775201360759, "train_loss": 4.355772495269775, "grad_norm": 0.9892471432685852, "tokens_seen": 80945152, "tokens_per_sec": 20976.635936508967, "vram_gb": 5.521900544, "lambda": 0.3163648185630639, "lambda_min": 0.07845333963632584, "lambda_max": 0.6330605149269104, "S_mean": 0.2538960774739583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9900, "lr": 0.00010111170446978885, "train_loss": 4.26826810836792, "grad_norm": 1.042472004890442, "tokens_seen": 81108992, "tokens_per_sec": 20976.630225112123, "vram_gb": 5.521900544, "lambda": 0.3164774135996898, "lambda_min": 0.0784231647849083, "lambda_max": 0.6334919929504395, "S_mean": 0.2596638997395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9920, "lr": 0.0001003523784087631, "train_loss": 4.308637619018555, "grad_norm": 1.018849492073059, "tokens_seen": 81272832, "tokens_per_sec": 20976.72957522235, "vram_gb": 5.521900544, "lambda": 0.3166297435139616, "lambda_min": 0.0784512609243393, "lambda_max": 0.6339848637580872, "S_mean": 0.24866231282552084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9940, "lr": 9.959956348195613e-05, "train_loss": 4.510919570922852, "grad_norm": 0.988740861415863, "tokens_seen": 81436672, "tokens_per_sec": 20976.839481782183, "vram_gb": 5.521900544, "lambda": 0.316782273662587, "lambda_min": 0.07845976948738098, "lambda_max": 0.6345532536506653, "S_mean": 0.2509307861328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9960, "lr": 9.885328103371709e-05, "train_loss": 4.397705554962158, "grad_norm": 1.0109199285507202, "tokens_seen": 81600512, "tokens_per_sec": 20977.046724326276, "vram_gb": 5.521900544, "lambda": 0.31691854167729616, "lambda_min": 0.07845701277256012, "lambda_max": 0.6349794268608093, "S_mean": 0.2374114990234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 9980, "lr": 9.81135522231818e-05, "train_loss": 4.366389751434326, "grad_norm": 0.968385636806488, "tokens_seen": 81764352, "tokens_per_sec": 20977.13949220097, "vram_gb": 5.521900544, "lambda": 0.3170410205299656, "lambda_min": 0.07847002148628235, "lambda_max": 0.6353473663330078, "S_mean": 0.25152587890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10000, "lr": 9.738039802367232e-05, "train_loss": 4.3604607582092285, "grad_norm": 0.9781030416488647, "tokens_seen": 81928192, "tokens_per_sec": 20977.41654189048, "vram_gb": 5.521900544, "lambda": 0.3171801393230756, "lambda_min": 0.07849739491939545, "lambda_max": 0.6357676982879639, "S_mean": 0.24802652994791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 10000, "val_loss": 4.420059251785278, "val_ppl": 83.10120910747025, "lambda": 0.31718500424176455, "lambda_min": 0.07849880307912827, "lambda_max": 0.635784924030304, "lambda_per_layer": [0.18106421828269958, 0.13682082295417786, 0.14372535049915314, 0.1547761857509613, 0.17042101919651031, 0.1461123675107956, 0.12366331368684769, 0.11278174072504044, 0.07849880307912827, 0.1285272240638733, 0.21820339560508728, 0.35210296511650085, 0.335437536239624, 0.5479781031608582, 0.552251935005188, 0.6232855916023254, 0.63029944896698, 0.33505627512931824, 0.4806286096572876, 0.635784924030304, 0.5144182443618774, 0.4822198152542114, 0.26524072885513306, 0.2631414830684662], "S_mean": 0.250762939453125, "G_mean": null, "G_std": null, "tokens_seen": 81928192} +{"event": "step", "step": 10020, "lr": 9.665383922210301e-05, "train_loss": 4.5105977058410645, "grad_norm": 1.1888737678527832, "tokens_seen": 82092032, "tokens_per_sec": 20969.55589297856, "vram_gb": 5.521900544, "lambda": 0.3172966210792462, "lambda_min": 0.07852984964847565, "lambda_max": 0.636222243309021, "S_mean": 0.2659556070963542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10040, "lr": 9.593389641839048e-05, "train_loss": 4.332789897918701, "grad_norm": 1.0279079675674438, "tokens_seen": 82255872, "tokens_per_sec": 20969.437475618346, "vram_gb": 5.521900544, "lambda": 0.3174222592885296, "lambda_min": 0.07860497385263443, "lambda_max": 0.6365774869918823, "S_mean": 0.2507375081380208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10060, "lr": 9.522059002486981e-05, "train_loss": 4.3124799728393555, "grad_norm": 1.0350990295410156, "tokens_seen": 82419712, "tokens_per_sec": 20969.518779118418, "vram_gb": 5.521900544, "lambda": 0.31750708787391585, "lambda_min": 0.07863999158143997, "lambda_max": 0.6368985772132874, "S_mean": 0.2542775472005208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10080, "lr": 9.45139402657161e-05, "train_loss": 4.361564874649048, "grad_norm": 1.0837901830673218, "tokens_seen": 82583552, "tokens_per_sec": 20969.760544412133, "vram_gb": 5.521900544, "lambda": 0.31764108687639236, "lambda_min": 0.07865149527788162, "lambda_max": 0.6372029781341553, "S_mean": 0.23799641927083334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10100, "lr": 9.381396717637045e-05, "train_loss": 4.368465423583984, "grad_norm": 0.9945093393325806, "tokens_seen": 82747392, "tokens_per_sec": 20970.160728360508, "vram_gb": 5.521900544, "lambda": 0.31775849498808384, "lambda_min": 0.07861838489770889, "lambda_max": 0.6377338171005249, "S_mean": 0.23920694986979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10120, "lr": 9.312069060297244e-05, "train_loss": 4.376124858856201, "grad_norm": 0.9669700860977173, "tokens_seen": 82911232, "tokens_per_sec": 20970.32519357679, "vram_gb": 5.521900544, "lambda": 0.3178792716935277, "lambda_min": 0.07862107455730438, "lambda_max": 0.6381607055664062, "S_mean": 0.24778238932291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10140, "lr": 9.243413020179702e-05, "train_loss": 4.1688432693481445, "grad_norm": 0.9885736107826233, "tokens_seen": 83075072, "tokens_per_sec": 20970.82851229905, "vram_gb": 5.521900544, "lambda": 0.31797652163853246, "lambda_min": 0.07860377430915833, "lambda_max": 0.6384826898574829, "S_mean": 0.2537638346354167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10160, "lr": 9.17543054386977e-05, "train_loss": 4.2584569454193115, "grad_norm": 1.0241597890853882, "tokens_seen": 83238912, "tokens_per_sec": 20971.010816496604, "vram_gb": 5.521900544, "lambda": 0.3180946158245206, "lambda_min": 0.07862576097249985, "lambda_max": 0.6389399766921997, "S_mean": 0.24452718098958334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10180, "lr": 9.108123558855408e-05, "train_loss": 4.457998991012573, "grad_norm": 1.1060062646865845, "tokens_seen": 83402752, "tokens_per_sec": 20971.26186334833, "vram_gb": 5.521900544, "lambda": 0.31821469341715175, "lambda_min": 0.078589528799057, "lambda_max": 0.6393975019454956, "S_mean": 0.2398223876953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10200, "lr": 9.041493973472571e-05, "train_loss": 4.33261775970459, "grad_norm": 1.0187137126922607, "tokens_seen": 83566592, "tokens_per_sec": 20971.227975598642, "vram_gb": 5.521900544, "lambda": 0.31834058091044426, "lambda_min": 0.07858801633119583, "lambda_max": 0.6397843360900879, "S_mean": 0.2512054443359375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10220, "lr": 8.97554367685109e-05, "train_loss": 4.414090394973755, "grad_norm": 1.1132607460021973, "tokens_seen": 83730432, "tokens_per_sec": 20971.63138977078, "vram_gb": 5.521900544, "lambda": 0.3184533677995205, "lambda_min": 0.07857560366392136, "lambda_max": 0.6401039958000183, "S_mean": 0.24675496419270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10240, "lr": 8.910274538861126e-05, "train_loss": 4.230705499649048, "grad_norm": 1.1497697830200195, "tokens_seen": 83894272, "tokens_per_sec": 20971.67370736619, "vram_gb": 5.521900544, "lambda": 0.31857854531457025, "lambda_min": 0.07857218384742737, "lambda_max": 0.6404882073402405, "S_mean": 0.2673288981119792, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10260, "lr": 8.845688410060126e-05, "train_loss": 4.400719404220581, "grad_norm": 1.04905104637146, "tokens_seen": 84058112, "tokens_per_sec": 20971.810029232667, "vram_gb": 5.521900544, "lambda": 0.3186947737510006, "lambda_min": 0.07855293899774551, "lambda_max": 0.640811026096344, "S_mean": 0.2396240234375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10280, "lr": 8.781787121640375e-05, "train_loss": 4.417839288711548, "grad_norm": 1.0081199407577515, "tokens_seen": 84221952, "tokens_per_sec": 20972.174173736676, "vram_gb": 5.521900544, "lambda": 0.31884903988490504, "lambda_min": 0.0785093680024147, "lambda_max": 0.6411857604980469, "S_mean": 0.251800537109375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10300, "lr": 8.718572485377082e-05, "train_loss": 4.285373687744141, "grad_norm": 0.978723406791687, "tokens_seen": 84385792, "tokens_per_sec": 20972.141667743417, "vram_gb": 5.521900544, "lambda": 0.31895187124609947, "lambda_min": 0.07849115133285522, "lambda_max": 0.6415368318557739, "S_mean": 0.240814208984375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10320, "lr": 8.656046293576982e-05, "train_loss": 4.438645362854004, "grad_norm": 1.068582534790039, "tokens_seen": 84549632, "tokens_per_sec": 20972.02404513436, "vram_gb": 5.521900544, "lambda": 0.31905988572786254, "lambda_min": 0.07847429066896439, "lambda_max": 0.6419004797935486, "S_mean": 0.2511545817057292, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10340, "lr": 8.59421031902756e-05, "train_loss": 4.366619348526001, "grad_norm": 1.126011848449707, "tokens_seen": 84713472, "tokens_per_sec": 20972.093956023054, "vram_gb": 5.521900544, "lambda": 0.3191622418041031, "lambda_min": 0.0784907341003418, "lambda_max": 0.642175018787384, "S_mean": 0.24463399251302084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10360, "lr": 8.533066314946748e-05, "train_loss": 4.433237552642822, "grad_norm": 1.0893348455429077, "tokens_seen": 84877312, "tokens_per_sec": 20972.21199372369, "vram_gb": 5.521900544, "lambda": 0.3192675669367115, "lambda_min": 0.07849689573049545, "lambda_max": 0.6423979997634888, "S_mean": 0.24236043294270834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10380, "lr": 8.47261601493324e-05, "train_loss": 4.3987884521484375, "grad_norm": 1.1429558992385864, "tokens_seen": 85041152, "tokens_per_sec": 20972.37108225967, "vram_gb": 5.521900544, "lambda": 0.31937368400394917, "lambda_min": 0.07850813120603561, "lambda_max": 0.6426882743835449, "S_mean": 0.24114481608072916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10400, "lr": 8.412861132917342e-05, "train_loss": 4.453961372375488, "grad_norm": 1.0111640691757202, "tokens_seen": 85204992, "tokens_per_sec": 20972.112935752382, "vram_gb": 5.521900544, "lambda": 0.31947052385658026, "lambda_min": 0.07853671908378601, "lambda_max": 0.6430615782737732, "S_mean": 0.2558746337890625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10420, "lr": 8.353803363112361e-05, "train_loss": 4.404626131057739, "grad_norm": 1.027212142944336, "tokens_seen": 85368832, "tokens_per_sec": 20972.483805313368, "vram_gb": 5.521900544, "lambda": 0.3195616928860545, "lambda_min": 0.07857482880353928, "lambda_max": 0.6435464024543762, "S_mean": 0.24630228678385416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10440, "lr": 8.295444379966585e-05, "train_loss": 4.26800274848938, "grad_norm": 0.9680227041244507, "tokens_seen": 85532672, "tokens_per_sec": 20972.416048091214, "vram_gb": 5.521900544, "lambda": 0.31962453449765843, "lambda_min": 0.07859436422586441, "lambda_max": 0.6439984440803528, "S_mean": 0.2584889729817708, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10460, "lr": 8.237785838115788e-05, "train_loss": 4.414725303649902, "grad_norm": 1.0970126390457153, "tokens_seen": 85696512, "tokens_per_sec": 20972.55986649056, "vram_gb": 5.521900544, "lambda": 0.3197386482109626, "lambda_min": 0.07858863472938538, "lambda_max": 0.6443900465965271, "S_mean": 0.24070231119791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10480, "lr": 8.18082937233634e-05, "train_loss": 4.273125648498535, "grad_norm": 1.0056774616241455, "tokens_seen": 85860352, "tokens_per_sec": 20972.450830054997, "vram_gb": 5.521900544, "lambda": 0.31984360950688523, "lambda_min": 0.07858987152576447, "lambda_max": 0.6446550488471985, "S_mean": 0.238739013671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10500, "lr": 8.12457659749885e-05, "train_loss": 4.353208541870117, "grad_norm": 1.0426721572875977, "tokens_seen": 86024192, "tokens_per_sec": 20972.539756507194, "vram_gb": 5.521900544, "lambda": 0.31994096903751296, "lambda_min": 0.07859151810407639, "lambda_max": 0.6448503732681274, "S_mean": 0.2298431396484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10520, "lr": 8.069029108522372e-05, "train_loss": 4.410834074020386, "grad_norm": 1.116071105003357, "tokens_seen": 86188032, "tokens_per_sec": 20972.58064331755, "vram_gb": 5.521900544, "lambda": 0.3200582650800546, "lambda_min": 0.07859817892313004, "lambda_max": 0.6452050805091858, "S_mean": 0.2552998860677083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10540, "lr": 8.014188480329182e-05, "train_loss": 4.337773084640503, "grad_norm": 1.0425492525100708, "tokens_seen": 86351872, "tokens_per_sec": 20972.76763281922, "vram_gb": 5.521900544, "lambda": 0.3201578613370657, "lambda_min": 0.0785970687866211, "lambda_max": 0.645563006401062, "S_mean": 0.240936279296875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10560, "lr": 7.960056267800137e-05, "train_loss": 4.3410632610321045, "grad_norm": 1.1028039455413818, "tokens_seen": 86515712, "tokens_per_sec": 20972.912527872282, "vram_gb": 5.521900544, "lambda": 0.32024145995577175, "lambda_min": 0.07861515879631042, "lambda_max": 0.6458523273468018, "S_mean": 0.24827067057291666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10580, "lr": 7.906634005730591e-05, "train_loss": 4.366626262664795, "grad_norm": 1.1081184148788452, "tokens_seen": 86679552, "tokens_per_sec": 20972.877768573835, "vram_gb": 5.521900544, "lambda": 0.32031300974388915, "lambda_min": 0.07862724363803864, "lambda_max": 0.6461049318313599, "S_mean": 0.23472086588541666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10600, "lr": 7.853923208786862e-05, "train_loss": 4.24438738822937, "grad_norm": 1.0366178750991821, "tokens_seen": 86843392, "tokens_per_sec": 20973.171119891576, "vram_gb": 5.521900544, "lambda": 0.3204116464282076, "lambda_min": 0.07863524556159973, "lambda_max": 0.6463832259178162, "S_mean": 0.24988810221354166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10620, "lr": 7.80192537146331e-05, "train_loss": 4.2697224617004395, "grad_norm": 1.0376720428466797, "tokens_seen": 87007232, "tokens_per_sec": 20973.212895368764, "vram_gb": 5.521900544, "lambda": 0.3204937456175685, "lambda_min": 0.07860887050628662, "lambda_max": 0.6467332243919373, "S_mean": 0.2507171630859375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10640, "lr": 7.750641968039935e-05, "train_loss": 4.27360725402832, "grad_norm": 1.101028323173523, "tokens_seen": 87171072, "tokens_per_sec": 20973.391294562025, "vram_gb": 5.521900544, "lambda": 0.32058369740843773, "lambda_min": 0.07863704115152359, "lambda_max": 0.6470680236816406, "S_mean": 0.23980204264322916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10660, "lr": 7.700074452540616e-05, "train_loss": 4.433136701583862, "grad_norm": 1.0765528678894043, "tokens_seen": 87334912, "tokens_per_sec": 20973.5571279821, "vram_gb": 5.521900544, "lambda": 0.32066657052685815, "lambda_min": 0.07868413627147675, "lambda_max": 0.6474331021308899, "S_mean": 0.24872843424479166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10680, "lr": 7.650224258691843e-05, "train_loss": 4.334486961364746, "grad_norm": 1.061069130897522, "tokens_seen": 87498752, "tokens_per_sec": 20973.572144691065, "vram_gb": 5.521900544, "lambda": 0.3207789333537221, "lambda_min": 0.07871808111667633, "lambda_max": 0.6478654742240906, "S_mean": 0.25408935546875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10700, "lr": 7.601092799882103e-05, "train_loss": 4.3055830001831055, "grad_norm": 1.2018921375274658, "tokens_seen": 87662592, "tokens_per_sec": 20973.67515007595, "vram_gb": 5.521900544, "lambda": 0.3208739658196767, "lambda_min": 0.07872425019741058, "lambda_max": 0.648102879524231, "S_mean": 0.2509867350260417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10720, "lr": 7.552681469121778e-05, "train_loss": 4.4448723793029785, "grad_norm": 1.3247259855270386, "tokens_seen": 87826432, "tokens_per_sec": 20973.86767246446, "vram_gb": 5.521900544, "lambda": 0.32099065215637285, "lambda_min": 0.07872722297906876, "lambda_max": 0.6484516263008118, "S_mean": 0.2557576497395833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10740, "lr": 7.504991639003676e-05, "train_loss": 4.416244983673096, "grad_norm": 1.0309644937515259, "tokens_seen": 87990272, "tokens_per_sec": 20974.140403127065, "vram_gb": 5.521900544, "lambda": 0.32108022769292194, "lambda_min": 0.07873093336820602, "lambda_max": 0.6488306522369385, "S_mean": 0.248321533203125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10760, "lr": 7.458024661664092e-05, "train_loss": 4.218538761138916, "grad_norm": 1.1321141719818115, "tokens_seen": 88154112, "tokens_per_sec": 20973.974356828552, "vram_gb": 5.521900544, "lambda": 0.3211616255963842, "lambda_min": 0.07870993763208389, "lambda_max": 0.6490769982337952, "S_mean": 0.253631591796875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10780, "lr": 7.411781868744484e-05, "train_loss": 4.25766658782959, "grad_norm": 1.074897289276123, "tokens_seen": 88317952, "tokens_per_sec": 20974.389644224215, "vram_gb": 5.521900544, "lambda": 0.321255587041378, "lambda_min": 0.07868406921625137, "lambda_max": 0.6493557691574097, "S_mean": 0.24004109700520834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10800, "lr": 7.366264571353706e-05, "train_loss": 4.330764293670654, "grad_norm": 1.1218575239181519, "tokens_seen": 88481792, "tokens_per_sec": 20974.584363115235, "vram_gb": 5.521900544, "lambda": 0.32135916501283646, "lambda_min": 0.07866186648607254, "lambda_max": 0.649765133857727, "S_mean": 0.24272664388020834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10820, "lr": 7.321474060030854e-05, "train_loss": 4.284181833267212, "grad_norm": 1.116031289100647, "tokens_seen": 88645632, "tokens_per_sec": 20974.580026139854, "vram_gb": 5.521900544, "lambda": 0.32144642248749733, "lambda_min": 0.0786726251244545, "lambda_max": 0.6500791311264038, "S_mean": 0.247528076171875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10840, "lr": 7.277411604708658e-05, "train_loss": 4.289013862609863, "grad_norm": 1.0900671482086182, "tokens_seen": 88809472, "tokens_per_sec": 20974.828884542727, "vram_gb": 5.521900544, "lambda": 0.32150103369106847, "lambda_min": 0.07868832349777222, "lambda_max": 0.6501566171646118, "S_mean": 0.2514851888020833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10860, "lr": 7.23407845467748e-05, "train_loss": 4.279543399810791, "grad_norm": 1.1290576457977295, "tokens_seen": 88973312, "tokens_per_sec": 20974.694419628508, "vram_gb": 5.521900544, "lambda": 0.3215882144868374, "lambda_min": 0.07871931046247482, "lambda_max": 0.6504799723625183, "S_mean": 0.24994913736979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10880, "lr": 7.19147583854989e-05, "train_loss": 4.254411220550537, "grad_norm": 1.1070548295974731, "tokens_seen": 89137152, "tokens_per_sec": 20974.969709173234, "vram_gb": 5.521900544, "lambda": 0.32168089381108683, "lambda_min": 0.07875587791204453, "lambda_max": 0.650739312171936, "S_mean": 0.24808756510416666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10900, "lr": 7.149604964225853e-05, "train_loss": 4.1930992603302, "grad_norm": 1.028834342956543, "tokens_seen": 89300992, "tokens_per_sec": 20975.204344145674, "vram_gb": 5.521900544, "lambda": 0.321766749334832, "lambda_min": 0.07875636219978333, "lambda_max": 0.6510125398635864, "S_mean": 0.24437459309895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10920, "lr": 7.108467018858448e-05, "train_loss": 4.347858667373657, "grad_norm": 1.1038362979888916, "tokens_seen": 89464832, "tokens_per_sec": 20975.25966852002, "vram_gb": 5.521900544, "lambda": 0.3218550933524966, "lambda_min": 0.07872611284255981, "lambda_max": 0.6512686610221863, "S_mean": 0.24876912434895834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10940, "lr": 7.068063168820236e-05, "train_loss": 4.343914031982422, "grad_norm": 1.13438081741333, "tokens_seen": 89628672, "tokens_per_sec": 20975.26195277905, "vram_gb": 5.521900544, "lambda": 0.32195209587613743, "lambda_min": 0.07869575172662735, "lambda_max": 0.6516901254653931, "S_mean": 0.250518798828125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10960, "lr": 7.028394559670175e-05, "train_loss": 4.305848121643066, "grad_norm": 1.1052440404891968, "tokens_seen": 89792512, "tokens_per_sec": 20975.34701774777, "vram_gb": 5.521900544, "lambda": 0.32202807379265624, "lambda_min": 0.07871616631746292, "lambda_max": 0.6520668268203735, "S_mean": 0.24815877278645834, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 10980, "lr": 6.989462316121147e-05, "train_loss": 4.478109836578369, "grad_norm": 1.1277387142181396, "tokens_seen": 89956352, "tokens_per_sec": 20974.776887030745, "vram_gb": 5.521900544, "lambda": 0.3221087222918868, "lambda_min": 0.07875292003154755, "lambda_max": 0.6523929238319397, "S_mean": 0.23954264322916666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11000, "lr": 6.95126754200807e-05, "train_loss": 4.39249849319458, "grad_norm": 1.0658725500106812, "tokens_seen": 90120192, "tokens_per_sec": 20974.752213601547, "vram_gb": 5.521900544, "lambda": 0.3221934602285425, "lambda_min": 0.07875682413578033, "lambda_max": 0.6526920795440674, "S_mean": 0.24561564127604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11000, "val_loss": 4.377960872650147, "val_ppl": 79.67539935972789, "lambda": 0.32219812646508217, "lambda_min": 0.07875614613294601, "lambda_max": 0.6527119874954224, "lambda_per_layer": [0.1835058480501175, 0.13690727949142456, 0.1439160704612732, 0.15586717426776886, 0.17173202335834503, 0.14683908224105835, 0.12370602041482925, 0.11298951506614685, 0.07875614613294601, 0.12879551947116852, 0.21920600533485413, 0.35391777753829956, 0.3375815749168396, 0.5558779835700989, 0.5591061115264893, 0.6340040564537048, 0.6443955302238464, 0.34042292833328247, 0.4926453232765198, 0.6527119874954224, 0.5278891324996948, 0.4926718473434448, 0.2704585790634155, 0.26885151863098145], "S_mean": 0.24739583333333334, "G_mean": null, "G_std": null, "tokens_seen": 90120192} +{"event": "step", "step": 11020, "lr": 6.913811320256596e-05, "train_loss": 4.198936462402344, "grad_norm": 1.0811591148376465, "tokens_seen": 90284032, "tokens_per_sec": 20968.10738390355, "vram_gb": 5.521900544, "lambda": 0.3222708376124501, "lambda_min": 0.07873313128948212, "lambda_max": 0.6530137062072754, "S_mean": 0.2515157063802083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11040, "lr": 6.877094712852416e-05, "train_loss": 4.298766136169434, "grad_norm": 1.13998544216156, "tokens_seen": 90447872, "tokens_per_sec": 20967.82834209355, "vram_gb": 5.521900544, "lambda": 0.3223801553249359, "lambda_min": 0.07872507721185684, "lambda_max": 0.653397262096405, "S_mean": 0.25225830078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11060, "lr": 6.841118760811133e-05, "train_loss": 4.456212520599365, "grad_norm": 1.103873610496521, "tokens_seen": 90611712, "tokens_per_sec": 20968.218277281612, "vram_gb": 5.521900544, "lambda": 0.32246169665207464, "lambda_min": 0.07873625308275223, "lambda_max": 0.6536113023757935, "S_mean": 0.2440948486328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11080, "lr": 6.805884484148771e-05, "train_loss": 4.311638355255127, "grad_norm": 1.0589436292648315, "tokens_seen": 90775552, "tokens_per_sec": 20968.38690547186, "vram_gb": 5.521900544, "lambda": 0.32255756855010986, "lambda_min": 0.07875324040651321, "lambda_max": 0.6537803411483765, "S_mean": 0.2542928059895833, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11100, "lr": 6.771392881852834e-05, "train_loss": 4.351041793823242, "grad_norm": 1.3007495403289795, "tokens_seen": 90939392, "tokens_per_sec": 20968.4793795358, "vram_gb": 5.521900544, "lambda": 0.32264599079887074, "lambda_min": 0.07877762615680695, "lambda_max": 0.6538245677947998, "S_mean": 0.2635396321614583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11120, "lr": 6.737644931853984e-05, "train_loss": 4.393187761306763, "grad_norm": 1.055832862854004, "tokens_seen": 91103232, "tokens_per_sec": 20968.718755663183, "vram_gb": 5.521900544, "lambda": 0.3227294208481908, "lambda_min": 0.07878047227859497, "lambda_max": 0.654034435749054, "S_mean": 0.24946085611979166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11140, "lr": 6.70464159099833e-05, "train_loss": 4.35635781288147, "grad_norm": 1.088732123374939, "tokens_seen": 91267072, "tokens_per_sec": 20969.021756252438, "vram_gb": 5.521900544, "lambda": 0.3228020553166668, "lambda_min": 0.07879241555929184, "lambda_max": 0.6544031500816345, "S_mean": 0.25177001953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11160, "lr": 6.672383795020283e-05, "train_loss": 4.249967575073242, "grad_norm": 1.0545952320098877, "tokens_seen": 91430912, "tokens_per_sec": 20969.086128712715, "vram_gb": 5.521900544, "lambda": 0.3228835665310423, "lambda_min": 0.07879117131233215, "lambda_max": 0.6547329425811768, "S_mean": 0.263641357421875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11180, "lr": 6.64087245851603e-05, "train_loss": 4.371442556381226, "grad_norm": 1.085361123085022, "tokens_seen": 91594752, "tokens_per_sec": 20969.16871098214, "vram_gb": 5.521900544, "lambda": 0.32297529994199675, "lambda_min": 0.0788034051656723, "lambda_max": 0.6549835205078125, "S_mean": 0.2523091634114583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11200, "lr": 6.6101084749176e-05, "train_loss": 4.350666522979736, "grad_norm": 1.097619891166687, "tokens_seen": 91758592, "tokens_per_sec": 20969.343091237508, "vram_gb": 5.521900544, "lambda": 0.3230682422096531, "lambda_min": 0.07881640642881393, "lambda_max": 0.6553276777267456, "S_mean": 0.2581125895182292, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11220, "lr": 6.580092716467547e-05, "train_loss": 4.218590974807739, "grad_norm": 1.0781183242797852, "tokens_seen": 91922432, "tokens_per_sec": 20969.33902630345, "vram_gb": 5.521900544, "lambda": 0.32313474143544835, "lambda_min": 0.07883124053478241, "lambda_max": 0.6556107997894287, "S_mean": 0.24650065104166666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11240, "lr": 6.550826034194197e-05, "train_loss": 4.425546646118164, "grad_norm": 1.0609560012817383, "tokens_seen": 92086272, "tokens_per_sec": 20969.45146283621, "vram_gb": 5.521900544, "lambda": 0.32321928534656763, "lambda_min": 0.07883941382169724, "lambda_max": 0.6559280753135681, "S_mean": 0.2387847900390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11260, "lr": 6.522309257887536e-05, "train_loss": 4.409454345703125, "grad_norm": 1.0871264934539795, "tokens_seen": 92250112, "tokens_per_sec": 20969.549385295202, "vram_gb": 5.521900544, "lambda": 0.3232995814954241, "lambda_min": 0.07887387275695801, "lambda_max": 0.6562374830245972, "S_mean": 0.246185302734375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11280, "lr": 6.494543196075678e-05, "train_loss": 4.415302991867065, "grad_norm": 1.2282451391220093, "tokens_seen": 92413952, "tokens_per_sec": 20969.743490985806, "vram_gb": 5.521900544, "lambda": 0.32340175782640773, "lambda_min": 0.07888863235712051, "lambda_max": 0.6565809845924377, "S_mean": 0.2501017252604167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11300, "lr": 6.467528636001934e-05, "train_loss": 4.27796745300293, "grad_norm": 1.309248924255371, "tokens_seen": 92577792, "tokens_per_sec": 20969.99876902235, "vram_gb": 5.521900544, "lambda": 0.3234774610027671, "lambda_min": 0.07892142981290817, "lambda_max": 0.6568957567214966, "S_mean": 0.2461395263671875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11320, "lr": 6.441266343602508e-05, "train_loss": 4.352540731430054, "grad_norm": 0.9782376885414124, "tokens_seen": 92741632, "tokens_per_sec": 20970.243781652527, "vram_gb": 5.521900544, "lambda": 0.32354277651757, "lambda_min": 0.07892584055662155, "lambda_max": 0.6570823788642883, "S_mean": 0.24547831217447916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11340, "lr": 6.415757063484764e-05, "train_loss": 4.589942216873169, "grad_norm": 1.1195251941680908, "tokens_seen": 92905472, "tokens_per_sec": 20970.47324113972, "vram_gb": 5.521900544, "lambda": 0.3236060934141278, "lambda_min": 0.07891719043254852, "lambda_max": 0.6573569774627686, "S_mean": 0.24603780110677084, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11360, "lr": 6.391001518906129e-05, "train_loss": 4.276684045791626, "grad_norm": 1.0983985662460327, "tokens_seen": 93069312, "tokens_per_sec": 20970.647547466877, "vram_gb": 5.521900544, "lambda": 0.32369110733270645, "lambda_min": 0.07891697436571121, "lambda_max": 0.6576681137084961, "S_mean": 0.2594146728515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11380, "lr": 6.367000411753565e-05, "train_loss": 4.34850549697876, "grad_norm": 1.2154210805892944, "tokens_seen": 93233152, "tokens_per_sec": 20971.001072431412, "vram_gb": 5.521900544, "lambda": 0.3237645809228222, "lambda_min": 0.07892780750989914, "lambda_max": 0.6579480171203613, "S_mean": 0.2536163330078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11400, "lr": 6.343754422523697e-05, "train_loss": 4.332664489746094, "grad_norm": 1.0587024688720703, "tokens_seen": 93396992, "tokens_per_sec": 20971.17165851834, "vram_gb": 5.521900544, "lambda": 0.32383892219513655, "lambda_min": 0.07894663512706757, "lambda_max": 0.6581951975822449, "S_mean": 0.24412027994791666, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11420, "lr": 6.321264210303496e-05, "train_loss": 4.276928901672363, "grad_norm": 1.110642910003662, "tokens_seen": 93560832, "tokens_per_sec": 20971.559084846504, "vram_gb": 5.521900544, "lambda": 0.3239480024203658, "lambda_min": 0.07896806299686432, "lambda_max": 0.6585614085197449, "S_mean": 0.241729736328125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11440, "lr": 6.299530412751607e-05, "train_loss": 4.355758428573608, "grad_norm": 1.1264513731002808, "tokens_seen": 93724672, "tokens_per_sec": 20971.669999709353, "vram_gb": 5.521900544, "lambda": 0.3240473549813032, "lambda_min": 0.07896382361650467, "lambda_max": 0.6589318513870239, "S_mean": 0.2607676188151042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11460, "lr": 6.278553646080253e-05, "train_loss": 4.273661375045776, "grad_norm": 1.138720154762268, "tokens_seen": 93888512, "tokens_per_sec": 20971.76166937594, "vram_gb": 5.521900544, "lambda": 0.3241238674769799, "lambda_min": 0.07898379862308502, "lambda_max": 0.6592174172401428, "S_mean": 0.2360382080078125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11480, "lr": 6.258334505037787e-05, "train_loss": 4.507925987243652, "grad_norm": 1.0831310749053955, "tokens_seen": 94052352, "tokens_per_sec": 20971.70314048966, "vram_gb": 5.521900544, "lambda": 0.3241839933519562, "lambda_min": 0.0789913535118103, "lambda_max": 0.6595258712768555, "S_mean": 0.23440043131510416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11500, "lr": 6.238873562891809e-05, "train_loss": 4.264666795730591, "grad_norm": 1.1374584436416626, "tokens_seen": 94216192, "tokens_per_sec": 20971.51945799497, "vram_gb": 5.521900544, "lambda": 0.3242741447562973, "lambda_min": 0.07899410277605057, "lambda_max": 0.6598014235496521, "S_mean": 0.2502695719401042, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11520, "lr": 6.220171371412917e-05, "train_loss": 4.440380096435547, "grad_norm": 1.144604206085205, "tokens_seen": 94380032, "tokens_per_sec": 20971.661010929914, "vram_gb": 5.521900544, "lambda": 0.3243582422534625, "lambda_min": 0.07898195087909698, "lambda_max": 0.6601257920265198, "S_mean": 0.2546437581380208, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11540, "lr": 6.202228460859072e-05, "train_loss": 4.3770692348480225, "grad_norm": 1.1132649183273315, "tokens_seen": 94543872, "tokens_per_sec": 20971.568412970268, "vram_gb": 5.521900544, "lambda": 0.3244298954183857, "lambda_min": 0.0790003165602684, "lambda_max": 0.6602475643157959, "S_mean": 0.24587504069010416, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11560, "lr": 6.185045339960554e-05, "train_loss": 4.350161075592041, "grad_norm": 1.084610104560852, "tokens_seen": 94707712, "tokens_per_sec": 20969.699817745506, "vram_gb": 5.521900544, "lambda": 0.3244766149049004, "lambda_min": 0.07900355756282806, "lambda_max": 0.6603127121925354, "S_mean": 0.24342854817708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11580, "lr": 6.16862249590554e-05, "train_loss": 4.454514980316162, "grad_norm": 1.0090994834899902, "tokens_seen": 94871552, "tokens_per_sec": 20969.477416888705, "vram_gb": 5.521900544, "lambda": 0.3245675954967737, "lambda_min": 0.07900688797235489, "lambda_max": 0.6606013178825378, "S_mean": 0.24074300130208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11600, "lr": 6.152960394326298e-05, "train_loss": 4.310502052307129, "grad_norm": 1.1332614421844482, "tokens_seen": 95035392, "tokens_per_sec": 20969.772738708078, "vram_gb": 5.521900544, "lambda": 0.3246582867577672, "lambda_min": 0.07902083545923233, "lambda_max": 0.6608773469924927, "S_mean": 0.24853515625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11620, "lr": 6.13805947928597e-05, "train_loss": 4.224703550338745, "grad_norm": 1.2086933851242065, "tokens_seen": 95199232, "tokens_per_sec": 20969.96843339596, "vram_gb": 5.521900544, "lambda": 0.32472286621729535, "lambda_min": 0.07901199162006378, "lambda_max": 0.6611139178276062, "S_mean": 0.24342854817708334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11640, "lr": 6.123920173265997e-05, "train_loss": 4.31818962097168, "grad_norm": 1.0798929929733276, "tokens_seen": 95363072, "tokens_per_sec": 20969.82676852321, "vram_gb": 5.521900544, "lambda": 0.3248024582862854, "lambda_min": 0.07900586724281311, "lambda_max": 0.6613902449607849, "S_mean": 0.2532094319661458, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11660, "lr": 6.110542877154136e-05, "train_loss": 4.222339153289795, "grad_norm": 1.0845052003860474, "tokens_seen": 95526912, "tokens_per_sec": 20970.257340275533, "vram_gb": 5.521900544, "lambda": 0.3248705168565114, "lambda_min": 0.07902920991182327, "lambda_max": 0.6616891622543335, "S_mean": 0.24944559733072916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11680, "lr": 6.097927970233088e-05, "train_loss": 4.366001605987549, "grad_norm": 1.0565544366836548, "tokens_seen": 95690752, "tokens_per_sec": 20970.476188451186, "vram_gb": 5.521900544, "lambda": 0.3249536106983821, "lambda_min": 0.07904656231403351, "lambda_max": 0.6620430946350098, "S_mean": 0.2545166015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11700, "lr": 6.086075810169747e-05, "train_loss": 4.2660112380981445, "grad_norm": 1.0910773277282715, "tokens_seen": 95854592, "tokens_per_sec": 20970.51105061155, "vram_gb": 5.521900544, "lambda": 0.32502557275195915, "lambda_min": 0.07904893904924393, "lambda_max": 0.662377119064331, "S_mean": 0.2602437337239583, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11720, "lr": 6.074986733005065e-05, "train_loss": 4.309810161590576, "grad_norm": 1.1182861328125, "tokens_seen": 96018432, "tokens_per_sec": 20970.59561231957, "vram_gb": 5.521900544, "lambda": 0.3251111240436633, "lambda_min": 0.07905000448226929, "lambda_max": 0.662787914276123, "S_mean": 0.2366790771484375, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11740, "lr": 6.064661053144516e-05, "train_loss": 4.255445241928101, "grad_norm": 1.1139941215515137, "tokens_seen": 96182272, "tokens_per_sec": 20970.562084604975, "vram_gb": 5.521900544, "lambda": 0.32518199644982815, "lambda_min": 0.07906637340784073, "lambda_max": 0.6630865335464478, "S_mean": 0.24591064453125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11760, "lr": 6.0550990633491926e-05, "train_loss": 4.345399379730225, "grad_norm": 1.1808844804763794, "tokens_seen": 96346112, "tokens_per_sec": 20970.797314114967, "vram_gb": 5.521900544, "lambda": 0.3252531063432495, "lambda_min": 0.07906676828861237, "lambda_max": 0.6631871461868286, "S_mean": 0.237701416015625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11780, "lr": 6.0463010347274876e-05, "train_loss": 4.578226804733276, "grad_norm": 1.2223358154296875, "tokens_seen": 96509952, "tokens_per_sec": 20970.68413714455, "vram_gb": 5.521900544, "lambda": 0.32534991142650443, "lambda_min": 0.07906542718410492, "lambda_max": 0.6634941697120667, "S_mean": 0.2528940836588542, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11800, "lr": 6.038267216727428e-05, "train_loss": 4.445030450820923, "grad_norm": 1.0823729038238525, "tokens_seen": 96673792, "tokens_per_sec": 20970.835415958667, "vram_gb": 5.521900544, "lambda": 0.32541584751258296, "lambda_min": 0.07906121015548706, "lambda_max": 0.6636941432952881, "S_mean": 0.24709065755208334, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11820, "lr": 6.0309978371295866e-05, "train_loss": 4.453354358673096, "grad_norm": 1.1148176193237305, "tokens_seen": 96837632, "tokens_per_sec": 20971.046090195214, "vram_gb": 5.521900544, "lambda": 0.3254708554595709, "lambda_min": 0.07905783504247665, "lambda_max": 0.6639188528060913, "S_mean": 0.2686055501302083, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11840, "lr": 6.024493102040633e-05, "train_loss": 4.433189392089844, "grad_norm": 1.0280065536499023, "tokens_seen": 97001472, "tokens_per_sec": 20971.255716462885, "vram_gb": 5.521900544, "lambda": 0.3255678865437706, "lambda_min": 0.07907599955797195, "lambda_max": 0.6642324924468994, "S_mean": 0.23923238118489584, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11860, "lr": 6.018753195887489e-05, "train_loss": 4.258578777313232, "grad_norm": 1.1498230695724487, "tokens_seen": 97165312, "tokens_per_sec": 20971.09983042326, "vram_gb": 5.521900544, "lambda": 0.3256492270156741, "lambda_min": 0.07906904071569443, "lambda_max": 0.6646232604980469, "S_mean": 0.24541727701822916, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11880, "lr": 6.013778281412088e-05, "train_loss": 4.3138957023620605, "grad_norm": 1.1597739458084106, "tokens_seen": 97329152, "tokens_per_sec": 20971.18869305494, "vram_gb": 5.521900544, "lambda": 0.3257286244382461, "lambda_min": 0.0790579617023468, "lambda_max": 0.6649728417396545, "S_mean": 0.24219767252604166, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11900, "lr": 6.009568499666779e-05, "train_loss": 4.308275938034058, "grad_norm": 1.191818356513977, "tokens_seen": 97492992, "tokens_per_sec": 20971.323377022054, "vram_gb": 5.521900544, "lambda": 0.325783577747643, "lambda_min": 0.07904201745986938, "lambda_max": 0.6651725769042969, "S_mean": 0.270751953125, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11920, "lr": 6.006123970010321e-05, "train_loss": 4.302724838256836, "grad_norm": 1.1345648765563965, "tokens_seen": 97656832, "tokens_per_sec": 20971.48167737898, "vram_gb": 5.521900544, "lambda": 0.3258594358339906, "lambda_min": 0.0790097787976265, "lambda_max": 0.6654186248779297, "S_mean": 0.2565714518229167, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11940, "lr": 6.003444790104484e-05, "train_loss": 4.469978332519531, "grad_norm": 1.0998772382736206, "tokens_seen": 97820672, "tokens_per_sec": 20971.644587692663, "vram_gb": 5.521900544, "lambda": 0.3259110553190112, "lambda_min": 0.0789899155497551, "lambda_max": 0.6656562685966492, "S_mean": 0.2546488444010417, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11960, "lr": 6.001531035911301e-05, "train_loss": 4.379061222076416, "grad_norm": 1.0858638286590576, "tokens_seen": 97984512, "tokens_per_sec": 20971.851419400136, "vram_gb": 5.521900544, "lambda": 0.3259719687824448, "lambda_min": 0.0790245458483696, "lambda_max": 0.6658188700675964, "S_mean": 0.25738525390625, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "step", "step": 11980, "lr": 6.000382761690909e-05, "train_loss": 4.282325267791748, "grad_norm": 1.0561312437057495, "tokens_seen": 98148352, "tokens_per_sec": 20971.98624617221, "vram_gb": 5.521900544, "lambda": 0.3260646794612209, "lambda_min": 0.07903794944286346, "lambda_max": 0.6661189794540405, "S_mean": 0.23602294921875, "G_mean": null, "G_std": null, "skipped": 0} +{"event": "eval", "step": 11999, "val_loss": 4.352607107162475, "val_ppl": 77.68072112072338, "lambda": 0.3261538151030739, "lambda_min": 0.07901874929666519, "lambda_max": 0.6664481163024902, "lambda_per_layer": [0.18580907583236694, 0.13702261447906494, 0.14401938021183014, 0.15660253167152405, 0.1727752536535263, 0.14749565720558167, 0.12383189052343369, 0.11321159452199936, 0.07901874929666519, 0.12895987927913666, 0.21961605548858643, 0.355231910943985, 0.3398432731628418, 0.5606324672698975, 0.5649167895317078, 0.6426031589508057, 0.6551628708839417, 0.3456721603870392, 0.5013359785079956, 0.6664481163024902, 0.5370559692382812, 0.5014464855194092, 0.2753685414791107, 0.2736111581325531], "S_mean": 0.2514292399088542, "G_mean": null, "G_std": null, "tokens_seen": 98304000} +{"event": "done", "step": 11999, "elapsed_s": 4688.767492055893, "nan_failures": 0} diff --git a/tokens.bin b/tokens.bin new file mode 100644 index 0000000000000000000000000000000000000000..73738ad4fbc359d8f2a0f6a23773002b177cc955 --- /dev/null +++ b/tokens.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbf14d2291c5f4ea1e26d48c7b65f33d0c5002c9c7e682df1bdd8afa42d3554c +size 800000000 diff --git a/val.bin b/val.bin new file mode 100644 index 0000000000000000000000000000000000000000..f4f39ce297b896856cfd91fd82c20ccfd7a76ef7 --- /dev/null +++ b/val.bin @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c9e54ad59fa5032a116f27c05fa63790f52789f52711fa6aeb3fd8d3c74f06cb +size 4000000